I want expand an L2-norm with some matrix operation inside.
Assume I have a regression $Y=X\beta+\epsilon$.
I want to solve (meaning expand),
$$\displaystyle\|Y-X\beta \|_{2}^2$$
Should I do:
1)
$$\displaystyle\|Y\|_{2}^2+2\beta^TX^TX\beta+\|X\|_{2}^2$$
or 2) $$\sum_{i=1}^{k}(y_i-x_i\beta_i)^2$$
$$\sum_{i=1}^{k}(y_i^2-2x_i\beta_i+x_i^2)$$
X: n by k matrix
$\beta$: k by 1 vector
$Y$: n by 1 vector
I think I have seen both: keeping the norm notation as I did in 1), and expand in summation for as I did in 2).
Could someone show me how to do it?
2 Answers
$\begingroup$The FOIL rule with column vectors $u, v \in \mathbb R^n$ tells us that\begin{align} \|u - v\|^2 &= (u - v)^T (u - v) \\ &= u^T u - u^T v - v^T u + v^T v\\ &= \|u\|^2 - 2 u^T v + \|v\|^2. \end{align}(In the last step, we used the fact that $u^Tv = v^Tu = \sum_{i=1}^n u_iv_i$.)
Applying this to our particular problem, we find that\begin{align} \|Y - X \beta \|^2 &= \|Y\|^2 - 2 Y^T X \beta + \| X \beta \|^2. \end{align}
If we'd like, we could rewrite the final term as\begin{align*} \|X \beta \|^2 &= (X \beta)^T (X \beta) \\ &= \beta^T X^T X \beta. \end{align*}
$\endgroup$ 3 $\begingroup$It's hard to know what you're trying to achieve. When possible, avoid coordinates until the end. What do you know here and what do you not know? What do you mean by "solve"? Your first equation would be my preference, dotting $Y-X\beta$ with itself, but you have to get the formula correct.
$\endgroup$ 5
