4 ms·
The normal equation: https://essentialproject.files.wordpress.com/2013/12/normal-equation.jpg https://essentialproject.files.wordpress.com/2013/12/normal-...
by latenightcoding 10y ago
The normal equation:
https://essentialproject.files.wordpress.com/2013/12/normal-equation.jpg https://essentialproject.files.wordpress.com/2013/12/normal-...
used to find the line of best fit.
The first time I saw it I was in high school and I was watching Andrew Ng's machine learning class for the first time.
My Math background was not the best and I didn't fully understand it until a couple of years later when I took MIT's free online course on linear algebra (OCW 18.06)
- harpocrates 10y agoFor the interested: take `B` to be the matrix minimizing `(Y - XB)'(Y - XB)` (the least squares aka the "best fit"), then `B = (X'X)^-1 X'Y`. Proof: Since `B` is a minimum, the derivative of the expression minimized wrt `B` must be zero (the "derivative" is taken in the matrix sense here). 0 = d/dB [ (Y - XB)'(Y - XB) ] = d/dB [ Y′Y − Y′XB − B′X′Y + B′X′XB ] = d/dB [ Y′Y − 2Y′XB + B′X′XB ] Y′XB and B′X′Y are just scalars :) = − 2Y′X + 2′X′XB Solving for `B` you get the result. [1] https://isites.harvard.edu/fs/docs/icb.topic515975.files/OLSDerivation.pdf https://isites.harvard.edu/fs/docs/icb.topic515975.files/OLS...
- latenightcoding 10y agoOr watch this video: https://www.youtube.com/watch?v=Y_Ac6KiQ1t0 https://www.youtube.com/watch?v=Y_Ac6KiQ1t0 One of my favourite math lectures.
- tnecniv 10y agoThis is also known as the (Moore-Penrose) Pseudoinverse.