Generalizing from a straight line (first degree polynomial) to a kᵗʰ degree polynomial:
y = a₀ + a₁x + ⋯ + akxᵏ
Leads to equations in matrix form:
[n Σxᵢ ⋯ Σxᵢᵏ ] [a₀] [Σyᵢ ]
[Σxᵢ Σxᵢ² ⋯ Σxᵢᵏ⁺¹] [a₁] [Σxᵢyᵢ ]
[ ⋮ ⋮ ⋱ ⋮ ] [ ⋮ ] = [ ⋮ ]
[Σxᵢᵏ Σxᵢᵏ⁺¹ ⋯ Σxᵢ²ᵏ ] [ak] [Σxᵢᵏyᵢ]
This is a Vandermonde matrix:
[1 x₁ ⋯ x₁ᵏ] [a₀] [y₁]
[1 x₂ ⋯ x₂ᵏ] [a₁] [y₂]
[⋮ ⋮ ⋱ ⋮ ] [ ⋮ ] = [ ⋮]
[1 xn ⋯ xnᵏ] [ak] [yn]
Can write as equation:
y = Xa
Solve by premultiplying by transpose Xᵀ:
XᵀWy = XᵀWXa
Invert to yield vector solution:
a = (XᵀWX)⁻¹XᵀWy
(http://mathworld.wolfram.com/LeastSquaresFittingPolynomial.html)
For reference, the traditional way to do least squares:
_ _ __
x y - xy _ _
m = _________ b = y - mx
_ __
(x)² - x²