If you wish to contribute or participate in the discussions about articles you are invited to join Navipedia as a registered user

Block-Wise Weighted Least Square

From Navipedia
Jump to navigation Jump to search


FundamentalsFundamentals
Title Block-Wise Weighted Least Square
Author(s) J. Sanz Subirana, J.M. Juan Zornoza and M. Hernández-Pajares, Technical University of Catalonia, Spain.
Level Advanced
Year of Publication 2011

Let's consider two linear [math][m_1\times n], [m_2\times n][/math] equations systems, sharing the same unknown parameters vector [math]{\mathbf X}[/math]:

[math] \begin{array}{l} {\mathbf Y_1}={\mathbf G_1}\,{\mathbf X};{\mathbf R_1}\\[0.3cm] {\mathbf Y_2}={\mathbf G_2}\,{\mathbf X};{\mathbf R_2}\\ \end{array} \qquad \mbox{(1)} [/math]


where [math]{\mathbf R_1}[/math] and [math]{\mathbf R_2}[/math] are the covariance matrices of measurement vectors [math]{\mathbf Y_1}[/math], [math]{\mathbf Y_2}[/math].


Thence the two systems can be combined into a common [math][(m_1+m_2)\times n][/math] system as:

[math] \left[ \begin{array}{c} {\mathbf Y_1} \\ {\mathbf Y_2} \end{array} \right] = \left[ \begin{array}{c} {\mathbf G_1}\\[0.2cm] {\mathbf G_2} \end{array} \right] {\mathbf X} ;\qquad {\mathbf R}=\left[ \begin{array}{cc} {\mathbf R_1} & {\mathbf 0} \\[0.2cm] {\mathbf 0} & {\mathbf R_2} \end{array} \right] \qquad \mbox{(2)} [/math]


where no correlation between the two measurement vectors [math]{\mathbf Y_1}[/math] and [math]{\mathbf Y_2}[/math] is assumed in matrix [math]{\mathbf R}[/math].


From (3) and (4) (see Best Linear Unbiased Minimum-Variance Estimator (BLUE))

[math] \hat{\mathbf X}=({\mathbf G}^T\,{\mathbf R}^{-1}\,{\mathbf G})^{-1}{\mathbf G}^T\,{\mathbf R^{-1}}\,{\mathbf Y} \qquad \mbox{(3)} [/math]
[math] {\mathbf P}=({\mathbf G}^T\,{\mathbf R}^{-1}\,{\mathbf G})^{-1} \qquad \mbox{(4)} [/math]


it is easy to show that taking the corresponding augmented matrices [math]{\mathbf Y}[/math] and [math]{\mathbf G}[/math], the WLS solution of previous system (2) yields:

[math] \hat{\mathbf X}=\left [{\mathbf G_1}^T\,{\mathbf R_1}^{-1}\,{\mathbf G_1} + {\mathbf G_2}^T\,{\mathbf R_2}^{-1}\,{\mathbf G_2} \right ]^{-1} \left [{\mathbf G_1}^T\,{\mathbf R_1^{-1}}\,{\mathbf Y_1} + {\mathbf G_2}^T\,{\mathbf R_2^{-1}}\,{\mathbf Y_2} \right ] \qquad \mbox{(5)} [/math]
[math] {\mathbf P}=\left [{\mathbf G_1}^T\,{\mathbf R_1}^{-1}\,{\mathbf G_1} + {\mathbf G_2}^T\,{\mathbf R_2}^{-1}\,{\mathbf G_2} \right ]^{-1} \qquad \mbox{(6)} [/math]


Comments:

  • Recursive computation: From previous approach, the following recursive computation of estimate [math]{\mathbf X}[/math] can be written:
[math] \begin{array}{rl} {\mathbf P_1}=&\left [ {\mathbf G_1}^T\,{\mathbf R_1}^{-1}\,{\mathbf G_1} \right ]^{-1}\\[0.2cm] \hat{\mathbf X}_{(1)}=&{\mathbf P_1} \cdot \left [{\mathbf G_1}^T\,{\mathbf R_1^{-1}}\,{\mathbf Y_1} \right ]\\[0.4cm] {\mathbf P_2}=&\left [{\mathbf P_1}^{-1}+ {\mathbf G_2}^T\,{\mathbf R_2}^{-1}\,{\mathbf G_2} \right ]^{-1}\\[0.2cm] \hat{\mathbf X}_{(2)}=& {\mathbf P_2} \cdot \left [{\mathbf P_1^{-1}}\,{\mathbf X_{(1)}} + {\mathbf G_2}^T\,{\mathbf R_2^{-1}}\,{\mathbf Y_2} \right ]\\ \end{array} \qquad \mbox{(7)} [/math]


Note: If only the final estimate is desired, it is best not to process data sequentially using (7), but instead to apply (see Best Linear Unbiased Minimum-Variance Estimator (BLUE))
[math] \hat{\mathbf X}=({\mathbf G}^T\,{\mathbf R}^{-1}\,{\mathbf G})^{-1}{\mathbf G}^T\,{\mathbf R^{-1}}\,{\mathbf Y} \qquad \mbox{(8)} [/math]
and (6), that accumulates the equations without solving until the end [Bierman, 1976] [1]. This could be especially useful in case of numerical instabilities, avoiding the propagation of the numerical inaccuracies along the recursive steps.


  • Constrains: A priory information can be added to the linear system (1) as constrain equations [math]{\mathbf \Lambda}={\mathbf A} {\mathbf X}[/math] with a given weight [math]{\mathbf W}={\mathbf R_\Lambda}^{-1}[/math]. Indeed:
[math] \begin{array}{l} {\mathbf Y}={\mathbf G}\,\,{\mathbf X}~;~{\mathbf R}\\[0.1cm] {\mathbf \Lambda}={\mathbf A}\,\,{\mathbf X}~;~{\mathbf R_\Lambda} \end{array} \qquad \mbox{(9)} [/math]


References

  1. ^ [Bierman, 1976] Bierman, G., 1976. Factorization Methods fro Discrete Sequential estimation. Academic Press, New York, New York, USA.