If you wish to contribute or participate in the discussions about articles you are invited to contact the Editor

# Block-Wise Weighted Least Square

Fundamentals
Title Block-Wise Weighted Least Square
Author(s) J. Sanz Subirana, J.M. Juan Zornoza and M. Hernández-Pajares, Technical University of Catalonia, Spain.
Year of Publication 2011

Let's consider two linear $\displaystyle{ [m_1\times n], [m_2\times n] }$ equations systems, sharing the same unknown parameters vector $\displaystyle{ {\mathbf X} }$:

$\displaystyle{ \begin{array}{l} {\mathbf Y_1}={\mathbf G_1}\,{\mathbf X};{\mathbf R_1}\\[0.3cm] {\mathbf Y_2}={\mathbf G_2}\,{\mathbf X};{\mathbf R_2}\\ \end{array} \qquad \mbox{(1)} }$

where $\displaystyle{ {\mathbf R_1} }$ and $\displaystyle{ {\mathbf R_2} }$ are the covariance matrices of measurement vectors $\displaystyle{ {\mathbf Y_1} }$, $\displaystyle{ {\mathbf Y_2} }$.

Thence the two systems can be combined into a common $\displaystyle{ [(m_1+m_2)\times n] }$ system as:

$\displaystyle{ \left[ \begin{array}{c} {\mathbf Y_1} \\ {\mathbf Y_2} \end{array} \right] = \left[ \begin{array}{c} {\mathbf G_1}\\[0.2cm] {\mathbf G_2} \end{array} \right] {\mathbf X} ;\qquad {\mathbf R}=\left[ \begin{array}{cc} {\mathbf R_1} & {\mathbf 0} \\[0.2cm] {\mathbf 0} & {\mathbf R_2} \end{array} \right] \qquad \mbox{(2)} }$

where no correlation between the two measurement vectors $\displaystyle{ {\mathbf Y_1} }$ and $\displaystyle{ {\mathbf Y_2} }$ is assumed in matrix $\displaystyle{ {\mathbf R} }$.

From (3) and (4) (see Best Linear Unbiased Minimum-Variance Estimator (BLUE))

$\displaystyle{ \hat{\mathbf X}=({\mathbf G}^T\,{\mathbf R}^{-1}\,{\mathbf G})^{-1}{\mathbf G}^T\,{\mathbf R^{-1}}\,{\mathbf Y} \qquad \mbox{(3)} }$
$\displaystyle{ {\mathbf P}=({\mathbf G}^T\,{\mathbf R}^{-1}\,{\mathbf G})^{-1} \qquad \mbox{(4)} }$

it is easy to show that taking the corresponding augmented matrices $\displaystyle{ {\mathbf Y} }$ and $\displaystyle{ {\mathbf G} }$, the WLS solution of previous system (2) yields:

$\displaystyle{ \hat{\mathbf X}=\left [{\mathbf G_1}^T\,{\mathbf R_1}^{-1}\,{\mathbf G_1} + {\mathbf G_2}^T\,{\mathbf R_2}^{-1}\,{\mathbf G_2} \right ]^{-1} \left [{\mathbf G_1}^T\,{\mathbf R_1^{-1}}\,{\mathbf Y_1} + {\mathbf G_2}^T\,{\mathbf R_2^{-1}}\,{\mathbf Y_2} \right ] \qquad \mbox{(5)} }$
$\displaystyle{ {\mathbf P}=\left [{\mathbf G_1}^T\,{\mathbf R_1}^{-1}\,{\mathbf G_1} + {\mathbf G_2}^T\,{\mathbf R_2}^{-1}\,{\mathbf G_2} \right ]^{-1} \qquad \mbox{(6)} }$

• Recursive computation: From previous approach, the following recursive computation of estimate $\displaystyle{ {\mathbf X} }$ can be written:
$\displaystyle{ \begin{array}{rl} {\mathbf P_1}=&\left [ {\mathbf G_1}^T\,{\mathbf R_1}^{-1}\,{\mathbf G_1} \right ]^{-1}\\[0.2cm] \hat{\mathbf X}_{(1)}=&{\mathbf P_1} \cdot \left [{\mathbf G_1}^T\,{\mathbf R_1^{-1}}\,{\mathbf Y_1} \right ]\\[0.4cm] {\mathbf P_2}=&\left [{\mathbf P_1}^{-1}+ {\mathbf G_2}^T\,{\mathbf R_2}^{-1}\,{\mathbf G_2} \right ]^{-1}\\[0.2cm] \hat{\mathbf X}_{(2)}=& {\mathbf P_2} \cdot \left [{\mathbf P_1^{-1}}\,{\mathbf X_{(1)}} + {\mathbf G_2}^T\,{\mathbf R_2^{-1}}\,{\mathbf Y_2} \right ]\\ \end{array} \qquad \mbox{(7)} }$

Note: If only the final estimate is desired, it is best not to process data sequentially using (7), but instead to apply (see Best Linear Unbiased Minimum-Variance Estimator (BLUE))
$\displaystyle{ \hat{\mathbf X}=({\mathbf G}^T\,{\mathbf R}^{-1}\,{\mathbf G})^{-1}{\mathbf G}^T\,{\mathbf R^{-1}}\,{\mathbf Y} \qquad \mbox{(8)} }$
and (6), that accumulates the equations without solving until the end [Bierman, 1976] [1]. This could be especially useful in case of numerical instabilities, avoiding the propagation of the numerical inaccuracies along the recursive steps.

• Constrains: A priory information can be added to the linear system (1) as constrain equations $\displaystyle{ {\mathbf \Lambda}={\mathbf A} {\mathbf X} }$ with a given weight $\displaystyle{ {\mathbf W}={\mathbf R_\Lambda}^{-1} }$. Indeed:
$\displaystyle{ \begin{array}{l} {\mathbf Y}={\mathbf G}\,\,{\mathbf X}~;~{\mathbf R}\\[0.1cm] {\mathbf \Lambda}={\mathbf A}\,\,{\mathbf X}~;~{\mathbf R_\Lambda} \end{array} \qquad \mbox{(9)} }$

## References

1. ^ [Bierman, 1976] Bierman, G., 1976. Factorization Methods fro Discrete Sequential estimation. Academic Press, New York, New York, USA.