3. Modified regression estimation for changing survey frames

John Preston

Previous | Next

The MR estimators can be extended to the situation of changing survey frames by adding "births� into the population at the previous time period, and adding "deaths� into the population at the current time period to create a "pseudo-population� (Diagram 3.1). These "pseudo-populations� will satisfy the requirement that the units in the population remain unchanged between the previous and current time periods. A full description of the extension to the MR estimator for changing survey frames is outlined below.

Consider a dynamic population which changes over time due to the addition of "births� and the deletion of "deaths�. At time $t,$ the union of $U_h^{\left(t\right)}$ and $U_h^{\left(t-1\right)}$ can be divided into three components. The first component consists of units in the population in stratum $h$ at time $t-1$ but not at time $t,$ referred to as the "death� population $U_{dh}^{\left(t-1\right)}$ in stratum $h,$ comprised of $N_{dh}^{\left(t-1\right)}$ units. The second component consists of units in the population in stratum $h$ at time $t-1$ and time $t,$ referred to as the "common� population $U_{ch}^{\left(t-1\right)}=U_{ch}^{\left(t\right)}$ in stratum $h,$ comprised of $N_{ch}^{\left(t-1\right)}=N_{ch}^{\left(t\right)}$ units. The third component consists of units in the population in stratum $h$ at time $t$ but not at time $t-1,$ referred to as the "birth� population $U_{bh}^{\left(t\right)}$ in stratum $h,$ comprised of $N_{bh}^{\left(t\right)}$ units. Those units in the population which change stratum between time $t-1$ and $t$ are included in the "death� population $U_{dh}^{\left(t-1\right)}$ under their stratum at time $t-1$ and are also included in the "birth� population $U_{bh}^{\left(t\right)}$ under their stratum at time $t.$

Diagram 3.1 Standard and pseudo populations and samples

Description for Diagram 3.1

At time $t>1$ define the "pseudo-population� $U_h^{*\left(t-1\right)}=U_h^{*\left(t\right)}$ in stratum $h$ as the union of $U_h^{\left(t\right)}$ and $U_h^{\left(t-1\right)},$ comprised of $N_h^{*\left(t-1\right)}=N_h^{*\left(t\right)}=N_{dh}^{\left(t-1\right)}+N_{ch}^{\left(t-1\right)}+N_{bh}^{\left(t\right)}$ units. It is important to note that the "pseudo-population� $U_h^{*\left(t-1\right)}$ at time $t-1$ is different to the "pseudo-population� $U_h^{*\left(t-1\right)}$ at time $t,$ as the "pseudo-population� $U_h^{*\left(t-1\right)}$ at time $t-1$ is based on the union of $U_h^{\left(t-2\right)}$ and $U_h^{\left(t-1\right)},$ while the "pseudo-population� $U_h^{*\left(t-1\right)}$ at time $t$ is based on the union of $U_h^{\left(t-1\right)}$ and $U_h^{\left(t\right)}.$ Hence the "pseudo-populations� for the current and previous time periods need to be calculated at each time period. Define the "pseudo-values� for the variable of interest $y$ for unit $i$ at time $t-1$ and time $t$ as:

$$y_i^{*\left(t-1\right)}=\{\begin{array}{ll}{y}_i^{\left(t-1\right)},\hfill & \text{if }i\in U_{ch}^{\left(t-1\right)}\hfill \\ 0,\hfill & \text{if }i\in U_{bh}^{\left(t\right)}\hfill \end{array}$$

$$y_i^{*\left(t\right)}=\{\begin{array}{ll}{y}_i^{\left(t\right)},\hfill & \text{if }i\in U_{ch}^{\left(t\right)}\hfill \\ 0,\hfill & \text{if }i\in U_{dh}^{\left(t-1\right)}\hfill \end{array}$$

and define the "pseudo-values� for the auxiliary variables $x$ for unit $i$ at time $t-1$ and time $t$ as:

$$x_i^{*\left(t-1\right)}=\{\begin{array}{ll}{x}_i^{\left(t-1\right)},\hfill & \text{if }i\in U_{ch}^{\left(t-1\right)}\hfill \\ 0,\hfill & \text{if }i\in U_{bh}^{\left(t\right)}\hfill \end{array}$$

$$x_i^{*\left(t\right)}=\{\begin{array}{ll}{x}_i^{\left(t\right)},\hfill & \text{if }i\in U_{ch}^{\left(t\right)}\hfill \\ 0,\hfill & \text{if }i\in U_{dh}^{\left(t-1\right)}.\hfill \end{array}$$

At time $t>1$ denote $s_h^{*\left(t-1\right)}$ and $s_h^{*\left(t\right)}$ as the "pseudo-samples� in stratum $h,$ where $s_h^{*\left(t-1\right)}$ consists of all units selected in the original sample $s_h^{\left(t-1\right)}$ in stratum $h$ at time $t-1$ plus a random sample of units $s_{bh}^{\left(t\right)}$ from the "birth� population $U_{bh}^{\left(t\right)}$ in stratum $h$ at time $t$ selected with inclusion probabilities ${\pi }_i^{\left(t-1\right)}=n_h^{\left(t-1\right)}/N_h^{\left(t-1\right)}\left(i\in U_h^{\left(t-1\right)}\right),$ and $s_h^{*\left(t\right)}$ consists of all units selected in the original sample $s_h^{\left(t\right)}$ in stratum $h$ at time $t$ plus a random sample of units $s_{dh}^{\left(t-1\right)}$ from the "death� population $U_{dh}^{\left(t-1\right)}$ in stratum $h$ at time $t-1$ selected with inclusion probabilities ${\pi }_i^{\left(t\right)}=n_h^{\left(t\right)}/N_h^{\left(t\right)}\left(i\in U_h^{\left(t\right)}\right).$ Let $n_h^{*\left(t-1\right)}$ and $n_h^{*\left(t\right)}$ denote the sample sizes in the "pseudo-samples� $s_h^{*\left(t-1\right)}$ and $s_h^{*\left(t\right)}$ respectively. Once again it is important to note that the "pseudo-sample� $s_h^{*\left(t-1\right)}$ at time $t-1$ is different to the "pseudo-sample� $s_h^{*\left(t-1\right)}$ at time $t,$ as the "pseudo-sample� $s_h^{*\left(t-1\right)}$ at time $t-1$ includes a random sample of units from the "birth� population at time $t-1,$ while the "pseudo-sample� $s_h^{*\left(t-1\right)}$ at time $t$ includes a random sample of units from the "death� population at time $t-1.$ Hence the "pseudo-samples� for the current and previous time periods need to be calculated at each time period.

The choice of an appropriate sample selection technique, for the selection of the additional random samples of units from the "birth� and "death� populations, will depend on the sample selection technique used to select the original samples. Many repeated business surveys select their samples using a permanent random number (PRN) selection technique, to enable some control of the rotation of units into and out of sample from one time period to the next. Consider the simplest case where the original samples $s_h^{\left(t-1\right)}$ and $s_h^{\left(t\right)}$ in stratum $h$ described by $\left\{i\in U_h^{\left(t-1\right)}\&R_i\in \left[S_h^{\left(t-1\right)},E_h^{\left(t-1\right)}\right)\right\}$ and $\left\{i\in U_h^{\left(t\right)}\&R_i\in \left[S_h^{\left(t\right)},E_h^{\left(t\right)}\right)\right\},$ where $S_h^{\left(t\right)}$ and $E_h^{\left(t\right)}$ are the selection interval start and end points in stratum $h$ at time $t,$ and $R_i$ is the permanent random number for unit $i.$ In this case the "pseudo-samples� $s_h^{*\left(t-1\right)}$ and $s_h^{*\left(t\right)}$ in stratum $h$ are described by $\left\{i\in U_h^{*\left(t-1\right)}\&R_i\in \left[S_h^{\left(t-1\right)},E_h^{\left(t-1\right)}\right)\right\}$ and $\left\{i\in U_h^{*\left(t\right)}\&R_i\in \left[S_h^{\left(t\right)},E_h^{\left(t\right)}\right)\right\}.$ This selection technique will give a similar amount of overlap between the samples from the "death� population at time $t-1$ and $t$ and between the samples from the "birth� population at time $t-1$ and $t$ as between the samples from the "common� population at time $t-1$ and $t.$ Clearly the amount of overlap between the samples from the "death� and "birth� populations will affect the behaviour of the estimates and optimising the amount of overlap could be investigated.

Define the "pseudo-design weights� $w_i^{*\left(t-1\right)}=1/{\pi }_i^{\left(t-1\right)}$ for all units in the "pseudo-sample� $s_h^{*\left(t-1\right)}$ and $w_i^{*\left(t\right)}=1/{\pi }_i^{\left(t\right)}$ for all units in the "pseudo-sample� $s_h^{*\left(t\right)}.$ Since the "pseudo-design weights� for the original sampled units are equal to the original design weights and the "pseudo-values� for the variable of interest are equal to zero for the additional sampled units from the "birth� and "death� populations, then the HT estimator ${\widehat{Y}}_{\text{HT}}^{*\left(t\right)}={\displaystyle {\sum }_{h=1}^H{\displaystyle {\sum }_{i\in s_h^{*\left(t\right)}}{w}_i^{*\left(t\right)}{y}_i^{*\left(t\right)}}}$ based on the "pseudo-sample�, "pseudo-values� and "pseudo-design weights� is equivalent to the HT estimator ${\widehat{Y}}_{\text{HT}}^{\left(t\right)}={\displaystyle {\sum }_{h=1}^H{\displaystyle {\sum }_{i\in s_h^{\left(t\right)}}{w}_i^{\left(t\right)}{y}_i^{\left(t\right)}}}$ based on the original sample, original values and original design weights. Hence the inclusion of these additional sampled units into the "pseudo-sample� from the "birth� and "death� populations will not introduce any extra variability into the point-in-time estimates.

The proposed MR estimator for the special case of changing survey frames can be written as:

$${\widehat{Y}}_{\text{MR}}^{*\left(t\right)}={\displaystyle \sum _{i\in s^{*\left(t\right)}}{\stackrel{⌣}{w}}_i^{*\left(t\right)}{y}_i^{*\left(t\right)}}(3.1)$$

where ${\stackrel{⌣}{w}}_i^{*\left(t\right)}=w_i^{*\left(t\right)}{\stackrel{⌣}{g}}_i^{*\left(t\right)}$ and ${\stackrel{⌣}{g}}_i^{*\left(t\right)}$ is the "pseudo $-g-$ weight� for unit $i$ at time $t$ given by:

$$\begin{array}{lll}{\stackrel{⌣}{g}}_i^{*\left(t\right)}\hfill & =\hfill & 1+{\left(\left(X^{\left(t\right)},{\tilde{Z}}^{*\left(t\right)}\right)-\left({\widehat{X}}_{\text{HT}}^{\left(t\right)},{\widehat{Z}}_{\text{HT}}^{*\left(t\right)}\right)\right)}^T\hfill \\ \hfill & \times \hfill & {\left({\displaystyle \sum _{i\in s^{*\left(t\right)}}\frac{w_i^{*\left(t\right)}\left(x_i^{*\left(t\right)},z_i^{*\left(t\right)}\right){\left(x_i^{*\left(t\right)},z_i^{*\left(t\right)}\right)}^T}{c_i^{\left(t\right)}}}\right)}^{-1}\frac{\left(x_i^{*\left(t\right)},z_i^{*\left(t\right)}\right)}{c_i^{\left(t\right)}}(3.2)\hfill \end{array}$$

and the MR1, MR2 and MRR values for the "pseudo-composite auxiliary variables� are given by:

$$z_{\left(\text{MR}1\right)i}^{*\left(t\right)}=\{\begin{array}{ll}{R}_h^{\left(t-1,t\right)}{y}_i^{*\left(t-1\right)},\hfill & \text{if }i\in s_h^{*\left(t\right)}\cap s_h^{*\left(t-1\right)}\text{ and }{s}_h^{*\left(t\right)}\backslash s_h^{*\left(t-1\right)}\ne \varnothing \hfill \\ R_h^{\left(t-1,t\right)}\left(\frac{{\displaystyle \sum _{i\in s_h^{\left(t\right)}}{w}_i^{\left(t\right)}}}{{\displaystyle \sum _{i\in s_h^{*\left(t\right)}\cap s_h^{\left(t-1\right)}}{w}_i^{*\left(t\right)}}}\right)y_i^{*\left(t-1\right)},\hfill & \text{if }i\in s_h^{*\left(t\right)}\cap s_h^{*\left(t-1\right)}\text{ and }{s}_h^{*\left(t\right)}\backslash s_h^{*\left(t-1\right)}=\varnothing \hfill \\ R_h^{\left(t-1,t\right)}\left(\frac{\left({\displaystyle \sum _{i\in s_h^{\left(t\right)}}{w}_i^{\left(t\right)}}-{\displaystyle \sum _{i\in s_h^{*\left(t\right)}\cap s_h^{\left(t-1\right)}}{w}_i^{*\left(t\right)}}\right)}{{\displaystyle \sum _{i\in s_h^{*\left(t\right)}\backslash s_h^{*\left(t-1\right)}}{w}_i^{*\left(t\right)}}}\right){\overline{Y}}_{\left(\text{MR}\right)h}^{\left(t-1\right)},\hfill & \text{if }i\in s_h^{*\left(t\right)}\backslash s_h^{*\left(t-1\right)}.\hfill \end{array}(3.3)$$

$$z_{\left(\text{MR}2\right)i}^{*\left(t\right)}=\{\begin{array}{ll}{R}_h^{\left(t-1,t\right)}\{\left(\frac{{\displaystyle \sum _{i\in s_h^{\left(t\right)}}{w}_i^{\left(t\right)}}}{{\displaystyle \sum _{i\in s_h^{*\left(t\right)}\cap s_h^{\left(t-1\right)}}{w}_i^{*\left(t\right)}}}\right)y_i^{*\left(t-1\right)}+\left(1-\left(\frac{{\displaystyle \sum _{i\in s_h^{\left(t\right)}}{w}_i^{\left(t\right)}}}{{\displaystyle \sum _{i\in s_h^{\left(t\right)}\cap s_h^{*\left(t-1\right)}}{w}_i^{*\left(t\right)}}}\right)\right)y_i^{*\left(t\right)}\},\hfill & \text{if }i\in s_h^{*\left(t\right)}\cap s_h^{*\left(t-1\right)}\hfill \\ R_h^{\left(t-1,t\right)}{y}_i^{*\left(t\right)},\hfill & \text{if }i\in s_h^{*\left(t\right)}\backslash s_h^{*\left(t-1\right)}.\hfill \end{array}(3.4)$$

$$z_{\left(\text{MRR}\right)i}^{*\left(t\right)}=\{\begin{array}{ll}{R}_h^{\left(t-1,t\right)}{y}_i^{*\left(t-1\right)},\hfill & \text{if }i\in s_h^{*\left(t\right)}\cap s_h^{*\left(t-1\right)}\text{ and }{s}_h^{*\left(t\right)}\backslash s_h^{*\left(t-1\right)}\ne \varnothing \hfill \\ R_h^{\left(t-1,t\right)}\left({\displaystyle \sum _{i\in s_h^{\left(t\right)}}{w}_i^{\left(t\right)}}/{\displaystyle \sum _{i\in s_h^{*\left(t\right)}\cap s_h^{\left(t-1\right)}}{w}_i^{*\left(t\right)}}\right)y_i^{*\left(t-1\right)},\hfill & \text{if }i\in s_h^{*\left(t\right)}\cap s_h^{*\left(t-1\right)}\text{ and }{s}_h^{*\left(t\right)}\backslash s_h^{*\left(t-1\right)}=\varnothing \hfill \\ R_h^{\left(t-1,t\right)}\{y_i^{*\left(t\right)}-[\left({\displaystyle \sum _{i\in s_h^{\left(t\right)}\backslash s_h^{*\left(t-1\right)}}{w}_i^{*\left(t\right)}}/{\displaystyle \sum _{i\in s_h^{*\left(t\right)}\backslash s_h^{*\left(t-1\right)}}{w}_i^{*\left(t\right)}}\right)\hfill & \hfill \\ \times \left({\displaystyle \sum _{i\in s_h^{\left(t\right)}\cap s_h^{*\left(t-1\right)}}{w}_i^{*\left(t\right)}{y}_i^{*\left(t\right)}}/{\displaystyle \sum _{i\in s^{\left(t\right)}\cap s^{*\left(t-1\right)}}{w}_i^{*\left(t\right)}}\right)]\hfill & \hfill \\ +[\left(\left({\displaystyle \sum _{i\in s_h^{\left(t\right)}}{w}_i^{\left(t\right)}}-{\displaystyle \sum _{i\in s_h^{*\left(t\right)}\cap s_h^{\left(t-1\right)}}{w}_i^{*\left(t\right)}}\right)/{\displaystyle \sum _{i\in s_h^{*\left(t\right)}\backslash s_h^{*\left(t-1\right)}}{w}_i^{*\left(t\right)}}\right)\hfill & \hfill \\ \times \left({\displaystyle \sum _{i\in s_h^{*\left(t\right)}\cap s_h^{\left(t-1\right)}}{w}_i^{*\left(t\right)}{y}_i^{*\left(t-1\right)}}/{\displaystyle \sum _{i\in s_h^{*\left(t\right)}\cap s_h^{\left(t-1\right)}}{w}_i^{*\left(t\right)}}\right)]\},\hfill & \text{if }i\in s_h^{*\left(t\right)}\backslash s_h^{*\left(t-1\right)}.\hfill \end{array}(3.5)$$

where $R_h^{\left(t-1,t\right)}=\left({\displaystyle {\sum }_{i\in s_h^{\left(t-1\right)}}{w}_i^{\left(t-1\right)}}/{\displaystyle {\sum }_{i\in s_h^{\left(t\right)}}{w}_i^{\left(t\right)}}\right)$ is a correction factor applied to the MR1, MR2 and MRR values to account for the relative change in the population size in stratum $h$ between time $t-1$ and time $t.$ The other adjustments to the MR2 and MRR values were made to ensure that the HT estimator for the "pseudo-composite auxiliary variables� ${\widehat{Z}}_{\text{HT}}^{*\left(t\right)}={\displaystyle {\sum }_{h=1}^H{\displaystyle {\sum }_{i\in s_h^{*\left(t\right)}}{w}_i^{*\left(t\right)}{z}_i^{*\left(t\right)}}}$ at time $t$ is unbiased for the corresponding key survey variables $Y^{\left(t-1\right)}$ at time $t-1.$ A simple proof of the unbiasedness of the HT estimator for the "pseudo-composite auxiliary variables� is shown in the Appendix.

The HT estimator ${\widehat{Y}}_{\text{HT}}^{*\left(t\right)}={\displaystyle {\sum }_{h=1}^H{\displaystyle {\sum }_{i\in s_h^{*\left(t\right)}}{w}_i^{*\left(t\right)}{y}_i^{*\left(t\right)}}}$ is equivalent to ${\widehat{Y}}_{\text{HT}}^{\left(t\right)}={\displaystyle {\sum }_{h=1}^H{\displaystyle {\sum }_{i\in s_h^{\left(t\right)}}{w}_i^{\left(t\right)}{y}_i^{\left(t\right)}}}$ since the "pseudo-values� for the variable of interest are equal to zero for the additional sampled units from the "birth� and "death� populations. Similarly the GR estimator ${\widehat{Y}}_{\text{GR}}^{*\left(t\right)}={\displaystyle {\sum }_{h=1}^H{\displaystyle {\sum }_{i\in s_h^{*\left(t\right)}}{\tilde{w}}_i^{*\left(t\right)}{y}_i^{*\left(t\right)}}}$ is equivalent to ${\widehat{Y}}_{\text{GR}}^{\left(t\right)}={\displaystyle {\sum }_{h=1}^H{\displaystyle {\sum }_{i\in s_h^{\left(t\right)}}{\tilde{w}}_i^{\left(t\right)}{y}_i^{\left(t\right)}}}$ since the "pseudo-values� for the variable of interest and the auxiliary variables are equal to zero for the additional sampled units from the "birth� and "death� populations. However, the MR estimator ${\widehat{Y}}_{\text{MR}}^{*\left(t\right)}={\displaystyle {\sum }_{h=1}^H{\displaystyle {\sum }_{i\in s_h^{*\left(t\right)}}{\stackrel{⌣}{w}}_i^{*\left(t\right)}{y}_i^{*\left(t\right)}}}$ is not equivalent to ${\widehat{Y}}_{\text{MR}}^{\left(t\right)}={\displaystyle {\sum }_{h=1}^H{\displaystyle {\sum }_{i\in s_h^{\left(t\right)}}{\stackrel{⌣}{w}}_i^{\left(t\right)}{y}_i^{\left(t\right)}}}$ since the "pseudo-values� for the composite auxiliary variables are not equal to zero for the additional sampled units from the "birth� and "death� populations.

The proposed procedure of adding "births� into the population at the previous time period and adding "deaths� into the population at the current time period is performed independently at each time period, so there is no accumulation of "births� and "deaths� in the "pseudo-population� over time.

Previous | Next