7. Discussion and conclusions

Alina Matei and M. Giovanna Ranalli

Previous

We have proposed a reweighting system to compensate for non-ignorable nonresponse based on a latent auxiliary variable. This variable is computed for each unit in the sample using a latent model assuming the existence of item nonresponse and that the same latent structure is hidden behind item and unit nonresponse. Unit response probabilities are then estimated by a logistic model that uses as a covariate the latent trait extracted by the response patterns using a latent trait model. The proposed reweighting system is then used in a three-phase estimator to handle nonresponse, together with a replication method to estimate its uncertainty. The main goal is to reduce nonresponse bias in the estimation of the population total. The proposed estimator performs well in our simulation studies compared with the naive estimator, and the gain in efficiency is substantial in certain cases. Reductions in bias are also seen when the correlation between the latent trait and the variable of interest is modest.

By design, the estimated latent variable ${\widehat{\theta }}_k$ is related to the response indicators $x_{kj}$ for the variable of interest $y_j;$ since nonresponse is assumed to be non-ignorable, $y_{kj}$ and $x_{kj}$ are related as well. If the following condition holds,

$${\rho }_{y_j\mathrm{,}{x}_j}^2+{\rho }_{\widehat{\theta }\mathrm{,}{x}_j}^2>1,$$

where the correlation coefficients ${\rho }_{y_j\mathrm{,}{x}_j}\mathrm{,}{\rho }_{\widehat{\theta }\mathrm{,}{x}_j}>0,$ then $y_j$ and $\widehat{\theta }$ are positively correlated (see Langford, Schwertman and Owens 2001). Note that the minimum degree of correlation between the variable of interest and the latent variable capable of reducing the nonresponse bias was found to be 0.3 in simulation setting 2 (Section 6.2). Of course, bias reduction depends on model assumptions. If response indicators are not good predictors of unit response behavior, then model misspecification is present and, of course, reduction in bias may not be present and variance could be introduced in estimation. Nonetheless, diagnostic tools from item response theory can be used to assess the goodness of fit of the latent trait model employed to estimate values for ${\theta }_k.$

We have considered the case in which no auxiliary information is available at the sample or population level to reduce nonresponse bias. Observed covariates (if available) and the latent variable can be, however, used together in the estimation of response probabilities. Moreover, latent trait models can, themselves, be fitted with covariates. The introduction of covariates in these models should be carried out with increasing prudence on variance.

The proposed estimator is a three-phase estimator using a reweighting system based on ${\widehat{p}}_k$ and ${\widehat{q}}_{kj}.$ It is known that small values of ${\widehat{p}}_k$ and ${\widehat{q}}_{kj}$ may lead to unstable reweighted estimators because of large nonresponse weights. To overcome this problem, the propensity score method (e.g., Eltinge and Yansaneh 1997) is often used in practice, providing a good solution against extreme weights adjustments. In order to apply this method in our framework, the respondents to $y_j$ should be grouped in different classes given by the quantiles of $1/\left({\widehat{p}}_k{\widehat{q}}_{kj}\right).$ The final step is the calculation of a weight for each class.

Final remarks concern the conditional independence assumption in latent trait models. In nonresponse literature, it is usual to use Poisson sampling to model unit response behavior by assuming that units in the set $r$ are selected with unknown response probabilities and that response is independent from unit to unit. The conditional independence assumption in the latent trait models is a similar condition applied to items. Both assumptions are strong, sometimes they are in doubt, yet they are necessary in the statistical inferential process.

Different methods were developed in psychometric literature to relax the conditional independence assumption. We cite here the partial independence approach by Reardon and Raudenbush (2006), developed for the case where responses to earlier questions determine whether later questions are asked or not, and where the usual conditional independence assumption of standard models fails. This approach could be used in our framework for the case where $q_{k\ell }$ is defined as $P(x_{k\ell }=1|x_{kj}\mathrm{,}$ for some $j\in \left\{\mathrm{1,}\dots \mathrm{,}m\right\}\mathrm{,}\ell \ne j\mathrm{,}{\theta }_k)$ instead of $P\left(x_{k\ell }=1|{\theta }_k\right)\mathrm{,}k\in r\mathrm{.}$ Another useful approach for cases where items are clustered is the latent trait hierarchical modeling. A random effect is introduced into a latent trait model to account for potential residual dependence due to the common sources of variation shared by clusters of items (see e.g., Scott and Ip 2002). Further research should be done to accommodate these approaches in the survey sampling framework.

Acknowledgements

The work of M. Giovanna Ranalli has been developed partially under the support of the project PRIN-SURWEY (grant 2012F42NS8, Italy).

References

Bartholomew, D.J., Steele, F., Moustaki, I. and Galbraith, J.I. (2002). The Analysis and Interpretation of Multivariate Data for Social Scientists. Chapman and Hall/CRC.

Beaumont, J.-F. (2000). An estimation method for nonignorable nonresponse. Survey Methodology, 26, 2, 131-136.

Bethlehem, J. (1988). Reduction of nonresponse bias through regression estimation. Journal of Official Statistics, 4, 3, 251-260.

Biemer, P.P., and Link, M.W. (2007). Evaluating and Modeling early Cooperator Effects in RDD Surveys. New York: John Wiley & Sons, Inc.

Bond, T., and Fox, C. (2007). Applying the Rasch model: Fundamental Measurement in the Human Sciences (2^nd Ed.). Lawrence Erlbaum Associates, Inc, Mahwah, N.J.

Cassel, C.M., Särndal, C.-E. and Wretman, J.H. (1983). Some uses of statistical models in connection with the nonresponse problem. In Incomplete Data in Sample Surveys, (Eds., W.G. Madow and I. Olkin), New York: Academic Press. 3, 143-160.

Chambers, R.L., and Skinner, C. (2003). Analysis of Survey Data. New York: John Wiley & Sons, Inc.

Copas, A.J., and Farewell, V.T. (1998). Dealing with non-ignorable non-response by using an ‘enthusiasm-to-respond’ variable. Journal of the Royal Statistical Society, Series A, 161, 385-396.

De Menezes, L.M., and Bartholomew, D.J. (1996). New developments in latent structure analysis applied to social attitudes. Journal of Royal Statistical Society, Series A, 159, 213-224.

Deville, J.-C., Särndal, C.-E. and Sautory, O. (1993). Generalized raking procedure in survey sampling. Journal of the American Statistical Association, 88, 1013-1020.

Drew, J.H., and Fuller, W.A. (1980). Modeling nonresponse in surveys with callbacks. Proceedings of the Survey Research Methods Section, American Statistical Association.

Eltinge, J.L., and Yansaneh, I.S. (1997). Diagnostics for formation of nonresponse adjustment cells, with an application to income nonresponse in the U.S. Consumer Expenditure Survey. Survey Methodology, 23, 1, 33-40.

Greenlees, J.S., Reece, W.S. and Zieschang, K.D. (1982). Imputation of missing values when the probability of response depends on the variable being imputed. Journal of the American Statistical Association, 77, 251-261.

Groves, R.M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70, 5, 646-675.

Groves, R.M., Couper, M., Presser, S., Singer, E., Tourangeau, R., Acosta, G.P. and Nelson, L. (2006). Experiments in producing nonresponse bias. Public Opinion Quarterly, 70, 5, 720-736.

Kim, J.K., and Kim, J.J. (2007). Nonresponse weighting adjustment using estimated response probability. Canadian Journal of Statistics, 35, 501-514.

Kim, J.K., Navarro, A. and Fuller, W.A. (2006). Replication variance estimation for two-phase stratified sampling. Journal of the American Statistical Association, 101, 473, 312-320.

Kott, P.S. (2006). Using calibration weighting to adjust for nonresponse and coverage errors. Survey Methodology, 32, 2, 133-142.

Langford, E., Schwertman, N. and Owens, M. (2001). Is the property of being positively correlated transitive? The American Statistician, 55, 4, 322-325.

Legg, J.C., and Fuller, W.A. (2009). Two-phase sampling. Handbook of Statistics, 29, 55-70.

Little, R.J., and Vartivarian, S. (2005). Does weighting for nonresponse increase the variance of survey means? Survey Methodology, 31, 2, 161-168.

Little, R.J.A., and Rubin, D.B. (1987). Statistical Analysis with Missing Data. New York: John Wiley & Sons, Inc.

Moran, P.A.P. (1986). Identification problems in latent trait models. British Journal of Mathematical and Statistical Psychology, 39, 2, 208-212.

Moustaki, I., and Knott, M. (2000). Weighting for item non-response in attitude scales using latent variable models with covariates. Journal of Royal Statistical Society, Series A, 163, 445-459.

Oh, H.L., and Scheuren, F.J. (1983). Weighting adjustments for unit non-response. In Incomplete Data in Sample Surveys, (Eds., W.G. Madow, I. Olkin and D.B. Rubin). New York: Academic Press. 2, 143-184.

Olsson, U., Drasgow, F. and Dorans, N. (1982). The polyserial correlation coefficient. Psychometrika, 47, 337-347.

Qin, J., Leung, D. and Shao, J. (2002). Estimation with survey data under nonignorable nonresponse or informative sampling. Journal of the American Statistical Association, 97, 193-200.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. The Danish Institute of Educational Research, Copenhagen.

Reardon, S.F., and Raudenbush, S.W. (2006). A partial independence item response model for surveys with filter questions. Sociological Methodology, 36, 1, 257-300.

Rizopoulos, D. (2006). ltm: An R package for latent variable modelling and item response theory analyses. Journal of Statistical Software, 17, 5, 1-25.

Rosenbaum, P.R., and Rubin, D.B. (1983). The central role of the propensity score inobservational studies for causal effects. Biometrika, 70, 41-55.

Särndal, C.-E., and Lundström, S. (2005). Estimation in Surveys with Nonresponse. New York: John Wiley & Sons, Inc.

Scott, S.L., and Ip, E.H. (2002). Empirical bayes and item-clustering effects in a latent variable hierarchical model: A case study from the national assessment of educational progress. Journal of American Statistical Association, 97, 459, 1-11.

Skrondal, A., and Rabe-Hesketh, S. (2007). Latent variable modelling: A survey. Scandinavian Journal of Statistics, 34, 712-745.

Tillé, Y., and Matei, A. (2012). Sampling: Survey Sampling. R package version 2.5.

Wright, B. (1996). Local dependency, correlations and principal components. Rasch Meas Trans, 10-3, 509-511.

Zhang, L.C. (2002). A method of weighting adjustment for survey data subject to nonignorable nonresponse. DACSEIS research paper no. 2, http://w210.ub.unituebingen.de/dbt/volltexte/2002/451.

Previous