1. Introduction
Phillip S. Kott and Dan Liao
Survey sampling is a tool used primarily for estimating the parameters of a finite population based on a randomly drawn sample of its members. Probability samples come with design (sampling) weights, which are often the inverses of the individual member selection probabilities. As long as each population element has a positive selection probability, it is a simple matter to produce an estimator for the population total of a survey variable that is unbiased with respect to the probability-sampling mechanism. The ratio of two unbiased estimators of totals or any other smooth function of estimated totals, while not necessarily unbiased, is asymptotically unbiased and often consistent since its relative variance, like its relative bias, tends to zero as the sample size grows arbitrarily large.
Deville and Särndal (1992) introduced calibration weighting as a tool for adjusting design weights in such a way that the weighted sums of certain “calibration” variables equal their known (or better-estimated) population totals. As a consequence of these calibration equations holding, the standard error of an estimated total for a variable without a known population total is often reduced while remaining nearly (i.e., asymptotically) unbiased under the probability sampling mechanism.
Although originally developed to reduce standard errors, calibration weighting has also been used to remove selection biases resulting from unit nonresponse under certain assumptions (e.g., Folsom 1991; Fuller, Loughin and Baker 1994; Lundström and Särndal 1999; Folsom and Singh 2000). To this end, whether (or not) an element selected for the sample responds to a survey is treated as an additional phase of Poisson random sampling with unknown, but positive, selection probabilities. Calibration weighting estimates these Poisson selection probabilities implicitly and produces estimated totals that are nearly unbiased under the combined sample- and response-selection mechanisms, which is often called the “quasi-sampling design”. See Oh and Scheuren (1983).
An important caveat is that although the sample-selection mechanism is fully under the control of the statistician, the response-selection mechanism is unknown. The response mechanism is assumed to have a particular form, and the failure of this assumption can result in biased estimators.
An alternative justification for calibration weighting involves a different type of modeling. It is easy to show that calibration weighting produces an estimator that is unbiased under a linear prediction (outcome) model if the expected value of the survey variable under the prediction model is a linear function of the calibration variables so long as the sampling and response mechanisms are ignorable, that is to say, the same prediction model applies whether or not the population element is sampled or whether it responds when sampled.
Unlike the selection model governing the response mechanism, it is possible for the linear prediction model to hold for one survey variable and not another. That is why most survey samplers prefer to assume a selection model when adjusting for unit nonresponse. Nevertheless, it is reassuring to know that if either model is correct, then the estimated total is nearly unbiased (i.e., has a relative bias that vanishes asymptotically), a property Kim and Park (2006) called “double protection” against nonresponse bias.
It is possible to simultaneously remove the selection bias and decrease standard error under the probability-sample mechanism in a single step by adjusting the design weights of unit respondents so that the estimated totals for a set of calibration variables equal their known population totals. Nevertheless, there are reasons for preferring the use of two calibration-weighting steps even when the sets of calibration variables used in both steps are the same or a subset of the calibration variables in a single step: the first step from the respondent sample to the original sample to remove selection bias and the second from the original sample to the population to decrease the variances of the resulting estimators.
Although Folsom and Singh (2000) and others have pointed out that calibration weighting can also be used to remove the selection bias due to under- or over-coverage of the sampling frame, we will direct our attention here on a single-stage sample drawn from a complete list frame without duplication. That is to say, we will assume that the sampling frame is identical to the target population (i.e., each population unit is listed on the frame).
The paper is structured as follows. Section 2 reviews some background theory on calibration weighting. Section 3 introduces a slightly new variance estimator that, like the variance estimator in Kott (2006), can be used to measure both the mean squared error of a calibration-weighted estimator under the quasi-sampling design and the variance under either the prediction model or the combination of the prediction model and original sampling mechanism, thus making the double protection against nonresponse bias arguably more useful for inference. The variance estimator in Kott applies only when calibrating to the population. Here we follow Folsom and Singh (2000) and allow the possibility that calibration is to the original sample.
Section 4 discusses the limitations of calibrating weighting in a single step and develops some theory for a two-step approach. Although our main purpose here is to argue the benefits of using two steps even when similar sets of calibration variables are employed in both steps, the calibration estimator we treat in this section is broader. Section 5 describes the results of some simulation experiments, while Section 6 offers a few concluding remarks.
- Date modified: