3 A dynamic adaptive survey design: Re-assigning interviewers in a follow-up survey

Barry Schouten, Melania Calinescu and Annemieke Luiten

Previous | Next

In this section, we provide an example of a dynamic adaptive design: the re-assignment of interviewers based on observations of the propensity to cooperate. The example is based on hypothetical response propensities and cost functions. Interviewers are assigned to sample cases that have refused once, based on an assessment of the propensity to respond made during a first phase of the survey. The assessment is made for respondents and refusers, but it is not available for sample units who were not contacted during the first phase. It provides a judgement on the propensity that the sample unit participates in the survey when contacted again. The assessment is made on a three point scale: easy, medium, difficult. Easy means that there is a high probability that if contacted again the sample unit would respond.

After a first phase of data collection, the intermediate survey results are evaluated and sample units are divided into respondents, refusers and noncontacts. Refusers receive a different treatment. Interviewers are rated based on their historic performance and grouped in good and less good interviewers. Refusers are re-assigned to one of the two groups of interviewers. Since there is no assessment available for non-contacts, the treatment for this group is not altered.

We use the R-indicator given by (2.7) as the quality objective function. We split the sample using $X=(\text{age})$ into two groups, labelled as young and old. The goal in the second phase is to assign refusers to the two interviewer groups such that the R-indicator with respect to age is maximized.

Let $n$ be the sample size of the survey. The population proportions of the two subpopulations, young and old, are denoted by $q(1)$ and $q(2).$ We let $q(\tilde{x}|x)$ be the conditional probability that a sample unit from age subpopulation $x$ is of type $\tilde{x},$ where $\tilde{x}\in \{\text{easy, medium, difficult}}\text{.}$ Furthermore, let $\lambda (x,\tilde{x})$ be the probability that a sample unit of type $\tilde{x}$ from age subpopulation $x$ is a refusal. If a person is not a refuser, then $\mu (x,\tilde{x})$ is the probability that the person either was a respondent after the first phase or becomes a respondent when he/she was a noncontact after the first phase.

The total number of interviewers is $M$ and $p_{s}M$ represents the number of interviewers with skill $s\in S=\left\{\text{good, less good}\right\},$ $0\le p_s\le 1$ and $p_{\text{good}}+p_{\text{less good}}=1$. The set $S$ forms the set of strategies, i.e., we want to assign each refuser to either a good or a less good interviewer. We assume that each interviewer can handle at most $c$ refusal cases in the second phase of the survey. The probability that a refusal of type $\tilde{x}$ from subpopulation $x$ will respond if contacted by an interviewer of skill $s$ is denoted by $\rho (s,x,\tilde{x})$ and it is again assumed to be known from previous surveys.

Let $\{p{(s|x,\tilde{x})\}}_{x,\tilde{x}}$ be the set of decision variables, where $p(s|x,\tilde{x})$ represents the probability that a sample unit of type $\tilde{x}$ will be assigned to an interviewer of skill $s$ given that he/she belongs to subpopulation $x.$ In other words, we allow for a random assignment of sample units to the two interviewer groups.

In this example, we express costs in terms of the overall interviewer occupation rates. Since interviewers can handle at most $c$ cases, there are two constraints

$n{\displaystyle \sum _{x,\tilde{x}}q(x)q(\tilde{x}|x)p(s|x,\tilde{x})\lambda (x,\tilde{x})\le M{p}_{s}c},\forall s\in S.$

In other words, the total number of refusers that can be assigned to interviewers of skill $s$ is restrained to the maximum possible workload for that skill group.

The response propensity for a unit from subpopulation $x$ can now be derived as

${\displaystyle \sum _{\tilde{x}}q(\tilde{x}|x)\left[(1-\lambda (x,\tilde{x}))\mu (x,\tilde{x})+\lambda (x,\tilde{x}){\displaystyle {\sum }_{s}p(s|x,\tilde{x})\rho (s,x,\tilde{x})}\right]},$

and form the input to the R-indicator.

Now, consider the following input data for the example: a sample size of $n=$2,000, a total of 80 interviewers, $M=$80, a maximal workload of 30 cases per interviewer, $c=$30, an age distribution equal to $q(1)=q(2)=$0.5, conditional distributions of refusal type $q(\tilde{x}|1)=(0.2,0.3,0.5{)}^{\prime }$ and $q(\tilde{x}|2)=(1/3,1/3,1/3{)}^{\prime }$ and 25% of the interviewers are classified as good, $p_1=0.25=1-p_2.$

Tables 3.1 and 3.2 give the hypothetical response probabilities $\rho (s,x,\tilde{x})$ for the two subgroups when refusal conversion is applied, as well as the cooperation probabilities $\mu (x,\tilde{x})$ and refusal probabilities $\lambda (x,\tilde{x}).$

We optimize the R-indicator with respect to the two age groups. For two strata, it can be shown that the R-indicator is maximal when the absolute distance between the two strata response propensities is minimal. The optimal value of the R-indicator turns out to be 0.827. Table 3.3 shows the optimal values of the decision variables; all but one of the decision variables $p(s|x,\tilde{x})$ are either 0 or 1, i.e., the re-assignments are mostly non-probabilistic. The exception is the subpopulation of young persons with medium response propensity assessment.

Table 3.1
Response probabilities when refusal conversion is applied to young and old refusers given the assessment of propensity to respond.
Table summary
This table displays the results of response probabilities when refusal conversion is applied to young and old refusers given the assessment of propensity to respond. good interviewer and less good interviewer, calculated using easy, medium and difficult units of measure (appearing as column headers).

	Good interviewer			Less good interviewer
	Easy	Medium	Difficult	Easy	Medium	Difficult
Young refuser
$$\rho (s,1,\tilde{x})$$	0.8	0.6	0.4	0.7	0.5	0.3
Old refuser
$$\rho (s,2,\tilde{x})$$	0.9	0.7	0.5	0.8	0.6	0.4

Table 3.2
Refusal and cooperation probabilities in the first phase of data collection
Table summary
This table displays the results of refusal and cooperation probabilities in the first phase of data collection young and old, calculated using easy, medium and difficult units of measure (appearing as column headers).

	Young			Old
	Easy	Medium	Difficult	Easy	Medium	Difficult
$$\lambda (x,\tilde{x})$$	0.5	0.6	0.7	0.2	0.3	0.4
$$\mu (x,\tilde{x})$$	0.85	0.8	0.76	0.95	0.93	0.91

Table 3.3
Optimal assignment of cases to interviewers
Table summary
This table displays the results of optimal assignment of cases to interviewers young and old, calculated using easy, medium and difficult units of measure (appearing as column headers).

	Young			Old
	Easy	Medium	Difficult	Easy	Medium	Difficult
Good	1	0.83	1	0	0	0
Less good	0	0.17	0	1	1	1

It is useful to compare the optimal allocation to a random allocation of interviewers in order to see how much is gained. If we would randomly assign the refusals to the interviewers, then the value of the R-indicator equals 0.749. The optimal assignment, thus, leads to a considerable increase in the R-indicator. The response rates are, respectively, 72.0% and 70.1% for the optimal and the random assignment.

If we increase the number of interviewers, while fixing the maximal number of cases per interviewer as well as the other parameters, then for any interviewer number higher than $M=84$ the R-indicator does not improve. Both interviewer groups are sufficiently big to handle the entire sample and the cost constraint is no real constraint anymore. The R-indicator for $M=84$ is equal to 0.830 and the response rate is 72.1%. If we would maximize the response rate rather than the R-indicator, then the allocation of interviewers will converge towards assigning only good interviewers to all cases.

Previous | Next