Dealing with small sample sizes, rotation group bias and discontinuities in a rotating panel design 5. Redesign of the Dutch Labour Force Survey

The LFS was redesigned in 2010, as described in Section 2. Discontinuities induced by this redesign were quantified by conducting the first panel under the old and new design in parallel for a period of six months, from January through June 2010. Each month two separate samples with the regular monthly sample size were drawn from the target population according to the sample design of the LFS. One sample was assigned to the old and one to the new LFS design. This made a direct estimate possible for the discontinuities for the main parameters in the first panel.

Mainly due to budget constraints, the subsequent panels were not conducted in parallel under the old and the new design. Possible discontinuities were quantified using the intervention approach described in Section 3. In the time series model, the outcomes of the subsequent panels are benchmarked to the level of the first panel. It is therefore crucial that the first panel is measured as accurately as possible, including possible discontinuities due to a redesign. Therefore it was decided to conduct a sufficiently large parallel run for the first panel, and use the intervention approach for the remaining panels. The estimates for the discontinuities from the parallel run as well as the intervention variables of the time series model are the effect of all factors that changed simultaneously in the redesign of the survey.

In the parallel run, 19,150 responding households under the old design and 16,906 responding households under the new design were obtained. Table 5.1 compares the field work results of the new and old design, both for households with and without a listed phone number. Overall, the response rate is lower for households without a listed phone number. This can be explained by the fact that this part of the population typically consists of hard to reach groups like young people and migrants. Furthermore, the response rate is lower under CATI than under CAPI for households with a listed phone number. Both the percentages of no contact and of frame errors increase substantially when using CATI instead of CAPI. Frame errors under CAPI are mostly non-existing or unoccupied addresses, under CATI they are mostly closed phone lines. Other non-response includes, for example, illness.

Table 5.2 summarizes the estimation results of the parallel run for the unemployed labour force. At the national level, the change-over to the new design resulted in an increase of about 55,000 in the monthly unemployed labour force figures. The differences fluctuated considerably over the six months of the parallel run, probably caused by the large sampling errors of the GREG estimates. A strong increase in the differences was observed in the last two months of the parallel run, particularly at the national level. This can be explained partially by the low response under the new design during these two months.

The decision was made to produce official monthly figures using the data obtained under the old design until June 2010. After completion of the parallel run, all the available data obtained under the new design were used to compile official monthly figures. So since July 2010, the data in the first panel have been based on the new design from January 2010, while the data in the second panel are based on the new design from April 2010, and the data in the third panel are based on the new design from July 2010 and so on.

Table 5.1
Overview fieldwork results of the parallel run first panel (OLD)
Table summary
This table displays the results of Overview fieldwork results of the parallel run first panel (OLD). The information is grouped by Category (appearing as row headers), CAPI - phone, CAPI – no phone and total, calculated using households and % units of measure (appearing as column headers).
Category CAPI - phone CAPI – no phone total
households % households % households %
Total 20,813 10000.0 14,469 100.0% 35,282 100.0%
Frame errors 769 370.0 1,039 7.2 1,808 5.1
Not approached 618 3.0 463 3.2 1,081 3.1
Language problems 390 1.9 878 6.1 1,268 3.6
Refusal 4,909 23.6 3,112 21.5 8,021 22.7
No contact 889 4.3 1,455 10.1 2,344 6.6
Other non-response 921 4.4 689 4.8 1,610 4.6
Complete response 12,317 59.2 6,833 47.2 19,150 54.3

 

Table 5.1 (cont.)
Overview fieldwork results of the parallel run first panel (NEW)
Table summary
This table displays the results of Overview fieldwork results of the parallel run first panel (NEW). The information is grouped by Category (appearing as row headers), CATI, CAPI and total, calculated using households and % units of measure (appearing as column headers).
Category CATI CAPI total
households % households % households %
Total 20,234 100.0 13,345 100.0 33,579 100.0
Frame errors 1,539 7.6 982 7.4 2,521 7.5
Not approached 1 0.0 428 3.2 429 1.3
Language problems 317 1.6 788 5.9 1,105 3.3
Refusal 4,545 22.5 2,903 21.8 7,448 22.2
No contact 2,233 11.0 1,333 10.0 3,566 10.6
Other non-response 963 4.8 641 4.8 1,604 4.8
Complete response 10,636 52.6 6,270 47.0 16,906 50.3

 

Table 5.2
Comparison of GREG estimates new and old design for monthly unemployed labour force figures, first panel (×1,000), standard errors in brackets, significant difference at a 5% significance level indicated with *
Table summary
This table displays the results of Comparison of GREG estimates new and old design for monthly unemployed labour force figures National
level, Men
15-24 , Women
15-24 , Men
25-44 , Women
25-44 , Men
45-64 and Women
45-64 (appearing as column headers).
  National
level
Men
15-24
Women
15-24
Men
25-44
Women
25-44
Men
45-64
Women
45-64
Monthly unemployed labour force new design  
Mean over January – June 475 67 56 103 101 80 68
Difference new and old design monthly unemployed labour force  
Mean January – June 55*(17) 19*(6) 7 (6) -1 (9) 20*(8) 6 (8) 4 (7)
Difference per month  
January 56 (39) 13 (14) 1 (14) -15 (21) -16 (18) 52*(18) 22 (15)
February 38 (42) 41*(16) 9 (17) -10 (22) 24 (21) -41*(18) 15 (18)
March 1 (41) -2 (15) -11 (13) -18 (21) 29 (21) 6 (19) -4 (14)
April 55 (40) -2 (13) 17 (17) 17 (21) 36 (20) 0 (17) -13 (16)
May 70 (44) 20 (15) 17 (13) 12 (27) 14 (21) 4 (20) 3 (15)
June 110*(41) 41*(15) 10 (14) 6 (21) 35 (18) 13 (20) 5 (17)
Date modified: