The promise and challenge of pushing respondents to the Web in mixed-mode surveys
Section 1. Introduction
A surprising, but critical, development in survey design during the early 21st century is the extensive use of web-push data collection methods, i.e., the use of postal mail to obtain questionnaire responses from general public samples mostly over the Internet instead of paper questionnaires. Web-push methods are now being used as a replacement for paper mail-push procedures whereby an attempt is made to obtain responses by mail before using other modes of response such as telephone or personal interviews. Web-push methods are now being used in official government surveys and as a replacement for random digit dialing (RDD) voice telephone surveys.
For example, the American Community Survey, which serves as the major source of state and regional information on U.S. households, began using a web-push approach to data collection in 2013 that includes the possibilities of responding later in the implementation process by mail, telephone or in-person interview. Plans are now in place to use such a methodology for the 2020 U.S. Decennial Census. Web-push data collection with an initial mail request is also being used worldwide. Examples include the 2015 Japanese Census (Statistics Japan 2015), and the 2016 Censuses in Canada (Statistics Canada 2016) and Australia (Australian Bureau of Statistics 2016). Other examples of web-push procedures are household surveys in Switzerland (Roberts, Joye and Staehli 2016) and the United Kingdom’s Community Life Survey (United Kingdom Cabinet Office 2016), which is being transitioned from a personal interview mode. In addition, the U.S. College Graduates Survey, conducted every 2-3 years by the National Science Foundation, has completed the shift from mail and telephone data collection to a web-push approach followed by the other two modes of data collection (Finamore and Dillman 2013).These examples are only a few of the major survey efforts around the world that are now using this methodology.
Use of web-push data collection methods has been encouraged by a number of considerations, ranging from seemingly unfixable problems of RDD telephone surveys to the fact that Postal Service residential address lists or country-wide registration lists now provide the most complete coverage of households. In addition, there are no acceptable ways of drawing probability samples of household email addresses as a means of household contact. Even if email addresses could be sampled, it is not likely that reasonable response rates could be obtained through email-only contact (Lozar, Bosnjak, Berzelak, Haas and Vehovar 2008).
The current heavy reliance on mail contact is surprising, despite the demonstrated potential of mail surveys for obtaining reasonable response rates in the late 20th century (Dillman 2000). Until recently, mailing address sample frames have been mostly unavailable and inadequate. In addition the general availability of a telephone alternative prior to the late 1990’s meant that mail was infrequently used for government surveys, with the exception of official government censuses.
My purpose in this paper is first, in Section 2, to discuss the reasons that web-push survey methodologies have been developed and adopted worldwide. Secondly, in Section 3 and Section 4, I describe research efforts that have not only made web-push methodologies feasible, but are improving the effectiveness of such methods in producing reliable estimates of the opinions and behaviors of survey populations throughout the world.
This research has shown, see Section 5, that web-push methodologies are quite promising with regard to improving coverage and response rates, while reducing measurement differences across modes as well as total survey costs. It has also shown that there are many perils that threaten their use, ranging from respondent trust of the Internet to the plethora of devices now available for responding to such surveys, see Section 6. My focus in this paper is to present the substantial promise and many challenges associated with web-push methods for conducting sample surveys. Section 7 presents a summary and a conclusion.
- Date modified: