Assessment of validity of self-reported smoking status

View the most recent version.

by Suzy L. Wong, Margot Shields, Scott Leatherdale, Eric Malaison and David Hammond

The health risks associated with cigarette smoking are well-documented and widely recognized—cancer, respiratory illness, heart disease, and stroke.^1-3 In Canada, smoking contributes to more than 37,000 deaths a year.⁴ Tobacco-related health care expenditures amount to billions of dollars annually, with additional indirect costs such as lost productivity, longer-term disability and premature death.⁴

Self-reported data are typically used to monitor trends in cigarette smokiNg E.^5-7 However, estimates based on self-report, particularly of socially undesirable behaviours, are subject to reporting biases.⁸ The widespread implementation of legislation prohibiting smoking in workplaces and public areas⁹ and prominent health warnings on cigarette packages may reinforce the perception of smoking as socially undesirable, and thereby increased the tendency to underreport over time.

To validate self-reported smoking status, the urinary concentration of cotinine, a widely accepted objective measure of exposure to tobacco smoke,¹⁰ has been used. Cotinine is the major metabolite of nicotine, with a half-life of about 16 to 20 hours.¹¹ Because of its high sensitivity and specificity, cotinine is considered to be an accurate quantitative measure of recent exposure to tobacco smoke.¹²  Compared with estimates based on cotinine concentration, smoking prevalence based on self-report, is generally lower,¹³ although the extent of the difference varies by country.¹⁴

The validity of self-reported cigarette smoking data have yet to be determined for Canada. Thus, this study compares estimates of the prevalence of cigarette smoking based on self-report with estimates based on urinary cotinine concentrations. The data are from the 2007 to 2009 Canadian Health Measures Survey, which included self-reported smoking status and the first nationally representative measures of urinary cotinine. 

Methods

Data source

The Canadian Health Measures Survey (CHMS) is a nationally representative survey of the household population. Data for cycle 1 were collected from March 2007 through February 2009 at 15 sites across the country for respondents aged 6 to 79. Full-time members of the Canadian Forces and residents of Crown lands, Indian reserves, institutions and certain remote regions were excluded. The sample represented approximately 96% of the population.¹⁵

The CHMS consisted of a household interview during which information about socio-demographic characteristics, health and lifestyle was gathered. This was followed by a visit to a mobile examination centre where direct measurements, including the collection of urine samples, were taken.

Of the households selected for the survey, 69.6% agreed to participate. One or two members of each responding household were invited to take part in the survey. Of these, 88.3% responded to the household questionnaire, and 84.9% of those who completed the questionnaire visited the mobile examination centre. The overall response rate, after adjusting for the sampling strategy, was 51.7%. For adults aged 20 to 79, the overall response rate was 50.9%, and for youth aged 12 to 19, 52.7%. In total, 4,530 respondents aged 12 to 79 participated in the mobile examination centre component of the CHMS.

Ethics approval for conducting the CHMS was obtained from Health Canada's Research Ethics Board. Written informed consent was obtained from respondents. Participation was voluntary; respondents could opt out of any part of the survey at any time. Additional information about the survey is available in published reports^16-20 and on Statistics Canada's website ().

During the household interview, respondents were asked if they currently smoked cigarettes daily, occasionally or not at all. They were also asked if they had smoked cigars or a pipe or used snuff or chewing tobacco in the past month. To facilitate accurate reporting, when respondents aged 12 to 19 were being asked about sensitive topics including smoking, parents and guardians were requested to leave the room.

Respondents were asked if they had used prescription or over-the-counter medications in the past month. When they went to the mobile examination centre, they were asked to: confirm the medications they had previously reported;  report any other medications they were taking; and report the last time they had taken each medication. Drug Identification Numbers (DIN) were collected for these medications and coded using the Anatomical Therapeutic Chemical (ATC) classification system. ATC code N07BA01 refers to medications in which nicotine is an active ingredient.²¹ This code would identify smoking cessations aids (nicotine patches, gums and aerosols) that contain nicotine as the active ingredient.

In the introduction to the household interview, respondents were told that direct measurements, including urine samples, would be taken at the mobile examination centre, and were given a list of the laboratory tests that would be performed. However, whether they were aware that the results of the cotinine test could be used to assess smoking status is unknown.

Urinary cotinine analysis

During each respondent's visit to the mobile examination centre (one day to six weeks after the household interview, an average of 13 days), a spot midstream urine sample was collected in a 120 ml container. The samples were frozen at -20 °C and shipped on dry ice to the testing laboratory at the Institut national de santé publique du Québec (accredited under ISO 17025). Free cotinine was recovered by solid-phase extraction in a 96 well plate format on an automated robotic workstation.²² Deuterated cotinine was used as the internal standard. The extract was then redissolved into 250 μL of mobile phase, and 10 μL were injected into the ultra performance liquid chromatography-tandem mass spectrometric instrument, operated in the MRM mode with ion source in positive electrospray. The limit of detection was 1.1 μg/L. Details of the quality assurance program at the mobile examination centre and at the laboratory that performed the cotinine testing can be found elsewhere.²⁰

Statistical analysis

Certain exclusions were necessary to compare smoking status based on self-report versus urinary cotinine concentration. Respondents were excluded from the analyses if they:

• did not have a valid cotinine test result, for example, insufficient volume of urine collected; refused urine sample; etc. (n=48).
• reported using a medication with nicotine as an active ingredient (ATC code N07BA01) in the past month (n=4).
• reported smoking cigars or a pipe, or using snuff or chewing tobacco (n=258).

The latter two exclusions were necessary because it is possible that respondents who reported not being cigarette smokers could have been classified as smokers based on elevated cotinine concentrations that resulted from using these other nicotine-containing products.  Among the 4,530 CHMS mobile examination centre participants aged 12 to 79, these exclusions resulted in a loss of 307 cases, leaving a final sample size of 4,223 for the study. (Three records were flagged for exclusion for more than one reason.)

For smoking status based on self-report, respondents who reported that they currently smoked cigarettes "daily" or "occasionally" were classified as smokers. For smoking status based on cotinine concentrations, respondents with urinary concentrations greater than 50 ng/ml were classified as smokers. This is the cut-point recommended by the Society for Research on Nicotine and Tobacco to distinguish tobacco users from non-tobacco users, including those exposed to second-hand smoke.²³ It is highly unlikely that levels above this cut-point would be observed among non-users, even if they were regularly exposed to second-hand smoke.^23,24

The correlation between smoking prevalence based on self-report and cotinine concentrations was calculated.  The accuracy of self-reported smoking status was assessed by calculating sensitivity and specificity. Sensitivity is the percentage of true positives (the percentage of respondents who reported being smokers among those classified as smokers based on cotinine concentrations). Specificity is the percent of true negatives (the percentage of respondents who reported being non-smokers among those classified as non-smokers based on cotinine concentrations).

Comparisons were made between the self-reported prevalence of smoking based on the CHMS and on other Statistics Canada surveys that collect data on smoking status. To make meaningful comparisons, it was necessary to calculate smoking prevalence for the entire CHMS sample (n=4,530) without the exclusions in the sensitivity and specificity analyses. CHMS smoking prevalence estimates based on the entire sample are shown in Appendix Table A.

Table A Prevalence of cigarette smoking based on self-report, by age group and sex, household population aged 12 to 79, Canada, March 2007 to February 2009

Results are presented overall, by age group (12 to 19, 20 to 79), and by sex. Statistical analyses were performed using SAS and SUDAAN software. Standard errors, coefficients of variation and 95% confidence intervals were calculated with the bootstrap technique^25,26 using the replicate weights provided on the datafile. The number of degrees of freedom was specified as 11 to account for the sample design.¹⁵ Differences between estimates were tested for statistical significance, established at the level of p<0.05.  Additional information on the CHMS measured cotinine levels, arithmetic mean, geometric mean and percentile distributions for urinary cotinine, overall, by age group and sex, for smokers and non-smokers, respectively, is available in a published report²⁷ and on the Health Canada website (http://www.hc-sc.gc.ca).

Results

Smoking prevalence:  Self-report versus urinary cotinine

According to the CHMS, the prevalence of smoking was 18.8% based on self-report and 19.1% based on urinary cotinine concentration (Table 1). Differences between prevalences based on self-report versus cotinine concentration were not significant for any of the age/sex groups. Correlation results indicated strong agreement between smoking status based on self-report and cotinine (r=0.90, p<0.001).

Table 1 Prevalence of cigarette smoking based on self-report and urinary cotinine concentration, by sex and age group, household population aged 12 to 79, Canada, March 2007 to February 2009

Accuracy of self-reported smoking status

Sensitivity for self-reported smoking status was 91.6% (Tables 2 and 3). That is, among respondents classified as smokers based on their urinary cotinine concentration, 91.6% reported that they were cigarette smokers, and 8.4% were misclassified in that they reported that they did not smoke cigarettes. The mean cotinine concentration for the misclassified cases was 615.7 ng/ml (95% CI: 427.5 to 803.8), which was substantially lower than the mean (1,239.4 [95% CI: 1100.2 to 1378.7]) for properly classified cases. Among the misclassified cases, a high percentage (74.7% [95% CI: 54.6 to 87.9]) reported that they were former cigarette smokers.  The majority (60%) of these former smokers reported that they had quit during the past 5 years, and close to half (48%) reported they had quit in the past two years.

Table 2 Accuracy of classification of smoking status based on self-reports, household population aged 12 to 79, Canada, March 2007 to February 2009

Table 3 Sensitivity and specificity of self-reported smoking status, by sex and age group, household population aged 12 to 79, Canada, March 2007 to February 2009

Sensitivity estimates were similar for males and females. And although sensitivity tended to be lower for respondents aged 12 to 19 than for those aged 20 to 79, the difference did not attain statistical significance.

Specificity for self-reported smoking status was 98.3%, meaning that 1.7% of respondents whose cotinine concentration classified them as non-smokers reported that they smoked cigarettes. Of these, the majority (89%) reported that they were occasional smokers. Most (82%) of these occasional smokers said they had smoked on 10 or fewer days in the past month, and about half (51%) reported they usually smoked only one cigarette on the days that they smoked.

Discussion

This study is the first to examine the validity of estimates of cigarette smoking prevalence in Canada based on self-report. Urinary cotinine concentration measured by the CHMS provided a biomarker of tobacco smoke exposure with which to validate self-reported smoking status. The results indicated that smoking prevalence based on self-report closely approximates estimates based on cotinine concentration. 

Smoking prevalence was 0.3 percentage points lower based on self-report than on cotinine concentrations. This was consistent with results from the United States where smoking prevalence was 0.6 percentage points lower when based on self-report than on cotinine concentration.¹⁴ In England and Poland, smoking prevalence based on self-report was lower by 2.8 percentage points and 4.4 percentage points, respectively.¹⁴ The strong correlation and lack of significant differences between smoking prevalence based on self-report and cotinine concentration in the present study suggest that self-reported data provide a valid estimate of national smoking prevalence in Canada.

Although sensitivity was high (91.6%), 8.4% of respondents were classified as "false negatives" (their cotinine concentrations identified them as smokers although they reported that they did not smoke). The mean cotinine concentration was substantially lower among these false negatives (615.7 ng/ml) than among properly classified cases (1239.4 ng/ml), suggesting that heavy smokers are more likely than light smokers to report that they smoke.

Some misreporting would be expected due to social desirability bias. Although parents/guardians were asked to leave the room when the questions on smoking were administered to respondents aged 12 to 19, some of these younger respondents may have been reluctant to report that they smoked, resulting in the lower sensitivity estimates for this age group.

However, other reasons may explain some of the false negatives. Consistent with previous research,²⁸ a significantly higher percentage of the false negative cases reported being former smokers rather than never smokers, and the majority of these former smokers were recent quitters. Relapse is common among recent quitters.²⁹  If some of them relapsed in the period between their household interview and mobile examination centre visit, they would have been inappropriately classified as false negatives.  Similarly, smoking initiation or experimentation in this period may have resulted in some cases being inappropriately classified as false negatives, particularly among respondents aged 12 to 19.

Other studies have found varying levels of sensitivity for self-reported estimates of smoking, depending on the population studied, the type of biological specimen used in the measurement of cotinine, and the cut-points used to identify smokers.¹³  Similar to the CHMS findings, sensitivity estimates greater than 90% have frequently been reported,¹³ but studies based on pregnant women,³⁰ and on patients with smoking-related illnesses such as respiratory disease³¹ and cancer³² have yielded lower estimates of sensitivity. 

A small percentage (1.7%) of respondents were classified as "false positives" (their cotinine concentration classified them as non-smokers, but they reported that they smoked cigarettes).  Nearly all these false positive cases reported that they were occasional smokers, and most reported smoking on 10 or fewer days in the past month. Cotinine is a measure of recent exposure to tobacco smoke,²³ so it is likely that cotinine levels in some of these occasional smokers were too low to classify them as smokers.

An important question is the degree to which findings from this study apply to other Statistics Canada surveys that collect self-reported smoking data, such as the Canadian Community Health Survey (CCHS)⁶ and the Canadian Tobacco Use Monitoring Survey (CTUMS).³³ Survey respondents may be more likely to accurately report their smoking status if they know, or believe, that a biospecimen will also be collected to determine smoking status.³⁴ Unlike the CHMS, biospecimans are not collected by the CCHS or CTUMS. 

According to the CHMS, 20% of Canadians aged 12 to 79 were self-reported smokers (Appendix Table A). The prevalence of self-reported smoking among people aged 12 or older from the 2009 CCHS ³⁵ was 20%. These similar results suggest that self-reported CCHS data provide accurate estimates of cigarette smoking prevalence. 

In 2009, the prevalence of smoking estimated from CTUMS was 18% among the population aged 15 or older.³³  While trends in CTUMS data paralleled those derived from the CCHS, CTUMS smoking rates were consistently lower.⁵ However, unlike CTUMS, which is designed to monitor smoking prevalence, the smoking questions in the CCHS (and the CHMS) were asked in the context of a general health survey. A study of why smoking prevalence differs between the CCHS and CTUMS suggested that people are more inclined to talk frankly about smoking when the topic is part of a broad-based health survey.³⁶

The way in which data were collected might also contribute to differences in prevalence estimates between the surveys. CTUMS is conducted entirely by telephone; the CHMS is conducted entirely in person; and the CCHS uses in-person and telephone interviews. Nonetheless, a study comparing the effect of in-person and telephone interviews found that, overall, the interview mode was not associated with significantly different estimates of smoking prevalence.³⁷ 

Limitations

One limitation of this study was the relatively low overall CHMS response rate (52%). While the survey weights ensured that the sample was representative of the target population, bias might exist if non-respondents were more or less likely than respondents to be cigarette smokers and/or more or less likely to accurately self-report their smoking status. However, a comparison of the characteristics of those who responded to the household questionnaire with the characteristics of people who went on to complete the mobile examination centre component found the prevalence of smoking to be similar in the two groups.¹⁵ Furthermore, smoking prevalence based on self-report was similar in the CHMS and the CCHS, the latter of which had a higher response rate (73%).⁶

The examination centre visit occurred, on average, 13 days after the household interview. Although the number of cases would likely be small, a true change in smoking behaviour during this interval may have resulted in some respondents being erroneously classified as false negatives or false positives.

The use of cotinine concentrations to assess of the validity of self-reported smoking status may be inappropriate for occasional smokers and result in some respondents being erroneously classified as false positives.

Small sample sizes for respondents aged 12 to 19 resulted in estimates with high sampling variability.  Therefore, results for this age group should be interpreted with caution. Possibly because of small sample sizes, the lower sensitivity for the younger age group did not attain statistical significance. As future CHMS cycles become available, it will be possible to augment the sample and produce estimates with higher reliability.

Conclusion

Representative data for the Canadian population showed no significant difference between national estimates of smoking prevalence based on self-report versus urinary cotinine concentration. This suggests that self-reported data on smoking status provide a valid estimate of the prevalence of smoking in Canada.

Acknowledgments

The authors thank Tracey Bushnik and Johanne Levesque of Statistics Canada for their contributions to this study.  Scott Leatherdale is a Cancer Care Ontario Research Chair in Population Studies.