Bias in self-reported estimates of obesity in Canadian health surveys: An update on correction equations for adults

View the most recent version.

The health consequences of excess body weight have made obesity a public health challenge throughout the world.¹  Accurate monitoring of the prevalence of obesity is critical in the assessment of intervention programs.

Obesity prevalence estimates are commonly based on body mass index (BMI), a measure of weight in relation to height.  Each year, Statistics Canada's Canadian Community Health Survey collects self-reported height and weight data from respondents in order to monitor obesity trends at the national, provincial and health region levels.  However, self-reports overestimate height and underestimate weight.^2,3  Consequently, the prevalence of obesity based on self-reported data is underestimated.  Moreover, the magnitude of the bias has increased over time.⁴

Another problem with using self-reported data is that the relationship between obesity and obesity-related diseases is distorted.  The misclassification that occurs when BMI categories are based on self-reported height and weight results in elevated associations between obesity and diseases such as hypertension and diabetes,^5-7 and in underestimates of the health care burden of these conditions.⁷

In 2005, the Canadian Community Health Survey collected both self-reported and measured height and weight for a subsample of respondents.  Data for this subsample were used to develop correction equations to apply to the self-reported data to produce obesity prevalence estimates that approximated those derived from measured data.⁸

Statistics Canada planned to periodically collect both measured and self-reported height and weight from a subsample of Canadian Community Health Survey respondents to monitor the magnitude of the bias and adjust the correction equations.  Such data were, in fact, collected in 2008.  But around the same time, in partnership with Health Canada and the Public Health Agency of Canada, Statistics Canada  launched the Canadian Health Measures Survey,⁹ which collected both self-reported and measured height and weight.  Because this survey will be conducted every two years, a decision was made to drop the direct measurement component from the Canadian Community Health Survey and use the Community Health Measures Survey to correct for biases in the self-reported Canadian Community Health Survey data.  However, the context and methods of the two surveys differ:  before respondents to the Canadian Health Measures Survey report their height and weight, they are informed that they will later be measured; Canadian Community Health Survey respondents do not know this.  Thus, the bias in height, weight, BMI, and consequently, the prevalence of obesity may differ between the two data sources and possibly preclude the use of Canadian Health Measures Survey data to establish correction equations for the Canadian Community Health Survey.

The purpose of this study was to address the following questions:

1. Does the bias in height, weight and BMI differ depending on the context of the survey?
2. Does the bias vary over time (2005 versus 2008 Canadian Community Health Survey)?
3. Can correction equations be successfully applied to the self-reported 2008 Canadian Community Health Survey data:
   • established with 2007 to 2009 Canadian Health Measures Survey data?
   • established with 2005 Canadian Community Health Survey data?

Methods

Data sources

Data for this study were from the 2008 and 2005 Canadian Community Health Surveys and the 2007 to 2009 Canadian Health Measures Survey.

The Canadian Community Health Survey is an ongoing survey designed to provide cross-sectional estimates of health determinants, health status and health system use at a subprovincial level.¹⁰ The survey covers the non-institutional household population aged 12 or older in all provinces and territories, except members of the regular Canadian Forces and residents of Indian reserves, Canadian Forces bases (military and civilian), and some remote areas.  It is representative of 98% of the population.

In both 2008 and 2005, a subsample of respondents was selected in the ten provinces (the territories were excluded) for direct measurement.  The subsamples were randomly selected from the Canadian Community Health Survey area frame for which all interviews were conducted in person in the respondent's home.  These respondents were asked their height and weight, and later in the interview, their height and weight were measured.  Before they self-reported their height and weight, they had not been told that their measurements would be taken.  In 2008, the response rate to the subsample was 85.0% at the household level and 59.7% for the direct measurement component, for an overall response rate of 50.7%.  In 2005, the response rate to the subsample was 87.0% at the household level and 64.2% for the direct measurement component, for an overall response rate of 55.9%.

Data for the Canadian Health Measures Survey were collected at 15 sites across Canada from March 2007 through February 2009.  The survey covered the household population aged 6 to 79. Residents of Indian Reserves or Crown lands, institutions and certain remote regions, and full-time members of the Canadian Forces were excluded; 96.3% of Canadians were represented.  Technical details of the sample design can be found in a published report.¹¹  In addition to a detailed questionnaire administered in the respondent's home, the survey involved physical measures (including height and weight) at a mobile examination centre one day to six weeks after the home interview.  In the introduction to the home interview (before the questions on height and weight were asked), respondents were told that measurements would be taken ("... the second part of the survey involves a visit to a clinic to collect direct physical measures such as blood pressure, height and weight, and fitness levels").¹²  The household response rate was 69.6%—that is, in 69.6% of selected households, the sex and date of birth of all household members were provided by a household resident.  In each responding household, one or two members were chosen to participate in the survey; 88.3% of selected respondents completed the household questionnaire, and 84.9% of those who completed the questionnaire participated in the subsequent examination centre component.  The overall response rate was 51.7%.  Because two people were selected in some households, the overall response rate is not the result of multiplying the household and person response rates.¹³

Measures and definitions

This study is based on adults aged 18 to 79 for whom both measured and self-reported values of height and weight were collected.  Pregnant women were excluded.  Sample sizes for the Canadian Community Health Survey are 3,876 for 2008 and 3,895 for 2005.  The sample size for the Canadian Health Measure Survey is 3,625.

In each survey, self-reported height and weight were collected in the respondent's home with the questions:

• "How tall are you without shoes on?" Categories for height in feet and inches were listed on the questionnaire, with corresponding metric values in brackets.
• "How much do you weigh?" After reporting weight, respondents were asked if they had reported in pounds or kilograms; more than 90% reported in pounds.

The Canadian Community Health Survey interview lasted about 50 minutes.  Self-reported height and weight were collected close to the beginning, and the measurements were taken near the end.  Interviewers were trained to measure height and weight.  Height (without shoes) was measured to the nearest 0.5 cm with a measuring tape attached to a wall.  Weight was measured to the nearest 0.1 kg with a calibrated digital scale (ProFit UC-321 made by Lifesource). 

In the Canadian Health Measures Survey, the measures were taken at a mobile examination centre by specialists with a degree in kinesiology and certification from the Canadian Society for Exercise Physiology as either a Certified Exercise Physiologist or Certified Personal Trainer.  Height was measured to the nearest 0.1 cm using a ProScale M150 digital stadiometer (Accurate Technology Inc., Fletcher, USA), and weight, to the nearest 0.1 kg with a Mettler Toledo VLC with Panther Plus terminal scale (Mettler Toledo Canada, Mississauga, Canada).  Equipment was calibrated regularly.

Body mass index ( BMI ) is a measure of weight adjusted for height.  BMI is calculated by dividing weight in kilograms by the square of height in metres. "Measured BMI" refers to BMI calculated from measured height and weight, and "self-reported BMI," to BMI calculated from self-reported height and weight.  Corrected BMI values were derived from correction equations applied to self-reported values.  Based on Canadian guidelines,¹⁴ which are in line with those of the World Health Organization,¹⁵  respondents were categorized as underweight (BMI less than 18.5 kg/m²), normal weight (BMI 18.5 to 24.9 kg/m²), overweight (BMI 25.0 to 29.9 kg/m²), or obese (BMI 30.0 kg/m² or more).

Analytical techniques

The bias associated with using self-reported data for weight, height and BMI was estimated by calculating the difference between self-reported and measured values (self-reported minus measured).  A negative difference indicates under-reporting, and a positive difference, over-reporting.

The degree of misclassification that resulted from using self-reports to assign respondents to BMI categories was assessed by calculating sensitivity and specificity.  Sensitivity is the percentage of true positives (the percentage of obese, overweight, normal weight or underweight individuals, based on measured values, who were appropriately classified as such based on self-reported values).  Specificity is the percentage of true negatives (the percentage of non-obese, non-overweight, non-normal weight, or non-underweight individuals correctly classified as such based on self-reported values).

Previously established correction equations derived from the 2005 Canadian Community Health Survey data⁸ were applied to 2008 Canadian Community Health Survey self-reported values.  The original study tested four models:

• Model 1 (Height and Weight Full):  Measured height and weight were predicted based on self-reported values along with factors significantly associated with the bias in height and weight.  BMI was calculated using these corrected values of height and weight.
• Model 2 (BMI Full):  Measured BMI was predicted based on self-reported BMI as well as factors significantly associated with the bias in BMI.
• Model 3 (Height and Weight Reduced):  Measured height and weight were predicted based solely on the self-reported values, and BMI was calculated using these corrected values of height and weight.
• Model 4 (BMI Reduced):  Measured BMI was predicted based solely on self-reported BMI.

The variables considered in relation to the bias in height, weight and BMI in the full models were determined from a review of the literature and availability in the survey:  age group, education, employment status, immigrant status, race/ethnicity, household income, self-perceived general health, self-perceived mental health, chronic conditions (arthritis, hypertension, diabetes, heart disease, cancer and mood disorders), perceived stress, satisfaction with life, smoking status, perception of weight, leisure-time physical activity level, and end-digit preference.

All analyses were run separately for men and women.  Interactions and quadratic terms (including a quadratic term for BMI) were tested.  The four models were assessed by comparing corrected means for BMI, prevalence rates by BMI category, and sensitivity and specificity values.  As no model was consistently superior, the model based solely on self-reported BMI (BMI reduced) was recommended because it was the most parsimonious.⁸

For the current study, the methods used to generate the correction equations in the earlier study were replicated using 2007 to 2009 Canadian Health Measures Survey data.  The equations were then applied to self-reported values from the 2008 Canadian Community Health Survey.  As in the earlier study, the results for the four models were similar, and therefore, only the results of the BMI reduced models are presented here.  Thus, the current study evaluates the feasibility of correcting self-reported BMI values in the 2008 Canadian Community Health Survey using two BMI reduced models: one applying equations based on the BMI reduced model from the 2005 Canadian Community Health Survey data, and the other applying equations based on the BMI reduced model from the 2007 to 2009 Canadian Health Measures Survey data (Appendix Table A).

Table A Regression equations for correcting self-reported values of body mass index (BMI), by sex, BMI reduced models, based on 2005 Canadian Community Health Survey data and 2007 to 2009 Canadian Health Measures Survey data, household population aged 18 to 79, Canada

Corrected prevalence estimates of BMI categories for the 2008 Canadian Community Health Survey were produced based on the two models to see how closely they approximated estimates based on measured values.  Sensitivity and specificity estimates based on corrected values were generated for each model.

Correction equations were also developed based on half the 2008 Canadian Community Health Survey sample and then applied to the other half  (similar to the approach in the earlier study).  The results were similar to what was observed when the 2005 correction equations were applied to the 2008 Canadian Community Health Survey data (data not shown).

Data for all surveys were weighted, and all measures of variance were estimated with the bootstrap technique^16,17 to account for the complex survey designs.  For the Canadian Health Measures Survey, the number of degrees of freedom was specified as 11.  SAS (version 9.1) and SUDAAN (version 10) were used for all analyses.

Results

For both sexes in each survey, height was over-reported, and weight, under-reported (Table 1).  As a result, mean BMI and the prevalence of obesity were higher when based on measured than on self-reported data.

Table 1 Mean height, weight, body mass index (BMI) and prevalence of obesity, by collection method and sex, household population aged 18 to 79, Canada, 2008, 2007 to 2009, and 2005

The magnitude of the bias in height was similar in each survey.  This was not true for weight.  In the 2008 Canadian Community Health Survey, weight was under-reported by an average of 2.2 kg among men, and by 2.7 kg among women.  Results had been similar in 2005, with men under-reporting by an average of 1.9 kg, and women, 2.8 kg.  In the 2007 to 2009 Canadian Health Measures Survey, weight was under-reported to a lesser degree—0.6 kg among men and 1.6 kg among women.  Consequently, the bias in the prevalence of obesity was approximately twice as high in the two Canadian Community Health Surveys as in the Canadian Health Measures Survey (Table 1).

Sensitivity and specificity values were similar for the two Canadian Community Health Surveys (Table 2).  In the Canadian Health Measures Survey, sensitivity values were higher for overweight and obese men and obese women than in the 2008 Canadian Community Health Survey.  Specificity for normal-weight men and women was higher in the Canadian Health Measures Survey than in the 2005 and 2008 Canadian Community Health Surveys.

Table 2 Sensitivity and specificity values for self-reported data, by sex, household population aged 18 to 79, Canada, 2008, 2007 to 2009, and 2005

Corrections were made to the self-reported BMI values in the 2008 Canadian Community Health Survey based on two sets of regression equations (one generated from the 2005 Canadian Community Health Survey, and the other from the Canadian Health Measures Survey) (see Methods and Appendix Table A).

In the 2008 Canadian Community Health Survey,  based on self-reported height and weight, BMI was underestimated by 1.2 kg/m² for men and by 1.5 kg/m² for women (Table 3).  Application of the corrections based on the Canadian Health Measures Survey model reduced the bias to 0.6 kg/m² for men and to 0.7 kg/m² for women.  Use of the 2005 Canadian Community Health Survey correction equations further decreased the bias—to 0.2 kg/m² for men and to 0.3 kg/m² for women (Table 3).  Although both models reduced the BMI bias, means based on corrected values remained significantly lower than means based on measured values.

Table 3 Mean body mass index (BMI) for self-reported, measured and corrected data, by sex, household population aged 18 to 79, Canada, 2008

For both sexes, percentage distributions by BMI category differed significantly when based on self-reported versus measured values (Table 4).  The correction equations yielded distributions closer to those based on measured values.  However, for the distributions based on the Canadian Health Measures Survey correction equations, significant differences remained. 

Table 4 Percentage distribution of population, by body mass index (BMI) category and sex, based on self-reported, measured and corrected data, household population aged 18 to 79, Canada, 2008

Among men, the prevalence of obesity in the 2008 Canadian Community Health Survey was 26.1% based on measured values, and 18.5% based on self-reported values.  When the self-reported values were corrected using the Canadian Health Measures Survey correction equation, the prevalence of obesity was 22.0% (Table 4), significantly below  the measured value.  By contrast, the prevalence of obesity based on the 2005 Canadian Community Health Survey correction equation—24.9%—was not statistically different from the measured estimate.

Results were similar for women.  In the 2008 Canadian Community Health Survey, the prevalence of obesity among women was 23.3% based on measured values and 16.1% based on self-reported values; using the 2005 correction equation, the corrected self-reported estimate was 22.8%, which was not statistically different from the measured estimate.  Although the Canadian Health Measures Survey correction equation improved the estimate based on self-reported values, it remained significantly below the measured estimate.

Based on self-reported 2008 Canadian Community Health Survey data, sensitivity for the obese category was 67% for men and 65% for women, meaning that self-reports correctly classified about two-thirds of obese men and women (Table 5).  Corrections using the Canadian Health Measures Survey equations improved sensitivity to 77% for men and to 75% for women.  However, the 2005 correction equations yielded even higher sensitivity values:  84% for men and 82% for women.

Table 5 Sensitivity and specificity values for self-reported and corrected data, by sex, household population aged 18 to 79, Canada, 2008

The use of correction equations also improved sensitivity estimates for the overweight category.  However, for the normal-weight category, sensitivity estimates based on corrected values were lower than those based on self-reported values:  in some cases, respondents correctly classified as normal weight based on self-reports were erroneously classified as overweight based on the correction equations.

Correction equations, notably those based on the 2005 Canadian Community Health Survey, improved specificity estimates for the normal-weight category.  For the obese category, the correction equations slightly reduced specificity.

The ultimate goal of establishing correction equations for the Canadian Community Health Survey is to be able to apply them to the full sample in order to estimate obesity at provincial and health region levels.  To this end, the two sets of correction equations were applied to the full 2007 to 2008 Canadian Community Health Survey sample of 107,141 respondents aged 18 to 79, 38% of whom were interviewed in person, and the remaining 62%, by telephone.  For both sexes, full-sample obesity estimates corrected with the 2005 equations were similar to measured obesity estimates based on the 2008 subsample (Table 6).  In fact, corrected estimates for all BMI categories based on the 2005 correction equations were similar to measured estimates except for normal-weight women for whom the corrected estimate was somewhat higher.  Again, the Canadian Health Measures Survey correction equations resulted in some improvements, but they were less effective than the 2005 correction equations.

Table 6 Percentage distribution of population, by body mass index (BMI) category and sex, based on self-reported, measured and corrected data for full 2007 to 2008 Canadian Community Health Survey, household population aged 18 to 79, Canada

Discussion

Consistent with past research,² this study found biases when height and weight are based on self-reported values.  Because survey respondents tended to over-report height and under-report weight, the self-reported data underestimated the prevalence of obesity.

The magnitude of the bias in height was similar across the three surveys, but the bias in weight was lower in the Canadian Health Measures Survey than in the two Canadian Community Health Surveys.  As a result, the bias in the prevalence of obesity was approximately twice as high in the two Canadian Community Health Surveys as in the Canadian Health Measures Survey.  Sensitivity for the obese category was substantially higher in the Canadian Health Measures Survey, meaning that obese respondents to that survey were far more likely to be accurately identified as obese based on self-reported values.  Survey context likely played a role in these discrepancies.  Before they reported their height and weight, respondents to the Canadian Health Measures Survey were informed that their height and weight would later be measured.  By contrast, the subsample of respondents selected for the direct measurement component of the Canadian Community Health Survey had no prior indication that their measurements would be taken.  As suggested in other research,¹⁸ self-reports may be more accurate if respondents know that they will be weighed and measured.

The methodology and context of the Canadian Health Measures Survey are similar to the National Health and Nutrition Examination Survey (NHANES) conducted in the United States.¹⁹  The bias in height in the Canadian Health Measures Survey and the Canadian Community Health Surveys is similar to that in the 2003/2004 NHANES (1.2 cm for men and 0.5 cm for women aged 18 to 74).⁴  For women aged 18 to 74, the bias in weight in the NHANES (-1.3 kg) was similar to that in the Canadian Health Measures Survey (-1.6 kg), and substantially less than the bias in the 2008 Canadian Community Health Survey (-2.7 kg).  In the Canadian Health Measures Survey, men also under-reported weight but not as much as in the Canadian Community Health Survey; in the NHANES, men did not under-report weight.

The current study found no change in the bias between the 2005 and 2008 Canadian Community Health Surveys, although three years is a short period over which to assess change.  Nonetheless, a Swiss study found that the bias remained relatively constant in that population over three decades.²⁰  A study that compared changes in the bias in BMI across multiple NHANES cycles (1976 to 1980, 1988 to 1994, 2003/2004) with changes in the bias between the Canadian Heart Health Surveys (from 1986 to 1992) and the 2005 Canadian Community Health Survey concluded that the bias remained relatively stable in the United States but rose in Canada.⁴ However, the context of the Canadian Heart Health Surveys was similar to the NHANES and the Canadian Health Measures Survey in that respondents knew that they would be required to visit a clinic for physical measurements.  In the Canadian Heart Health Surveys, weight was underestimated by 1.8 kg among men and by 2.3 kg among women (based on the population aged 18 to 74 age-standardized to the 2001 Canadian census).  While this bias is less than in the 2005 and 2008 Canadian Community Health Surveys, it is substantially more than in the Canadian Health Measures Survey (weight was underestimated by 0.6 kg among men and by 1.6 kg among women aged 18 to 74).  In the Canadian Heart Health Surveys, height was overestimated by 0.6 cm among men and by 0.2 cm among women.  This bias in height is less than in the Canadian Health Measures Survey (1.2 cm for men and 0.7 cm for women aged 18 to 74) or either Canadian Community Health Survey.  In Canada, a lack of data points from surveys conducted in a similar fashion prevents tracking trends in the bias over time.

With some success, other studies have employed correction equations to adjust self-reported BMI values.^8,21-25  However, the external applicability of these correction equations depends on factors such as survey context, changes in the bias over time, and the population group studied.  Because the bias in the Canadian Health Measures Survey was significantly different from those in the Canadian Community Health Surveys, the use of regression equations based on Canadian Health Measures Survey data had limited success in correcting self-reported 2008 Canadian Community Health Survey estimates.  This was particularly true for the prevalence of obesity, with a 3- to 4-percentage-point difference remaining between the corrected and measured estimates.  Results were similar in an American study that applied regression equations based on data from the NHANES to data from the Behavioural Risk Factor Surveillance System, which collects only self-reported values for height and weight.¹⁸  A previous study based on data from a Dutch population survey also found that correction equations may not be applicable to other datasets.²⁶

In the current study, when the regression equations based on the 2005 Canadian Community Health Survey were applied to the self-reported 2008 data, the corrected obesity prevalence estimates approximated those based on measured data.  Although sensitivity for the normal-weight group was somewhat reduced, substantial improvements in sensitivity were realized for both sexes in the obese group.  Use of the 2005 correction equations would be particularly effective for studies based on the 2008 Canadian Community Health Survey that dichotomize BMI as obese or not obese.

When the 2005 correction equations were applied to the full 2007 to 2008 Canadian Community Health Survey sample, obesity estimates were statistically similar to those derived from measured values for the 2008 subsample.  These improvements were realized even though 62% of the interviews in the full sample were by telephone.  While further studies are needed to assess the applicability of the equations at provincial and subprovincial levels, the use of correction equations is recommended for all analyses using Canadian Community Health Survey data.

Limitations

The response rates were 51.7% for the Canadian Health Measures Survey, 50.7% for the 2008 Canadian Community Health Survey, and 55.9% for the 2005 Canadian Community Health Survey.  Sampling weights were adjusted to compensate for the various levels of non-response, but estimates could be biased if respondents' characteristics differed significantly from those of non-respondents.

Differential non-response may have resulted in the higher bias in the self-reported data from Canadian Community Health Survey than from the Canadian Health Measures Survey.  However, prevalence estimates by BMI categories based on measured height and weight did not differ between the 2008 Canadian Community Health Survey and the 2007 to 2009 Canadian Health Measures Survey, which suggests that the differences in the bias in the self-reported data from the two surveys were due to survey context rather than differential non-response.

An American study²⁷ and a study based on Canadian Community Health Survey data²⁸ found that telephone interviews resulted in a larger bias in self-reported obesity estimates than did in-person interviews.  However, in the current study, the self-reported 2008 Canadian Community Health Survey obesity estimates were similar for the subsample, for which only on in-person interviews were conducted, and for the full 2007 to 2008 sample, for which 62% of interviews were conducted by telephone.  When the 2005 regression equations were applied to the full 2007 to 2008 Canadian Community Health Survey sample, the corrected obesity estimates approximated those based on measured data.  The differential bias in telephone versus in-person interviews may be changing over time.

Conclusion

Although directly measured height and weight provide the most accurate estimates of the prevalence of obesity, cost and logistical considerations oblige the Canadian Community Health Survey to continue to collect self-reported data.  The use of Canadian Health Measures Survey data to develop equations to correct for the bias in these self-reports is less effective than equations from a subsample of Canadian Community Health Survey respondents whose height and weight were measured.  Nonetheless, it is important to monitor differences in measured estimates of obesity from the Canadian Health Measures Survey and corrected estimates from the Canadian Community Health Survey over time.  Differences would indicate that the bias in self-reported values is changing, resulting in the need to develop new equations to minimize bias and approximate measured values.