Findings

View the most recent version.

Methods
Results

Population health surveys often base estimates of the prevalence of obesity on calculations of body mass index (BMI), which is a measure of weight in relation to height. Since the mid-1990s, Statistics Canada's two major health surveys, the Canadian Community Health Survey (CCHS) and the National Population Health Survey (NPHS), have generally relied on respondents to report their weight and height and used these data to estimate BMI.

A recent systematic review of the literature substantiated the existence of a bias associated with self-reported weight and height data.¹ Most studies have found that self-reports underestimate weight and overestimate height. Therefore, estimates of the prevalence of obesity based on self-reports tend to be lower than those based on measured data. As well, some evidence indicates that associations between obesity and morbidity differ depending on whether BMI is calculated with self-reported or measured data.^2,3

In 2005, the CCHS collected both self-reported and measured height and weight from a subsample of respondents. Using these data, this study documents the magnitude of the bias that exists for the Canadian population when height, weight and BMI are based on self-reports rather than on physical measures. Factors associated with reporting error are examined.

Methods

Data source

Data are from the 2005 CCHS. The CCHS covers the population aged 12 years or older living in private households. It excludes residents of Indian reserves, of institutions, and of some remote areas; full-time members of the Canadian Armed Forces; and civilian residents of military bases. Interviews for the 2005 CCHS were conducted between January and December of that year. The response rate was 79%, yielding a sample of 132,947 respondents.

Three sampling frames were used to select the sample of households for the 2005 CCHS: 49% of households came from an area frame; 50% from a list frame of telephone numbers; and the remaining 1%, from a Random Digit Dialing (RDD) sampling frame. Because of cost considerations, measured height and weight were collected for only a subsample ("subsample 2") of respondents, all of whom were from the area frame. Residents of the territories were not included in this subsample.

In total, 7,376 CCHS respondents were selected for subsample 2. Measured height and weight were obtained for 4,735 of them. The main reason for non-response was refusal. Because measured height and weight were recorded for only 64% of the selected respondents in subsample 2, an adjustment was made to minimize non-response bias. A special sampling weight was created by redistributing the sampling weights of non-respondents to respondents, using response propensity classes. The variables used to create these classes were: region (British Columbia, Prairies, Ontario, Quebec, Atlantic provinces), age, sex, household size, marital status, rural/urban indicator, and quarter of collection.

Of the 4,735 respondents for whom measured height and weight were collected, 125 were excluded from this analysis because self-reported height or weight was missing, and 43 women were excluded because they were pregnant at the time of the survey. This left 4,567 respondents.

A detailed description of the CCHS methodology is available in a published report.⁴

Analytical techniques

The bias associated with using self-reported data for weight, height and BMI was estimated by calculating the difference between measured and self-reported values (measured minus self-reported). A positive difference indicates under-reporting, and a negative difference, over-reporting.Respondents whose measured minus self-reported value was five or more standard deviations from the mean were considered outliers and dropped from the analysis (28 records were dropped for weight, 30 for height, and 32 for BMI).

Because the validity of self-reported data differs between the sexes,^5-9 separate analyses were conducted for males and females. To identify factors associated with reporting bias, differences between measured and self-reported values were examined in relation to: age, household income, immigrant status, leisure-time physical activity level, and measured weight, height and BMI. Multiple linear regression models were used to determine which factors were independently associated with the bias.

Respondents were classified into BMI categories (see Definitions). The degree of misclassification that resulted from using self-reports to estimate the prevalence of the various BMI categories was assessed by calculating sensitivity and specificity. Sensitivity is the percent of true positives, and specificity, the percent of true negatives. For example, for obesity (BMI 30 kg/m² or more), sensitivity is the percentage of respondents classified as obese based on self-reported values among those classified as obese based on measured values; in other words, the percentage of obese respondents who reported themselves as such. Specificity is the percentage of respondents classified as not obese (BMI less than 30 kg/m²) based on self-reported values among those who were not obese based on measured values; that is, the percentage of respondents who reported that they were not obese and among those who actually were not obese.

All estimates were weighted to represent the household population aged 12 years or older in 2005 (using the weight created to adjust for non-response to measured height and weight in subsample 2). To account for the survey design effect of the CCHS, standard errors, coefficients of variation, and 95% confidence intervals were estimated using the bootstrap technique.^10-12 Differences between estimates were tested for statistical significance, which was established at the 0.05 level.

Definitions

Self-reported height and weight were collected with the questions:

• "How tall are you without shoes on?" Categories for height in feet and inches were listed on the questionnaire, with corresponding metric values in brackets. Interviewers were instructed to round up to the closest inch for respondents who reported half-inch measures.

• "How much do you weigh?" If asked, interviewers told respondents to report weight without clothing. After reporting their weight, respondents were asked if they had reported in pounds or kilograms. Most respondents (94%) reported in pounds.

The majority of respondents (73% of males and 67% of females) reported values for their weight that ended in 0 or 5, although it would be expected that by chance only about 20% of respondents would have end-digits of 0 or 5 (10% for each value). This end-digit preference is another factor that was examined in relation to reporting bias.

CCHS interviewers were trained to measure the height and weight of respondents. Height was measured to the nearest 0.5 cm, and weight, to the nearest 0.1 kg. Calibrated scales (ProFit UC-321 made by Lifesource) and measuring tapes were used to ensure accuracy and consistency. The interview lasted about 50 minutes—respondents were asked their height and weight near the beginning, and measurements were taken close to the end.

Body mass index (BMI) is a measure of weight adjusted for height. In this analysis, BMI was derived from both measured and self-reported values. BMI is calculated by dividing weight in kilograms by the square of height in metres. Based on Canadian guidelines,¹³ which are in line with those of the World Health Organization,¹⁴BMI for adults is classified into six categories:

Category                   BMIkg/m² range
Underweight               (BMI less than 18.5)
Normal weight            (BMI 18.5 to 24.9)
Overweight                 (BMI 25.0 to 29.9)
Obese class I              (BMI 30.0 to 34.9)
Obese class II             (BMI 35.0 to 39.9)
Obese class III            (BMI 40.0 or more)

For adults aged 18 or older, respondents were assigned to height and weight quartiles based on weighted distributions. Separate quartile cut-points were established for men and women.

The International Obesity TaskForce (IOTF) has recommended that overweight and obesity among children and adolescents be determined by extrapolating the adult cut-points of 25 kg/m² for overweight and 30 kg/m² for obese to create sex- and age-specific values.¹⁵ In this analysis, 12- to 17-year-olds were classified as normal weight, overweight or obese based on these IOTF criteria; all obese adolescents were assigned to obese class I.

Immigrants were defined as those who were born outside of Canada and were not Canadian citizens by birth. Immigrant respondents were categorized into two groups according to length of residence in Canada: 0 to 10 years, and 11 or more years.

Leisure-time physical activity level was based on total energy expenditure (EE) during leisure time. EE was calculated from the reported frequency and duration of all of a respondent's leisure-time physical activities in the three months before the 2005 CCHS interview and the metabolic energy demand (MET value) of each activity, which was independently established.¹⁶

EE = ∑(Ni*Di*METi / 365 days) where
Ni = number of occasions of activity i in a year,
Di = average duration in hours of activity i, and
METi = a constant value for the metabolic energy cost of activity i.

An EE of 3 or more kilocalories per kilogram per day (KKD) was defined as active; 1.5 to 2.9 KKD, moderately active; and less than 1.5 KKD, inactive.

Household income groups were derived by calculating the ratio between the total household income from all sources in the previous 12 months and Statistics Canada's' low-income cutoff (LICO) specific to the number of people in the household, the size of the community, and the survey year. These adjusted income ratios were grouped into deciles (10 groups, each containing one-tenth of Canadians). Household income was missing for 253 records (8%) on the analysis file. To maximize sample sizes, a category for missing income values was created and included in the regression analysis.

Results

Height

On average, self-reported height was 0.7 cm more than measured height (Table 1). Males over-reported their height by an average of 1 cm, compared with 0.5 cm for females.

The tendency to over-report height increased with age, particularly among seniors (Table 2).  Men and women aged 65 to 79 years over-reported by 2.3 and 1.6 cm, respectively, and those aged 80 years or older, by 2.6 and 3.3 cm.

The shortest people (those whose measured height placed them in the lowest quartile of the distribution) were the least accurate: males in this group over-reported their height by an average of 2.3 cm, and females, by 1.9 cm. There was no significant difference between measured and self-reported height for males in the highest quartile (tallest), and for females in the two highest quartiles.

Over-reporting of height varied by measured BMI.  For people in the normal weight category, self-reported and measured height did not differ, but those who were overweight or obese tended to over-report. Discrepancies were pronounced among people in obese class III, with males over-reporting their height by an average of 2.1 cm, and females, by 2.8 cm.

Multiple linear regression was used to identify variables associated with differences between self-reported and measured height. Measured height, measured weight and age were independently associated with differences for both sexes (Appendix Table A). In general, height was over-reported. Therefore, positive regression coefficients (for example, height) signal a reduction in this over-reporting bias, and negative coefficients (for example, weight), an increase in the bias. Associations between height discrepancies and household income, immigrant status and physical activity in the univariate analysis did not persist in the multivariate analysis.

Weight

Self-reported weight was, on average, 2.1 kg less than measured weight. The bias was greater among females, who under-reported by an average of 2.5 kg, compared with 1.8 kg for males.

Females in all four measured weight quartiles under-reported their weight, with the difference rising from an average of 0.6 kg for those in the lowest quartile to 5.1 kg for those in the highest (Table 3). The self-reported and measured weight of males in the lowest quartile did not differ. Males in the remaining quartiles under-reported, with the difference rising from 1.1 kg for those in the second quartile to 4.1 kg for those in the highest.

End-digit preference (reporting a weight ending in 0 or 5) was associated with under-reporting for females, but not for males. Females with an end-digit preference tended to round their weight down, whereas males were as likely to round up as to round down.

Differences between self-reported and measured weight were strongly associated with measured BMI. Underweight males over-reported their weight by an average of 6.9 kg. Self-reported and measured weight did not differ significantly for males in the normal weight range, but those who were overweight or obese tended to under-report, with the greatest difference among the obese. For underweight females, self-reported and measured weight were not significantly different. Females in the normal, overweight and obese categories all under-reported, with discrepancies increasing at successively heavier BMI categories.

When differences between self-reported and measured weight are displayed graphically (Figure 1), the increase in bias associated with BMI category is evident. As BMI moves from underweight to obese, the distribution of average differences shifts to the right of zero, showing that the extent of under-reporting rises with BMI.

In the multivariate analysis, the strongest predictor of a difference between self-reported and measured weight was measured weight (Appendix Table B), as evidenced by the standardized regression coefficients. In this case, the positive value of the regression coefficient for weight indicated an increase in the bias. The negative regression coefficient for measured height for males shows that as measured height increased, under-reporting of weight decreased. For females, an association with leisure-time physical activity level emerged—active females were slightly more likely to under-report their weight. Age and immigrant status were significant in the univariate analysis, but these associations did not persist in the multivariate analysis.

Body mass index

BMI based on self-reported height and weight was, on average, 1.1 kg/m² less than BMI based on measured values. Underestimation occurred for both sexes, but was slightly greater for females (1.2 kg/m²) than for males (0.9 kg/m²).

The extent of the difference between BMI based on self-reported rather than on measured height and weight was strongly associated with measured BMI (Table 4). For underweight males, BMI based on self-reported values was overestimated, and for underweight females, BMI based on self-reported and measured values did not differ significantly. For all other BMI categories, self-reported BMI underestimated measured BMI, with the degree of underestimation increasing with successively higher BMIs.  For obese class III, underestimation was, on average, 4.0 kg/m² among males, and 5.0 kg/m² among females.

In the multivariate analysis, the strongest predictors of BMI differences were measured weight and height (Appendix Table C). There was also a weak association with age. Among females, an association with leisure-time physical activity level emerged: underestimation of BMI was slightly greater among active and moderately active females, compared with inactive females

Misclassification of BMI categories

The degree of misclassification that results when BMI categories are based on self-reported height and weight was assessed by calculating sensitivity and specificity (Table 5).

Sensitivity was high for those who, according to measured height and weight, were in the normal weight category. That is, 95% of males and 93% for females whose measured height and weight put them in the normal weight BMI category were correctly placed in this category based on their self-reported height and weight. For people who were overweight, sensitivity fell to 70% among males and to 63% among females. Sensitivity was low for males and females who were obese: 51% and 54% for those in obese class I, and 45% and 57% for those in obese class II/III. Among people who were underweight, sensitivity was particularly low for males at 40%, but higher for females at 78%.

For the obese category overall (BMI 30 kg/m² or more), sensitivity was 63%, and was somewhat higher for females than for males (Table 6). Sensitivity was particularly low for seniors.

Specificity was very high (more than 95%) for the obese categories, indicating that very few respondents reported height and weight that put them in the obese category unless they really were obese.

Prevalence of obesity

Prevalence estimates of BMI categories differed substantially when calculated with measured rather than self-reported height and weight (Table 7).The prevalence of obesity based on measured data was 7 percentage points higher than the estimate based on self-reported data (22.6% versus 15.2%). Among males, the prevalence was 9 percentage points higher, and among females, 6 percentage points higher.

Differences were particularly pronounced among people aged 65 years or older (Figure 2). For elderly men, the estimate of obesity based on measured values was 15 percentage points higher than the estimate based on self-reported values, and for elderly women, 13 percentage points higher.

Discussion

This is the first nationally representative study to compare self-reported and measured height, weight and BMI for the Canadian population. Consistent with other research,¹ systematic errors emerged, with height over-reported, and weight under-reported.

As in other studies,^5,7,8,17,18 over-reporting of height rose with age for both sexes and was substantial at age 65 years or older. Loss of stature commonly occurs among seniors as a result of aging-related processes such as osteoporosis and loss of muscle tone,¹⁹ and they may report their height as it was in earlier years.

The degree of under-reporting of weight in the 2005 CCHS was greater than in studies based on population health surveys conducted in the past in other countries, including the United States,^5,20,21 England,^22, 23 Scotland,²⁴ Wales,⁷ Spain,¹⁷ New Zealand,¹⁸ Mexico,²⁵ Finland,²⁶ and Brazil.²⁷ Most of the data for these studies were collected at least 10 years ago.

As well, for Canada, two decades ago, self-reported weight from the 1985 Health Promotion Survey was compared with measured weight from the 1981 Canada Fitness Survey.²⁸ For those aged 20 to 69 years, males' average weight did not differ between the two surveys, and for females, average weight based on measured values was actually 0.6 kg lower than that based on self-reports. These results are similar to findings from a contemporaneous American study,²⁹ and indicate that the reporting bias for weight has increased in the intervening years.

In recent years, the percentage of Canadians with excess weight has risen considerably,^30,31 mirroring a worldwide trend.³² Because the extent to which weight is under-reported increases with BMI, the greater overall bias may reflect the higher percentages of Canadians in the overweight and obese categories in 2005.  Another possibility is the stigma associated with obesity. The increasing prevalence of obesity does not seem to have made excess weight more acceptable, and some evidence suggests that the stigma is intensifying.³³ This may also explain the greater tendency to under-report weight among females, who may feel more pressure to conform to "desirable" standards.³⁴

Limitations

For various reasons, measured height and weight were obtained for only 64% of the respondents who were selected for the physical measures component (subsample 2) of the CCHS. A special sampling weight was created to minimize the non-response bias associated with factors such as age, sex and region of the country (see Data source). Nonetheless, estimates of obesity based on measured values could still be biased if the height and weight of non-respondents differed systematically from the height and weight of those for whom measured data were obtained. However, because self-reported height and weight were collected for both respondents and non-respondents to the physical measures, it was possible to partially evaluate the extent of this bias by comparing obesity estimates based on these self-reported data.  Among all respondents selected for the physical measures, the prevalence of obesity based on self-reported values was 15.9% (Appendix Table D). The prevalence was substantially higher among non-respondents than among those whose height and weight were measured (19.1% versus 14.0%), indicating that heavier people were less likely to agree to be measured. But when the special sampling weight was applied to respondents to the physical measures, the prevalence of obesity based on self-reported data was 15.2%, fairly close to the estimate for all respondents selected for physical measures.

Some of the bias associated with under-reporting weight may be due to clothing. Respondents were weighed fully clothed, but people may weigh themselves at home with minimal or no clothing, and if asked, interviewers told respondents to report their weight without clothing.

Some of the bias associated with over-reporting height may be due to rounding. Interviewers were instructed to round up to the nearest inch for respondents who reported half-inch values, while for the measurement component, height was measured to the nearest 0.5 cm.

A number of other studies have been designed to ensure participants were unaware that measurements would be taken,^2,18 because it is believed that if respondents know they are going to be measured, they may report more accurate values. Although CCHS interviewers were not instructed to ensure that respondents in subsample 2 did not know that they would be measured, this did not seem to have affected the self-reported values—there were no differences between the average self-reported height and weight of respondents from the area frame who were selected to be measured and those who were not.

Although measured height and weight were considered "true" values, some factors may have limited their accuracy. Trained Statistics Canada interviewers measured the height and weight of respondents; measures taken by health technicians, as have been used in other studies, may be more accurate.^5,29 The Statistics Canada interviewers used identically calibrated scales and measuring tapes, but validity and reliability studies to assess inter- and intra-interviewer accuracy and reproducibility were not performed. Stadiometers might have provided more accurate measures of height than measuring tapes.

Finally, this study compares measured height and weight with self-reported values obtained in face-to-face interviews. Self-reports from face-to-face interviews may yield higher prevalence estimates of obesity than do data collected by telephone.³⁵ Even so, the estimate of obesity based on self-reports for the sample from the telephone frame was only one percentage point lower than the estimate for subsample 2, which was based on self-reported data from interviews conducted in person.

Conclusion

For fiscal and logistical reasons, the collection of self-reported height and weight data will continue in large-scale health surveys conducted by Statistics Canada. As this study reveals, this practice yields biased values for height and weight, which result in substantial misclassification of the population by BMI category. The prevalence of obesity based on measured data was 7 percentage points higher than the estimate based on self-reported data (22.6% versus 15.2%).

The implications of this study are relevant to policy-makers, researchers and data users. Until now, trends in the prevalence of obesity in Canada have generally been based on self-reports, but the use of such data means that the accuracy of estimates and true changes in prevalence over time are unknown.

As well, the results raise the question of whether associations between BMI and obesity-related health conditions are distorted when BMI is derived from self-reported data.  It is often suggested that underestimating the prevalence of obesity may diminish associations between obesity and health outcomes. However, a second study, also based on 2005 CCHS data,³⁶ found that associations between obesity-related conditions and overweight and obesity were exaggerated when BMI was based on self-reported rather than measured data. To correct the bias, researchers may wish to consider adjusting self-reported values or lowering BMI cut-points for the overweight and obese categories.

Finally, it will be important to measure the magnitude of the bias periodically to see if it changes over time. In 2007, Statistics Canada launched the Canadian Health Measures Survey (CHMS), the most comprehensive national survey using physical measurements ever conducted in Canada. The CHMS data will provide the opportunity for further analysis of the bias resulting from using self-reported measures in estimating the prevalence of obesity. As well, the data set will be used to study measured BMI in comparison with other anthropometric measures such as waist and hip circumference and skinfold measurements.