Analytical Studies: Methods and References
Imputing Postal Codes to Analyze Ecological Variables in Longitudinal Cohorts: Exposure to Particulate Matter in the Canadian Census Health and Environment Cohort Database Analytical Studies: Methods and References
Imputing Postal Codes to Analyze Ecological Variables in Longitudinal Cohorts: Exposure to Particulate Matter in the Canadian Census Health and Environment Cohort Database

View the most recent version.

by Philippe Finès, Lauren Pinault and Michael Tjepkema
Health Analysis Division

Skip to text

Text begins

Abstract

This paper describes a method of imputing missing postal codes in a longitudinal database. The 1991 Canadian Census Health and Environment Cohort (CanCHEC), which contains information on individuals from the 1991 Census long-form questionnaire linked with T1 tax return files for the 1984-to-2011 period, is used to illustrate and validate the method. The cohort contains up to 28 consecutive fields for postal code of residence, but because of frequent gaps in postal code history, missing postal codes must be imputed. To validate the imputation method, two experiments were devised where 5% and 10% of all postal codes from a subset with full history were randomly removed and imputed. The proportion of discrepancies in displacements and mean exposure were consistently higher in the experiment with 10% removed postal codes.

Keywords:  CanCHEC, census follow-up cohort; data linkage; environmental exposure; PM_2.5; geographic information systems; imputation; longitudinal studies; pollution; postal codes; residential mobility

1. Introduction

The 1991 Canadian Census Health and Environment Cohort (CanCHEC), which contains information on more than 2.5 million respondents to the 1991 Census long-form questionnaire, was linked with T1 tax return data for the 1984-to-2011 period. Thus, the linked file contains up to 28 consecutive fields for postal code of residence. However, for many respondents, the postal code history is incomplete: individuals may not have filed a tax form, or they may have left the country. In fact, missing residential information is common in longitudinal databases.

Full postal code histories are important for environmental health research. Missing postal codes must be imputed in order to assign historical exposures to environmental hazards or to the ecological variables under study. Imputation must be done so that values of ecological variables assigned in years with missing postal codes likely represent real exposures. The method must be plausible, simple and parsimonious, and should yield reliable results.

This paper describes a method of imputing postal codes and assesses its validity. The method is demonstrated using a specific database, CanCHEC, and for a specific ecological variable, exposure to particulate matter 2.5 micrometres in diameter (PM_2.5). The concept of gap is used throughout the paper. A gap is defined as a string of consecutive years with missing postal codes.

2. Data

The Historical Tax Summary File

The Historical Tax Summary File (HTSF) is an annual compilation of tax return data representing individuals for whom a tax declaration was filed in a given year. For the 1984-to-2011 period, the HTSF provides a history of individuals’ residential locations, with 28 consecutive fields for postal code (Wilkins et al. 2008; Peters et al. 2013). The HTSF was linked to the 1991 Canadian Census Cohort and the Canadian Mortality Database (CMDB) using social insurance number, thereby creating a new database—CanCHEC—that provides cohort members’ postal code histories. These postal codes are used in environmental health research to assign exposure data to cohort members over time.

Postal codes

A postal code is a six-character alpha-numeric identifier devised and maintained by the Canada Post Corporation for sorting and delivering mail. The characters are arranged in the form “ $ANANAN$,” where “A” represents a letter, and “N,” a single-digit number (for example, K1A 0T6). The first three characters refer to a set of stable, well-defined areas, known as forward sortation areas (FSAs); the last three characters identify routes known as local delivery units (Statistics Canada 2014). 

The first character represents a large region or province/territory. Postal codes with zero (0) as the second character are rural; by default, those that do not have a 0 in the second position may be suburban or urban (hereafter urban).

Postal codes have a hierarchical structure. Areas with the same first two characters are in the major region designated by the first character. This hierarchy continues with each additional character (Statistics Canada 2014). Postal codes do not include the letters D, F, I, O, Q or U, and the first position does not use the letters W or Z (Table 1).

3. Methods

Preliminary steps on postal codes

The CanCHEC is used to demonstrate the method of imputing postal codes, but the method can be applied to other longitudinal cohorts.

The CanCHEC database contains 2,734,835 observations, 2,644,370 of which (those with a valid history) were used for analyses. With the Postal Code Conversion File Plus (PCCF+) program, probable non-residential (business) postal codes (Statistics Canada 2014) were flagged and removed from the database because they were unlikely to relate to a private residence. Among the remaining observations, 1,238,825^{Note 1} (47%) had a full postal code history, that is, a postal code in each year of follow-up. The others were missing at least one postal code between the year of entry (1984) and the year of exit (2011 or year of death).

Corrections and adjustments were then applied to the database.

1. Censoring: A cohort member not identified as dead was considered to be alive up to the final year of follow-up (2011). In such cases, if the last year with a postal code was before the final year of follow-up:
   • for (up to) the first two years after the last year with a postal code, the last postal code was assigned;
   • for the years thereafter until the final year of follow-up, a postal code was assigned based on imputation Case 2b (see below).
2. Postal code at death: For cohort members identified as dead (from the CMDB or HTSF), the following rules were applied:
   • if the CMDB contained a postal code, that postal code was assigned in that year and used as a surrounding postal code (“following”) for a gap, and imputation was based on Case 1 (see below);
   • if the CMDB did not contain a postal code, the postal code at death was missing. Imputation Case 2b (see below) was used for the last gap.

Imputing Historical Tax Summary File postal codes

The objective is to use imputation to fill the gaps in postal code histories in the HTSF. All postal codes in a gap are imputed during the same step. Consequently, the imputed values of the postal codes in the same gap are not necessarily related, and newly imputed postal codes are not used for imputation.

Because a large majority of cohort members do not move in a given year, missing postal codes can be largely based on the postal codes reported in the surrounding years. For example, if a postal code for a given year is missing, but the same postal code is recorded in the years before and after, it can be inferred that the address during the gap was the same as that of the surrounding years. However, there is always a non-null probability that the postal code for the gap should not be imputed based on the surrounding postal codes. For example, in a given year, a person may have temporarily not lived in their usual place of residence.

As well, the probability that missing postal codes can be imputed based on surrounding postal codes diminishes as the gap lengthens. The imputation method takes account of a probability threshold (p) that varies by gap length such that p is not 100%, and that p does not increase if gap length increases. Suggested values for this threshold are shown in Table 2-1.

Taking advantage of the hierarchical coding structure of postal codes, similarity (k) and dissimilarity (d) values of the postal codes surrounding a gap are defined as in Table 2-2. The similarity value (k) is the number of consecutive identical characters (beginning from the left) in the surrounding postal codes. The dissimilarity value (d) is equal to 6 minus k.

Definition of rules and cases

Two rules are defined using threshold (p), similarity (k) and dissimilarity (d):

• Rule A:  The missing postal code is imputed based on the postal codes surrounding the gap. The imputed value will contain (from left to right): the k characters common to both sides of the gap, followed by d times character “*”.
• Rule B: The missing postal code is given a value unrelated to the surrounding postal codes, that is, the value “ $DUMMYd$.”

Case 1: For gaps with surrounding postal codes:

• For each year in the gap, a random number (u) is drawn from a random uniform distribution $U(0,1)$. If $u\le p$, apply Rule A; otherwise, apply Rule B.

Case 2: For gaps missing at least one surrounding postal code, Rule B is always applied.

• If there is no postal code before AND after the gap (all history lacks postal codes), assign all missing postal codes to “ $DUMMY7$ ”;
• If only the postal code after the gap is missing, assign all missing postal codes to “ $DUMMY8$ ”;
• If only the postal code before the gap is missing, assign all missing postal codes to “ $DUMMY9$.”

The rules and cases are illustrated with examples in the Appendix. The imputation method thus replaces a missing postal code with: (1) a complete postal code; (2) a value that contains characters from a postal code followed by a certain number of “*”; (3) or a value that starts by “$DUMMY$” followed by a number from 0 through 9.

Validation

The purpose of validating the imputation method is to determine if exposure data calculated using imputed postal codes are similar to those calculated using the original, complete postal code data. As an example, estimates of PM_2.5 are calculated.

Cohort member residences were spatially linked to estimates of a surface layer of PM_2.5 concentration for all mainland North America, derived from a model that provides average PM_2.5 concentrations at an approximately 1 km² resolution from 2004 to 2011 (van Donkelaar et al. 2015; Pinault et al. 2016). Estimates were backcast for 1998 to 2003 using the inter-annual variation in Boys et al. (2014). Outliers with PM_2.5 values greater than 20 micrograms per cubic metre (µg/m³) were excluded from analysis (fewer than 1% of cohort members in any year) (Pinault et al. 2016). Air pollution data for exposure were not available before 1998 (fifteenth year of follow-up). When multiple observations of the same postal code were available in the exposure database, one was randomly selected.

Since the exposure file used in this analysis contained exposure data for all postal codes with at least the first 3 characters present (for example, postal codes such as $A0A$, $A0A1$, $A0A1A$, $A0A1A0$ ), it follows that (1) all postal codes built with Rule A and containing at least 4 “*” and (2) all postal codes built with Rule B are defined as uninformative postal codes, that is, imputed postal codes for which no exposure data are available. For all uninformative postal codes, exposure data are assigned to a missing value. By contrast, imputed postal codes ending with $d<4$ “*” are partially informative. The environmental exposure for these postal codes is assigned as the average for postal codes starting with the same k similarity characters.

Consider a missing postal code that belongs to a gap for which the surrounding postal codes are present: Case 1 is applied. If the surrounding postal codes have no common characters, either Rule A is applied: the resulting postal code will be “******”; or Rule B is applied: the resulting postal code will be “ $DUMMY6$.” Therefore, even though the two resulting postal codes differ (because they are built with different rules), they each represent an uninformative postal code.  In other words, all postal codes in gaps surrounded by postal codes with a similarity equal to 0 are always uninformative.

Validation was performed on the 1,238,825 observations with complete postal code histories. A percentage of these postal codes was randomly erased, and the imputation method was applied to the missing postal codes. The percentages erased were 5% in the first experiment (Experiment A) and 10% in the second (Experiment B). These percentages approximate the actual percentage of missing postal codes in the original database (8.1%). Imputation used the threshold values in Table 2-1. The two new files with imputed postal code histories were compared with the original dataset with complete postal code histories. The percentages of postal codes imputed according to each rule and the percentages of imputed postal codes matching the originals were calculated.

Results for individuals were also examined. The discrepancy per individual between the original and new datasets was used to validate the imputation method. The measures were:

• mean number of moves (changes in postal code);
• mean geographic displacement (based on latitude and longitude coordinates^{Note 2} of the centroids of the postal code); and
• mean exposure during history (based on exposure to PM_2.5 by postal code).

The relevant statistics were the percentages of observations for which the absolute value of the difference between each of the two new files and the original file reached or exceeded a threshold for large values. For mean number of moves, mean displacement and mean exposure, the threshold was set at 0.1; for number of displacements, it was set at 2. These thresholds roughly corresponded to the upper 5% tail of the distributions of the variables.

Analyses of the postal codes were performed globally, then according to the first character of the postal code (which defines large regions), then according to the rural/urban designation of the second character of the postal code. Analyses of individuals were performed globally, then according to the first character of the first postal code in the history, then according to the rural/urban designation of the second character of that code.

4. Results

Global results

In the whole CanCHEC database of 2,644,370 observations,

• the mean length of history was about 26 years (results not shown);
• about 2.4 million postal codes were missing (Table 3), which represent about 8% of postal codes;
• distribution of the lengths of gaps was highly skewed; 55% of gaps were one or two years long (Table 3);
• when imputation was performed, Cases 1 (Rule A) and 2c were the most frequent (results not shown).

Analyses of postal codes

A total of 1,735,620 gaps were created in Experiment A, and 3,468,405 in Experiment B (Table 4-1. In Experiment A, 91% of missing postal codes belonged to gaps which were one year long and 9% to gaps which were two years long; in Experiment B, the figures were respectively 82% and 16%. Rules A and B were applied in proportions corresponding to the parameters in Table 2-1. The overall percentage of perfectly matched postal codes (i.e., situations where imputed and original postal code are the same) was 76%; percentages were higher for shorter (one or two years) than for longer (five or more years) gaps. Results by region and geography showed the same patterns (Tables 4-2 and 4-3).

Analyses of individuals

In Experiments A and B, the mean number of moves differed by at least 0.1 in 1.2% and 4.7% of observations, respectively; the mean number of latitude-longitude coordinates was different by at least 2 in 3.5% and 11.5% of observations; mean displacement differed by at least 0.1 degree of latitude-longitude in 2.4% and 4.5% of observations; and mean exposure differed by at least 0.1 µg/m³ in 4.1% and 8.1% of observations (results not shown).

Mean displacement

Differences in mean displacement between the experimental datasets and the original dataset were examined by region (defined by first character of first postal code in history), and urban versus rural location (defined by second character of first postal code in history). Generally, the percentage of observations where the absolute difference in mean distance was at least 0.1 degree did not vary systematically between regions, except for the Northwest Territories and Nunavut, where it was higher (Table 5). The percentages of observations where the absolute difference in mean distance was at least 0.1 degree were slightly higher for rural than urban postal codes.

Mean exposure

The percentage of observations where the absolute difference in mean PM_2.5 exposure was at least 0.1 µg/m³ did not vary systematically between regions (Table 6); it was slightly higher for observations with the first postal code indicating an urban region.

5. Discussion

Validation was conducted on a subset of the database in which all postal codes were present, but from which small percentages (5% in Experiment A; 10% in Experiment B) were erased and then imputed. The percentage of postal codes erased and imputed fit with the percentage observed in the subset.

Results for postal codes showed that Rules A and B were applied according to the a priori threshold p, and that the percentage of perfectly matching postal codes was usually greater than two-thirds for gaps that did not exceed two years.

Postal code results for individuals (number of moves, number of latitude-longitude coordinates, displacement based on latitude-longitude coordinates) revealed discrepancies in 1.2% to 3.5% of observations for Experiment A (4.5% to 11.5% for Experiment B). Results for PM_2.5 exposure revealed discrepancies in 4.1% of observations (Experiment A) and 8.7% of observations (Experiment B). In the context of assigning environmental health exposures, these percentages were considered satisfactory. Also, they did not vary drastically across geographic region or rural/urban location.

Based on these results, the imputation method was considered to be valid. However, in the subsets used for validation, most of the randomly created gaps were short (one or two years), whereas in the original database, only 55% of gaps were one or two years long (Table 3). Thus, the validation method yielded a lower percentage of long gaps than in the original file. This is because the random rule to render postal codes missing in the histories does not take account of some correlation that could exist between successive missing postal codes. Consequently, performance of the imputation could be slightly overestimated. This is not a limitation of the method, but rather, a limitation of the validation due to the database used for the analyses.

Nevertheless, for any longitudinal database in which gaps generally do not exceed two years, the imputed postal codes would be similar to those of the original database. It is the nature of CanCHEC that long gaps will occur. The only control is the choice of the a priori threshold (p). Depending on the situation, analysts using databases with long gaps could apply much lower values of p (for instance, 0.2) when the gap becomes too long, thereby increasing the percentage of occurrences of Rule B. This suggests that the presence of long gaps (at least four years) may make imputation of postal codes arduous.

Other methods could be used to impute the long postal code gaps that frequently occur in databases. Postal codes at the ends of the gaps might be imputed first, and then, working inward, those in the interior of the gap could be imputed in subsequent steps. However, this would generate postal codes that are highly dependent on the first ones imputed, and for which the level of confidence would vary with the distance to the ends of the gap. Another possibility is to impute postal codes based not only on the two surrounding the gap, but also on those that are one or two years further away. This could involve many hypotheses and a series of complex rules.

6. Conclusion

This paper describes a method to impute postal codes in a longitudinal cohort. Imputation was largely based on the postal codes immediately surrounding the gaps. Validation was conducted by randomly erasing a percentage of postal codes from a subset of full histories, imputing the postal codes that were erased, and evaluating the results. This method of imputing postal codes is fully functional for the Canadian Census Health and Environment Cohort database and is considered valid. It can be adapted for any longitudinal file and for any pollutant or ecological variable.

The SAS programs used to implement the methods described in this paper are available from the authors on request.  A user guide is in preparation.

7. Appendix

Illustration of imputation rules

To illustrate the imputation rules, five years of follow-up and seven examples are presented (first six columns in Appendix Table 1, showing examples of postal codes before imputation). For each gap identified in each example, the table provides a description of the gaps, identification of the case and how the surrounding postal codes compare (the two central columns in Appendix Table 1). According to the rules, for gaps belonging to Case 1, a random assignment would apply to Rule A or B. For examples belonging to Cases 2a, 2b or 2c, Rule B would be used. Results (postal codes after imputation) are shown in the last columns of Appendix Table 1. Example 4 illustrates what has already been explained: the three missing postal codes in the gap are imputed simultaneously and independently. Also, had the random generator produced different random numbers, imputed postal codes for examples 1, 2 (both imputed postal codes), 3 (year 3 only) and 4 (the 3 of them) could have been different: Rule A could have been applied instead of Rule B, and vice versa.

References

Boys, B.L., R.V. Martin, A. van Donkelaar, R.J. MacDonell, N.C. Hsu, M.J. Cooper, R.M. Yantosca, Z. Lee, D.G. Streets, Q. Zhang, and S.W. Wang. 2014. “Fifteen-year global time series of satellite-derived fine particulate matter.” Environmental Science and Technology 48 (19): 11109–11118.

Peters, P.A., M. Tjepkema, R. Wilkins, P. Fines, D. L. Crouse, P.C.W. Chan, and R. T Burnett, 2013. “Data Resource Profile: 1991 Canadian Census Cohort.” International Journal of Epidemiology 42 (5): 1319–1326.

Pinault, L., M. Tjepkema, D.L. Crouse, S. Weichenthal, A. van Donkelaar, R.V. Martin, M. Brauer, H. Chen, and R.T. Burnett. 2016. “Risk estimates of mortality attributed to low concentrations of ambient fine particulate matter in the Canadian Community Health Survey cohort.” Environmental Health 15 (1): 18.

Statistics Canada. 2014. Postal Code^OM Conversion File Plus (PCCF+) Version 6C, Reference Guide – November 2014 Postal Codes^OM. Statistics Canada Catalogue no. 82-F0086-XDB. Ottawa: Statistics Canada.

van Donkelaar, A., R.V. Martin, J.D. Spurr, and R.T. Burnett. 2015. “High-resolution satellite-derived PM_2.5 from optimal estimation and geographically weighted regression over North America.” Environmental Science and Technology 49 (17): 10482–10491.

Wilkins, R., M. Tjepkema, C. Mustard, and R. Choinière. 2008. “The Canadian census mortality follow-up study, 1991 through 2001.” Health Reports 19 (3): 25–43. Statistics Canada Catalogue no. 82-003-XPE.

Notes