Evaluation of the factor structure of the child-reported parenting questionnaire in the National Longitudinal Survey of Children and Youth

View the most recent version.

by Rübab G. Arim, Jennifer D. Shapka, V. Susan Dahinten and Brent F. Olson

Research has demonstrated significant relationships between parenting behaviours and child health and development.^1-6 For example, low parental nurturance and high parental rejection have been associated with anxiety, property offence and hyperactivity-inattention problems in adolescence.³  But despite the considerable number of studies, relatively little is known about the quality of measures based on child-reported parenting behaviours.^7-9  In the absence of empirically validated measures, it is possible that some findings reflect spurious associations.  This analysis addresses that gap by evaluating the factor structure of the child-reported parenting scales in Statistics Canada's National Longitudinal Survey of Children and Youth (NLSCY).

To date, no studies have evaluated the factor structure of the NLSCY child-reported parenting questionnaire.  Because the parenting behaviours measured by this questionnaire are often included as risk or protective factors in NLSCY-based health research,¹⁰ it is important that their quality be assessed.

The NLSCY is a high-profile survey, the results of which have the potential to influence policy on children's health and development.¹¹  It collects information about a representative sample of Canadian children and youth, which enhances the generalizability of findings.  As well, because the data are longitudinal, the factor structure of the child-reported parenting questionnaire can be evaluated across time, thereby providing a stronger analytical framework for construct validation than would cross-sections of a study population.

Data and methods

The biennial NLSCY, conducted jointly by Statistics Canada and Human Resources and Skills Development Canada, began in 1994/1995.  The target population for the first cycle was newborns through age 11. Households in Yukon, Nunavut, and the Northwest Territories, and children in institutional settings were excluded. The response rate was 87%, which resulted in 22,831 children in cycle 1 (1994/1995).¹² The response rates for these children in the second and third cycles were 67% and 65%, respectively.¹³

The household data were provided by the person most knowledgeable (PMK) about the child during a face-to-face or telephone interview.  With the PMK's permission and in a private setting to ensure confidentiality, children aged 10 or older completed a separate questionnaire.¹³

The initial sample for this analysis consisted of two longitudinal cohorts:  the first was made up of children aged 10 to 11 in 1998/1999 (cycle 3) who were re-interviewed at ages 12 to 13 in 2000/2001 (cycle 4), and at ages 14 to 15 in 2002/2003 (cycle 5); the second cohort was children aged 10 to 11 in 2000/2001 (cycle 4) who were re-interviewed at ages 12 to 13 in 2002/2003 (cycle 5), and at ages 14 to 15 in 2004/2005 (cycle 6). 

Before the data for the two cohorts were combined, a series of independent sample t-tests (or chi-square tests for dichotomous variables) was performed to ensure that there were no systematic differences between the cohorts in demographic factors such as gender, household income and PMK education, or in parental nurturance, rejection and monitoring at each age.  The findings indicated that socio-economic characteristics differed across cohorts, but these differences varied by age group and cohort.  For example, PMKs of children in cohort 1 reported lower household income than did those in cohort 2 at ages 10 to 11, but the reverse was found at ages 14 to 15.

Nurturance was the only parenting variable for which a statistically significant difference emerged, with a small effect size at ages 10 to 11 and 12 to 13 (children in cohort 2 reported higher levels of nurturance than did children in cohort 1).

The response format for the questionnaire was changed from a 4-point scale in cycle 1 to a 5-point scale in subsequent cycles; some items were removed after cycle 1; and new items were added after cycle 2.  To ensure the consistency of items and of the response scale, the sample was drawn from cycles 3 to 6. 

The final sample for the study consisted of 1,164 children who were interviewed at ages 10 to 11, 12 to 13, and 14 to 15, and who had complete data on the three parenting behaviour scales.  The impact of cycle-to-cycle attrition and attrition due to missing data was examined in a series of independent t-tests (or chi-square tests for dichotomous variables).  The final sample had a somewhat higher socio-economic status than did the initial sample, but the effect size measures were small.

Of the children in the analysis, 53% were female; 75% lived with their biological parents; 15% lived in a single-parent household; and 59% lived in a household with an annual income of $50,000 or more when they were aged 10 to 11.  Most PMKs (91%) had at least secondary graduation.

Parenting questionnaire

The child-reported  parenting questionnaire was developed by Lempers et al.,¹⁴ and was based on Schaefer's¹⁵ Children's Report of Parental Behavior Inventory and on the Child Rearing Practices Report of Roberts et al. ¹⁶  The original 29-item questionnaire measured three parenting behaviours:  nurturance, inconsistent rejection-oriented discipline, and monitoring.  Nurturance denotes positive evaluation, expression of affection and equalitarian treatment.  Inconsistent rejection-oriented discipline behaviours are negative affect, control, and hostility.  Monitoring involves parental direction and supervision.^15,16  These three dimensions of parenting were supported through an exploratory factor analysis (EFA) using varimax rotation.¹⁴  The alpha coefficient for all 29 items was .80.  The authors did not provide the internal consistency scores for the three subscales, nor did they provide information on validity.

In the NLSCY version of the questionnaire, the wording of one item was modified, and 10 items (6 nurturance, 3 rejection, and 1 monitoring) were excluded, resulting in a 19-item scale.  EFA based on the first cycle of data for children aged 10 to 11¹²  revealed three factors that were labelled nurturance (7 items; alpha = .77), rejection (7 items; alpha = .59), and monitoring (5 items; alpha = .54). 

Although the EFA revealed the underlying structure of this questionnaire, the low alphas suggest that a stronger empirical and conceptual evaluation of the measure is necessary to provide support for construct validity.¹⁷  Confirmatory factor analysis (CFA) has been shown to be a highly effective approach to providing support for construct validation.¹⁸  Unlike EFA, CFA of each parenting behaviour model (nurturance, rejection and monitoring) provides support for construct validity to the extent that the constructs are measured by the specified indicators, and are related in a theoretically predictable manner. 

Data analysis

The three parenting behaviours—nurturance (for example, "my parents smile at me"), rejection (for example, "my parents hit me or threaten to do so"), and monitoring (for example, "my parents want to know exactly where I am and what I am doing")—were assessed using a 5-point response scale ranging from 0(never) to 4 (always), with higher scores indicating that the child perceived more nurturing, rejection and monitoring.  (The item "[my parents] let me go out any evening I want" in the monitoring scale was reverse-coded.)

Ordinal coefficient alpha was used to estimate the reliability of the three scales.¹⁹  Across the three time points (at ages 10 to 11, 12 to 13, and 14 to 15, respectively), the estimates of reliability were "high" for the nurturance scale (.90, .92, .94), "good" for the rejection scale (.75, .79, .83), and "acceptable-to-satisfactory" for the monitoring scale (.63, .65, .70)

A separate set of CFAs was conducted to evaluate the factor structure of each parenting scale across the three time points.  Because of the construct-confirming nature of the study, the data were not weighted; the lack of generalizability of the findings at the population level is recognized.²⁰  The CFAs were run using the LISREL 8.80 program,²¹ with weighted least squares (WLS) estimation²² on polychoric covariance²³ and asymptotic variance/covariance matrices, which were computed using PRELIS²⁴ (version 2.80).  Model fit was evaluated using the following global goodness-of-fit indices: the root mean square error of approximation²⁵ (RMSEA) and the 90% confidence interval (CI) for RMSEA; the comparative fit index²⁶ (CFI); and the standardized version of the Root Mean Squared Residual²⁷ (SRMR).  An RMSEA less than or equal to .06, a CFI .95 or more, and a SRMR less than or equal to .08 indicated a good fit of the model to the data.²⁸  The 90% CI around the RMSEA statistics should contain .05 to indicate the possibility of a close fit.²⁹  In addition to the criteria for the goodness-of-fit statistics, the parameter estimates of all items³⁰ were considered, as well as the standardized residual matrix³¹ to evaluate model fit.  For this study, the standardized factor loading values were expected to be greater than or equal to .30,^17,32 and standardized residuals for each item to be consistently less than 4.0.³¹

Results

Nurturance

The 7-item NLSCY Parental Nurturance model (Table 1) was a good fit to the data for children aged 10 to 11 (RMSEA = .038, 90%CI = .024 to .053; CFI = .986; and SRMR = .035) and for children aged 12 to 13 (RMSEA = .039, 90%CI = .025 to .054; CFI = .989; and SRMR = .036), but not for those aged 14 to 15 (RMSEA = .078, 90%CI = .065 to .091; CFI = .981; and SRMR = .067).  Although all items loaded significantly on the factor at each age group, the fourth item—"[my parents] and I solve a problem together whenever we disagree about something"—had consistently lower correlations with other items (Appendix Table A).  This item taps into the construct of problem-solving and so may have a different meaning than nurturing.  An examination of the parental nurturance questionnaires that were recently reviewed by Locke and Prinz,³³ confirmed that most nurturance scales did not include problem-solving items.  In addition, at ages 14 to 15, five out of six standardized residuals were above 4.0 for the fourth item (range of residuals = -7.29 to 3.90), suggesting a high degree of error in prediction.  Item 4 was removed from the scale, and a new set of CFAs were performed.

Table 1 Factor loadings (FL), item uniqueness (IU), and range of residuals (RES) for items in single-factor models in child-reported parenting questionnaire, by age group of respondents, household population aged 10 to 15, Canada excluding territories, 1998/1999, 2000/2001, 2002/2003 and 2004/2005

Table A Polychoric correlation coefficients of child-reported parental behaviour scale items at three time points, by type of scale and age group of respondents, household population aged 10 to 15, Canada excluding territories, 1998/1999, 2000/2001, 2002/2003 and 2004/2005

The revised 6-item Parental Nurturance model was a good fit to the data (RMSEA = .037, 90%CI = .018 to .055; RMSEA = .024, 90%CI = .001 to .045; RMSEA = .039, 90%CI = .022 to .058; CFI = .991; .997; .996; and SRMR = .029; .021; .020, at ages 10 to 11; 12 to 13, and 14 to 15, respectively) (Table 2).  All items loaded significantly on the factor, and the standardized residuals were greatly reduced for each age group.

Rejection

The 7-item NLSCY Parental Rejection model was not a good fit to the data across any of the three time points (RMSEA = .069, 90%CI = .055 to .082; RMSEA = .078, 90%CI = .063 to .090; RMSEA = .070, 90%CI = .057 to .084; CFI = .889; .895; .936; and SRMR = .064; .079; .067, at ages 10 to 11, 12 to 13, and 14 to 15, respectively).  Consequently, the model was not confirmed for this sample of adolescents.  An inspection of the factor loadings and item uniqueness failed to identify specific items that were negatively influencing fit.  The items generally had low loadings (although all items significantly loaded on the factor), high item uniqueness, and a wide range of residuals across three time points (Table 1). 

Monitoring

The 5-item NLSCY Parental Monitoring model was a good fit to the data across all three time points (RMSEA = .035, 90%CI = .008 to .060; RMSEA = .001, 90%CI = .001 to .043; RMSEA = .041, 90%CI = .018 to .066; CFI = .982; .999; .988; and SRMR = .025; .012; .027, at ages 10 to 11, 12 to 13, and 14 to 15, respectively).  All items loaded significantly on the factor.  However, the second item—"[my parents] let me go out any evening I want"—had very low factor loadings and high item uniqueness across all three time points (Table 1).  The ambiguous wording of the item lends itself to various interpretations.  For example, some adolescents might regard being able to go out any evening they want as a lack of parental care, but others might view it as being granted appropriate independence and an indication of trust.  This ambiguity was noted by Lempers et al. ,¹⁴ who showed that the item loaded weakly on the Parental Nurturance scale (λ< .30) rather than on the Parental Monitoring scale.  The item was removed from the NLSCY scale, and a new set of CFAs were run.

The revised 4-item Parental Monitoring model was a good fit to the data across all three time points (RMSEA = .060, 90%CI = .028 to .098; RMSEA = .033, 90%CI = .001 to .075; RMSEA = .000, 90%CI = .001 to .052; CFI = .978; .994; 1.00; and SRMR = .027; .018; .010, at ages 10 to 11, 12 to 13, and 14 to 15, respectively).  All items significantly loaded on the factor, and the problems with the parameter estimates were resolved (Table 2).

Table 2 Factor loadings (FL), item uniqueness (IU), and range of residuals (RES) for items in revised parental nurturance and monitoring models in child-reported parenting questionnaire, by age group of respondents, household population aged 10 to 15, Canada excluding territories, 1998/1999, 2000/2001, 2002/2003 and 2004/2005

Discussion

The aim of this study was to assess the construct validity of the child-reported parenting measures in the NLSCY.  The only model with a good fit (based on global goodness-of-fit statistics criteria) across all three time points was the Parental Monitoring scale.  Further, removal of one conceptually and empirically weak item improved the measurement properties of the scale.  Based on these findings, use of the revised scale is recommended. 

From the perspective of terminological precision, "parental monitoring" may not capture the essence of the construct.  Monitoring has been defined as a "prevention or intervention" technique used by parents.^34,35  However, most measures of monitoring assess parental knowledge, which originates mainly from the child's willingness to disclose rather than parents' efforts to supervise.^36-38  Therefore, it is recommended that  "parental knowledge" or "monitoring efforts" be used as a new label for the current Parental Monitoring scale.

The Parental Nurturance scale was initially confirmed only for times 1 and 2 (at ages 10 to 11 and 12 to 13), but not at time 3 (at ages 14 to 15).  Inclusion of the problem-solving item could not be conceptually justified.  When this item was removed, the model was confirmed for all time points.  Therefore, use of the revised scale is recommended. 

The factor structure of the Parental Rejection scale was not confirmed.  In addition to rejection, the items that constitute the scale encompass behaviours such as inconsistency and harshness.  In fact, the original name was the "inconsistent rejection-oriented discipline" scale.¹⁴  Sabatelli and Waldron³⁹ have suggested that although an EFA may provide support for the interrelationships among specific items in a scale, those items may not represent a theoretically coherent set of indicators for a particular construct.  This appears to be the case for the Parental Rejection scale.  Research is warranted to establish the defining features of the rejection construct, and to review related constructs (for instance, harsh parenting) to clarify the conceptual relationships between them.

Limitations

This study has a number of methodological limitations.  First, although the sample was relatively large, it may not be representative.  Many participants were excluded because of longitudinal attrition or non-completion of the parenting questionnaire.  In fact, analyses of missing data revealed higher socio-economic status (SES) for the final sample compared with the initial sample.  Therefore, the findings may not apply to a sample with low SES.  As well, owing to the construct-confirming goal of the study, missing data were not imputed.  A replication study would be useful to compare these findings with those obtained from a larger, more representative sample.

A second potential limitation is use of the same sample to confirm the NLSCY models and the revised models.  From a strict CFA point of view, removing one item from a model may require a new, independent dataset to confirm the revised model.⁴⁰  However, the revisions to the original models were minor and not completely exploratory; they were conceptually driven based on careful inspection of items.

Conclusion

Although the results of this analysis raise concerns about the construct conceptualization and item content of the three NLSCY child-reported parenting scales, the two revised models appear to be useful in assessing nurturance and monitoring for adolescents aged 10 to 15.  Research is needed to evaluate the predictive utility of these scales by examining their association with child health and development.

Acknowledgements

This research was supported through a doctoral fellowship awarded to Rübab G. Arım by the Social Sciences and Humanities Research Council (SSHRC) and a research grant provided by the Initiative on the New Economy (INE) Collaborative Research Initiatives program developed by SSHRC.