5. Variation in PISA scores as a function of student and school characteristics

View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

Archived

This page has been archived on the Web.

PISA literature has demonstrated that there can be a strong interplay between student and school characteristics and results on the PISA reading assessment test. This section represents an initial examination of how PISA reading scores can vary as a function of selected student and school level characteristics as illuminated through the 2009 PISA data. Two types of data are reviewed in the following analyses: categorical characteristic data and indexed data. For an explanation of how this analysis was conducted see the accompanying text boxes for each section.

Selected categorical characteristics and PISA reading scores

Research literature has consistently shown that gender is an important explanatory variable in understanding variation in reading performance. Girls typically outperform boys on reading assessments. Although there were no significant differences between the distribution of males and females when comparing minority and majority-language students it was important to see whether the gender difference in reading performance seen in the general population was also true for the minority-language population.

In all provinces except Nova Scotia, reading performance was significantly different by gender, with female minority students outperforming male minority students by about a half a reading proficiency level or more. The same was true for the students in the majority-language school system. With both populations experiencing the same phenomenon, the differences in performance by gender do not help explain the overall differences between minority and majority students in PISA reading performance (Table 28.1).

It was expected that having a home language different from the language of instruction at school would explain some of the differences between the minority and majority populations on PISA results. At the Canada level, the average PISA score for minority students having a home language the same as the language of instruction at school was significantly higher than the average score for minority students having a different home language than the language of instruction – by about a half a reading proficiency level. The difference at the Canada level was driven primarily by a significant difference between the two minority sub-populations in Quebec. In all other provinces the difference between same and different home languages for minority-language students was not significant. The same scenario was evident for the majority-language students. Significant differences were found at the Canada level, and in Quebec, with students having the same language at home bettering those with a different language at home by half a reading proficiency level. For the majority-language population of students, Manitoba also had a significantly higher average score for students whose home language was the same as the language used at school. Given that minority students whose home language was different from school language tended to be using the other official language at home, while majority students with a different home language tended to be using an allophone language at home, these results are inconclusive when explaining differences between overall differences in PISA scores for minority and majority-language student populations (Table 28.1).

Table 28.1 Differences in PISA combined reading scores by selected student characteristics, minority and majority language school system students, Canada and selected provinces, 2009

Having grades of 70.0% or more in the language arts was associated with a significant difference on PISA reading performance. At both the Canada level and for all provinces covered in this report, students from the minority-language system who reported grades of 70.0% or more in the language arts had significantly better average PISA reading scores – typically by more than a proficiency level – when compared to those who had grades of less than 70.0%. The same results were found for the majority-language students at all geographies, with statistically significant differences evident between those who were doing well in the language arts and those who were not performing as well in that subject area (Table 28.2).

Post-secondary expectations were also associated with significant differences in average scores on the PISA reading assessment. Minority-language system students who expected to be attending some form of post-secondary education outperformed minority students on the PISA reading assessment relative to those who did not expect to attend post-secondary. The difference between these two groups was, again, about a half a reading proficiency level in PISA scores. Within the minority school student population, the differences were significant at the Canada level and for New Brunswick, Quebec, Ontario and Manitoba. In the majority student population the same phenomenon was evident; however, the differences in average PISA scores for those who expected to attend post-secondary and those who did not were significant for all geographies (Table 28.2).

Table 28.2 Differences in PISA combined reading scores by selected student characteristics, minority and majority language school system students, Canada and selected provinces, 2009

Differences in school characteristics were much less consistent in terms of their relationship to PISA reading scores, with one notable exception. When comparing differences in PISA reading scores by differences in community size, between communities with a population of 15,000 or less and those with populations of more than 15,000, significantly different reading outcomes were found for the Canada, New Brunswick, Quebec and Ontario geographies. In all instances, minority students from the larger communities performed better on PISA than did the minority students from smaller communities. Overall, minority-language students from larger communities had PISA scores that were, on average, 43 points higher than minority-language students from smaller communities. In Nova Scotia and Manitoba, minority students from larger communities also performed better by about a half a proficiency level although these differences were not significant. The overall impact of community size was much smaller for majority-language school systems. At the Canada level the overall difference in PISA scores as a function of community size was 20 points higher on average for the students from larger communities. Alberta was the only province where the higher PISA performance of the majority-language students from large communities was significantly different from that of students from smaller communities (Table 29.1).

Teacher shortages in the language arts coincided with a significant difference of about 33 points on PISA scores at the Canada level for schools attended by minority-language students. Schools experiencing no or very little shortage outperformed those who were experiencing shortages in this area. Provincial differences were non-significant across the board. In Nova Scotia and Quebec although the differences were not significant, minority students from schools experiencing no teacher shortages did better on the PISA reading assessment than those experiencing shortages - by almost half a proficiency level (29 and 32 points respectively). For majority-language students and schools, teacher shortages in the language arts were not related to large differences in PISA scores except in Ontario. In that province, majority-language students from schools not experiencing shortages of teachers in the language arts had PISA scores significantly higher, by about 41 points on average, than students from schools who did have these types of shortages (Table 29.1).

Table 29.1 Differences in PISA combined reading scores by selected school characteristics, minority and majority language school systems, Canada and selected provinces, 2009

The data on differences in PISA scores as a function of shortages in instructional material was extremely varied across the provinces. In Nova Scotia, minority students in schools with shortages of instructional material did more poorly than those with no shortages by about 45 points on the PISA scale -the only province to have a significant difference on this item in the minority student group. Differences on this item were small for the majority-language school systems in all provinces except Alberta. In that province shortages of instructional material were associated with significantly lower PISA reading scores (by about 39 points) relative to the average scores for students attending schools without this type of shortage (Table 29.2).

It was expected that large student populations whose first language was not the same as the test language (or language of instruction at school) would help explain some of the difference between minority and majority students on PISA. Results from the comparison of PISA scores for minority students in schools where more than 20% and less than 20% of the student body had a first language other than the test language were not significantly different however, for all provinces except Quebec. In schools attended by minority-language students in Quebec where more than 20% of the student body had a first language other than the PISA test language the average PISA score was about 29 points lower than for students from schools where less than 20% of students had another first language. For the majority-language students, it was found that the PISA reading scores were actually significantly higher in New Brunswick and Alberta when more than 20% of the student body had a first language other than the language used for the PISA test (Table 29.2).

Table 29.2 Differences in PISA combined reading scores by selected school characteristics, minority and majority language school systems, Canada and selected provinces, 2009

Selected indexed characteristics and PISA reading scores

As with the categorical analysis, the data from the effect of changes in scores on the selected indices was varied across indices and across geographies. The analysis on the effect of a 1 standard deviation (1 SD) increase in score on the index of social, cultural and economic status (SCES) was associated with an improvement of between 20 and 36 points in PISA scores. In all provinces and at the Canada level this effect was non-significant when comparing the impact of the change for minority-language students relative to the impact of a similar change for majority-language students within a given province.

Changes in the scores on the index of enjoyment of reading had a greater impact on PISA scores than did a change on the index of social, cultural and economic status (SCES). Students with enjoyment of reading scores 1 SD above the mean scored between 30 and 40 points higher on the PISA reading assessments. Although the overall effect was larger on this index than for the SCES index, there were no-significant differences between minority and majority student populations on the within province comparisons of changes on the enjoyment of reading index. Minority and majority populations were noticeably similar on the effect of an improvement in score on this index when doing the within province comparison. The largest difference in the effect of a 1 SD change on the index of enjoyment of reading when comparing minority and majority-language student populations was less than 5 points in PISA scores (Table 30).

A 1 SD change in scores on the index of quality of educational resources was associated with a negligible change in PISA scores for Canada – overall approximately 5 points on the PISA scale for both minority and majority-language student populations. Differences were non-significant for all geographies except Nova Scotia. The effect of the change in index score for minority students in that province was 25 points on the PISA scale, while for majority students in the same province, an increase in the index was associated with a negligible drop in PISA scores (3 points) (Table 30).

Table 30 Impact of a change in the indexed score on PISA reading scores, for students in minority and majority language school systems, Canada and selected provinces, 2009