Chapter 4
Measures of literacy and numeracy in 2003 and 2012

View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

Archived

This page has been archived on the Web.

PIAAC constitutes the latest in a series of international studies on adult skills that date back to the early 1990s. The most recent of these was undertaken in 2003: the Adult Literacy and Life Skills Survey (ALL).^{Note 1} In total, six countries participating in PIAAC also participated in ALL, including Canada.

From ALL to PIAAC: context, constructs and instruments

To permit comparisons of country measures in literacy and numeracy over time, efforts were made to employ assessment measures in PIAAC that correspond with those used in ALL. This involved a complex undertaking, both conceptually and technically, for the simple reason that the world of today is very different from that of 2003.

In 2003, social media was still in its infancy. The term “apps” meant very little to the general public, and smartphones were still years from widespread adoption. By the time OECD began preparing PIAAC, it was clear that any meaningful measure of skills would have to acknowledge a very different reality from that examined at the time of ALL.

PIAAC reflects this new reality in a number of ways. First, it broadened the concept of literacy. Given the growing importance of digital devices and applications as a means of generating, accessing and storing written text, the reading of digital texts became an integral part of literacy measured in PIAAC.

Second, where ALL reported literacy as two separate domains on two separate scales, covering prose literacy and document literacy, PIAAC reports literacy as a single domain reported on a single literacy scale that covers the reading of not only prose texts and document texts, but also of digital (such as websites, results pages from search engines and blog posts) and mixed format texts (i.e. texts containing both continuous and non-continuous elements).

Third, PIAAC employed adaptive testing in the computer-based assessment, which ALL was not able to do since it was entirely paper-based. Adaptive testing adjusts the difficulty of questions as the respondent takes the test.  In PIAAC, respondents were directed to different blocks of items based on their estimated proficiency.^{Note 2} This kind of testing leads to a more fine-grained distinction in scores.

Fourth, PIAAC gathered significantly more data for constructing the numeracy scale than was collected for ALL, although it uses essentially the same concept of numeracy.

These changes prohibit a direct and immediate comparison of results in PIAAC with those previously reported for ALL. For numeracy, results from 2003 have been re-estimated to refine the measure and to ensure better comparability with the results from PIAAC. For literacy, in PIAAC, a new scale was constructed to merge and incorporate the two measures of ALL into a single literacy measure to ensure better comparability with PIAAC. As a result, readers are cautioned against retrieving the results previously reported for ALL and comparing them with those in PIAAC: the scales used in 2003 and 2012 are not the same, and comparisons are therefore invalid.

Comparisons over time are only possible once the ALL results for prose literacy, document literacy and numeracy have been re-estimated and re-scaled. Results that follow provide a preliminary look at literacy and numeracy in Canada in 2003^{Note 3} and 2012. The information provided is limited and additional analysis is needed to better understand skill differences over time.

Canada’s skills distribution in 2003 and 2012

Charts 4.1a and 4.1b present the skills distribution in Canada for ALL in 2003 and PIAAC in 2012. For literacy, Canadians scored an average of 280 in 2003, with 14% at Level 1 or below, and 18% at Level 4 or 5. The average in 2012 was 273, with 17% at Level 1 or below, and 14% at Level 4 or 5.

For numeracy, the pattern is very similar. Canadians scored an average of 272 in 2003, with 18% at Level 1 or below, and 14% at Level 4 or 5. The average in 2012 was 266, with 23% at Level 1 or below, and 13% at Level 4 or 5.

 

More in-depth study is necessary to reach an understanding of what the distribution of skills described above reveals about skills and changes in skills over time.^{Note 4} Some potential areas under consideration for future analysis are outlined below.

• The compositional shifts of the Canadian population - Age, educational attainment, and labour force participation can all affect these results, as can language abilities, and levels of immigration. All or some of these factors may have changed considerably in the time between the two surveys, and the impact of theses changes is yet unclear.
• Use of skills in the information society - It should be remembered in this regard that skills on their own do not account for proficiency; how, when, and how often these skills are used — in the workplace, at home, and in the public domain — are also contributing factors. The acceleration of computer use in society, even within such a comparatively short period as 10 years, may have an impact on the use and maintenance of literacy and numeracy skills. This in turn could manifest itself in proficiency scores.

In light of the foregoing discussion, it is recognized that these results serve as the foundation for further analysis of literacy and numeracy both internationally and nationally.

---

Notes

1. Conducted in 2003, the International Adult Literacy and Skills Survey (IALSS) is the Canadian component of the Adult Literacy and Life Skills program (ALL). Throughout this chapter we will refer to the survey as ALL.
2. The OECD (2013c, Chapter 3) presents more information on the adaptive testing in PIAAC in their Reader’s Companion.
3. These results use the re-scaled literacy and numeracy score
4. It was found at the international level that the variation in country results, as well as the magnitude of those changes, signalled the need for further analysis (OECD 2013b). OECD will study the issue in a separate publication in order to obtain a better understanding of what the variations tell us. Canada is following suit with OECD, recognizing that deeper analysis is required both internationally and nationally before drawing conclusions about the differences in scores in literacy and numeracy over the past decade.