SDMX Metadata Concepts

Download PDF

Goal 4: Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all

Target 4.1: By 2030, ensure that all girls and boys complete free, equitable and quality primary and secondary education leading to relevant and effective learning outcomes

Indicator 4.1.1: Proportion of children and young people: (a) in grades 2/3; (b) at the end of primary; and (c) at the end of lower secondary achieving at least a minimum proficiency level in (i) reading and (ii) mathematics, by sex

Institutional information

Organization(s):

UNESCO Institute of Statistics (UIS)

Concepts and definitions

Definition:

Percentage of children and young people achieving at least a minimum proficiency level in (i) reading and (ii) mathematics during primary education (Grade 2 or 3), at the end of primary education, and at the end of lower secondary education. The minimum proficiency level will be measured relative to new common reading and mathematics scales currently in development.

Concepts:

Minimum proficiency level (MPL) is the benchmark of basic knowledge in a domain (mathematics, reading, etc.) measured through learning assessments. In September 2018, an agreement was reached on a verbal definition of the global minimum proficiency level of reference for each of the areas and domains of Indicator 4.1.1 as described in the Minimum Proficiency Levels (MPLs): Outcomes of the consensus building meeting.

Minimum proficiency levels defined by each learning assessment

To ensure comparability across learning assessments, a verbal definition of MPL for each domain and levels between cross-national assessments (CNAs) was established by conducting an analysis of the performance level descriptors (PLDs)[1] of cross-national, regional, and community-led tests in reading and mathematics. The analysis was led and completed by the UIS and a consensus among experts on the proposed methodology was deemed adequate and pragmatic.

The global MPL definitions for the domains of reading and mathematics are presented in Table 1.

Table 1. Minimum proficiency levels defined by each learning assessment

Reading

Educational Level

Descriptor

Grade 2

They read and comprehend most of written words, particularly familiar ones, and extract explicit information from sentences.

Grade 3

Students read aloud written words accurately and fluently. They understand the overall meaning of sentences and short texts. Students identify the texts’ topic

Grades 4 & 6

Students interpret and give some explanations about the main and secondary ideas in different types of texts. They establish connections between main ideas on a text and their personal experiences as well as general knowledge.

Grades 8 & 9

Students establish connections between main ideas on different text types and the author’s intentions. They reflect and draw conclusions based on the text.

Mathematics

Educational Level

Descriptor

Grades 2-3

Students demonstrate skills in number sense and computation, shape recognition and spatial orientation.

Grades 4-6

Students demonstrate skills in number sense and computation, basic measurement, reading, interpreting, and constructing graphs, spatial orientation, and number patterns.

Grades 8 & 9

Students demonstrate skills in computation, application problems, matching tables and graphs, and making use of algebraic representations.

1

PLD: Performance level descriptors are descriptions of the performance levels to express the knowledge and skills required to achieve each performance level, by domain. ↑

Rationale:

The indicator aims to measure the percentage of children and young people who have achieved the minimum learning outcomes in reading and mathematics during or at the end of the relevant stages of education.

The higher the figure the higher the proportion of children and/or young people reaching at least minimum proficiency in the respective domain (reading or mathematic) with the limitations indicated under the “Comments and limitations” section.

Comments and limitations:

Learning outcomes from cross-national learning assessment are directly comparable for all countries which participated in the same cross-national learning assessments. However, these outcomes are not comparable across different cross-national learning assessments or with national learning assessments. A level of comparability of learning outcomes across assessments could be achieved by using different methodologies, each with varying standard errors. The period of 2020-2021 will shed light on the standard errors’ size for these methodologies.

The comparability of learning outcomes over time has additional complications, which require, ideally, to design and implement a set of comparable items as anchors in advance. Methodological developments are underway to address comparability of assessments outcomes over time.

Methodology

Computation method:

The number of children and/or young people at the relevant stage of education n in year t achieving or exceeding the pre-defined proficiency level in subject s expressed as a percentage of the number of children and/or young people at stage of education n, in year t, in any proficiency level in subject s.

MPLt,n,s, = MPt,n,s / Pt,n

where:

MPt,n,s = the number of children and young people at stage of education n, in year t, who have achieved or exceeded the minimum proficiency level in subject s.

Pt,n = the number of children and young people at stage of education n, in year t, in any proficiency level in subject s.

n = the stage of education that was assessed

s = the subject that was assessed (reading or mathematics).

Harmonize various data sources

To address the challenges posed by the limited capacity of some countries to implement cross-national, regional, and national assessments, actions have been taken by the UIS and its partners. The strategies are used according to its level of precision and following a reporting protocol that includes the national assessments under specific circumstances.

Out-of-school children

In 2016, 263 million children, adolescents and youth were out of school, representing nearly one-fifth of the global population of this age group. 63 million, or 24% of the total, are children of primary school age (typically 6 to 11 years old); 61 million, or 23% of the total, are adolescents of lower secondary school age (typically 12 to 14 years old); and 139 million, or 53% of the total, are youth of upper secondary school age (about 15 to 17 years old). Not all these kids will be permanently outside school, some will re-join the educational system and, eventually, complete late, while some of them will enter late. The quantity varies per country and region and demands some adjustment in the estimate of Indicator 4.1.1. There is currently a discussion on how to implement these adjustments to reflect all the population. In 2017, the UIS proposed to make adjustments using the out-of-school children (OOSC)[2] and the completion rates.

2

UIS (2017a). More than one-half of children and adolescents are not learning worldwide. Montreal and UIS (2017b). Counting the number of children not learning: Methodology for a global composite indicator for education. Montreal. ↑

Treatment of missing values:

  • At country level:

Missing values are not imputed.

  • At regional and global levels:

Missing values are not imputed.

Regional aggregates:

Not yet applicable. Data are reported at the national level only. Population weighted average by region to be reported in 2020.

Sources of discrepancies:

Not yet applicable. Data are reported at the national level only.

Methods and guidance available to countries for the compilation of the data at the national level:

Information not available.

Quality assurance:

Information not available.

Data sources

Description:

Type of data sources: In school and population-based learning assessments.

Table 2. How interim reporting is structured?

 

In-school based

Household Based Surveys

Grade

Cross-national

National

Grade 2

or 3

LLECE

Yes

MICS6

2/3 plus one year when primary lasts more than 4 years according to ISCED level of the country

PASEC

EGRA

TIMSS

EGMA

PIRLS

PAL network 

End of primary

LLECE

Yes

PAL network

plus or minus one year of last year of primary according to ISCED level of the country

PASEC

TIMSS

PIRLS

PILNA

SEAMEO

SACMEQ

End of lower secondary

PISA

Yes

Young Lives

plus two or minus one of last year of lower secondary according to ISCED level of the country

PISA-D

TIMSS

Definition of minimum level until 2018 release

Those defined by each assessment by point of measurement and domain

Definition of minimum level from 2019

According to alignment as adopted by Global Alliance to Monitoring Learning (GAML) and Technical Cooperation Group (TCG)

Grade for end of primary and end of lower secondary

As defined by the ISCED levels in each country

Validation

Sent from UIS for countries’ approval

Collection process:

Information not available.

Data availability

Time series:

Data available since 2000. The indicator will be reported annually.

Disaggregation:

Indicator is published disaggregated by sex. Other disaggregation such as location, socio-economic status, immigrant status, ethnicity and language of the test at home are based on data produced by international organizations administering cross learning assessment detailed in the expanded metadata document and validated by countries. Parity indexes are estimated in the reporting of Indicator 4.5.1. Information on the disaggregation of variable for Indicator 4.1.1 are presented in the following tables.

Calendar

Data collection:

Data collection is ongoing.

Data release:

February 2020

Data providers

School Based assessments

  • International Large Scale Assessments are reported to the UIS by cross-national organisations (LLECE, PASEC, TIMSS, and PIRLS). Typically, Cross National Large Scale Assessment, either regional or international, define various performance levels, and report as well the mean and standard deviation. They choose as well one level as the cut-off point that defines what children/youth are below or above level.
  • Regional assessments: PASEC, SACMEQ, ERCE, PILNA, SEAMEO.
  • National Large-Scale Assessments either sample- or census- based. Countries should report the proportion of students by level of competency for each domain indicating as well the minimum proficiency level, when it is defined by the national assessment. EGRA and EGMA as reported by USAID or individual countries.

Household-Based survey

  • MICS6: reported to the UIS by UNICEF
  • Pal Network: reported to the UIS by Pal Network

Data compilers

UNESCO Institute of Statistics (UIS)