Skip to main content
Unlisted page
This page is unlisted. Search engines will not index it, and only users having a direct link can access it.

4. Questions, concepts, and classifications

Question: Can AI relate questions to concepts and standards?

Rationale​

Two surveys can both have a variable called educ with values 0 to 5 and mean different things: one follows ISCED 2011, the other a national scheme. A user comparing them needs the classification and its version. An AI system combining them needs the same, in a field it can read. Concepts and classifications are what make a variable comparable across surveys, years, and countries, and what let a system relate "highest education level" in a question to educ in a file.

The question text is the other half: it states exactly what was asked, in the words the respondent heard, which no label fully captures.

Target state​

  • Every collected variable carries the literal question text and, where relevant, the interviewer instruction.
  • Every variable that follows a classification names it, with the version and the level (ISCO-08 sub-major group).
  • A concept map links variables to concepts in a shared vocabulary, so that the same concept can be found across surveys.
  • Harmonized variable names are used for the concepts the organization measures repeatedly.

Maturity levels​

LevelSteps
FoundationalPublish the questionnaire with the data. Name the classifications used in the study record's method section.
AI-readyAttach the question text to each variable (var_qstn_qstnlit). Publish a concept map that links variables to classifications with versions and URIs, and store it in var_concept. Publish the crosswalk where a national classification is used.
AI-nativeMaintain a question bank and a concept scheme (SKOS or XKOS) shared across surveys, with harmonized names, so that a system can find every survey that measures a concept.
Running example

Four variables of EX-LFS-2025-Q2 follow international classifications: educ (ISCED 2011), lfs_status (the 19th ICLS resolution), occupation (ISCO-08), and industry (ISIC Rev.4). The concept map records each with its version, level, and URI, and gives each a harmonized name.

Recipes​

Recipe

4.1 Attach the question text to every collected variable

Level
Foundational
Skills
Data curator with the questionnaire
Indicative time
Two to four hours per survey

Result: each variable record states the question exactly as it was asked, so that a label can be checked against it and a user can see the wording that produced the data.

  1. Take the question text from the final questionnaire, including the question number and the placeholders as printed ("Q4. How old was [NAME] at last birthday?").
  2. Put it in the question column of the dictionary (recipe 3.1), which the converter maps to var_qstn_qstnlit. Add pre-question and post-question text and interviewer instructions where they change the meaning.
  3. For derived variables, put the derivation rule in the same column ("Derived from Q10 to Q16") and the full rule in var_derivation.
  4. Record the questionnaire version in the study record, and re-attach the text when the questionnaire changes.

Worked example. The variable hours has the label "Hours worked last week". Its question, taken from the questionnaire, reads "Q15. How many hours did [NAME] work last week in all jobs?". The question text tells a user two things the label does not: that the figure is the hours worked in the reference week, as distinct from usual hours, and that it covers all jobs. The pre-question text, "Ask of persons who worked at least one hour last week", is added as the interviewer instruction, because it is the skip rule that defines the universe. For the derived variable lfs_status, the question column reads "Derived from Q14 to Q18".

What this does not do. Question text in the dictionary does not make the questionnaire itself machine-readable (skip patterns, response options, routing). DDI Lifecycle and a question bank do that, and recipe 4.3 starts there.

Recipe

4.2 Map variables to concepts and classifications

Level
AI-ready
Skills
Subject-matter statistician; classification specialist
Indicative time
One day per survey the first time

Result: the survey has a concept map with one row per variable. Each row names the concept the variable measures, the classification it follows, the version, a web address for the concept, and a harmonized name, and the same information is stored in the variable records.

A concept is what a variable measures, stated independently of this survey: "educational attainment", "labour force status". A classification is the standard list of categories for it: ISCED for education, ISCO for occupation. The concept map is a CSV:

ColumnMeaning
nameThe variable name
conceptThe concept, in plain words
classification, versionThe standard the codes follow, and its version
uriA web address that identifies the concept or classification
harmonized_nameThe name used for this concept across all the organization's surveys (educ_isced11)
noteHow the variable's codes relate to the classification's levels
name,concept,classification,version,uri,harmonized_name,note
educ,Educational attainment,ISCED,2011,https://uis.unesco.org/en/topic/international-standard-classification-education-isced,educ_isced11,Levels 0 to 5 map to ISCED 2011 levels 0 to 5; level 6 and above collapsed into 5
lfs_status,Labour force status,ICLS resolution on work statistics,19th ICLS (2013),https://www.ilo.org/resource/19th-icls-resolution-i,lfs_status_icls19,Employed; unemployed; outside the labour force
occupation,Occupation,ISCO,ISCO-08,https://www.ilo.org/public/english/bureau/stat/isco/isco08/,occupation_isco08_2d,Sub-major group (2 digits)
industry,Economic activity,ISIC,Rev.4,https://unstats.un.org/unsd/classifications/Econ/isic,industry_isic4_2d,Division (2 digits)
sex,Sex,,,,sex,1 male; 2 female
age,Age,,,,age_years,Completed years
region,Reference area,National administrative regions,2024 gazette,https://stats.example/codes/regions,region_code,Codes R01 and R02 as published in the regions code list
  1. For each variable, name the concept and the classification it follows, with version and level. Where a national classification is used, link the crosswalk to the international one (chapter 3 of the dissemination cookbook gives the crosswalk format and XKOS).
  2. Store the concept in the variable record (var_concept, with its title, vocabulary, and URI) through the concept column of the dictionary, and publish the map with the data.
  3. Check that the value codes in the dictionary are consistent with the named classification (ISCED levels 0 to 5, ISCO-08 two-digit codes).

Worked example. The concept map of the running example has four rows. The row for educ names the concept "educational attainment", the classification ISCED, the version 2011, the URI of the ISCED 2011 scheme, and the harmonized name educ_isced11. Its note reads "codes 0 to 5 are ISCED levels 0 to 5; level 6 and above are merged into 5". The row for occupation names ISCO-08 at the sub-major group level, which is two digits, and the check confirms that every code in the dictionary is a valid two-digit ISCO-08 code. The row for lfs_status names the 19th ICLS resolution as its standard, because labour force status is defined by a resolution and has no code list of its own.

What this does not do. A concept map names the classification. It does not verify that the coding followed it. A sample check of coded values against the classification's definitions, or AI-assisted coding review (the program's classification and coding workstream), does that.

Recipe

4.3 Reuse questions and concepts across surveys

Level
AI-native
Skills
Methodologist; data curator
Indicative time
Ongoing; one week to set up

Result: each of the organization's recurring concepts has one definition, one harmonized name, and one question wording, so that a system can find every survey that measures a concept and compare them.

  1. List the concepts measured in more than one survey (labour force status, educational attainment, household composition) and give each a harmonized variable name (educ_isced11, lfs_status_icls19).
  2. Publish the concept definitions as a concept scheme (SKOS or XKOS, the web standards for such schemes) with labels in each language, and reference it from the variable records' concept URI.
  3. Keep the question wordings in a question bank, with the concept each measures and the surveys that use each wording. The program's Global Question Bank workstream develops this for development surveys.
  4. When a new survey is designed, draw questions from the bank, so that the concept links exist before the data do.

Worked example. The example organization measures educational attainment in the labour force survey, the household budget survey, and the census. The three questionnaires word the question differently, and two of the three use the national scheme. The methodologist defines the concept once, as "highest level of education completed, ISCED 2011", gives it the harmonized name educ_isced11, and publishes it in the organization's concept scheme with labels in English and French. Each survey's educ variable now points to that concept, and a search for "educational attainment" returns all three surveys, with the crosswalk from the national scheme attached to the two that need it.

What this does not do. Harmonized names and shared definitions do not make data from two surveys comparable when the samples, modes, or reference periods differ. The quality statements of chapter 7 carry that information.

Implementation options​

  • Classifications: ISCED, ISCO, ISIC, COICOP, and the ICLS resolutions on work statistics, each with a version.
  • Concept schemes: SKOS for concepts with multilingual labels; XKOS for statistical classifications and the correspondences between them.
  • DDI Lifecycle models questions, concepts, and variables as reusable objects across studies; DDI Codebook holds the links per study.
  • Question banks: a shared repository of question wordings keyed by concept; the Global Question Bank workstream is the program's contribution.

World Bank examples​

  • The microdata schema carries var_concept (title, vocabulary, URI) and var_std_catgry (standard categories) per variable.
  • The program's Global Question Bank and ontologies and knowledge graphs workstreams develop reusable questions and concept links (program overview).
  • The AI-ready data framework names semantic knowledge, including concepts and classifications, among the attributes of AI-ready data.

Resources​

Common mistakes​

  • A classification without a version. ISCED 1997 and ISCED 2011 differ; a system that does not know which was used will combine them.
  • Labels in place of question text. The label is a summary; the question is the evidence of what was measured.
  • Concepts only in the report. A concept that is not in the variable record is not findable by concept.
  • Harmonization by renaming. Giving two variables the same name without checking the definitions makes them look comparable when they are not.

Verification​

  • Question text. Every collected variable has its question text, and a sample compared with the questionnaire matches word for word.
  • Concept map. Every classified variable has a classification, a version, and a URI that resolves.
  • Code consistency. The value codes of each classified variable are valid codes of the named classification at the stated level.
  • Cross-survey search. A query for a concept returns every survey that measures it (chapter 5).

Checklist​

0 of 8 done

Ticks are saved in this browser only.