4. Dataset cards

Question: How is a dataset documented for the people who train models on it?

Rationale

A person who trains a model on a dataset needs to know things the dictionary and the loading record do not say. Why does the dataset exist? How were the examples collected and labelled, and how reliable are the labels? What was removed or changed? Whom do the examples represent, and whom do they leave out? What is the dataset for, what must it never be used for, under which terms, and how is it cited? A dataset card is the one document that answers these questions, in a fixed order, on the dataset's page. The idea comes from the datasheets proposal of Gebru and others (2018) and from the data cards of Pushkarna and others (2022); dataset repositories made the card the first thing a user sees.

For a statistical organization the card has two extra duties. It states that the dataset is a collection of examples and not a source of statistics, so that nobody computes a rate from it. And it carries the representativeness report of chapter 5, so that a model builder knows which groups the examples cover thinly. A card is written by the people who made the dataset and checked for completeness by a script, so that no release goes out with a section missing.

Target state

• Every dataset has a card with the twelve sections of this chapter, written from the decision record, the collection notes, and the measurements.
• The card opens with a machine-readable header (licence, language, task, size) that repositories and catalogs read.
• The card is checked for completeness on every release, and reviewed by a person for accuracy.

Maturity levels

Level	Steps
Foundational	Write the card from the template for one dataset; run the check.
AI-ready	Write the representativeness and known-gaps section from the report of chapter 5; publish the card on the dataset page and in the repository; review it with a subject-matter specialist.
AI-native	Generate the measured sections of the card at each release from the scripts, and keep the written sections under version control with the data.

Running example

The card of the occupation-coding dataset says where the job descriptions come from, how they were coded and with what agreement between coders, what was removed, how the records differ from the population, what the dataset is for, what it is not for, its licence, its version, and how to cite it.

Recipes

Recipe

4.1 Write the card from the template

Level

Foundational

Skills

Dataset owner; methodologist

Indicative time

Half a day per dataset

Result: the dataset has a card with a machine-readable header and twelve sections, which is its documentation for users.

The card is a Markdown file. It opens with a YAML header (a short block of key and value pairs between two lines of three dashes) that repositories read for filtering: the licence identifier, the languages, the task, the size, and tags. The twelve sections follow. The card of the running example:

---
license: cc-by-4.0
language:
  - en
pretty_name: "LFS occupation coding (example)"
task_categories:
  - text-classification
size_categories:
  - n<1K
tags:
  - tabular
  - official-statistics
  - occupation-coding
---

# Dataset card: LFS occupation coding (example)

## Dataset summary

Six hundred job descriptions from the example organization's labour
force survey, each with the ISCO-08 occupation code assigned by a
trained coder and the respondent's region, urban or rural residence,
sex, age group, education level, and survey weight. The task the
dataset supports is automatic occupation coding: predicting the
four-digit code from the job and industry descriptions. All records are
invented for this guide.

## Motivation

Occupation coding is the most time-consuming step of processing the
labour force survey. A model that proposes a code with a confidence
lets coders concentrate on the hard cases. A shared, documented dataset
lets the organization and its partners train and compare such models on
the same examples.

## Composition

600 records, one per employed person, from 344 households. Columns:
record_id, hhid, region, urban, sex, age_group, educ, job_title,
industry_text, isco_code (the label), isco_major, weight. The
dictionary file describes each column. Records are split into training
(69%), validation (11%), and test (20%) by household, so that no
household appears in two splits.

## Collection process

Job and industry descriptions were recorded by interviewers in face-to-
face interviews during the second quarter of 2025, as the respondent
gave them, in English. Codes were assigned by the organization's coding
unit with the ISCO-08 index; a second coder re-coded a ten percent
sample with 94 percent agreement at four digits.

## Preprocessing

Descriptions were trimmed of leading and trailing spaces and nothing
else; spelling was not corrected, because a coder sees the raw text.
Records with no job description (3 of the original extract) were
removed. Direct identifiers were removed; the household identifier was
replaced by a sequence number.

## Representativeness and known gaps

The dataset is not a sample of the employed population. Compared with
the 2024 population census (persons aged 15 to 64): the Northern Region
holds 14.3 percent of records against 27 percent of the population,
urban residents 66.2 percent against 41 percent, and men 63.5 percent
against 49 percent. Persons aged 15 to 24 are 21.0 percent of records
against 30 percent of the population.
The survey weight (weight) corrects for the sampling design, not for
these differences in the extract. Occupations in agriculture and
fishing are under-represented relative to their share of employment.
The 600 records cover 44 of the 436 ISCO-08 unit groups, none with 30
records or more; a model trained on this file alone will code most
occupations poorly, and the file serves as a worked example of the
format.

## Intended uses

- Training and evaluating automatic occupation coders for the
  organization's own surveys.
- Benchmarking coding models across organizations that use ISCO-08.
- Teaching and demonstrations of text classification on official
  statistics.

## Out-of-scope uses

- Estimating the occupational structure of the population. The extract
  is not representative and carries no design information beyond the
  weight.
- Identifying respondents. No direct identifier remains, and the terms
  of use prohibit re-identification attempts.
- Training models for other countries' classifications without
  re-coding; the codes follow ISCO-08 and the national index of the
  example organization.

## Licence and terms

Creative Commons Attribution 4.0 (CC BY 4.0). Use for training and
evaluating models is permitted; attribution is required in any
publication, model card, or product that used the data, in the form
given under Citation. The organization asks that models trained on the
data cite it in their model card.

## Versions

Version 1.0.0 (October 2026): first release. The test split is frozen;
a new version with a changed test split would change the version's
first number.

## Citation

National Statistics Office (example). 2026. LFS occupation coding
(example), version 1.0.0. https://stats.example/datasets/lfs-occupation-ml

## Contact

data@stats.example

The sections and what each answers:

Section	What it answers
Dataset summary	What the dataset is, in one paragraph, including the task it supports
Motivation	Why it was made and for whom
Composition	How many records, which columns, which splits
Collection process	Where the examples come from, how the labels were assigned, and how reliable they are
Preprocessing	What was removed, changed, or anonymized
Representativeness and known gaps	Whom the examples represent, compared with the population, and which groups are thin (chapter 5)
Intended uses	The uses the dataset was made for
Out-of-scope uses	The uses it must not be put to, including estimation and re-identification
Licence and terms	The licence, the attribution form, and the terms for training (chapter 7)
Versions	The current version, what changed, and what is frozen (chapter 8)
Citation	The citation text with a DOI or URL
Contact	Whom to ask

1. Fill the header: the licence identifier (cc-by-4.0), the languages of the text fields (ISO 639-1 codes), the task (text-classification for coding), the size category, and tags.
2. Write the summary, motivation, and intended and out-of-scope uses from the decision record of chapter 1.
3. Write the collection process with the facts a model builder needs: who wrote the inputs, who assigned the labels, with which instrument, and how well two labellers agree. The agreement between coders is the ceiling of what a model can reach.
4. Write the preprocessing section as a list of what was done to the source records and nothing else. "Spelling was not corrected" is as useful as what was.
5. Leave the representativeness section for chapter 5 and the licence section for chapter 7, and fill them when those chapters are done.

Worked example. The collection process section of the running example's card reads, in part: "Job descriptions were written by interviewers from respondents' answers to the question on the kind of work done in the main job in the reference week. Codes were assigned by six trained coders using the ISCO-08 index, with a second coder on a tenth of the records. The two coders agreed on 91 percent of four-digit codes and 97 percent of major groups." The preprocessing section lists four things. Employer names were replaced by a generic term. Place names below the region were removed. Spelling was not corrected. Records coded 9999 were kept. The out-of-scope section opens with "Do not estimate the share of any occupation or group from this dataset; it is a collection of examples and is not a sample of the population."

What this does not do. A card documents. It does not make the collection process sound or the labels reliable, and a card that overstates either misleads every user. The subject-matter review of recipe 4.3 is where accuracy is checked.

Recipe

4.2 Check the card before release

Level

Foundational

Skills

Python

Indicative time

Minutes

Result: a check runs on every release and confirms that the card has its header fields and its twelve sections. It also checks four statements: that the out-of-scope section lists a use, that the representativeness section states a number, that the citation has a DOI or a URL, and that the versions section names a version.

The script reads the card and reports each check:

"""Check a dataset card for the sections and statements a release needs.

Reads a dataset card (Markdown with a YAML front matter) and checks: the
front matter names a licence, a language, a task category, and a pretty
name; every required section is present; the out-of-scope section lists
at least one use; the representativeness section contains at least one
number; the citation section contains a DOI or a URL; and the versions
section names the current version. Exit status 1 when anything is
missing. Uses PyYAML for the header.

Usage:
    python check_dataset_card.py dataset_card.md

What this does not do: it checks presence. Whether the stated uses are
sensible, the known gaps complete, and the collection process described
truthfully are the authors' responsibility, reviewed by a person.
"""

from __future__ import annotations

import argparse
import re
import sys
from pathlib import Path

import yaml

FRONT_MATTER = ["license", "language", "task_categories", "pretty_name"]
REQUIRED = [
    "Dataset summary",
    "Motivation",
    "Composition",
    "Collection process",
    "Preprocessing",
    "Representativeness and known gaps",
    "Intended uses",
    "Out-of-scope uses",
    "Licence and terms",
    "Versions",
    "Citation",
    "Contact",
]


def split_card(text: str) -> tuple[dict, str]:
    """The YAML header as a dict and the Markdown body."""
    m = re.match(r"^---\n(.*?)\n---\n(.*)$", text, re.DOTALL)
    if not m:
        return {}, text
    return (yaml.safe_load(m.group(1)) or {}), m.group(2)


def section(body: str, name: str) -> str:
    m = re.search(
        rf"^##\s+{re.escape(name)}\s*$(.*?)(?=^##\s|\Z)",
        body,
        re.MULTILINE | re.DOTALL | re.IGNORECASE,
    )
    return m.group(1) if m else ""


def main(argv: list[str] | None = None) -> int:
    parser = argparse.ArgumentParser(
        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
    )
    parser.add_argument("card")
    args = parser.parse_args(argv)
    meta, body = split_card(Path(args.card).read_text(encoding="utf-8"))

    problems: list[str] = []
    if not meta:
        problems.append("no YAML front matter")
    for k in FRONT_MATTER:
        ok = bool(meta.get(k))
        print(f"  {'ok' if ok else 'MISSING':<8} front matter: {k}")
        if not ok:
            problems.append(f"front matter: {k}")
    for name in REQUIRED:
        ok = bool(section(body, name).strip())
        print(f"  {'ok' if ok else 'MISSING':<8} {name}")
        if not ok:
            problems.append(name)
    checks = [
        (
            "out-of-scope section lists a use",
            bool(
                re.search(
                    r"^\s*[-*]\s+\S", section(body, "Out-of-scope uses"), re.MULTILINE
                )
            ),
        ),
        (
            "representativeness section states a number",
            bool(re.search(r"\d", section(body, "Representativeness and known gaps"))),
        ),
        (
            "citation has a DOI or URL",
            bool(re.search(r"10\.\d{4,}/|https?://", section(body, "Citation"))),
        ),
        (
            "versions section names a version",
            bool(re.search(r"\b\d+\.\d+(\.\d+)?\b", section(body, "Versions"))),
        ),
    ]
    for label, ok in checks:
        print(f"  {'ok' if ok else 'MISSING':<8} {label}")
        if not ok:
            problems.append(label)
    print(
        f"result: {'PASS' if not problems else 'FAIL'}"
        + (f" ({len(problems)} problem(s))" if problems else "")
    )
    return 1 if problems else 0


if __name__ == "__main__":
    sys.exit(main())

python check_dataset_card.py dataset_card.md

Output for the running example:

  ok       front matter: license
  ok       front matter: language
  ok       front matter: task_categories
  ok       front matter: pretty_name
  ok       Dataset summary
  ok       Motivation
  ok       Composition
  ok       Collection process
  ok       Preprocessing
  ok       Representativeness and known gaps
  ok       Intended uses
  ok       Out-of-scope uses
  ok       Licence and terms
  ok       Versions
  ok       Citation
  ok       Contact
  ok       out-of-scope section lists a use
  ok       representativeness section states a number
  ok       citation has a DOI or URL
  ok       versions section names a version
result: PASS

The first four lines confirm the header fields, the next twelve confirm the sections, and the last four confirm the statements. The verdict follows.

1. Run the check on the card. A missing section or statement blocks the release.
2. Add the organization's own rules to the script: a sentence that must appear ("not a source of published statistics"), a maximum age of the benchmark used in the representativeness section, the contact's domain.
3. Keep the check in the release pipeline with the Croissant check and the manifest.

Worked example. The first draft of the card failed the check on two lines: the representativeness section was a placeholder with no number, because chapter 5 had not yet run, and the citation had no URL. After the representativeness report was written in and the citation gained the dataset's URL, the check passed. The data manager then added one organization-specific rule to the script, which requires the sentence "not a source of published statistics" somewhere in the card, and the check now fails any card that omits it.

What this does not do. The check reads presence. A section that is present and wrong passes. Recipe 4.3 is the review for that.

Recipe

4.3 Review the card and keep it current

Level

AI-ready

Skills

Subject-matter specialist; dataset owner

Indicative time

One hour per release

Result: a specialist has confirmed the card's statements against the source, and the card is updated with every version of the dataset.

1. Have a specialist who knows the source survey read the collection process, preprocessing, and representativeness sections against the survey's own documentation. Record the review and the date in the versions section.
2. Regenerate the measured sections (composition, representativeness) from the scripts at each release, so that the numbers in the card are the numbers of the files beside it.
3. When a user reports something the card did not say (a group the model fails on, a labelling rule they did not know), add it to the known gaps or the collection process. Release the corrected card as a new patch version with the same data.
4. Keep the card under version control with the data, and publish it on the dataset page, in the repository, and as the description of the catalog record.

Worked example. The labour statistician who runs the survey reads the card against the survey's documentation and corrects two statements. The coders used index version 4, where the card said version 3, and the second-coder sample was an eighth of the records, where the card said a tenth. The review is recorded in the versions section with her initials and the date. Three months after release, a partner reports that the model it trained fails on job descriptions in French, which the card had not mentioned. The known gaps section gains the sentence "about 12 percent of job descriptions are in French, concentrated in the Northern Region", and the card is released as version 1.0.1 with the same data.

What this does not do. A card can only say what its authors know. The representativeness report measures part of the rest, and users' findings supply the remainder over time, which is why the card has a contact and a version.

Implementation options

• Template: the twelve sections of this chapter; the Hugging Face dataset card template, which has similar sections and the same header; a datasheet with its question list.
• Header: the YAML keys that repositories read (license, language, task_categories, size_categories, pretty_name, tags).
• Where it lives: the release folder as dataset_card.md; the dataset page; the catalog record's description; the repository's README.

World Bank examples

• The synthetic data cookbook's synthesis record is the same idea for synthetic files: a fixed set of sections, checked by a script before release.
• The model-use statement of the production cookbook documents the model side of the same workflow.

Resources

• Hugging Face dataset cards: the card format repositories render, the header keys, and the card template.
• Datasheets for Datasets (Gebru and others, 2018): the questions on motivation, composition, collection, preprocessing, uses, distribution, and maintenance that a card answers.
• Data Cards (Pushkarna and others, 2022): structured summaries of a dataset's essential facts for decisions across its lifecycle.
• Metadata Editor: where the catalog record that the card's description feeds is kept.

Common mistakes

• A card written once. Numbers in the card drift from the files; the measured sections are regenerated.
• Label reliability unstated. A model builder cannot tell a model's error from the coders' error.
• No out-of-scope section. Someone computes an employment rate from the examples.

Verification

• Completeness. The check passes on every release.
• Accuracy. A specialist's review is recorded in the versions section.
• Currency. The composition and representativeness numbers match the current files.

Checklist

0 of 5 done

• Card with the YAML header and the twelve sections
• Collection process states who labelled, with which instrument, and the agreement between labellers
• Out-of-scope uses include estimation and re-identification
• Check passes on every release
• Specialist review recorded; measured sections regenerated at each release

Ticks are saved in this browser only.