Skip to main content

Practical Guide to AI-Assisted Metadata Schema Development

A cookbook to help technical practitioners start building a metadata schema for their own materials and use case. Version 0.1 (working draft), November 2026.

Working draft

This guide is a working draft from the AI for Data – Data for AI program. All seven chapters contain substantive recipes and worked examples for editorial review. Chapter 1 defines the scope and samples; Chapters 2–3 include Python worked examples. Chapters 4–7 add concept review, field design, validation decisions, and standards comparison. The complete draft is ready for a coordinated editorial and technical review. It does not represent official World Bank Group guidance or policy. Corrections are welcome through GitHub issues; the contributing page describes the review expectations.

To help people find and use reports, datasets, images, or recordings, organizations need a consistent way to describe them. A metadata schema defines what information to record about each item. The challenge is deciding which fields will be useful for the materials and the task at hand.

This cookbook guides practitioners from examples of their own materials to a schema suited to their use case. AI can propose candidate fields, identify related concepts, and help compare meanings or draft documentation. Human reviewers make the schema decisions.

This cookbook adapts the approach presented in the Data Snapshot research paper by Dy & Solatorio (2026), using that study as a worked example.

Purpose and learning outcomes​

  • The practitioner can define what the schema needs to describe and select examples that reflect its intended uses.
  • The practitioner can use AI to suggest and organize metadata fields, then review and refine those suggestions.
  • The practitioner can write field definitions and usage guidance, then test and improve the schema using new examples.
  • The practitioner can use relevant standards to refine the schema while preserving the requirements of their materials and use case.

Prerequisites​

  • Manual review requires representative materials and a reviewer who understands their intended use.
  • The response-inspection and profiling examples require Python 3.10 or newer and the ability to read CSV and JSON files. They use only the standard library.
  • Hosted LLM execution requires suitable provider credentials and configuration, and may require paid access.
  • Local LLM execution requires an available model, a compatible interface, and sufficient computing resources; provider API credentials may not be needed.

Running the supplied scripts on recorded study outputs needs no LLM credentials or paid calls. The request examples describe logical tasks; adapt inputs and response checks to the chosen interface before a new run. Rechecking standards requires authoritative sources. New runs also need resource planning and checks against the materials.

First steps​

  1. Read chapter 1 to define what gets a metadata record and select development and holdout examples. Allow about 30 minutes for reading and the scope worksheet.
  2. Inspect metadata suggestions in chapter 2, then group names and propose concepts in chapter 3. Allow about 60–90 minutes for reading, response inspection, and the profiling example.
  3. Review concepts and turn them into fields in chapters 4 and 5. Allow about 60–90 minutes for reading and selected worksheet tasks.
  4. Test the schema and review findings in chapter 6, then compare relevant standards and refine the specification in chapter 7. Allow about 90–120 minutes for reading and selected evidence checks.

These are indicative reading and worked-example estimates. Applying the methodology to a new collection requires sampling, domain review, standards research, and additional work. The self-assessment provides one checkpoint per chapter.

Chapters and questions​

ChapterQuestionPractical topics
1. Artifact Characterization and SamplingWhat must the schema describe, and which artifacts provide credible evidence of its requirements?A characterization statement and frozen sample manifest will supply the inputs to independent discovery.
2. Metadata DiscoveryHow can descriptive requirements emerge independently from artifacts?Evidence-bearing metadata observations will provide the input to deterministic profiling.
3. Field Profiles and Metadata Concept SynthesisHow do local observations become manageable evidence for shared concepts?Field profiles and a candidate metadata concept inventory will provide the evidence for expert refinement.
4. Expert Concept Refinement and AssessmentWhich concepts should remain, and how does the development evidence support them?A refined inventory, human decision record, and concept coverage evidence will support operational schema design.
5. Operational Metadata Schema DesignHow do meaningful concepts become fields that people can use consistently?An initial operational schema and field-decision record will be frozen for held-out validation.
6. Schema ValidationWhat does new artifact evidence reveal about missing concepts, unclear guidance, and representation limits?Validation evidence, human dispositions, a documentation-revised schema, and a representation backlog will support standards-informed refinement.
7. Standards-Informed Schema RefinementHow can standards improve an artifact-derived schema while preserving its established requirements?A standards-informed operational schema, crosswalk, and human decision record will complete the documented methodology.

Maturity levels​

An organization can reach different levels in different chapters. Human review remains necessary at every level. Greater automation does not grant authority to accept schema changes.

LevelMeaning
FoundationalThe practice exists in a form a person can use and a crawler can read. Practitioners inspect evidence and record human decisions.
AI-readyThe practice is structured, identified, and accessible through documented interfaces. Observations, mappings, and decisions support repeatable checks.
AI-nativeAI-oriented methods have provenance and verification built in. Bounded model assistance supports discovery, comparison, and assessment under human adjudication.

Running example​

The running example comes from a real use case examined by Dy & Solatorio (2026): developing a metadata schema for data snapshots, such as tables, charts, and maps from reports. The chapters follow selected examples and decisions from that study. Source images and document metadata are available in the public dataset.

Scope of the examples​

Recorded study results, selected excerpts, preparation-time deterministic worked examples, and proposed extensions retain distinct labels. A simplified example does not claim an independent execution of the study. The study sample size is not a universal minimum.

The evidence supports bounded conclusions for the tested collections. It does not establish universal generalizability, superiority over alternatives, or a false-negative rate. Production metadata extraction, document layout detection, formal ontology construction, and knowledge graph infrastructure are outside this guide. Post-study machine-readable implementation remains separate from the research endpoint.

Files​

The downloads support the response-inspection and profiling examples in Chapters 2–3, plus the manual worksheets and worked reviews. A sample-overlap checker is available as a supplemental tool. Python 3.10 or newer is sufficient for the scripts; no additional packages or model credentials are required. The downloads below supply the case-study inputs; applying the method elsewhere requires the reader's own artifact evidence.

FileUsed in
Evidence package notesAll chapters use source, chronology, evidence-label, and reuse notes.
Research evidenceChapters 1–7 use observations, profiles, concepts, decisions, and study summaries.
Source-image registerAll chapters use identities, image and document links, attribution, and visual limits.
Chapter-to-example mapAll chapters use the precise evidence selections assigned to each stage.
Concept review templateChapter 4 uses the existing blank worksheet for human concept decisions.
Gap review templateChapter 6 uses the existing blank worksheet for schema adjudication.
Filled characterization worksheetChapter 1 uses a teaching statement of artifact scope and boundaries.
Manifest checkerRecipe 1.3 shows how to check sample overlap using the selected register.
Selected response excerptsChapter 2 inspects four retained title observations from three recorded responses.
Teaching discovery requestChapter 2 uses this request specification to guide a new discovery run; recorded examples come from the study.
Response inspectorChapter 2 checks provenance, execution status, and selected observation structure.
Geographic observation inputChapter 3 profiles three unchanged recorded observations.
Exact-name profiling scriptChapter 3 creates a new JSON profile with explicit counts, denominators, and evidence.
Teaching synthesis requestChapter 3 prepares an unexecuted concept-synthesis specification.
Teaching assessment requestChapter 4 inspects fixed-vocabulary classification and its historical boundary.
Initial field excerptsChapter 5 uses five exact v1.1 definitions and their original schema examples.
Field-specification worksheetChapter 5 records definitions, teaching guidance, evidence, and reviewer dispositions.
Validation request specificationsChapter 6 compares unexecuted teaching adaptations of open residual and consequential-gap assessment.
Standards-review worksheetChapter 7 records authoritative comparisons, candidate changes, and human dispositions.
Final field inventoryChapter 7 reconciles 36 baseline fields with 37 final fields and their final cardinalities.
ResourcePurpose
Metadata Curation with Language ModelsCuration of records within an existing schema complements this guide to developing the schema.
Extracting Data from DocumentsArtifact location and numerical extraction provide related guidance outside this scope.
Publishing ML-Ready DatasetsDataset documentation and split management support reusable research collections.
Evaluation Suites for Statistical AIEvaluation design supplies related methods for future controlled assessments.

Citation​

World Bank. 2026. Practical Guide to AI-Assisted Metadata Schema Development. Version 0.1 (working draft). AI for Data – Data for AI.

Research reference​

Dy & Solatorio (2026). Artifact-Driven Metadata Schema Development: A Human-in-the-Loop Methodology with Multimodal LLMs. Working paper.

The manuscript names AJ Carl P. Dy and Aivin V. Solatorio as its authors. Its arXiv publication is pending; this citation has no assigned public identifier.

Version history​

  • 0.1 (November 2026). Seven chapters provide 22 recipes, two Python worked examples, a supplemental sample checker, and human-review, validation, and standards-comparison materials. The complete working draft awaits coordinated editorial and technical review.