2.6 Generating Metadata Compliant with Standards
Generating metadata in accordance with the Bank’s metadata standards requires the creation of a structured, machine-readable file based on a prescribed set of metadata elements. Metadata is maintained in JSON format, with the flexibility to export to XML, CSV, or additional formats as necessary.
The Bank’s metadata standards accommodate multiple implementation methods. A dedicated metadata editor application is available for the creation, management, and storage of metadata, featuring full API support to facilitate programmatic import and export where automation enhances efficiency. Additionally, metadata can be produced through other specialized tools, provided that compliance with established standards is maintained. The ensuing sections detail these three approaches to metadata production.
2.6.1 Generating Metadata Using a Specialized Metadata Editor
The Bank utilizes a dedicated metadata editor application, which is compatible with all metadata standards outlined in this Guide. [1] Accessible to all Bank staff members via the Bank’s intranet, the application features an extensive permission and role-management system to facilitate collaborative workflows and robust metadata governance.
This purpose-built solution provides comprehensive capabilities for the creation and management of metadata related to development data assets:
Multi-standard support – Delivers a unified interface for constructing metadata for development data aligned with all metadata standards utilized by the Bank.
Automated metadata extraction – Facilitates dataset importation and automates the creation of essential metadata components, including variable-level data dictionaries for microdata, or feature characteristics for geographic vector datasets.
Template management – Enables the development and maintenance of standardized templates customized to specific data and documentation requirements.
API-driven architecture – Supports automated metadata generation, bulk modifications, and seamless integration with analytical and cataloging workflows via programmatic access (for example, through R or Python libraries).
Flexible export options – Accommodates various export formats (such as JSON, XML, and CSV) to enhance interoperability with external tools and systems.
Role-based access control – Implements user roles (viewer, editor, administrator) to effectively manage contributions, uphold quality assurance protocols, and strengthen metadata governance.
AI-readiness – Designed to interface with external artificial intelligence tools for metadata quality assessment, enrichment, and automation processes.
System compatibility – Operates efficiently with other Bank applications, including those supporting data management, processing, and cataloging, thereby fostering an integrated and efficient data ecosystem.
2.6.2 Generating Metadata Programmatically
Leveraging R and Python for metadata management streamlines workflows, offering both efficiency and scalability while adhering to metadata standards. By employing programmatic methods, users can batch process and automate tasks, which reduces manual work and enhances precision. This is particularly useful for datasets with frequently changing metadata. Additionally, these tools facilitate regular quality checks, diagnostics, and the integration of AI-driven features to boost completeness and consistency. Secure API keys are used to manage access, defining permissions for each user or script. Metadata that aligns with Bank standards can then be uploaded to compatible applications through an API. [2]
Using R
In R, metadata is generated by constructing a list object comprising nested lists that correspond to the groups of metadata elements defined in the standard. The organization of elements and their associated data types must adhere strictly to the schema specifications set forth in the Bank’s metadata standards. Within the description of standards, curly brackets { } signify that a group of elements must be represented as a list, while square brackets [ ] indicate repeatable elements which are implemented as lists of lists. Figure 1 uses the DOCUMENT metadata standard to illustrate the mapping between schema notation and the corresponding R list structure.

Figure 1 – Documentation of the JSON metadata standard (left) and its correspondence to an R list of lists
Using Python
In Python, metadata is organized as dictionaries, with nested dictionaries and lists reflecting the metadata standard's hierarchy. Non-repeatable elements use dictionaries, while repeatable ones are stored as lists of dictionaries. In the documentation of standards, { } denotes a dictionary and [ ] indicates repeatable items as lists of dictionaries. Figure 2 provides the Python equivalent of the example in Figure 1.

Figure 2 - Documentation of the JSON metadata schema (left) and its correspondence to a Python dictionary
2.6.3 Metadata Generated by Other Software Applications
Metadata standards may be integrated within data management and processing platforms. For instance, Survey Solutions, developed by the Bank, and CSPro, a public-domain software from the U.S. Census Bureau, are capable of exporting data dictionaries using the DDI Codebook format.[3] Similarly, ArcGIS Pro (ESRI) and the open-source QGIS facilitate the creation of metadata adhering to ISO 19115 and ISO 19139 requirements for geographic datasets. [4]
Although these applications and comparable tools can generate metadata that is broadly consistent with the Bank’s standards, their outputs are generally limited to structural metadata—including files, variables, and attributes. Critical contextual information, such as methodological details, data collection processes, quality indicators, and usage notes, is typically omitted.
2.6.4 Using Artificial Intelligence to Generate and Enhance Metadata
Artificial Intelligence (AI), particularly through advanced large language models (LLMs), offers significant capabilities for both the creation and enhancement of metadata.
For new metadata generation, AI technologies can extract pertinent information from datasets, documents, survey instruments, scripts, and other digital assets. These models convert unstructured data into structured metadata elements that adhere to recognized standards, such as generating abstracts, defining geographic coverage, or summarizing research methodology. The Office of the WBG Chief Statistician is actively engaged in developing and implementing such AI-powered solutions, aiming to facilitate scalable and high-quality metadata creation throughout the Bank’s data ecosystem.
Additionally, AI can be leveraged to enhance human-curated metadata. AI-driven tools enable the enrichment of descriptions, the completion of minor gaps, the improvement of keyword accuracy, and the automation of quality control processes. Examples of metadata augmentation include:
Generating keywords for datasets: LLMs can analyze abstracts, definitions, and dataset descriptions to propose relevant keywords, thereby improving searchability and indexing.
Extracting and classifying topics: AI can identify key topics within datasets, either by freely generating new topics for metadata or tagging content according to established classification systems.
Creating and refining indicator definitions: AI can generate or improve indicator definitions, ensuring alignment with standardized terminology.
Populating the “relevance” metadata element: For indicators, AI can suggest succinct relevance statements to help users appreciate the significance and applicability of the data, thereby enhancing semantic search capabilities.
Grouping variables by themes: For microdata, AI can classify variables into thematic groups and composes descriptive summaries of variable groupings.
Automating translations: AI-based translation solutions can facilitate the production of metadata in multiple languages, advancing accessibility for global audiences.
Extraction of descriptive content from digital images: AI methods can examine digital images and automatically derive descriptive details—including objects, individuals, scenes, or textual elements—enabling richer metadata.
Generation of transcriptions and quality metrics for audio and video files: AI systems can efficiently extract technical and descriptive metadata from media files, capturing aspects such as language, speaker or scene identification, quality metrics, and full transcripts.
While AI technologies substantially advance both metadata generation and enhancement, human supervision remains indispensable. These AI-driven tools should serve as assistants, expediting documentation and improving metadata completeness, whereas data curators and subject matter experts are responsible for validating the accuracy and relevance of all AI-generated outputs.
The application was released as an open-source software application under an MIT license. See https://worldbank.github.io/metadata-editor-docs/ and https://github.com/worldbank/metadata-editor ↩︎
Programmers can refer to the documentation of the standards available at https://worldbank.github.io/metadata-schemas/. ↩︎
See https://mysurvey.solutions/en/ and https://www.census.gov/data/software/cspro.html ↩︎
See https://pro.arcgis.com/en/pro-app/latest/help/metadata/create-iso-19115-and-iso-19139-metadata.htm ↩︎