Introduction

Microdata are detailed, unit-level data collected from surveys, censuses, and administrative systems. Each record represents a specific unit of observation, such as a person, household, business, or institution, and contains information about its characteristics, activities, or conditions. Because collecting, processing, and maintaining microdata requires a significant investment of time, expertise, and resources, these data are valuable institutional assets.

To ensure that microdata can be understood, discovered, accessed, and reused over time, they must be accompanied by comprehensive metadata. Metadata provide the context needed to find/identify a dataset, interpret the dataset, describe what the data measure, how they were collected, processed, and organized, who produced them, and under what conditions they can be accessed and used. The Data Documentation Initiative Codebook (DDI-C) and the Dublin Core Metadata Initiative (DCMI) are internationally recognized ISO standards - useful for providing a structured framework for documenting microdata and related resources in a consistent, machine-readable^[1], and interoperable^[2] manner.

In organizations that generate and process microdata, microdata archivists and data stewards play a critical role in applying these standards to preserve, document, organize, manage, and disseminate microdata while protecting respondent privacy and confidentiality. Through effective stewardship and standardized metadata, microdata remain understandable, discoverable, secure, accessible, and reusable, maximizing their value for research, policy development, and evidence-based decision-making.

This Quick Reference Guide for Microdata Archivists provides data archivists with guidelines for documenting micro-datasets in compliance with The Data Documentation Initiative Codebook (DDI Codebook/DDI-C) and The Dublin Core Metadata Initiative (DCMI) metadata standards^[3].

This Guide summarizes the process in the following chronological steps:

1. Organizing the folder structure
2. Gathering and preparing the data set
3. Gathering and preparing the documentation
4. Documenting the study, data file(s) and variables
5. Documenting external resources
6. Metadata Quality assessment
7. Generating the output for publication
8. Cataloging data, metadata and resources

Microdata archiving is part of the wider Data Management Lifecycle. It supports verification and processing; protection, documentation and packaging; storage and preservation; dissemination; and later evaluation and use.

This guide introduces the key concepts and practices for microdata archiving, including preparing data and organizing resources for cataloging and dissemination. It covers the creation of structured, machine-readable metadata, as well as the cataloging of microdata and related resources. Where relevant, the guide provides references to World Bank-supported open-source tools, including sdcMicro for data anonymization, the Metadata Editor for documenting datasets and resources and generating machine-readable metadata, and NADA for cataloging and disseminating data collections.

This guide is not intended to serve as a user manual for these tools. Comprehensive instructions for their installation, configuration, and use are available in their respective documentation. Additional information about these tools can be found here:

• sdcMicro - Statistical Disclosure Control
• Metadata Editor Documentation
• NADA Cataloging Software

To get started, the next section outlines the key principles and best practices for organizing datasets and related resources within a repository. Establishing a well-structured repository is an essential first step that facilitates efficient data curation, metadata creation, cataloging, and dissemination.

---

1. Machine-readable metadata is structured metadata that software can process automatically. ↩︎

2. Interoperability refers to the ability of different systems, tools, and platforms to exchange, interpret, and use data and metadata consistently, enabling seamless integration and collaboration across various data management and analysis environments. ↩︎

3. DDI-Codebook (DDI-C) and DCMI (Dublin Core Metadata Initiative) are international XML metadata specifications. For more information on these standards, please visit https://ddialliance.org/ddi-codebook and https://www.dublincore.org/specifications/dublin-core/dcmi-terms/. ↩︎