Standards used in this guide
The recipes use published standards and codes of practice wherever one
exists. Where the World Bank's Development Data Group has adopted one for
its own catalogs, the guide uses that one.
Usage metrics and citations
| Standard | Use | World Bank use | In this guide |
|---|
| Scholix | Exchange of dataset-literature links between services | | Chapter 3 |
| COUNTER Code of Practice for Research Data | Rules for counting dataset views and downloads, with machine access separated from regular access; developed with Make Data Count | | Chapter 8 |
| DataCite | DOIs and citation metadata for datasets; citation events for datasets with DOIs | The Microdata Library assigns DataCite DOIs | Chapters 1, 3, 4, and 8 |
| Crossref | DOIs and reference lists of publications | | Chapters 3 and 4 |
| OpenAlex | Open index of scholarly works with an API | | Chapters 3 and 4 |
Collection
| Standard | Use | In this guide |
|---|
| RFC 9309 (robots.txt) | The rules a crawler reads before collecting from a site | Chapter 5 |
| PyMuPDF | Text extraction from PDFs, as in the program's document parser | Chapters 4 and 5 |
Extraction and harmonization
| Resource | Use | In this guide |
|---|
| Monitoring of Data Use | The program's extraction, deduplication, and harmonization pipeline (ai4data.data_use) | Chapters 6 and 7 |
| GLiNER | Zero-shot named-entity recognition, the model family the extractor uses, with the program's adapter for dataset mentions | Chapter 6 |
| RapidFuzz | Fuzzy string matching used in the program's harmonization | Chapter 7 |
| sentence-transformers | Sentence embeddings for semantic matching in the program's harmonization | Chapter 7 |
Records and links
| Standard | Use | World Bank use | In this guide |
|---|
| World Bank metadata schemas | data_sources in document records; aliases and citation_requirement in series records | The Development Data Group's catalogs | Chapters 1 and 9 |
| NADA | Catalog with citations lists per study | Runs the Microdata Library | Chapters 8 and 9 |
| schema.org Dataset | citation on dataset pages | NADA writes schema.org on catalog pages | Chapter 9 |
Corrections
Standards change versions and URLs. If a reference here is wrong or out of
date, open an issue with the
row and the corrected source.