2. Retrieval of data by software

Question: Can AI retrieve our data reliably?

Rationale

An AI system that cannot reach the numbers will use whatever it can reach. That may be an outdated table in a PDF, a copy of the figures on a third-party site, or a value it remembers from its training data, and the system presents all three with the same confidence as the official figure. Reliable retrieval means that the authoritative values are available in a form that software can request directly, by series, area, and period, and receive together with their unit, their period, and their source.

The most common barrier is data that only a person can get at: tables inside PDFs, scanned pages, and web interfaces that require clicking through forms. The second most common is data that software can reach but cannot rely on, because the file layout changes with every release or the address changes with every website redesign. This chapter addresses both, in three steps that build on one another: files, an API, and an agent interface over the API.

Target state

• Data are available in open, structured formats.
• Every series, dataset, and release has a stable identifier that does not change when a page is redesigned.
• A documented interface lets software request a specific series, geography, and period.
• Responses include the values together with their unit, period, and source.

Maturity levels

Level	Steps
Foundational	Publish data as CSV or another open format in addition to Excel and PDF. Use one tidy layout per file, with one header row and consistent codes. Keep download URLs stable.
AI-ready	Provide a documented REST API with an OpenAPI description. Support filtering by series, geography, and period. Return units, period, and a source URL with the values. Publish a bulk download. State the terms of use and rate limits.
AI-native	Expose an MCP server so AI assistants can discover and call data tools through a standard protocol. Return provenance with every response. Document which tools are read-only.

Running example

The example organization publishes its values as one tidy CSV whose columns are SDMX cross-domain concepts, describes a three-endpoint API in OpenAPI, and wraps the same data in an MCP server with two read-only tools.

Recipes

Recipe

2.1 Publish tidy CSV files at stable URLs

Level

Foundational

Skills

Data manager

Indicative time

One day to set the convention, then part of each release

Result: every series is published as one file, in a layout that any tool can read, at a web address that stays the same from one release to the next.

A tidy file has one row per observation and the same columns for every series. The column names come from the SDMX standard for exchanging statistical data, so that the file maps directly to SDMX tools:

Column	Meaning
SERIES	The series identifier
REF_AREA	The area the value refers to, as a code (NAT for national)
TIME_PERIOD	The period, as an ISO date or an SDMX period (2023, 2025-Q1)
OBS_VALUE	The value
OBS_STATUS	The status code: A normal, P provisional, B break in series, E estimated
UNIT_MEASURE	The unit
RELEASE	The date of the release the value comes from
SOURCE_URL	The page of the series

SERIES,REF_AREA,TIME_PERIOD,OBS_VALUE,OBS_STATUS,UNIT_MEASURE,RELEASE,SOURCE_URL
FS_UNDERNOURISH_PCT,NAT,2022,8.4,A,% of population,2025-03-14,https://stats.example/series/FS_UNDERNOURISH_PCT
FS_UNDERNOURISH_PCT,NAT,2023,7.9,A,% of population,2025-03-14,https://stats.example/series/FS_UNDERNOURISH_PCT
FS_UNDERNOURISH_PCT,NAT,2024,7.6,A,% of population,2025-03-14,https://stats.example/series/FS_UNDERNOURISH_PCT
ED_PRIMARY_NER_PCT,NAT,2023,91.2,A,% of age group,2025-02-01,https://stats.example/series/ED_PRIMARY_NER_PCT
ED_PRIMARY_NER_PCT,NAT,2024,92.0,A,% of age group,2025-02-01,https://stats.example/series/ED_PRIMARY_NER_PCT
HE_U5_MORT_RATE,NAT,2015,48,A,deaths per 1000 live births,2022-11-30,https://stats.example/series/HE_U5_MORT_RATE
HE_U5_MORT_RATE,NAT,2020,41,A,deaths per 1000 live births,2022-11-30,https://stats.example/series/HE_U5_MORT_RATE
LF_UNEMP_PCT,NAT,2025-Q1,6.3,A,% of labour force,2025-08-20,https://stats.example/series/LF_UNEMP_PCT
LF_UNEMP_PCT,NAT,2025-Q2,6.1,P,% of labour force,2025-08-20,https://stats.example/series/LF_UNEMP_PCT
PV_HEADCOUNT_NPL_PCT,NAT,2018,23.5,A,% of population,2023-06-10,https://stats.example/series/PV_HEADCOUNT_NPL_PCT
PV_HEADCOUNT_NPL_PCT,NAT,2021,21.9,A,% of population,2023-06-10,https://stats.example/series/PV_HEADCOUNT_NPL_PCT

1. Use one row per observation and the columns above, named after the SDMX cross-domain concepts of the Content-Oriented Guidelines. The SDMX-CSV format uses the same names.
2. Keep codes in the file (NAT, 2025-Q1) and put the labels in the code lists (chapter 3). Use UTF-8, a period as the decimal separator, and ISO 8601 dates and SDMX period formats.
3. Fill OBS_STATUS from the SDMX observation status code list. The example marks the latest quarter of unemployment as provisional.
4. Publish each series at a predictable path, for example /series/FS_UNDERNOURISH_PCT/data.csv, and the whole catalog as one bulk file. When a new release replaces the file, the URL stays the same and the RELEASE column changes.
5. Offices that run an SDMX web service can publish SDMX-CSV from it directly. For files outside SDMX, describe the columns once with a Frictionless table schema or CSV on the Web metadata, so that tools know the column types.

Worked example. The example organization's values file has one row per observation. The row for the prevalence of undernourishment in 2024 reads FS_UNDERNOURISH_PCT,NAT,2024,7.6,A,% of population,2025-03-14,https://stats.example/series/FS_UNDERNOURISH_PCT. The row for the unemployment rate in the second quarter of 2025 carries the status P, because the quarter is provisional. The file for the undernourishment series lives at https://stats.example/series/FS_UNDERNOURISH_PCT/data.csv. When the 2025 figure is published the following March, the same address serves the new file, the RELEASE column changes to the new date, and every script that downloads the file keeps working.

What this does not do. A tidy file at a stable URL serves one series. A catalog of hundreds needs the bulk file and the API of recipe 2.2 for selection by area and period. The example has five columns; each added dimension (sex, age group) is a column with its own code list in chapter 3.

Recipe

2.2 Describe the API with OpenAPI

Level

AI-ready

Skills

Backend developer

Indicative time

One to two days for an existing API

Result: the API has a machine-readable description that developers and AI systems can read, and every data response carries the unit, the period, and the source in the same place.

An API (application programming interface) is the address at which a program asks for data and gets it back in a structured form. OpenAPI is the standard way to describe one: which addresses exist, what parameters they take, and what they return. The minimal description below has three endpoints: list series, get one series, get observations. Series records use the field names of the World Bank indicator schema and data responses use SDMX concept names, so the API speaks the same vocabulary as the files:

openapi: 3.0.3
info:
  title: Statistics API (example)
  version: "1.0"
  description: >
    Read-only access to published series. Series records use the field names
    of the World Bank indicator metadata schema; data responses use SDMX
    cross-domain concept names (SERIES, REF_AREA, TIME_PERIOD, OBS_VALUE,
    OBS_STATUS, UNIT_MEASURE). Offices that run an SDMX web service already
    have the SDMX REST API as their documented interface and do not need this
    description.
  license:
    name: CC BY 4.0
    url: https://creativecommons.org/licenses/by/4.0/
  contact:
    email: stats@example
servers:
  - url: https://stats.example/api/v1
paths:
  /series:
    get:
      summary: List published series
      parameters:
        - name: q
          in: query
          description: Free-text filter on name and definition
          schema: {type: string}
      responses:
        "200":
          description: Series records
          content:
            application/json:
              schema:
                type: array
                items: {$ref: "#/components/schemas/Series"}
  /series/{idno}:
    get:
      summary: Metadata for one series
      parameters:
        - {name: idno, in: path, required: true, schema: {type: string}}
      responses:
        "200":
          description: Series record
          content:
            application/json:
              schema: {$ref: "#/components/schemas/Series"}
        "404": {description: Unknown series}
  /series/{idno}/data:
    get:
      summary: Observations for one series
      parameters:
        - {name: idno, in: path, required: true, schema: {type: string}}
        - {name: REF_AREA, in: query, schema: {type: string}, description: Code from the published reference area code list}
        - {name: startPeriod, in: query, schema: {type: string}, description: First period, inclusive (ISO 8601 or SDMX period)}
        - {name: endPeriod, in: query, schema: {type: string}, description: Last period, inclusive}
      responses:
        "200":
          description: Observations with their unit, status, and release
          content:
            application/json:
              schema: {$ref: "#/components/schemas/DataResponse"}
components:
  schemas:
    Series:
      type: object
      required: [idno, name, measurement_unit, periodicity, sources, date_last_update]
      properties:
        idno: {type: string, example: FS_UNDERNOURISH_PCT}
        name: {type: string}
        definition_long: {type: string}
        measurement_unit: {type: string, example: "% of population"}
        periodicity: {type: string, example: annual}
        time_period_start: {type: string}
        time_period_end: {type: string}
        geographic_units: {type: string}
        sources: {type: string}
        date_last_update: {type: string, format: date}
        definition_references: {type: string, format: uri}
        license: {type: string, format: uri}
        citation_requirement: {type: string}
    DataResponse:
      type: object
      required: [SERIES, UNIT_MEASURE, RELEASE, SOURCE_URL, observations]
      properties:
        SERIES: {type: string}
        UNIT_MEASURE: {type: string}
        RELEASE: {type: string, format: date}
        SOURCE_URL: {type: string, format: uri}
        observations:
          type: array
          items:
            type: object
            required: [REF_AREA, TIME_PERIOD, OBS_VALUE]
            properties:
              REF_AREA: {type: string, example: NAT}
              TIME_PERIOD: {type: string, example: "2024"}
              OBS_VALUE: {type: number, example: 7.6}
              OBS_STATUS:
                type: string
                description: "Code from the SDMX observation status code list, for example A (normal), P (provisional), B (break), E (estimated)"

A data response then looks like this, with the series, the unit, the release, and the source at the top and a status on each observation:

{
  "SERIES": "FS_UNDERNOURISH_PCT",
  "UNIT_MEASURE": "% of population",
  "RELEASE": "2025-03-14",
  "SOURCE_URL": "https://stats.example/series/FS_UNDERNOURISH_PCT",
  "observations": [
    {"REF_AREA": "NAT", "TIME_PERIOD": "2023", "OBS_VALUE": 7.9, "OBS_STATUS": "A"},
    {"REF_AREA": "NAT", "TIME_PERIOD": "2024", "OBS_VALUE": 7.6, "OBS_STATUS": "A"}
  ]
}

Offices that run an SDMX web service already have a documented interface: the SDMX REST API specification, with SDMX-ML, SDMX-JSON, and SDMX-CSV responses. The World Bank publishes the World Development Indicators this way through its SDMX API. The steps below are for an organization with a custom REST API.

1. Start from the minimal description and extend it to the API's own endpoints.
2. Make every data response carry SERIES, UNIT_MEASURE, RELEASE, and SOURCE_URL at the top level and OBS_STATUS on each observation, as above.
3. Publish the description at a stable URL (for example /api/v1/openapi.yaml) and render it with any OpenAPI viewer.
4. State the terms of use, the rate limit, and a contact in the info block.
5. Test: a colleague who did not build the API retrieves one known value using only the description.

Worked example. The example organization's description has three endpoints. GET /series lists the five series with their names and units. GET /series/{idno} returns one series record, and GET /series/{idno}/observations returns values, filtered by the query parameters area, from, and to. The colleague who tests it reads the description, forms the request /series/FS_UNDERNOURISH_PCT/observations?area=NAT&from=2023&to=2024, and receives the response shown above, with the two values of 7.9 and 7.6 and the release date. The info block states a limit of 60 requests a minute and the address of the data team.

What this does not do. The minimal description covers three endpoints and leaves out authentication, pagination beyond a limit parameter, and bulk queries. An SDMX web service covers those through the standard; a custom API adds them as the agents cookbook describes for access control.

Recipe

2.3 Wrap the API as an MCP server

Level

AI-native

Skills

Python developer

Indicative time

One day for a prototype; a week to harden

Result: any AI client that speaks the Model Context Protocol can search the catalog and fetch values through two read-only tools, and every response it receives carries its source.

The Model Context Protocol (MCP) is the open standard by which AI assistants call external tools. An MCP server offers a small set of named operations (tools) with descriptions the assistant reads. The example server below offers two: search the catalog, and get the values of a series. It reads the two CSV files of the running example; in production the tool bodies call the API of recipe 2.2:

"""A minimal MCP server over a statistics catalog.

Exposes two read-only tools to any MCP-compatible AI client: one to search
the catalog and one to fetch values for a series. The data come from the two
example CSV files next to this script, whose columns follow the World Bank
indicator metadata schema (catalog) and SDMX cross-domain concepts (values).
In production the tools would call the office's existing SDMX or REST API.

The World Bank's Data360 MCP server (github.com/worldbank/data360-mcp) is a
production example of the same pattern, with search, metadata, data, and
analysis tools.

Requires the official Python SDK:  pip install "mcp[cli]"
Run locally:                       python mcp_server_example.py
Try it with the inspector:         mcp dev mcp_server_example.py
"""

import csv
from pathlib import Path

try:  # mcp 2.x
    from mcp.server import MCPServer as Server
except ImportError:  # mcp 1.x, where the class was called FastMCP
    from mcp.server.fastmcp import FastMCP as Server

HERE = Path(__file__).parent
mcp = Server("statistics-example")


def _read(name):
    with open(HERE / name, newline="", encoding="utf-8") as fh:
        return list(csv.DictReader(fh))


CATALOG = _read("example_catalog.csv")
VALUES = _read("example_values.csv")


@mcp.tool()
def search_series(query: str, limit: int = 5) -> list[dict]:
    """Search published series by words in the name or definition.

    Returns idno, name, measurement_unit, periodicity, and coverage for each
    match.
    """
    words = query.lower().split()
    hits = []
    for rec in CATALOG:
        text = (rec["name"] + " " + rec["definition_long"]).lower()
        score = sum(1 for w in words if w in text)
        if score:
            hits.append((score, rec))
    hits.sort(key=lambda h: h[0], reverse=True)
    keys = (
        "idno",
        "name",
        "measurement_unit",
        "periodicity",
        "time_period_start",
        "time_period_end",
    )
    return [{k: rec[k] for k in keys} for _, rec in hits[:limit]]


@mcp.tool()
def get_values(series: str, ref_area: str = "NAT") -> dict:
    """Return the published values for a series.

    The response carries the unit, the observation status, the release date,
    and the source URL so that the client can cite them.
    """
    rows = [v for v in VALUES if v["SERIES"] == series and v["REF_AREA"] == ref_area]
    if not rows:
        return {"error": f"no values for {series} / {ref_area}"}
    return {
        "SERIES": series,
        "REF_AREA": ref_area,
        "UNIT_MEASURE": rows[0]["UNIT_MEASURE"],
        "RELEASE": rows[0]["RELEASE"],
        "SOURCE_URL": rows[0]["SOURCE_URL"],
        "observations": [
            {
                "TIME_PERIOD": r["TIME_PERIOD"],
                "OBS_VALUE": float(r["OBS_VALUE"]),
                "OBS_STATUS": r["OBS_STATUS"],
            }
            for r in rows
        ],
    }


if __name__ == "__main__":
    mcp.run()

1. Install the SDK (pip install "mcp[cli]") and run mcp dev mcp_server_example.py to try the tools in the inspector.
2. Keep the tools read-only and limit them to published data. Tool descriptions are read by the model, so write them as documentation.
3. Return the unit, the release, and the source URL in every response so that the client can cite them (chapter 5).
4. Connect an AI assistant to the server and ask for five known values. Compare each with the CSV.
5. Before public deployment, add authentication, rate limits, and logging, and run the governance checks of chapter 8.

Worked example. The developer runs the example server in the inspector and calls search_catalog with the query "hunger". The tool returns the undernourishment series, because the word appears in its keywords. The developer then calls get_values with the identifier FS_UNDERNOURISH_PCT and receives the three values for 2022 to 2024, with the unit, the release date of 14 March 2025, and the source URL. An assistant connected to the server is asked for the five known values of the question set, and each answer matches the CSV file, with the source URL quoted.

Two World Bank implementations exist. NADA AI exposes a NADA catalog to AI clients through an MCP server, beside NADA's REST API (documented in OpenAPI); it is being integrated into NADA, so an organization that runs NADA will have the catalog side of this recipe with the software. The Data360 MCP server is a production example of the data side of this pattern. It exposes discovery tools, metadata tools, data tools, and analysis tools over the Data360 API, and the program's MCP page describes the architecture and the tools an MCP server for official statistics can expose.

What this does not do. The example reads two CSV files and matches words. A production server calls the organization's API, paginates large responses, returns the structure (code lists, dimensions) so that the client can form valid requests, handles disaggregations, caches, and runs behind authentication and rate limits. The agents cookbook covers all of that.

Implementation options

• Formats: SDMX-CSV or SDMX-JSON for aggregate data; CSV with SDMX concept names and a table schema where SDMX tooling is not in place; Parquet for large files.
• APIs: the SDMX REST API for statistical data (the World Bank serves WDI this way), or a REST API described by OpenAPI, as the Data360 API is. Many organizations already run one of these.
• Identifiers: persistent URLs, DOIs, or SDMX structure identifiers. Choose one scheme and keep it unchanged across site redesigns.
• Agent interface: the Model Context Protocol is an open standard. A single MCP server can serve any compatible AI client. It wraps the API that already exists and does not replace it.
• Documents: when data exist only in PDFs, extraction tools can recover tables and figures. Publish the extracted data next to the document, and mark them as extracted.

World Bank examples

• The World Development Indicators SDMX API serves WDI through the SDMX 2.1 REST interface, and the Data360 API serves indicator search, metadata, and data.
• The Data360 MCP server exposes Data360 to AI clients through the Model Context Protocol.
• Model Context Protocol for AI-centric data dissemination describes the architecture and the tools an MCP server for official statistics can expose.
• Data Snapshots covers layout detection models that locate figures and tables in PDF documents so that the data inside them can be extracted. The home page shows an example.

Resources

• SDMX standards and the SDMX REST API: the exchange format and the API for aggregate statistics; sdmx-csv for the file form.
• OpenAPI and Swagger UI: describing a REST API and publishing its documentation from the description.
• CSV on the Web and Frictionless Table Schema: column descriptions published beside a CSV file.
• Model Context Protocol and the Python SDK: the agent interface standard and its server library; the agents cookbook covers the full design.
• Data360 MCP server: a server in operation over a statistical API.
• Example values file, OpenAPI description, and MCP server example: the running example's files.

Common mistakes

• Publishing a PDF and calling it open data. A table in a PDF cannot be requested by series and period. Publish the CSV next to it.
• Wide layouts with one column per year. Every release changes the columns. One row per observation keeps the schema stable.
• Labels where codes belong. "Q1 2025", "2025 Q1", and "2025-Q1" are three strings for one period. Codes in the file, labels in the code list.
• Changing identifiers in a redesign. Every citation and integration breaks. Keep identifiers and redirect old URLs.
• An MCP server before an API. The agent interface is a thin layer; the API and the data behind it are the work.

Verification

• Direct request. Someone who did not build the API retrieves a known value with a single call written from the documentation alone.
• Stability. The same identifiers requested a month later still resolve.
• Completeness of response. Every response includes the unit, the period, and the source.
• Agent check. Where an MCP server exists, an assistant asked for five known values returns the official figures.

Checklist

0 of 10 done

• Open, tidy file formats available for each dataset
• Columns and status codes follow SDMX cross-domain concepts and code lists
• Stable identifiers and download URLs
• Table schema published for CSV files outside SDMX
• Documented API with an OpenAPI or SDMX description
• Responses include unit, period, and source
• Bulk download available
• Terms of use and rate limits published
• Data in PDFs also published in structured form
• Agent interface (MCP) evaluated for the highest-demand datasets

Ticks are saved in this browser only.