Data Management and Archiving

How to keep your data and code organized, reproducible, and shareable

SharePoint is the shared space

wkpepfiles/ -> WKPEP Documents

MML’s dedicated space:

Micro Modeling/
├── _data-repo/        ← NO CODE! Reference data, by <country>/<survey> (read-only)
├── _projects-archive/ ← Past projects: code, data, outputs (snapshots only)
├── _projects-shared/  ← NO CODE! Active projects results data and outputs (live)
└── <other>/           ← Other folders to be decided later

Acess rights:

  • Micro Modeling folder is accessible to the MML team only.
  • _data-repo/ is only visible to MML team members.
    • some data under _data-repo/ is restricted to MML admins team only
  • _projects-archive/ is only visible to MML team members. Some subfolders only to MML Admins.

_data-repo/: Archive read-only data

_data-repo/
├── _AAA-template/          copy this to start a new deposit (sorts to the top)
│   ├── _data/              empty — the shape to fill
│   └── readme.md           the deposit-log format
├── {ISO}/                  country data — one folder per country
│   ├── readme.md           REQUIRED: dated log of every deposit
│   ├── {ISO}{YYYY}{v01}/        one folder per survey round
│   └── {ISO}{YYYY}{v02}/        a new round is a NEW folder, never an overwrite
└── {SOURCE}/               global reference data — named by provider
    ├── readme.md           REQUIRED: URL, retrieval date, what the script does
    └── clean/              derived; deletable because it is regenerable

Checklist

The log entry is the part that survives you. Vintage, access, and known issues are what a future analyst needs and cannot reconstruct from the files themselves.

README.md template for a new deposit:

# 2026-06-15 — GNB2021 deposited

**Person:** John Doe <jdoe@worldbank.org>
**Data:** DHS 2021, national, households + individuals + consumption goods
**Source:** Received from Jane Roe <jroe@worldbank.org>
            (or: <URL>, downloaded 2026-06-15)
**Vintage:** provider release 2023-04, harmonized v3
**Access:** WB-internal only — NSO agreement
**Restrictions:** no redistribution, no publication without permission
**State:** as received, unmodified
**Contents (optional):** 26 files, 179 MB
**Context:** Fiscal incidence analysis for Country FY2026. TTL: Jane Roe <jroe@worldbank.org>
**Known issues and limitations:**
- value labels in Portuguese;
- s07 module not provided;
- weights do not sum to population totals;
**Notes:** if needed

Reading archived data in Stata

The archive sits at a different absolute path on every machine: SharePoint syncs under each analyst’s own OneDrive root. Hardcoding your own path is what makes a do-file unrunnable for the next person.

So the path appears exactly once, in a block that branches on the Stata username, and every file reference downstream starts from the $archive global:

if "`c(username)'" == "jdoe" {
    global archive "C:\Users\jdoe\OneDrive - WBG\mml\_data-repo"
}
if "`c(username)'" == "jroe" {
    global archive "C:\Users\jroe\OneDrive - WBG\mml\_data-repo"
}

use "$archive/<ISO>/<ISO>{YYYY}{v01}/households.dta", clear

Paths under the global mirror the archive layout — $archive/{ISO}/{ISO}{YYYY}{v01}/filename.dta. A new collaborator adds one branch to the block at the top and runs the rest untouched.

_projects-archive/: Archive past projects

When a project ends (delivery, publication, or hand-over), deposit a self-contained, reproducible package in _projects-archive/ on SharePoint. The test is blunt: another analyst opens the folder years later and re-runs the analysis without asking anyone what happened.

Raw data. From the Archiving point of view, it is relevant to keep a copy of the RAW data along with the self-contained bundle.

  • One folder per country and fiscal year and analysis type.
  • Each folder contains a README.md — who built it, when, for whom, and how to reproduce it.
  • Inside, either version subfolders (vYYYY-MM-DD/) or the project as-is in bulk if there is only one vintage.
  • Data, code, and output stay separated inside each version.

Naming convention

{ISO}-{FYXX}-{TYPE}

  • ISO — three-letter country code (GNB, GTM, TJK)
  • FYXX — World Bank fiscal year of the analysis (FY25, FY26)
  • TYPE — optional analysis type (MicroMacro, CCDR, PolicySim)
  • Drop the { }. Example: GNB-FY26, GTM-FY26-CCDR
  • version subfolders use vYYYY-MM-DD — the date that version was completed, e.g. v2026-05-01.

Files and folders structure

_projects-archive/
│
├── README.md               conventions for the whole archive
├── _AAA-template/          copy this to start a new project archive
│   ├── README.md           project README template (optional)
│   └── vYYYY-MM-DD/
│       ├── README.md       version README template
│       └── **/             subfolders as per project structure
├── GNB-FY26-MFMode/        one country, one fiscal year, one analysis type
│   ├── README.md           WHO built it, WHEN, for WHICH TTL, HOW to rerun
│   └── v2026-05-01/        one dated version of the analysis
│       └── **/
└── TJK-FY25/

Checklist

README.md template

One per {ISO}-{FYXX} folder. It answers who, when, for whom, and what the work was.

# Name of the project

**Date completed:** 2026-01-20

**Status:** delivered / superseded by <folder> / abandoned

**Developed by:** Jane Roe <jroe@worldbank.org>

**TTL history:**
- Latest: John Doe <jdoe@worldbank.org>
- Previous: Jane Roe <jroe@worldbank.org>

**Previous author(s):**
- Jane Roe <jroe@worldbank.org>
- Original author (for Chad, adapted here): John Doe

**GitHub repo:** <URL> (COMPULSORY)

**What this is:** One paragraph: the question, the macro scenario, the survey, the deliverable.