Skip to the content.

PRWP Reproducibility Protocol

This protocol outlines the step-by-step process for verifying the reproducibility of research packages submitted to the World Bank’s PRWP verification team. It ensures that research findings can be independently replicated using the provided code, data, and instructions. The protocol covers the entire workflow—from receiving submissions, verifying completeness and data access, running the package in a clean environment, tracking changes with version control, and checking consistency with the original manuscript, to publishing the final reproducibility package.

1. Receive submission

2.a Verify completeness

2.b Verify data access

3. Start Clean Environment

4. Version control with Git

This will help you see if the outputs are changing after you run the code.

5. Run the package

Document these critical aspects as you prepare and run the package:

  1. Dependencies
    • List all libraries and packages required, saved in ado folder (Stata), requirements.txt (Python), or renv.lock (R).
  2. Changes Made
    • Note any adjustments, such as installing packages, modifying paths, or code changes. Document substantive changes in the GitHub issue.
  3. System Information
    • Record the OS, processor, memory, and software version (including Stata edition). You will include this in the report (step 9)
  4. Time Spent
    • Log time spent at each stage as a comment in the GitHub issue, tagging the Project coordinator.

6. Verify stability

7. Confirm initial run

8. Verify consistency with the manuscript

9. Draft Reproducibility Report

10. Prepare Metadata and Publication

11. Publish to RRR

12. Send Report and Package to Authors