Date: 10/2/2019
Updated: 9/26/2024
This is first part in a series on accessing the International Debt Statistics (IDS) database through the World Bank Data API. In this part, we will query the World Bank Data API to retrieve the indicator names (including the new ones!) and location codes.
The following code in this guide will show step-by-step how to:
- Setup up your environment with the needed Python packages
- Create Indicator API Queries
- Create Location API Queries (for both debtor and creditor locations)
1. Setup¶
To start, make sure you have the following packages installed on your machine. If you aren't familiar with how to install a Python package, visit each of the linked packages for instructions.
Then, open up your preferred mode of writing Python. This could be in a Jupyter Notebook using Jupyter Lab, using a code editor (like Cursor or Visual Studio) + command line, or just from the command line. Then import the required packages:
# Import packages
import requests
import json
import pandas as pd
2. Indicator API Queries¶
To get a data series from the World Bank Data API, you first to use an "indicator code." For example, the indicator code for long-term external debt stock is "DT.DOD.DLXF.CD." These indicator codes can be found in the World Bank Data Catalog, but you can also use the API to explore and select indicators. In this section, we will guide you through going from a topic idea to using the World Bank Data API to select an indictor code.
More information on indicator API queries can also be found through the Data Help Desk.
Getting the Source ID for International Debt Statistics¶
To find the indicator code, we will first need to specify the API source. The World Bank Data API has numerous sources, including one specific to International Debt Statistics.
The following GET request will get us every source from the World Bank Data API. However, the request is returned in a json format that is difficult to read as is, so we will include some code to parse through it. Then we can see each source name and ID available in the World Bank Data API to find IDS.
# Get all sources from the World Bank API
sources = requests.get("https://api.worldbank.org/v2/sources?per_page=100&format=json")
sourcesJSON = sources.json()
#print(sourcesJSON) # if you want to view the JSON response as is, remove the "#" at the beginning of the print command
# Parse through the response to see the source names and ID numbers.
for i in sourcesJSON[1]:
if i["name"] == "International Debt Statistics":
print("The source ID for International Debt Statistics is " + i["id"])
else:
pass
#print(i["id"],i["name"]) # to see all the source names and IDs, remove the # at the beginning of this line
The source ID for International Debt Statistics is 6
From the above result, you can see that "International Debt Statistics" has a source ID of 6. We will add that source ID to the end of our API queries.
Getting the Indicator code¶
Now that we have source ID for International Debt Statistics we can make another request to the World Bank API to receive all of the indicator names and codes associated with that source.
# Requesting the indicators for the topic External Debt
indicators = requests.get("https://api.worldbank.org/v2/indicator?format=json&source=6")
indicatorsJSON = indicators.json()
Now that we have the indicators, we can view the names and codes. The default number of results on each page is 50, but what if we want to see all the indicators on one page? Let's first print the "total" number of indicators. Then using that number we can adjust our API query.
# Print the total number of indicators
print("There are " + str(indicatorsJSON[0]["total"]) + " IDS indicators")
There are 572 IDS indicators
The above code shows the number of indicators in the External Debt topic. When submitting our new query to the API, we will take this into account by setting the "per_page" parameter to 500. This will allow us to view all the results with one query, instead of having to request multiple pages. Then we can parse through the result to explore the different indicator names and corresponding IDs.
# Get all External Debt indicators, with a per_page parameter of 500.
indicators = requests.get("https://api.worldbank.org/v2/indicator?format=json&source=6&per_page=500")
indicatorsJSON = indicators.json()
#print(indicatorsJSON) # to view ALL of the indicators as is, remove the "#" at the beginning of the print command
# Parse through the response to see the Indicator IDs and Names
for i in indicatorsJSON[1]:
IDSindicators = (i["id"],i["name"])
#print(IDSindicators) # to view the indicator ids and names, remove the "#" at the beginning of the print command
The result gives us all of the External Debt indicators and their codes. You can also view the IDS indicators and codes in their hierarchical order on our data tables. The Analytical view shows the select indicators from the IDS publication and the Standard view shows all available indicators. The indicator and code we want is "DT.DOD.DLXF.CD External debt stocks, long-term (DOD, current US$)." Before using this data, we need to understand its full definition. You can use an API query to get that information as well.
# Use the indicator code to define the "indicator" variable
indicator = "DT.DOD.DLXF.CD"
# Parse through the response to get the "sourceNote" or definition for the desired indicator
for dict_entity in indicatorsJSON[1]:
if dict_entity["id"] == indicator:
print(dict_entity["sourceNote"])
else:
pass
Long-term debt is debt that has an original or extended maturity of more than one year. It has three components: public, publicly guaranteed, and private nonguaranteed debt. Data are in current U.S. dollars.
By using the API queries above, we were able to sort through the World Bank Data API sources to find IDS, explore the corresponding indicators, and then select one indicator of interest and find its ID and definition.
3. Location API Queries*¶
*The 11/16/20 update to this guide added how to also get the full list of creditor locations¶
Now that we have the indicator code, we need to select debtor and creditor locations. To select a location by country, region, or income level category you will need to know its 3 letter code. This section will show you how to use similar API queries to pull the location names and codes.
For more information on location API queries visit the Data Help Desk.
When pulling the list of debtor locations, I will set the per_page result to 300.
# Requesting the locations
dlocations = requests.get("https://api.worldbank.org/v2/sources/6/country?per_page=300&format=JSON")
dlocationsJSON = dlocations.json()
# Parse through the response to see the location IDs and names
dlocations = dlocationsJSON["source"][0]["concept"][0]["variable"]
listLen = int(len(dlocations))
# Create a list to store each row
rows = []
for i in range(0, listLen):
code = dlocations[i]['id']
name = dlocations[i]['value']
rows.append({"id": code, "value": name})
dlocationsList = pd.DataFrame(rows)
# See first few items in the dataframe
print(dlocationsList.head(n=10))
id value 0 AFG Afghanistan 1 AGO Angola 2 ALB Albania 3 ARG Argentina 4 ARM Armenia 5 AZE Azerbaijan 6 BDI Burundi 7 BEN Benin 8 BFA Burkina Faso 9 BGD Bangladesh
IDS 2021 added a 4th dimension to the data - creditor country. You can read more about that on the World Bank Data Blog. To get the full list of the creditor codes and names, you use a query like the one above, but in the get request you replace "country" with "counterpart-area."
# Requesting the locations
clocations = requests.get("https://api.worldbank.org/v2/sources/6/counterpart-area?per_page=300&format=JSON")
clocationsJSON = clocations.json()
# Parse through the response to see the location IDs and names
clocations = clocationsJSON["source"][0]["concept"][0]["variable"]
listLen = int(len(clocations))
# Initialize an empty list to collect rows
rows = []
for i in range(0, listLen):
code = clocations[i]['id']
name = clocations[i]['value']
rows.append({"id": code, "value": name})
clocationsList = pd.DataFrame(rows)
# See first few items in the dataframe
print(clocationsList.head(n=10))
id value 0 001 Austria 1 002 Belgium 2 003 Denmark 3 004 France 4 005 Germany, Fed. Rep. of 5 006 Italy 6 007 Netherlands 7 008 Norway 8 009 Portugal 9 010 Sweden
The results from these two queries are also saved as location-codes csv*.