Skip to main content

Session 02 assets

This folder holds the recorded data behind the overview page, the slides, and the two notebooks. The notebooks read the files of occupation/ and topics/ from the repository, so they run without a key for a large model.

Overview page​

FileContentSource
overview.jsonThe numbers, requests, sentences, and reports behind the figures of the overview page and the report-tagging slidesRecorded runs of the report-tagging task

Occupation coding (notebook 01)​

FileContentSource
occupation/titles.jsonl.gz10,134 job titles, each with an ISCO-08 unit group codetabiya/occupation_titles_esco, CC BY 4.0
occupation/test-titles.jsonl.gz1,000 further job titles with codes, kept for the testSame dataset
occupation/answers.jsonl.gz30,299 survey-style answers written from the 10,134 job titles, three per title. Each row holds the code of the title, the style of the answer, the code chosen by a second model, and the result of the check (kept)Written by Qwen3.8-27B and checked by Gemma 4 31B
occupation/test-answers.jsonl.gzThe 1,000 test titles, each rewritten as one survey-style answerWritten by Gemini 3.8 Flash
occupation/definition-titles.jsonl.gz15,865 job titles for all 436 unit groupsWritten by a large model from the ISCO-08 group definitions and checked by a second model
occupation/illustrative-answers.json29 short answers in the style of a household survey, each with an expected codeComposed for the slides of this session
occupation/results.jsonThe request sent to the large model, the token counts of the rewriting step, and the recorded scores of models trained on five combinations of the setsRecorded runs

The job titles come from the job board hahu.jobs in Ethiopia and were published by the Tabiya project. The job board assigned each title an occupation of the European classification ESCO. The first four digits of an ESCO occupation code are the ISCO-08 unit group, and the files keep those four digits. No trained coder reviewed the codes.

The survey-style answers, the titles written from the definitions, and the illustrative answers are synthetic. None of them is an answer from a survey, and no trained coder reviewed their codes.

The ISCO-08 structure and definitions are not stored in this folder. Notebook 01 downloads the file from the website of the International Labour Organization when it runs.

Report tagging (notebook 02)​

FileContentSource
topics/topics.csvThe 86 topics, each with its parent topic where it has oneThe World Bank's topic list
topics/sentences.jsonl.gz20,296 sentences, each with one or more topics and the identifier of the request that produced itWritten by Qwen3.8-27B in 1,000 requests
topics/reports-threshold.jsonl.gz488 abstracts of Policy Research Working Papers from 2016 to 2020 with their topics, used to choose the threshold. The topics are the subjects of the catalogue record, mapped to the list of 86World Bank Documents and Reports database; each row holds the URL of its record
topics/reports-test.jsonl.gz300 further abstracts with their topics, used for the testSame database

The sentences are synthetic. The abstracts and their topics are real records.