We start with the task: a survey response needs one code from a fixed list.
1
1Tasks
A survey response
Someone has to read this response and give it one of 436 codes.
The task is narrow, it repeats with every response, and each code comes from a fixed list.
The responses and the coders' codes were written for this session as illustrations. The suggestions are the recorded output of the small model built for this task. Group names: ISCO-08, International Labour Organization.
Tasks for a small model
The same kind of task appears across development work.
The tasks illustrate where the method could apply, and the practice cases are illustrations. A card says so where a model was built or tested.
Three options
An organization has three ways to get such a task done.
OPTION 1
By hand
Coders read every response and look up its code.
Coding by hand is slow when a survey or a census brings in responses in the thousands.
OPTION 2
A large AI service
A large model at a service company reads each response and returns a code.
Every response is sent to the company that runs the service.
OPTION 3
A small model of its own
A small model on the office's own computer suggests five codes, and a coder chooses.
The model has to be built first, and building it requires many examples with the right answers.
This session is about option 3 and about where the examples come from.
Overview of the process
A small model learns from practice cases that a large model writes.
Part 2 of 6
Small model
We compare a small model with a large one and give the reasons to build one.
2
2Small model
Large language models
The chat assistants in wide use run on very large models in data centers.
A phone or laptop
You type a question.
the text travels
The internet
The question leaves the organization.
to the service
A data center
A with more than 100 billion writes the answer. It needs more than 200 GB of memory.
A large language model can write, translate, summarize, and answer questions.
Memory ranges follow World Bank, World Development Report 2026, spotlight 1, table S1.1. The companies behind hosted chat assistants generally do not publish the size of their main models.
Small AI
A small model has far fewer parameters and is often trained for one task.
Small model142 million parametersLarge model27 billion parametersVery large modelmore than 100 billion parameters
The area of each square is in proportion to the number of parameters.
World Development Report 2026
The report uses the term for AI that is small in size or narrow in scope and is built to run where power, internet connectivity, and digital devices are scarce.
Each small model in this session does one task. One suggests occupation codes for a survey response, and one assigns topics to a report.
World Bank, World Development Report 2026, Box O.1 and spotlight 1. Both small models of this session are this size. The large model is the size of Qwen3.8-27B.
Why a small model
A small model runs inside the organization once it has been built.
Once built
It works without an internet connection.
It runs without a paid service or a .
The documents stay on the organization's own computer.
To build it
A large model writes the examples.
The runs on a GPU.
The model does one task only, and it has to be built for that task.
Building it requires many examples with the right answers.
Part 3 of 6
Synthetic data
A small model needs labelled examples. In this part a large model writes them.
3
3Synthetic data
Labelled examples
A model learns a task from examples with the right answer.
ExampleRight answer
sells roasted maize by the roadside
→5212 Street food salespersons
class 4 teacher, government primary school
→2341 Primary school teachers
grows maize and beans to feed the family
→6310 Subsistence crop farmers
An example with its code is a . The model needs labelled examples of every group.
The missing examples
For these tasks the labelled examples are often missing.
Never collected
After a classification is revised, no past response carries a code of the new version. The model is needed for the first survey that uses it.
Private
Survey responses are confidential. Sending them to an outside service for labelling is often not allowed.
Rare
A classification has hundreds of groups, and a survey meets some of them only a few times.
boda boda rider mama lishe fundi wa simu
Local language
Real responses use local words, abbreviations, and spelling errors. Labelled examples in local languages are scarce.
Synthetic data
Synthetic data are examples written by a computer program.
Responses and codes written by a large model
sell chapati and tea at the bus stand every morning
→5212 Street food salespersons
i teach standard 3 at the mission school
→2341 Primary school teachers
plant cassava and maize, we eat what we grow
→6310 Subsistence crop farmers
World Development Report 2026
The report describes as data that mimic real-world patterns. It presents them as an alternative where real data are too scarce or too sensitive to use.
In this session a large language model writes them.
World Bank, World Development Report 2026, chapter 7, p. 248. The three cases were written for this session as illustrations.
An office comparison
A new staff member learns from practice cases written by a colleague.
In an office
An experienced colleagueknows the work and has no past files to share
writes
Practice casesrealistic and varied cases with their answers
studied by
A new staff memberlearns the answers
checked on
Real past filesreviewed by a supervisor
With the models
A large modeltrained on a very large amount of text
writes
Synthetic datathousands of examples with their answers
train
A small modellearns to give the same answers
tested on
Real caseswith answers given by people
The final test always uses real cases.
Distillation
In distillation a large model passes a skill to a small one.
Teacher
A large model
27 billion parameters about 31 GB of memory
its answers on many examples
Student
A small model
142 million parameters 0.57 GB of memory
The original form
The student learns to match the teacher's probability for every possible answer. The probabilities carry more information than the teacher's top answer alone.
In this session the teacher passes its skill through the examples that it writes.
Hinton, Vinyals, and Dean (2015), "Distilling the Knowledge in a Neural Network." The memory of the teacher is that of its 8-bit version.
Labelled data from another source
A large model can rewrite labelled job titles as survey answers.
The job title comes from the job board hahu.jobs (dataset published by Tabiya, CC BY 4.0), with the code assigned by the job board.
Part 4 of 6
Nine steps
We go through the process one step at a time, with examples from occupation coding.
4
4Nine steps
The process
The process from a task to a small model has nine steps.
Write the cases
Train and test
Improve and maintain
The large model writes more cases where the small model fails
Specific to synthetic data
Standard practice
We follow one example through the nine steps: coding the occupations written in a household survey.
Step 1 of 9 · Describe the task
The team writes down the possible answers and defines each one.
In an office
The manager writes down the decisions the new staff member will make.
With the models
The team writes the list of possible answers, with a definition of each.
Step 2 of 9 · Show a few real cases
A few real cases show the large model how people describe their work.
In an office
The colleague who writes the practice cases first reads a few real files.
With the models
A few dozen real examples go into the request to the large model. Personal details are removed first.
Step 3 of 9 · Vary the cases
Each request to the large model asks for a different kind of case.
In an office
The colleague writes cases of every kind: short and long, calm and angry, easy and hard.
With the models
Each request sets a different combination of answer, length, style, language, and difficulty.
Step 4 of 9 · Write and check
A second model checks every case before it is used.
In an office
A second colleague reads each practice case and its answer before it is used.
With the models
A second model labels each case on its own. Scripts remove repeats, answers outside the list, and cases where the two models disagree.
Step 5 of 9 · Train the small model
The small model studies the checked cases until its answers match.
In an office
The new staff member studies the practice cases.
With the models
The small model is trained on the checked cases until its answers match theirs.
Step 6 of 9 · Test on real cases
The test uses real cases that no model has trained on.
In an office
Before the new staff member works alone, a supervisor checks their decisions on real past files.
With the models
A few hundred real cases with answers given by people are set aside for testing only.
Step 7 of 9 · Practise the weak spots
Extra cases are aimed at the kinds of cases the model gets wrong.
In an office
The colleague writes more practice cases of the kinds the new staff member gets wrong.
With the models
A program finds where the model makes most mistakes and asks the large model for more cases of those kinds. The model is trained again.
Step 8 of 9 · Ask people which is better
Experienced staff choose the better of two cases.
In an office
Experienced staff compare two practice cases, or two answers, and say which one is better.
With the models
People choose between pairs. The choices improve the request to the large model and filter the cases.
Step 9 of 9 · Watch and refresh
A model in use needs regular review by people.
In an office
The supervisor keeps reviewing a sample of the staff member's decisions.
With the models
People review a sample of the model's answers at regular intervals. When the answers, the language, or the situation change, new cases are written and the model is trained again.
Part 5 of 6
Occupation coding
We apply the steps to one occupation model that was built and tested.
5
5Occupation coding
Steps 2 to 4 · The training texts
Three sets of texts take the place of a coded survey in training.
After repeated texts are removed, the model trains on 44,058 different texts.
The job titles come from the job board hahu.jobs (dataset published by Tabiya, CC BY 4.0), and the job board assigned their codes.
Step 5 · The training run
In one training run the small model goes from 0 to 68 percent of first codes right.
The scores are the mean of three training runs on 1,000 survey-style test answers. One pass reads 35,515 of the 44,058 training texts, and the other texts are held out.
Step 6 · The model on test responses
A coder chooses among five codes that the model suggests.
The first six responses come from the test, where a large model wrote them from real job titles. The last two come from 29 informal responses written as illustrations, and no occupation coder has checked their codes.
Steps 6 and 7 · Scores by training set
Adding the rewritten answers raised the share of right first codes from 46 to 66 in 100.
Each score is the mean of three training runs. Each test set holds 1,000 texts from job titles set aside before training.
Step 6 · Compared with large models
On survey-style answers the small model scored about the same as the best of three large models.
The 1,000 test answers were written by Gemini 3.8 Flash from real job titles, because no survey answers coded by people were available. The small model's scores are the mean of three training runs.
Speed and computer
The small model answers in 0.05 seconds on an ordinary processor.
Each time is the median for one answer. The large models were timed over 50 requests each. The times of the hosted services include the network and the service's queue. The CPU time was measured with four threads of a server processor.
Part 6 of 6
Small or large
We end with the choice between a small and a large model for a task.
6
6Small or large
Small or large
A small model suits a narrow task that repeats many times.
A small model is a good candidate when
the answer comes from a fixed list, such as topics or codes;
the same task is repeated many times;
the data should stay inside the organization;
the internet connection is poor.
A large model is the better choice when
the answer is new text, such as a summary or a reply;
the questions cannot be listed in advance.
On survey-style answers about occupation, our small model scored about the same as the best of three large models and answered in 0.05 seconds on an ordinary processor. On 300 real reports, a second small model that assigns topics also scored about the same as three large models.
Before relying on a small model
A team checks four points before it relies on a small model.
1The model was tested on real documents from the place of use, with known answers.
2The test covers the topics or codes that matter.
3The score was compared with a large model on the same test.
4A person reviews a sample of the output at regular intervals.
A first attempt
A first attempt needs a subject specialist, real cases with known answers, and one GPU.
People
2roles in the team
A subject specialist describes the groups and reviews samples of the written examples. A second person runs the notebooks.
Real cases
1,000real job titles with codes set aside for the occupation test
The report-tagging model was tested on 300 real reports. Survey answers coded by people would make a stronger test for the occupation model.
Computer
1 GPUfor one to four minutes for each training run
The large model that writes the examples can be a hosted service or an open model on the organization's server.
The two notebooks of the session run both builds from the examples to the test.
The training times were measured on one server GPU. The notebooks are listed on the last slide.
Your turn
Which kind of model suits each of these six tasks?
Summary
A large model writes the examples, a small model learns from them, and real cases test the result.
The final test always uses real cases.
Sources and further reading
The overview page holds the full text, the figures, and the terms.
World Bank. 2026. World Development Report 2026: The Promise of Artificial Intelligence. Washington, DC: World Bank. doi:10.1596/978-1-4648-2331-2. Box O.1; spotlight 1, table S1.1 and p. 103; chapter 7, p. 248.
Tabiya. occupation_titles_esco: job titles from the job board hahu.jobs with the ESCO occupation assigned by the job board. huggingface.co/datasets/tabiya/occupation_titles_esco. CC BY 4.0. Used to train and test the occupation-coding models.
International Labour Office. 2012. International Standard Classification of Occupations 2008 (ISCO-08): Structure, Group Definitions and Correspondence Tables. Geneva: ILO.
United Nations. 2008. International Standard Industrial Classification of All Economic Activities (ISIC), Revision 4. New York: United Nations.
Solatorio, Aivin V. 2024. "GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning." arXiv:2402.16829. Code: github.com/worldbank/GISTEmbed
Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. 2015. "Distilling the Knowledge in a Neural Network." arXiv:1503.02531.
Contact: aifordata@worldbank.org
Annex
Further detail
These slides add detail to the main part. Each one links back to its slide.
A
Annex · Where a model can run
A model has to fit in the memory of the computer that runs it.
One square stands for 1 GB of memory. Memory ranges are rounded orders of magnitude for typical computers, following World Bank, World Development Report 2026, spotlight 1, table S1.1.
Annex · Rare cases
A model sees only a few real cases of a rare group.
Annex · Step 4 · The request
One request turns a job title into three answers of a respondent.
The run made one request for each of 10,134 job titles and received 30,299 answers. Qwen3.8-27B wrote them.
Annex · Step 4 · The check
A second model removed about a quarter of the answers.
Gemma 4 31B read each answer without the job title and without the code. It chose among the group of the title, the groups next to it in the classification, groups drawn at random, and the reply that none of the listed groups fits.
Annex · Cost of building
Each build took less than an hour on one server.
Requests, tokens and times come from the logs of the runs. Open models wrote and checked the examples on one server with two GPUs, so no price for each token was paid. The price is an example for comparison with the current price list of a provider. The time of the people who wrote the requests and reviewed the examples was not recorded.
Annex · Report-tagging model
A second small model tags each report with topics from a fixed list of 86.
How the model was built
TrainingA large model wrote more than 30,000 labelled sentences. No real report was used.
Threshold488 real reports set the score above which the model assigns a topic.
Test300 other real reports with recorded topics test the model.
The report and its topics come from the World Bank documents database.
Annex · Report-tagging scores
On 300 real reports the small model scored about the same as three large models.
F1 on 300 real report abstracts, on a scale from 0 to 1. The small model's scores are the mean of three training runs. The test covers 44 of the 86 topics. The small model takes 0.19 seconds per report on an ordinary processor.