Session 02 · Building specialized small AI using synthetic data
This session is in preparation. This page introduces its ideas for readers without a background in AI: small AI, synthetic data, and model distillation. It opens with tasks from development work that suit a small model, explains why the examples for training such a model are often written by a large model, and describes the process in nine steps. A worked example then follows the process on one task, with measured results. Linked terms are defined in the Terms section at the end of the page.
Tasks for a small model
A small model can take on a narrow task that repeats many times, on an organization's own computer. The figure shows six such tasks from development work: matching job seekers to vacancies at a public employment office, sorting complaints sent by text message to a project's grievance mechanism, picking out requests for help from messages after a flood, answering farmers' questions from an extension guide, flagging errors in benefit applications, and coding the occupations written in a household survey. For each task, the figure shows why no training data exist and one practice case with its answer. The tasks and practice cases were written for this page as illustrations.
Tasks for a small model
The figure shows six tasks from development work. Select a task to see what the model does, why no training data exist, one practice case, and where the model runs.
Sorts each complaint by issue and urgency and sends it to the office that handles it.
Why no training data exist
- Few labelled Few past cases have the right answer recorded.
- Private The real records hold personal data and cannot be sent to an outside service.
- Local language The cases are written in a local language, with abbreviations and mixed languages.
Where the model runs
On a server of the project team. The messages and phone numbers stay with the team.
One practice case
pls help, maji hayajatoka tangu jumatatu. contractor broke the pipe near the school
Right answer- Issue
- Water supply
- Urgency
- High
- Send to
- Site engineer
A large model writes thousands of cases like this one, each with its answer. The case shown was written for this page as an illustration.
The tasks illustrate where the method could apply. They do not describe systems in use.
Why a small model
The chat assistants in wide use today run on . These models can write, translate, summarize, and answer questions on almost any subject. The largest have more than 100 billion and run in data centers, so each request travels over the internet to the company that runs the model. The companies behind the best-known chat assistants generally do not publish the size of their main models.
A small model has far fewer parameters and is often trained for one task. The World Bank's World Development Report 2026 uses the term for AI that is small in size or narrow in scope and is built to run where power, internet connectivity, and digital devices are scarce. A small model can run on a laptop or on an organization's own server, so the documents do not leave the organization. Once built, it works without an internet connection, and it runs without a paid service or a . Building it has a cost: a large model writes the examples, and the training runs on a GPU.
A model has to fit in the memory of the computer that runs it. The report gives rounded memory ranges for three kinds of computer: about 2 to 20 GB for a phone or laptop, 20 to 200 GB for a single server, and more than 200 GB in a data center. The figure draws memory as squares of 1 GB. The small model of the worked example needs about half of one square. Qwen3.8-27B, one of the teachers of the worked example, needs about 31 squares in the 8-bit version used here, more than a laptop holds.
Where a model can run
A model has to fit in the memory of the computer that runs it. Each square is 1 GB of memory. Choose a model to fill the three kinds of computer.
Small model: 142 million parameters, 0.57 GB of memory. The student in the worked example is this size.
Phone or laptop
About 2 to 20 GB
✓ The model fits
- Internet
- Not needed
- The text
- Stays on the device
A single server
About 20 to 200 GB
✓ The model fits
- Internet
- Local network only
- The text
- Stays in the organization
Data center
More than 200 GB
✓ The model fits
- Internet
- Constant connection
- The text
- Travels to the data center, usually run by a service company
Memory ranges are rounded orders of magnitude for typical computers, following World Bank, World Development Report 2026, spotlight 1, table S1.1. The companies behind hosted chat assistants generally do not publish the size of their main models.
A small model does one task only, and it has to be built for that task. Building it requires many examples with the right answers.
Why synthetic data
A model learns a task from , which are examples together with the right answer. For the tasks above, such examples are often missing. Messages sent after a flood do not exist before it. Complaints, applications, and job seekers' profiles hold personal data and cannot be sent to an outside service. The cases that matter most, such as an emergency among routine messages, are rare. Real messages mix local languages and abbreviations, and few labelled examples exist in most local languages.
are examples written by a computer program. The World Development Report 2026 describes them as data that mimic real-world patterns and as an alternative where real data are too scarce or too sensitive to use. In this session a large language model writes them. The figure shows four reasons for writing them; under "Rare", a button adds written cases to a set of messages.
Why there are no training examples
The figure gives four common reasons why a team has no labelled examples for its task. Select a reason.
The cases that matter most are often the rarest: an emergency among routine messages, a wrong entry among correct forms. A large model can write many more of them.
Among 400 real messages, 4 are emergencies. A model trained on them sees too few to learn what an emergency looks like.
The figure ends with a comparison from office work: a new staff member who learns from practice cases written by an experienced colleague, and who is then tested on real files.
How synthetic data are made
The figure describes the process in nine steps. For each step it gives the office comparison, what happens, why the step is needed, and a demonstration based on the complaints example. A mark under each step shows whether the worked example used it.
From a task to a small model in nine steps
Select a step. Each step has an office comparison, what happens, why it is needed, and a demonstration based on the complaints example. The marks under the steps show which ones the worked example of this page used.
Describe the task
The manager writes down the decisions the new staff member will make.
- What happens
- The team writes the list of possible answers, with a definition of each, or the format of the answer.
- Why it is needed
- The large model that writes the cases, the people who check them, and the test all need the same definitions.
- Used in the worked example
- A list of 86 topics.
The list of answers for the complaints example
Water supply. Water cut, damaged pipes, or a dry well caused by the works.
Select an answer to see its definition. The large model and the people who check its cases read the same definitions.
- Describe the task. The team writes the list of possible answers with a definition of each. The large model, the people who check its cases, and the test all use the same definitions.
- Show a few real cases. A few dozen real examples, with personal details removed, go into the request to the large model as . Without them, a large model writes tidy, generic text that differs from real messages. A check makes sure that the output does not copy them. The worked example did not use this step.
- Vary the cases. Each request sets a different combination of attributes, such as the answer, the length, the style, the language, and the difficulty. A small model learns only from the kinds of cases in its training set. In the experiments behind this session, Qwen3.8-27B was asked many times for job titles of the same occupation groups, and 6,612 of the 18,000 titles written (37%) repeated an earlier title.
- Write and check. The large model writes the cases with their answers. A second model labels each case on its own, and scripts remove repeats, answers outside the list, and cases where the two models disagree.
- Train the small model on the checked cases.
- Test on real cases. A few hundred real cases with answers given by people are set aside for testing only. Practice cases share the writing habits of the large model, so a high score on them can fail to hold on real cases.
- Practise the weak spots. The model is tested on a second set of real cases, and a program finds the kinds of cases with the most mistakes. The large model writes more cases of those kinds, and the model is trained again. In the worked example, these targeted rounds scored 0.037 and 0.041 higher than rounds of the same size spread evenly over all topics, in two repeats of the experiment. On job titles the difference was 0.012, and its 95% interval included zero. Choosing what to practise from the model's mistakes is related to , which orders the training cases from easy to hard; the ordering itself was not tested.
- Ask people which is better. Experienced staff choose the better of two practice cases or two answers. This improves the request to the large model and filters the cases. Where the small model writes text, such as an answer to a farmer, it can also train the model towards the preferred answers. The experiments did not use this step.
- Watch and refresh. People review a sample of the model's answers at regular intervals, and new practice cases are written when the answers, the language, or the situation change. The World Development Report 2026 describes a screening tool for diabetic eye disease that rejected more than one-fifth of the images nurses took, because the lighting in the clinic differed from that in its training images.
Steps 5 to 9 are standard practice in machine learning, the field that builds models from labelled examples. A team that has trained a model on labelled examples has carried them out before. Steps 1 to 4 are specific to synthetic data: a large model writes the examples that people would otherwise collect and label. In step 7 the large model also writes the additional cases.
The worked example
An organization holds thousands of reports and wants each one tagged with its topics, such as Education or Transport, from a fixed list of 86. Tagging by hand is slow. A large AI service could do the tagging, but every report would then be sent to the company that runs the service. The organization also has no tagged reports from which to train a of its own.
The figure shows the three steps of the worked example. They cover steps 4 to 6 of the process above: writing and checking the examples, training, and testing. The worked example also used steps 1 and 7, and step 3 in part. The reports in the example are World Bank reports, and the model reads the of each one.
The method in three steps
Each step links to its section below. The switch replaces the description of each step with a comparison from office work.
- Step 1 · Synthetic data
A large model writes practice examples
A large AI model writes thousands of example sentences and lists the topics of each one.
Read step 1 - Step 2 · Distillation
A small model learns from the examples
A small model is trained on the examples until it gives the same topics to the same sentences.
Read step 2 - Step 3 · Small AI
The small model works alone
The small model tags the reports on the organization's own computer, without the large model and without an internet connection.
Read step 3
The large model is needed in steps 1 and 2 only. In step 3 the reports do not leave the organization.
Step 1: synthetic data
For the tagging task, a labelled example is a sentence together with its topics. The organization in the worked example has none, so a large language model writes them. Each request names eight of the 86 topics and asks for 20 sentences about them, with the topics of each sentence listed. A thousand requests give about 20,000 labelled sentences. The figure shows one request and the 20 sentences that came back.
A large model writes practice examples
One request from the worked example. The request names eight topics; the large model Qwen3.8-27B writes 20 sentences and lists the topics of each one.
Write 20 sentences about these topics:
- Biodiversity
- Community-Driven Development
- Disability Inclusion
- Financial Sector
- Labor Markets
- Migration
- Trade Facilitation and Logistics
- Urban Development
Qwen3.8-27B
Practice sentences
Press the button to show the sentences one by one.
The worked example repeats this for 1,000 requests, each with eight of the 86 topics. Together they give about 20,000 labelled sentences. The same request was sent twice with one rule changed. Switch between the two versions; topic names in the sentences are highlighted. Use the words a report on the subject would use, including the topic names or their key words where an author would use them. Sentences that name one of their topics, all 1,000 requests Score of a small model trained on each version, 300 real abstracts (F1, 0 to 1) The score for names allowed is the mean of three training runs; the score for names avoided comes from one run.Go deeper: two wordings of the same request
A script checks the sentences before they are used. It removes sentences without a topic, topics outside the list of 86, and repeated sentences. In the worked example these checks found nothing to remove.
The wording of the request also affects the result. The "go deeper" part of the figure shows a second version of the request, in which the sentences had to avoid the topic names. The small model trained on that version did worse on real reports.
Step 2: the small model learns
a model means showing it labelled examples until its answers match them. The figure trains a very simple model inside the browser, on the sentences of step 1 for four topics. The model is then tested on 60 real reports whose topics are known.
A model learns from examples
A simple model learns four topics from example sentences. It counts the words in the examples of each topic and gives a new text the topic whose words match best. Choose how many examples it reads.
Words the model links to each topic
- none yet
- digital
- none yet
- none yet
Test on 60 real reports
17 of 60 given the right topic
Each square is one report, filled when the model gives it the right topic. A random guess among the four topics gets about 15 right.
The sentence below is about Transport. Compare the model's answer with 10 and with 40 examples per topic, or write another sentence.
Go deeper: any number of examples, the learning curve, and the examples themselves
Share of the 60 reports with the right topic
The dashed line at 25% is a random guess. After about 20 examples per topic this model stops improving. The model built in the session learns more from each example and reads more than 30,000 of them.
The last example of each topic
- Education The study examines the correlation between teacher quality in basic education and long-term labor market outcomes.
- Health The success of digital health platforms depends on robust data privacy frameworks and widespread digital literacy among healthcare providers.
- Transport The new transport corridor aims to reduce logistics costs and improve market access for smallholder farmers in the rural interior.
- Energy A surge in renewable energy adoption in Southeast Asia has significantly lowered carbon footprints while enhancing grid reliability for industrial sectors.
With one example per topic, the model gives 17 of the 60 reports the right topic, close to the 15 expected from a random guess. With ten examples per topic it gets 31 right, and with forty it gets 34. The model in the figure counts words only, and it stops improving after about 20 examples per topic. The small model built in the session is a more capable kind of model, which takes the meaning of whole sentences into account. It learns all 86 topics from about 20,000 examples, and from another 12,000 to 15,000 in further rounds.
This way of training is called . A large model passes a skill to a small one: the large model acts as the teacher and the small model as the student. In its original form, described by Hinton, Vinyals, and Dean (2015), the student learns to match the teacher's probability for every possible answer, which carries more information than the teacher's top answer alone. In this session the teacher also writes the examples of step 1.
Step 3: the small model at work
After training, the student works alone. It needs neither the teacher nor an internet connection, and the reports stay on the organization's computer. The figure shows the student at work on one real report.
The student at work on one real report
The student reads the abstract of a report and gives every topic a score. It assigns the topics with a clear score. A check mark shows a topic recorded for this report in the World Bank documents database.
This paper measures the impact of low-cost transport by rail in Malawi on the dispersion of agricultural commodities prices across markets by exploiting the quasi-experimental design of the nearly total collapse of domestic transport by rail in January 2003 due to the destruction of a railway bridge at Rivirivi, Balaka. Estimations are based on monthly market prices of four agricultural commodities (maize, groundnuts, rice, and beans) in 27 local markets for the period 1998–2006. Market pairs connected by rail when the railway line was operational are intervention observations. Railway transport services explain a 14 percent to 17 percent reduction in price dispersion across markets. Geographical reach of trade varies by crop, most likely related to storability and geographical spread of production. Perishability appears to increase impact reflecting limited scope for arbitrage. Overall, impacts are remarkably similar in size across commodities.
Student's highest scores
Result
- ✓ Agriculture and Food
- ✓ Macroeconomics
- ✓ Trade
- ✓ Transport
- ✓ Transport
- ✓ Trade
- ✓ Agriculture and Food
- ✓ Macroeconomics
The student assigned 4 topics, and 4 of them are recorded for the report.
Go deeper: five more abstracts, the score threshold, and the answers of three large models
Select an abstract. The student scores every topic from 0 to 1 and assigns the topics above the dashed line. Move the slider to raise or lower the line. ✓ marks a topic recorded for the report in the World Bank documents database.
Electrification and Women's Empowerment : Evidence from Rural India (2019)
Student: the eight highest scores
The student assigns at most 5 topics. The starting line of 0.03 was chosen on a separate set of abstracts.
Answers
- ✓ Education
- ✓ Energy
- ✓ Gender
- ✓ Energy
- ✓ Gender
- ✓ Education
- Social Inclusion
- Social Sustainability and Inclusion
- ✓ Energy
- ✓ Gender
- Infrastructure
- ✓ Energy
- ✓ Gender
- Infrastructure
- Jobs and Development
- Social Inclusion
- Digital Development
- ✓ Energy
- ✓ Gender
- Labor Markets
- Social Inclusion
Below the student, three large models asked directly for the topics, with the list of topics in the prompt. Grey topics do not appear among the topics recorded for the 300 test abstracts, so they are not scored.
The student and three large models were tested on the same 300 report abstracts. On a score from 0 to 1, the student scored 0.52 and the large models scored 0.52 to 0.53. The student takes 0.19 seconds per abstract on an ordinary processor. The large models take 0.5 to 3.9 seconds on a GPU or through an online service.
The student compared with large models
The student and three large models assigned topics to the same 300 report abstracts. The large models were asked directly for the topics of each abstract.
Accuracy
The student scores about the same as the large models.
Student0.52
Large models0.52–0.53
Score from 0 to 1. A higher score means more correct topics and fewer missed ones.
Speed
The student is about 3 to 20 times faster.
Student0.19 s
Large models0.5–3.9 s
Seconds per abstract.
Computer
The student runs on an ordinary processor, without an internet connection.
StudentProcessor (CPU)
Large modelsGPU or hosted service
The student needs 0.57 GB of memory. A GPU is a graphics processor, now also used to run AI models.
Go deeper: the score and the time of each model
How well each model assigns topics
F1 score on 300 report abstracts, from 0 to 1. The score is high when the assigned topics are correct and few of the recorded topics are missed. Student scores are the mean of three training runs. The first student learned from the first 20,296 sentences. In further rounds, a program read the student's mistakes on a separate set of abstracts and asked the teachers for more sentences on the topics involved.
Scale from 0 to 0.6.
Time to assign the topics of one abstract
Each mark on the scale is ten times the one before.
Each abstract was sent alone; the student reads its sentences together. The times of the hosted services include the network and the service's queue. The CPU time was measured with four threads of a server processor on 100 of the abstracts, and the time of Qwen3.8-27B on 50.
The results come from one task. The right answers are the topics recorded for each report in the World Bank documents database, and a record can omit a topic that the abstract covers. The 300 test reports cover 44 of the 86 topics, so the scores say nothing about the other 42. On a second task in the same experiments, coding job titles to an occupation classification, the student scored 0.15 below the best large model.
Does a task suit a small model?
A small model is a good candidate when the answer comes from a fixed list, such as topics or codes, and the same task is repeated many times. It also suits work where the data should stay inside the organization or where the internet connection is poor. A large model is the better choice when the answer is new text, such as a summary or a reply, or when the questions cannot be listed in advance. The figure lists six more tasks from development work, different from those at the top of the page. For each task, the reader chooses a small or a large model and sees the suggested answer with its reason.
Which kind of model suits the task?
Choose a small or a large model for each task to show the suggested answer and the reason for it.
Recognize a crop disease from a photo of a leaf, in a field without internet
Read the number on a water meter from a photo taken by a meter reader
Recognize the language of each message received by a hotline
Write a personal reply to each complaint
Draft the summary of a 150-page project document
Answer any question a citizen asks about public services
Before a team relies on a small model, it checks the following points:
- The model has been tested on real documents from the place where it will be used, with known answers.
- The test covers the topics or codes that matter for the work.
- The model's score has been compared with that of a large model on the same test.
- A person reviews a sample of the model's output at regular intervals.
Notebooks
Two notebooks follow the process on one task each. Notebook 01 codes answers to a survey question on occupation with ISCO-08 unit groups. Notebook 02 assigns topics to reports from their abstracts. The examples written by the large model are stored in the repository, so both notebooks run in Google Colab without a key for a large model.
| Notebook | Open in Colab |
|---|---|
| 01 · Coding survey answers on occupation | Open |
| 02 · Tagging reports with topics | Open |
The files that the notebooks read are described in assets/README.md.
Terms
- Abstract: The short summary at the start of a report.
- Curriculum learning: Training a model on cases in a planned order, usually from easy to hard. In a wider sense, choosing which cases to practise next from the model's results.
- Distillation: Training a small model to reproduce the answers of a large model. The large model is called the teacher and the small model the student.
- GPU: A graphics processor, now also used to run AI models. Large models run on GPUs or similar chips, which cost more and use more power than an ordinary processor.
- Labelled example: An example together with the right answer. In this session, a sentence together with its topics.
- Large language model: A model trained on a very large amount of text. It can write, translate, summarize, and answer questions. Chat assistants run on such models.
- Model: A computer program that has learned a task from examples. A programmer does not write its rules.
- Parameter: One of the internal numbers of a model, adjusted during training. The number of parameters measures the size of a model.
- Preference feedback: Choices made by people between two outputs, such as two practice cases or two answers. The choices are used to filter cases, to improve a request, or to train a model towards the preferred outputs.
- Seed example: A real example placed in the request to a large model, so that its output resembles real cases.
- Small AI: The term of the World Development Report 2026 for AI that is small in size or narrow in scope and is built to run where power, internet connectivity, and digital devices are scarce. The report notes that a large model in the cloud, reached by voice call or text message from a basic phone, can serve the same purpose.
- Synthetic data: Examples written by a computer program. In this session a large language model writes them.
- Training: The step in which a model adjusts its parameters until its answers match a set of labelled examples.
Sources
- World Bank. 2026. World Development Report 2026: The Promise of Artificial Intelligence. Washington, DC: World Bank. doi:10.1596/978-1-4648-2331-2. Box O.1, "Small AI, big impact"; spotlight 1, table S1.1 and p. 103; chapter 7, p. 248.
- Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. 2015. "Distilling the Knowledge in a Neural Network." arXiv:1503.02531.
- The data behind the figures are in assets/overview.json.
Source files
The session's slides, demonstration script, worked-example files, and notebook sources are in the session folder on GitHub.