Skip to main content

01 · From a language model to an application

Open in Colab View the notebook on GitHub

The outputs on this page are the ones committed with the notebook. Without an API key, the notebook replays recorded model answers, and the same recordings appear here.

Session 01 · Building AI applications for research and data · notebook 1 of 6

Objective​

A language model writes from two sources: the text in its prompt and what it learned in training. This notebook shows the difference. The same research question is asked with nothing but the question, then with evidence placed in the prompt. The two answers establish the first two components of an application: the instructions, which set the model's behaviour, and the context, which supplies the evidence. The application writes both, and the model reads them.

The application after this notebook​

Diagram: the question and the context feed the prompt, which goes to the model and produces the answer.

The prompt is the only component this notebook builds. Later notebooks add the document store, retrieval, data tools, the agent loop, and the verifier around it.

The whole session uses one research question, so that each method is judged on the same case:

How have internet access and economic development evolved in Rwanda and Kenya? Compare the recent available data, summarize the main patterns, and cite the sources.

Contents​

  1. Section 1 defines the three parts of a prompt and who writes each.
  2. Section 2 asks the question with no evidence. Four of the six figures checked are close to the official values, two are off, and none can be traced to a source.
  3. Section 3 places four country briefs in the prompt. Every figure now carries a citation.
  4. Section 4 asks for a year the brief does not cover. The model says so and gives the closest year.
  5. Section 5 asks an ambiguous question. With one added instruction, the model asks which meaning is intended.
  6. Section 6 prints the messages as the model received them.
  7. Section 7 counts the tokens the context cost and explains why a whole collection cannot be supplied this way.

How to run it​

Run the cells in order. Without an API key, the model's outputs are recorded ones, labelled [recorded output: ...]. Each cell names its recorded scenario with w.use(...) before it calls the model. With a key in WORKSHOP_API_KEY, the same cells call the model live, the answer streams into its bubble as the model writes, and the wording differs from the recorded answers.

The next cell installs the session's tools and loads its helper module. The first run takes about a minute. The cell is setup only.

# Setup. In Colab this clones the repository and installs the dependencies; locally it uses the folder as is.
import os, sys, subprocess, pathlib

REPO_URL = "https://github.com/worldbank/ai4data.git" # the program's repository
BRANCH = "main" # the branch the Colab setup clones
SESSION = "training/session-01-building-ai-applications/notebooks"

if not pathlib.Path("workshop.py").exists():
subprocess.run(["git", "clone", "-q", "--depth", "1", "--branch", BRANCH, REPO_URL, "/content/ai4data"], check=True)
os.chdir(f"/content/ai4data/{SESSION}")
subprocess.run([sys.executable, "-m", "pip", "install", "-q", "-r", "requirements.txt"], check=True)
sys.path.insert(0, os.getcwd())

# Optional: a key for an OpenAI-compatible API. Without one, the notebook replays recorded outputs and says so.
try:
from google.colab import userdata # type: ignore
for name in ("WORKSHOP_API_KEY", "WORKSHOP_BASE_URL", "WORKSHOP_MODEL"):
if userdata.get(name):
os.environ[name] = userdata.get(name)
except Exception:
pass

import workshop as w
model = w.make_model()
print("Model:", w.mode(model))
Model: recorded outputs (no WORKSHOP_API_KEY set)

The stack​

NeedLibrary
Documents, splitting, embeddings, vector store, prompts, tools, structured outputLangChain
The agent loopLangGraph, through langchain.agents.create_agent
Tool server and client over the Model Context ProtocolFastMCP
Number verificationpcn-core, the Python core of the Proof-Carrying Numbers protocol
Output schemasPydantic

workshop.py holds what these do not: the worked-example data, a chat model that replays recorded outputs when no API key is configured, and a few printing and evaluation helpers.

The setup cell imported the helper module as w and created model, a LangChain chat model: a live one when a key is set, otherwise one that replays recorded answers. The helper functions used in this notebook, in the order they appear:

CallWhat it does
w.use(name)Names the recorded scenario for the next model call, so that the replay returns the right answer. Has no effect on a live model.
w.show(message, role=...)Draws a message as a chat bubble, with the text rendered as Markdown.
w.answer(chain, inputs)Calls the chain and draws the answer as a bubble. With a live model the tokens stream into the bubble; without a key the recorded answer is drawn at once. Returns the message.
w.indicator_lookup(indicator, country, start=...)Returns the observations of a World Development Indicators series for one country, from the session's data file.
w.latest(result)The last observation of such a result.
w.show_table(...), w.show_claim_check(...), w.show_prompt(...)Draw a table, the claims comparison, and the prompt as HTML.
w.load_documents()Loads the eight session documents as LangChain Document objects, with the file's header as metadata.

1. The three parts of a prompt​

A call to a chat model sends a list of messages. The first is the system message, which the model treats as standing instructions for the whole exchange. The others are user messages. An application composes three parts into these messages:

  • Instructions: how the model should behave, written by the application's builders.
  • Context: the evidence for this question, selected by the application. Here it is documents; later it is passages and data records.
  • Question: the user's text.

The parts are kept in separate slots because a different person changes each one, at a different time.

The next cell writes three sets of instructions. The basic set only names the role. The grounded set adds the three rules this session depends on: answer from the context, cite the document after each figure, and say when something is missing. The clarifying set adds one more rule, for section 5.

INSTRUCTIONS_BASIC = "You are a research assistant. Answer the question."

INSTRUCTIONS_GROUNDED = """You are a research assistant for development data.
Answer only from the supplied context. Cite the document id in square brackets after each figure.
If the context does not contain what is needed, say so and give the closest available information."""

INSTRUCTIONS_CLARIFY = INSTRUCTIONS_GROUNDED + """
If the question is ambiguous, say which readings are possible and ask which one is meant before answering."""

A prompt template holds the three slots. ChatPromptTemplate.from_messages takes a list of (role, text) pairs, and each {name} in a text is a slot that is filled at call time. The instructions become the system message. The context and the question go together into one user message, in a fixed layout, so that the model can tell them apart. prompt | model joins the two steps into one object: the output of the template becomes the input of the model. That object, chain, is the whole application at this stage.

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
("system", "{instructions}"), # the system message: the instructions slot
("human", "CONTEXT:\n{context}\n\nQUESTION: {question}"), # one user message: the context and the question slots
])
chain = prompt | model # fill the template, then send the messages to the model

w.show(w.MAIN_QUESTION, role="user")
user

How have internet access and economic development evolved in Rwanda and Kenya? Compare the recent available data, summarize the main patterns, and cite the sources.

Animation: instructions set by the developer, context supplied by the application and a question asked by the user are combined into one prompt, which goes to the model and produces an answer that cites its source.

The animation's text is this notebook's: the instructions from the cell above, the context and the answer from section 3, and the question from the worked example.

2. The model without context​

Without context, the model answers from training data: text it read before its training ended, compressed into its parameters. It holds no records, only patterns of text, so its figures are reconstructions.

The next cell makes the first call. w.answer passes one dictionary with a value for each slot to the chain, and draws the reply. The context slot gets the text (none), so the model sees that no evidence was supplied. w.use names the recorded scenario first, so that the replay returns the answer recorded for this exact call.

w.use("nb01_zero_context")
answer = w.answer(chain, {"instructions": INSTRUCTIONS_BASIC, "context": "(none)", "question": w.MAIN_QUESTION})
assistant · recorded output: nb01_zero_context

Internet Access and Economic Development in Rwanda and Kenya

I can't access live databases, so the figures below come from my training data. They are approximate and may be outdated, and definitions differ across sources (for example, subscriptions versus unique users). Check the latest releases before citing numbers in formal work.

Snapshot comparison (approximate, ~2022–2024)

IndicatorKenyaRwanda
Population~55 million~14 million
GDP (current US$)~$110–115 billion~$13–14 billion
GDP per capita~$2,000–2,200~$950–1,050
Real GDP growth~5% (2023), ~4–5% (2024)~8% (2023), ~9–10% (2024)
Internet users (share of population)roughly 40%roughly 30–40% (estimates vary widely)
Mobile broadbandLarge base of mobile data subscriptions, widely usedExpanding; 4G population coverage is high
Mobile moneyVery high adoption (M-Pesa is a global reference case)High adoption, growing via national platforms
Human Development Index~0.58 (mid-tier)~0.53 (low-medium tier, improving)
Poverty~36% national poverty line (2020 survey)Fell from ~77% (2001) to ~38% (2016/17); further decline reported in later surveys

Main patterns

1. Both countries have expanded access quickly, mostly through mobile. Mobile phones, not fixed broadband, drive connectivity in both. Fixed broadband and household internet access remain low, especially outside cities.

2. Kenya's model is market-led and mobile-money-driven. Competition among mobile operators, a dense private sector, and M-Pesa have made Kenya a reference case for mobile-enabled finance and digital services. Its growth has been broad-based, though poverty and inequality remain high.

3. Rwanda's model is state-led and infrastructure-focused. The government built a national fiber backbone, pursued universal coverage targets, and promoted digital services through institutions such as Smart Rwanda and the Kigali Innovation City initiative. Its growth has been faster, but from a lower income base.

4. Affordability is a binding constraint in both countries. Data prices relative to income remain high for many households, so access does not automatically translate into use. Usage gaps are largest among women, rural residents, and low-income groups.

5. Connectivity and growth move together, though causality is hard to isolate. Cross-country studies consistently link broadband and mobile adoption to productivity and GDP growth, and both countries' digital strategies assume this relationship. Disentangling the effect of internet access from other reforms in these two countries requires careful econometric work.

Suggested sources

  • ITU, Facts and Figures (2024) and Measuring Digital Development: Facts and Figures (ITU DataHub): internet use and affordability indicators.
  • World Bank, World Development Indicators (databank.worldbank.org): GDP, GDP per capita, poverty, and growth.
  • IMF, World Economic Outlook (October 2024 edition): growth forecasts and country data.
  • Kenya National Bureau of Statistics (KNBS), Economic Survey 2024 and Kenya Poverty Report.
  • Communications Authority of Kenya, Quarterly Sector Statistics Reports: mobile and internet subscriptions. Note that these count subscriptions, not individuals.
  • Rwanda Utilities Regulatory Authority (RURA), annual reports and ICT sector statistics.
  • National Institute of Statistics of Rwanda (NISR), Integrated Household Living Conditions Survey (EICV) and ICT household surveys: poverty and household internet access.
  • Smart Rwanda Master Plan 2050 and related Rwandan ICT strategy documents.
  • GSMA, The Mobile Economy Sub-Saharan Africa (annual): mobile connectivity and economic contribution.
  • Alliance for Affordable Internet (A4AI), Affordability Report: data-price comparisons.
  • DataReportal (Digital 2024/2025 country reports): internet user estimates.
  • UNDP, Human Development Report: HDI values.

If you tell me which years or indicators matter most for your purpose, I can narrow the comparison and suggest the most appropriate source for each figure.

The answer cannot be judged without reference values. The next cell prints the official values from the World Development Indicators for the two measures the answer leans on most: the share of people using the internet and GDP per capita. w.indicator_lookup returns the observations of one series for one country, and the loop keeps the 2023 and 2024 values.

rows = []
for ind in ["IT.NET.USER.ZS", "NY.GDP.PCAP.CD"]: # internet users (% of population), GDP per capita (current US$)
fmt = "{:,.0f}" if ind == "NY.GDP.PCAP.CD" else "{:.1f}"
for country in ["Kenya", "Rwanda"]:
r = w.indicator_lookup(ind, country, start=2023) # observations from 2023 on
by_year = {o["year"]: fmt.format(o["value"]) for o in r["observations"]}
rows.append((r["name"], country, by_year[2023], by_year[2024]))
w.show_table(["Indicator", "Country", "2023", "2024"], rows, source="Source: World Bank, World Development Indicators.")
IndicatorCountry20232024
Individuals using the Internet (% of population)Kenya32.135.0
Individuals using the Internet (% of population)Rwanda28.831.7
GDP per capita (current US$)Kenya1,9432,133
GDP per capita (current US$)Rwanda1,0611,060

Source: World Bank, World Development Indicators.

The next cell takes six claims from the answer and sets each beside the official value. Each row holds the claim, the model's words, the value from the data, and a verdict. value returns the latest observation of a series, and growth returns the growth rate of a given year.

def value(ind, country):
o = w.latest(w.indicator_lookup(ind, country, start=2023))
return o["value"], o["year"]


def growth(country, year):
return next(o["value"] for o in w.indicator_lookup("NY.GDP.MKTP.KD.ZG", country, start=year)["observations"] if o["year"] == year)


rows = [ # (claim, what the model said, what the data says, verdict)
("Kenya · internet use", "“roughly 40%” of the population", "{:.1f}% of people online in {}".format(*value("IT.NET.USER.ZS", "Kenya")), "above the data"),
("Rwanda · internet use", "“roughly 30–40% (estimates vary widely)”", "{:.1f}% of people online in {}".format(*value("IT.NET.USER.ZS", "Rwanda")), "close"),
("Kenya · GDP per capita", "“~$2,000–2,200”", "{:,.0f} US dollars in {}".format(*value("NY.GDP.PCAP.CD", "Kenya")), "close"),
("Rwanda · GDP per capita", "“~$950–1,050”", "{:,.0f} US dollars in {}".format(*value("NY.GDP.PCAP.CD", "Rwanda")), "close"),
("Kenya · GDP growth", "“~5% (2023), ~4–5% (2024)”", f"{growth('Kenya', 2023):.1f} percent in 2023, {growth('Kenya', 2024):.1f} percent in 2024", "close"),
("Rwanda · GDP growth", "“~8% (2023), ~9–10% (2024)”", f"{growth('Rwanda', 2023):.1f} percent in 2023, {growth('Rwanda', 2024):.1f} percent in 2024", "2024 above the data"),
]
w.show_claim_check(rows)
ClaimWhat the model saidWhat the data saysVerdictSource named
Kenya · internet use“roughly 40%” of the population35.0% of people online in 2024above the datanone
Rwanda · internet use“roughly 30–40% (estimates vary widely)”31.7% of people online in 2024closenone
Kenya · GDP per capita“~$2,000–2,200”2,133 US dollars in 2024closenone
Rwanda · GDP per capita“~$950–1,050”1,060 US dollars in 2024closenone
Kenya · GDP growth“~5% (2023), ~4–5% (2024)”5.7 percent in 2023, 4.7 percent in 2024closenone
Rwanda · GDP growth“~8% (2023), ~9–10% (2024)”8.6 percent in 2023, 7.2 percent in 20242024 above the datanone

4 of 6 claims land close to the data. The others are marked above. None names a source, so a reader cannot tell which is which.

Four of the six claims are close to the official values. Two are off: Kenya's internet share, given as roughly 40 percent against 35.0 in 2024, and Rwanda's 2024 growth, given as 9 to 10 percent against 7.2. The answer is hedged, structured, and ends with a reading list, and none of that helps a reader find the two wrong figures. Every figure is a rounded range, most carry no year, and the sources are institutions to consult, so no figure can be traced to a record. A different model or different instructions would change the wording. The figures would still be reconstructions, because the prompt contains no data.

3. The same question with context​

Context is text that the application places in the prompt for the model to read. Under the grounded instructions, the model answers only from it and cites it.

The next cell builds the context. w.load_documents returns the eight session documents, each with an id in its metadata, and the cell keeps the four country briefs. Each brief is prefixed with its id in square brackets, which is the form the instructions ask the model to cite. The size is printed as characters and as an estimate of tokens, at four characters per token.

docs = {d.metadata["id"]: d for d in w.load_documents()}
context = "\n\n".join(f"[{i}]\n{docs[i].page_content}" for i in ["kenya-digital", "rwanda-digital", "kenya-economy", "rwanda-economy"])
print(f"{len(context):,} characters of context, roughly {len(context)//4:,} tokens")
4,098 characters of context, roughly 1,024 tokens

The call is the same as in section 2, with the grounded instructions in the first slot and the four briefs in the second.

w.use("nb01_with_context")
answer = w.answer(chain, {"instructions": INSTRUCTIONS_GROUNDED, "context": context, "question": w.MAIN_QUESTION})
assistant · recorded output: nb01_with_context

Internet access and economic development in Kenya and Rwanda, 2010 to 2024

Latest comparison (2024):

IndicatorKenyaRwanda
Internet use (% of population)35.0 [kenya-digital]31.7 [rwanda-digital]
Mobile subscriptions (per 100 people)126.5 [kenya-digital]93.2 [rwanda-digital]
GDP per capita (current US$)2,133 [kenya-economy]1,060 [rwanda-economy]
Real GDP growth4.7% [kenya-economy]7.2% [rwanda-economy]
Population56.4 million [kenya-digital]14.3 million [rwanda-digital]

Internet use. Both countries have grown quickly from low bases. Kenya rose from 7.2% in 2010 to 35.0% in 2024 [kenya-digital]. Rwanda rose from 8.0% to 31.7% over the same period [rwanda-digital]. Kenya's share more than doubled between 2010 and 2015 [kenya-digital], while Rwanda's nearly doubled between 2015 and 2020 [rwanda-digital]. Kenya leads in 2024, but the gap is small.

Mobile subscriptions. Rwanda started lower, at 34.4 per 100 people in 2010 [rwanda-digital], compared with 60.0 in Kenya [kenya-digital]. Kenya's figure passed 100 around 2020 (117.6) [kenya-digital], and Rwanda's stayed below 100 throughout [rwanda-digital]. The source notes that values above 100 reflect multiple subscriptions per person and inactive SIM cards, so they do not show what share of people own a phone [kenya-digital].

Economic development. Kenya's GDP per capita is roughly double Rwanda's in 2024: 2,133 against 1,060 [kenya-economy; rwanda-economy]. Rwanda's rose by about 80% between 2010 and 2024 from a lower starting point (587 to 1,060) [rwanda-economy]. Kenya's roughly doubled from 1,092 to 2,133 [kenya-economy]. Both series are in current dollars, so exchange rates and prices affect them as well as real output. Kenya's 2023 decline coincides with a shilling depreciation [kenya-economy].

Growth. Rwanda has grown faster recently: 9.8% in 2022, 8.6% in 2023, and 7.2% in 2024 [rwanda-economy]. Kenya recorded 4.9%, 5.7%, and 4.7% over the same years [kenya-economy]. Both economies contracted in 2020, Kenya by 0.3% [kenya-economy] and Rwanda by 2.8% [rwanda-economy].

Main patterns: - Internet use roughly tripled or quadrupled in both countries between 2010 and 2024 but remains around a third of the population. - Mobile subscriptions have grown faster than internet use. Kenya's subscriptions exceed one per person, while internet use is still about a third of the population [kenya-digital]. - Kenya has the higher income level, while Rwanda has recently grown faster.

Limitations. The context does not analyse causal links between connectivity and income, so it cannot show whether internet access drives growth or the reverse. The Rwanda document states that growth exceeded 7% in 10 of 15 years [rwanda-economy], but it gives values for only six years, so I could not verify that count. Similarly, the Kenya document's statement that its economy grew in every year except 2020 [kenya-economy] is not checkable against the years it lists.

Diagram: the same question without context gets a hedged answer of rounded ranges with no citation, two of its figures off; with the four country briefs as context it gets a dated answer that cites its source.

Shortened excerpts of the recorded answers of sections 2 and 3.

Every figure now carries a citation to a brief, the comparison is specific, and the model notes two statements in the briefs that it could not verify from the years listed. The application chose the four briefs and put them in the prompt. The model wrote from that text. For the rest of the session, the application decides what evidence the model sees, and the quality of the answer follows from that decision.

4. A question the evidence cannot answer​

An instruction sets the model's behaviour when the evidence falls short. The grounded instructions say: if the context does not contain what is needed, say so and give the closest available information. The next cell supplies only the Kenya brief, prefixed with its id so that the model can cite it, and asks for 2025. The brief ends at 2024.

w.use("nb01_missing_info")
brief = f"[kenya-digital]\n{docs['kenya-digital'].page_content}" # one brief, with its id so that the model can cite it
answer = w.answer(chain, {"instructions": INSTRUCTIONS_GROUNDED, "context": brief,
"question": "What share of Kenya's population used the internet in 2025?"})
assistant · recorded output: nb01_missing_info

The supplied context does not include a 2025 figure. It states that 2024 is the latest year available. The closest figure is 35.0 percent of the population using the internet in 2024 [Kenya: internet use and mobile subscriptions, 2010 to 2024].

The model says that 2025 is absent, names 2024 as the latest year, and gives 35.0 percent with a citation. The behaviour came from one sentence in the instructions.

5. An ambiguous question under two sets of instructions​

"How connected is Kenya?" has at least three readings: internet use, mobile subscriptions, or broadband coverage. A model picks a reading unless it is told to ask. The next cell asks the question twice with the same context: first with the basic instructions, then with the clarifying set, whose added rule tells the model to list the readings and ask which one is meant.

print("Without an instruction to clarify:\n")
w.use("nb01_ambiguous_no_instruction")
first = w.answer(chain, {"instructions": INSTRUCTIONS_BASIC, "context": brief, "question": "How connected is Kenya?"})

print("\n" + "=" * 100 + "\nWith the instruction:\n")
w.use("nb01_ambiguous_with_instruction")
second = w.answer(chain, {"instructions": INSTRUCTIONS_CLARIFY, "context": brief, "question": "How connected is Kenya?"})
Without an instruction to clarify:
assistant · recorded output: nb01_ambiguous_no_instruction

Kenya is partly connected. Internet use reaches a little over a third of the population, while mobile subscriptions are numerous enough to exceed the population.

  • Internet use: 35.0% of people in 2024, up from 7.2% in 2010. That leaves roughly two-thirds of Kenyans (about 37 million of 56.4 million) not using the internet.
  • Mobile subscriptions: 126.5 per 100 people in 2024. This is a count of subscriptions, not people. Many individuals hold multiple SIM cards, and some cards are inactive, so this figure does not show how many people have a phone.
  • Growth: The share of internet users has risen steadily. The context says the fastest absolute growth was in 2010–2015, but the data suggest otherwise. Internet use rose about 9.4 points in 2010–2015, but about 10.3 points in just four years, 2020–2024. The recent pace is therefore the fastest.

Overall, mobile access is widespread, but regular internet use is still limited to about a third of the population. The gap between the two indicators suggests that cost of data and devices, not basic network access, may be the main constraint.


====================================================================================================
With the instruction:
assistant · recorded output: nb01_ambiguous_with_instruction

"Connected" could mean a few different things here, so which do you mean?

  1. Internet use: the share of the population using the internet. This is the narrower measure of online access.
  2. Mobile subscriptions: mobile cellular subscriptions per 100 people. This measures subscriptions, not how many people have a phone, since the figure can exceed 100 and counts inactive SIMs.
  3. Both together: a comparison of the two, which the context suggests is a gap where mobile coverage has grown faster than internet use.

Which one would you like me to answer? If you want the comparison, I can give the 2010 to 2024 figures for both indicators.

With the basic instructions, the model answers at length, covers two readings, and adds a growth analysis the question did not ask for. With the clarifying instruction, it names three readings and asks. An answer to the model's own reading looks as confident as an answer to the user's reading, so the user cannot tell them apart. For a research application, asking costs one exchange and removes that risk.

6. The prompt as sent to the model​

prompt.format_messages fills the template with values and returns the message list without calling the model. The next cell does this for the last call of section 5 and draws the messages in the colours of section 1. The instructions arrive as the system message. The context and the question arrive together in one user message. This is everything the model sees.

messages = prompt.format_messages(instructions=INSTRUCTIONS_CLARIFY, context=brief, question="How connected is Kenya?")
w.show_prompt(messages)
Instructions · the system message, 354 characters
You are a research assistant for development data. Answer only from the supplied context. Cite the document id in square brackets after each figure. If the context does not contain what is needed, say so and give the closest available information. If the question is ambiguous, say which readings are possible and ask which one is meant before answering.
Context · 1,317 characters, the first lines shown
[kenya-digital] # Kenya: internet use and mobile subscriptions, 2010 to 2024 ## Internet use The share of individuals using the internet in Kenya rose from 7.2 percent of the population in 2010 to 16.6 percent in 2015 and 24.7 percent in 2020. Growth continued after 2020: 29.4 percent in 2022, 32.1 percent in 2023, and 35.0 …
Question · from the user
How connected is Kenya?

7. The cost of context​

A model reads a limited amount of text per call, called its context window and measured in tokens. A token is a word or a part of a word, about four characters of English on average. The four briefs took about a thousand tokens. A collection of thousands of documents exceeds any window, the price of a call grows with its tokens, and the passages that answer one question are a small part of the whole. An application therefore selects, for each question, the part of the collection that the model should read. Notebooks 02 and 03 build that selection.

Diagram: the four briefs of this notebook, about 1,024 tokens, fit inside the model's context window, while a real collection of thousands of documents cannot, so the application must choose what the model reads.

The figure of about 1,024 tokens in the diagram is the size that section 3 printed for the four briefs.

Key points​

  • A prompt has three parts: instructions, context, and the question. A different person writes each, so they are kept in separate slots.
  • Without context, the figures are reconstructions from training data. Four of the six checked were close, two were off, and none could be traced to a record.
  • With the briefs as context, every figure cited a brief. The application chose the evidence, and the model wrote from it.
  • Instructions set behaviour: cite, say when information is missing, ask when a question is ambiguous. They cannot supply data.
  • The context window limits what one call can read, so the application has to select the evidence. The next two notebooks build the selection.

Exercise​

  1. Read the answers of sections 2 and 3 side by side. For each number, note whether it can be checked and where. Then decide which answer could be sent to a colleague, and give the reason.
  2. In the first cell of section 1, add a sentence to INSTRUCTIONS_GROUNDED, for example "Answer in no more than three sentences." Run that cell again, then run the cell of section 6. The Instructions block now shows the added sentence exactly as the model would receive it. The recorded answers do not react to new instructions. With a live key in WORKSHOP_API_KEY, rerun section 3 to see how the answer changes.

Where this connects to the program​

Next: 02 · Preparing knowledge for an application