AI coursesMain page
Back to the library

Agents and workflows: from one model call to a system

What makes a system an agent, and three workflow patterns built in plain Python: prompt chaining, routing and parallelization.

On this page
  1. What one model call can and cannot do
  2. Agents, workflows and who decides
  3. The stand-in model and the tiny web
  4. Prompt chaining
  5. Routing
  6. Parallelization
  7. Choosing and combining workflow patterns
  8. Failure modes and evaluation
  9. Key takeaways
  10. Summary
  11. Notes
  12. Problems
  13. References

Parts 1 and 2 built up to the transformer (article 12): a model trained to continue text, one token at a time. Used on its own, such a model is a function from text to text. You send it a prompt and it returns a reply, and that is all it can do. It cannot look anything up, it cannot check its answer, and it cannot act in the world. Most useful applications of language models wrap this one call inside a larger program that gathers information, calls the model several times, checks what comes back and decides what to do next. Part 3 is about how such programs are designed. The recurring designs have names, agentic design patterns, and this part follows the patterns of Antonio Gulli's book of that title [1]. This article sets out what makes a system an agent and introduces the question that separates the patterns: who decides the next step, the program or the model. We then build the three simplest patterns in plain Python on one running example: prompt chaining, routing and parallelization. In all three, the program decides. Articles 14–18 hand more and more of that decision to the model.

1. What one model call can and cannot do

A chat model is called with two pieces of text. The system prompt sets the model's role and rules ("You are a research assistant. Answer with citations."). The user message holds the request. The model returns one reply. Nothing persists between calls: if a program calls the model twice, the second call knows about the first only if the program pastes the first reply into the second prompt. The model's knowledge is whatever was in its training data, frozen at the date training stopped.

The running example of this part is a research question about a fictional city, Larkford:

Did Larkford's bike share reduce car trips in the city centre?

The question is small, but it has the shape of real research. The answer depends on documents the model has never seen. The obvious number, a 6% fall in car traffic, is misleading. And a good answer must say where each claim comes from. Here is what one call returns when asked for a short report with citations:

Yes. Larkford's bike share cut city-centre car trips by about 20% in its first year [1].
[1] Larkford Transport Authority (2024). Bike Share Impact Report.

Both the figure and the source are invented. Larkford has no "Transport Authority", and no such report exists. In this course the reply is scripted, as section 3 explains, but the failure is a real and well-known one. A language model is trained to produce plausible text, and a confident answer with a citation is plausible text. Without access to sources, the model has nothing to be faithful to. More careful prompting does not fix this, because the information is not in the model. The system has to fetch the sources and put them in front of the model.

2. Agents, workflows and who decides

The classic textbook definition says that an agent is anything that perceives its environment through sensors and acts on it through actuators [2]. For language-model systems, Gulli describes the same idea as a loop [1]:

  1. get a goal;
  2. gather the information it needs;
  3. decide on a plan;
  4. act;
  5. observe the result and adapt.

A thermostat fits this definition, and so does a chess program. What is new with language models is that the deciding step can be done by a model reading text, so one system can handle goals and situations nobody programmed in advance.

There are many ways to place a model inside a loop. Practitioners at Anthropic proposed an architectural distinction that is now widely used [3]:

  • A workflow is a system in which language models and tools are combined along code paths fixed in advance. The program decides which steps run and in which order, and the model fills in the content of each step.
  • An agent is a system in which the model directs its own process. On each turn it chooses the next action (which tool to call, with which arguments, or whether to stop), sees the result, and chooses again.

Figure 13.1 shows the difference. In a workflow, the number of model calls is known before the run starts. In an agent, it depends on what the model sees along the way. The six articles of this part move along this axis. This article covers workflows. Article 14 gives the model tools and the choice of which one to call. Articles 15–17 add checking, planning and memory, and article 18 lets several agents work together.

Two diagrams side by side. a) A workflow: a question goes to a search tool, then to a blue model box labelled take notes, one call per source, then to a model box labelled write report, which produces the report. A caption reads: same three steps for every question; the model fills in each step but never chooses one. b) An agent: a goal enters a model box labelled model decides next action. An arrow labelled action leads down to an orange box, act: search, fetch, calculate. A dashed orange arrow labelled observation leads back up to the model. An arrow labelled done leads from the model to the answer. A caption reads: the number and order of steps depend on what the model sees after each action.

Figure 13.1 Workflow and agent. Blue rounded boxes are calls to the language model, orange boxes are tools that touch the outside world, and white square boxes are ordinary code. a) In a workflow the program fixes the sequence: for every question it searches, takes notes on each source and writes a report. b) In an agent the model sits inside a loop. It picks an action, the program executes it and returns the result as an observation, and the model picks again until it decides it is done.

Gulli also sorts agents into four levels of capability [1]:

  • Level 0 is a model on its own, answering from its training data. That is the single call of section 1.
  • Level 1 connects the model to tools such as search or a database, so it can use current information.
  • Level 2 adds planning across several steps. It also adds context engineering: deciding what information goes into each call.
  • Level 3 is a team of specialized agents working together.

Two research frameworks offer vocabulary for the parts of an agent. A survey by Wang and colleagues splits an agent into a profile, memory, planning and action [5]. The CoALA framework of Sumers and colleagues describes a language agent as a model plus memory modules, plus a set of actions: external actions that affect the world, and internal actions that read or write memory. These are tied together by a repeated decision cycle [4]. The patterns in this part are concrete ways of building those parts.

A practical rule follows from the trade-off. Every extra step costs model calls, money and time, and every decision handed to the model is one more place where it can go wrong. The advice from practice is to use the simplest design that does the job: a single call where it suffices, a workflow where the steps are predictable, and an agent only where they are not [3].

3. The stand-in model and the tiny web

The examples in this part are plain Python with no frameworks. To make every run reproducible, so that what you see here is exactly what the code prints, they call a stand-in for the model:

def llm(system: str, user: str) -> str:
    """Stand-in for a chat-model call: same inputs and output as a real one."""

llm() has the same inputs and output as a real chat call, but its replies come from a script. Each example registers replies with script(role, reply, when=...). A reply is used when the system prompt contains role and the user message contains when. The script contains only replies a real model could plausibly give. When an example needs the model to make a mistake, as in section 1, the script makes that mistake on purpose and the text says so.

To run the examples against a real model, replace the body of llm() with an API call. With Anthropic's Python SDK, as of October 2026, it looks like this. Other providers' chat APIs have the same shape.

import anthropic

client = anthropic.Anthropic()  # reads the API key from the environment


def llm(system: str, user: str) -> str:
    response = client.messages.create(
        model="claude-opus-5-5",
        max_tokens=16000,
        system=system,
        messages=[{"role": "user", "content": user}],
    )
    return "".join(b.text for b in response.content if b.type == "text")

The "web" is seven short pages about Larkford, all fictional:

  • the city's annual review of the bike share;
  • a news story;
  • a university study;
  • an enthusiast's blog;
  • an old consultation document;
  • a page announcing a new tram line;
  • a notice of higher parking charges.

search(query, k) ranks the pages by how many words of the query they contain and returns the top k. fetch(url) returns a page's text. Both are ordinary Python functions, so their results are real.

Two costs are counted rather than measured, so that they come out the same on every run. Tokens are estimated as characters divided by four, a common rule of thumb for English text. Latency is simulated as 0.5 seconds per call plus 50 output tokens per second, which is roughly the speed of a mid-sized hosted model.

4. Prompt chaining

The problem. We could put everything into one prompt: paste the three most relevant pages after the question and ask for a cited report. That works for three short pages, but it scales badly. As documents grow and instructions pile up, a single prompt asks the model to do several jobs at once: read, select, reconcile and write. Gulli lists what goes wrong [1]:

  • instructions get neglected;
  • the model loses track of the original context;
  • early mistakes carry through to the end;
  • more of what it writes is invented.

A single prompt also gives the program no point at which it can check intermediate results.

The pattern. Prompt chaining (also called the pipeline pattern) breaks a task into a fixed sequence of steps, and feeds the output of each step into the next [1]. A chain is defined by:

  1. Steps: each step is a model call with a narrow prompt, or ordinary code.
  2. Hand-offs: the output of one step, preferably in a structured format such as JSON, becomes part of the next step's input.
  3. Gates: checks in code between steps. A gate can validate a format, a count or a rule, and stop or retry when the check fails.

Our chain has three steps. The first is code, not a model call: search for the question and keep the top three pages. The second takes notes on each page separately, asking for JSON. The third writes the report from the notes alone.

def research_chain(question):
    urls = [hit["url"] for hit in search(question, k=3)]  # step 1: code, not a model
    notes = {url: take_notes(question, url) for url in urls}  # step 2: one call each
    return write_report(question, notes)  # step 3: one call

The note-taking step gives the model one page and one job:

def take_notes(question, url):
    reply = llm(
        "You are a note-taker. Return JSON: "
        '{"claims": [...]} with the facts relevant to the question.',
        f"Question: {question}\nSource {url}:\n{fetch(url)}",
    )
    claims = json.loads(reply)["claims"]  # gate: must be valid JSON
    return [c for c in claims if c.strip()]

Asking for JSON is what makes the hand-off reliable. The next step receives a list of claims, not free text that it would have to interpret. json.loads is the first gate: if the model returns anything other than valid JSON, the chain stops with an error at this step, instead of passing garbage on. A production chain would retry once with the error message included in the prompt.

The writing step sees only the numbered notes, never the raw pages:

def write_report(question, notes):
    sources = "\n".join(
        f"[{i}] " + " ".join(claims) for i, claims in enumerate(notes.values(), 1)
    )
    report = llm(
        "You are a report writer. Use only the numbered notes. "
        "Cite every sentence with [n].",
        f"Question: {question}\nNotes:\n{sources}",
    )
    # gate: every sentence cites a source, and only sources that exist
    for sentence in re.split(r"(?<=\.)\s+", report):
        cited = {int(n) for n in re.findall(r"\[(\d+)\]", sentence)}
        assert cited, f"uncited sentence: {sentence}"
        assert cited <= set(range(1, len(notes) + 1)), f"unknown source: {sentence}"
    return report

The second gate checks two things. Every sentence must cite something, and it may only cite sources that exist. That rules out the invented "Transport Authority" of section 1: a citation to a source that was never retrieved fails the check. The gate cannot tell whether a cited source actually supports the sentence. That needs a model or a person to read the source, which is the subject of article 15.

Running the chain prints:

City-centre car trips fell by 6% between 2022 and 2024 [2], but tram Line 2 opened and parking charges rose by 40% over the same period [3]. A comparison with similar districts attributes about 2.5% of the fall to the bike share (95% CI 0.8% to 4.2%) [1]. So the bike share probably did reduce car trips, but by much less than the 6% headline figure [1][2].
  note-taker    200 tokens in   56 out
  note-taker    203 tokens in   55 out
  note-taker    150 tokens in   29 out
  report writer  167 tokens in   90 out
4 model calls, 720 tokens in, 230 tokens out

Compare this with section 1. Every number now comes from a page that exists:

  • the 6% fall comes from the city's review;
  • the tram line and the parking charges come from the news story;
  • the 2.5% estimate, with its confidence interval, comes from the university study.

The report also does what the single call could not do. It separates the headline fall from the part of it that the comparison study attributes to the bike share.

The cost is four model calls instead of one, about 950 tokens in total. The cost lines also show something useful: the writer's prompt (167 tokens) is smaller than any note-taker's prompt, because the writer sees the notes, not the pages. Choosing what each step sees is what Gulli calls context engineering [1]. A model given only the information its step needs is less likely to be distracted by the rest.

Where chains fit. Chains suit any task whose stages are known in advance:

  • Documents. Extract fields from an invoice, validate them in code, then ask again for only the fields that failed. This is a chain with a gate and a retry.
  • Writing. Outline, then draft each section with the previous one in context, then edit. Each stage can use a different prompt or even a different model.
  • Code. Write a function, run its tests in code, and pass the failures to a repair step. This is a chain in which one step is a tool, and it leads directly to the reflection pattern of article 15.

At a glance: prompt chaining

What: A task with several distinct stages overloads a single prompt. Instructions get missed, early errors carry through, and there is nowhere to check intermediate results.

Why: A chain gives each stage its own focused prompt and passes structured output from stage to stage. Gates in code check each hand-off.

Rule of thumb: Use a chain when the stages are known in advance and each can be described in one sentence. Do not split a task that one call already does well. Every step adds latency and one more place to fail.

5. Routing

The problem. A chain does the same thing for every input. Real requests differ. A question that needs a cited report should not be handled the same way as a request for a single number, and a request outside the system's purpose should not be researched at all. Running the full chain on every request wastes calls and gives worse answers.

The pattern. Routing adds a decision at the start, or at any point in a workflow. A router classifies the input, and the program sends it down the matching path [1]. The router can be any of the following:

  • A model call that returns a label. This is the most flexible, and it costs one call.
  • Embedding similarity: the input is compared, as a vector, with descriptions of each route. This is fast and cheap. Article 17 covers embeddings.
  • Rules: keywords or regular expressions. These are deterministic and free, but brittle on inputs nobody anticipated.
  • A small trained classifier. This is accurate on its training distribution, but it needs labelled examples.

Our router is a model call with three labels:

ROUTES = {"report", "fact", "out_of_scope"}


def route(request):
    label = llm(
        "You are a router. Reply with one word: report (needs several sources), "
        "fact (one number or name), or out_of_scope.",
        request,
    ).strip()
    return label if label in ROUTES else "out_of_scope"  # gate: unknown labels


def handle(request):
    label = route(request)
    if label == "report":
        return label, research_chain(request)
    if label == "fact":
        url = search(request, k=1)[0]["url"]
        return label, llm(
            "You are a fact finder. Answer in one sentence from the source.",
            f"{request}\nSource [1] {url}:\n{fetch(url)}",
        )
    return label, "Sorry, I can only research questions about Larkford's transport."

Notice the gate in route. A model asked for "one word" sometimes returns a sentence, a different word or a label in capitals. The program never trusts the label blindly: anything outside the known set goes to the safest route. The decision belongs to the model, but the set of possible paths and the fallback are fixed in code. That is what keeps routing a workflow pattern.

Three requests take three paths:

Did Larkford's bike share reduce car trips in the city centre?
  -> report, 5 call(s): City-centre car trips fell by 6% between 2022 and 2024 [2], but tr ...
How many bike-share stations does Larkford have?
  -> fact, 2 call(s): Larkford Bike Share had 120 stations in December 2024 [1].
Write me a poem about bicycles.
  -> out_of_scope, 1 call(s): Sorry, I can only research questions about Larkford's transport.

The number of calls follows the label. The report costs the router call plus the four calls of the chain. The fact costs two calls, and the refusal one. Routing pays for itself when cheap requests are common.

Where routing fits.

  • Support systems. A support system routes tickets to billing, technical help or a human.
  • Model routing. A cost-conscious system routes easy requests to a small, cheap model and hard ones to a large model.
  • Tool selection. A coding assistant routes a snippet by language and intent: explain, debug or translate.

In each case the routes are known in advance. If the set of possible paths cannot be listed, routing is the wrong tool, and the model needs to choose actions freely, as in article 14.

At a glance: routing

What: Different inputs need different handling, and a fixed chain treats them all alike. That wastes calls on simple requests and handles some requests badly.

Why: A router classifies each input with a model call, embeddings, rules or a classifier, and the program sends it down the matching path. Unknown labels go to a safe fallback.

Rule of thumb: Use routing when inputs fall into a few categories that need different handling, and you can list the categories. Measure the router on labelled examples (section 8). A misrouted request fails no matter how good the path behind it is.

6. Parallelization

The problem. In the chain of section 4, the three note-taking calls run one after another, but none of them needs the others' results. Waiting for each in turn adds their latencies together.

The pattern. Parallelization runs independent steps at the same time and joins their results when all have finished [1]. Two variants are common [3]:

  • Sectioning: split a task into independent parts and handle each separately, as with our three sources.
  • Voting: run the same step several times, possibly with different prompts, and combine the answers by majority or by taking the best one.

Voting goes back to self-consistency, which samples several reasoning paths for a question and takes the most common final answer. On arithmetic and commonsense reasoning benchmarks it improved accuracy over a single path [7].

In Python, a thread pool runs the three note-taking calls concurrently. The program waits for all three before writing:

def research_parallel(question):
    urls = [hit["url"] for hit in search(question, k=3)]
    with ThreadPoolExecutor() as pool:
        notes = dict(zip(urls, pool.map(lambda u: take_notes(question, u), urls)))
    return write_report(question, notes)

The report is identical to section 4, and so are the token counts. Parallelization changes when the calls happen, not what they cost. What changes is the wall-clock time. With the simulated latencies of section 3:

note-taking calls (s): 1.62, 1.60, 1.08
report writer (s): 2.30
sequential: 6.60 s; parallel: 3.92 s

Run in sequence, the calls take the sum of their latencies: 1.62+1.60+1.08+2.30=6.601.62 + 1.60 + 1.08 + 2.30 = 6.60 seconds. Run in parallel, the note-taking stage takes as long as its slowest call. The total is then 1.62+2.30=3.921.62 + 2.30 = 3.92 seconds, 41% less. The general rule is that a parallel stage costs its maximum, not its sum. So the more independent calls a stage has, the more parallelism saves, but only up to the slowest call. The writer still waits.

Where parallelization fits.

  • Gathering information from several sources at once: news, financial data and a company database for one company.
  • Running independent checks on the same input: format, policy and facts. Running them in parallel is faster than in sequence and gives the same answers.
  • Generating several candidates to pick from, which is the voting variant.

The limits are practical. API rate limits cap how many calls can run at once. Merging several outputs needs its own step, here the writer. And when one of several parallel calls fails, the program must decide whether to continue without that result, retry it or stop. Article 16 returns to that decision.

At a glance: parallelization

What: Independent steps run one after another, so the user waits for the sum of their latencies.

Why: Running independent steps concurrently cuts a stage's wall-clock time to its slowest step. Voting runs the same step several times to make the answer more reliable.

Rule of thumb: Parallelize when steps do not depend on each other's outputs and latency matters. It does not reduce token cost. Voting multiplies it.

7. Choosing and combining workflow patterns

The three patterns combine freely. In section 5 the router's "report" path is the chain of section 4, and that chain could use the parallel note-taking of section 6. Figure 13.2 shows the three building blocks side by side, and table 13.1 compares them on the running example.

Three diagrams. a) Prompt chain: input goes to step 1, then to a white gate box asking valid JSON?, then on pass to step 2 and to the output; a fail arrow from the gate leads to retry or stop. b) Routing: a request goes to a router, which sends it along one of three arrows to a report chain, a fact lookup, or a polite refusal box. c) Parallelization: sources fan out to three model boxes, notes on source 1, 2 and 3, which all feed a write report box; a note says the three calls run at the same time and the writer waits for all of them.

Figure 13.2 The three workflow patterns. a) A chain runs steps in a fixed order; a gate in code checks each hand-off and stops or retries when the check fails. b) A router sends each request down one of a fixed set of paths. c) Parallelization runs independent calls at the same time, and the next step waits for all of them. In all three, the program decides which steps run; the model only fills them in.

Table 13.1 The three workflow patterns on the running example.

Pattern Who decides the path Model calls per question Main benefit Main cost
One call — 1 cheapest no sources; invents facts
Prompt chain the program 4 grounded, checkable steps more calls and latency
Routing + chain the router's label, within fixed paths 1–5 cheap requests stay cheap misrouting
Parallel chain the program 4 lower latency (3.92 s vs 6.60 s) rate limits; merging

None of these systems is an agent in the sense of section 2. The number of steps is fixed before the run, or chosen from a fixed list. That is a strength. Every path can be tested, every cost bounded and every failure traced to a step. Workflows fall short when the right next step depends on what has just been found: a source that contradicts another, a page that fails to load, or a question that turns out to need a calculation. Then the model has to choose actions. Article 14 lets it.

8. Failure modes and evaluation

Chains propagate errors. Each step trusts its input. If a note-taker misreads "2.5%" as "25%", the writer will repeat the error faithfully, with a citation. Gates catch errors of form, such as invalid JSON, missing citations or unknown sources. They do not catch errors of content. Content errors need either a check against the source (article 15) or tests on the final output (below).

Routers misroute. A router is a classifier, and classifiers make mistakes, especially on inputs near the boundary between routes. "Roughly how much did ridership grow?" could be a fact or a report. Two defences help:

  • a fallback for labels the router does not recognise;
  • measuring the router like any classifier: on a few dozen labelled requests, count how often it picks the right route, and which wrong routes it picks. That is the confusion matrix of article 6.

Parallel steps fail partially. With three concurrent calls, one may time out. The program has to choose what to do, and an unexamined default, usually an exception that discards the other two results, is rarely the right choice.

Evaluating a workflow means comparing it with something simpler on the same questions. For a research assistant:

  1. Write a test set of questions with the facts a good answer must contain and the claims it must not make. Example: "must mention the tram; must not attribute the whole 6% to the bike share".
  2. Score each system's answers on those facts, by hand or with a model given the checklist, and check every citation against its source.
  3. Record the cost per question: model calls, tokens and seconds.

A design earns its extra calls only if it scores clearly better than the single call. As with any model evaluation (article 6), the test questions must not be the ones used to write the prompts. Article 18 describes public benchmarks for agents.

Key takeaways

  • A language model call maps text to text. Anything beyond that, such as sources, checks, actions or memory, is supplied by the program around it.
  • The central design question is who decides the next step. In a workflow the program decides, and the model fills in steps. In an agent the model chooses actions in a loop.
  • Prompt chaining splits a task into focused steps with structured hand-offs. Gates between steps catch errors of form, not errors of content.
  • Routing sends each input down one of a fixed set of paths, chosen by a model, embeddings, rules or a classifier. Its weak point is misclassification, which can be measured.
  • Parallelization cuts a stage's latency from the sum of its calls to the slowest call, at the same token cost. Voting trades extra calls for reliability.
  • Start with the simplest design that works, and add steps only when an evaluation against the simpler design shows they help.

Summary

A single model call can only produce text from what it was trained on, so asked about documents it has not seen, it invents them fluently. An agent, in the classic sense, perceives and acts in a loop. For language-model systems, the useful distinction is between workflows, where code fixes the steps, and agents, where the model chooses them. This article built three workflow patterns on a research question about a fictional city. A prompt chain searched, took structured notes on each source and wrote a report, with gates in code checking each hand-off. It turned an invented answer into a cited one at the cost of four calls. A router sent each request down one of three paths, so cheap requests stayed cheap. Parallelization ran independent note-taking calls at the same time, cutting simulated latency from 6.60 to 3.92 seconds without changing the token cost. In all three, every path could be listed and tested in advance. That is their strength, and their limit when the next step depends on what has just been found.

Notes

Terminology. "Agent" is used loosely in industry, often for any system with a model in it. The workflow–agent distinction used here follows Anthropic's engineering guidance [3]. That guidance also describes two more workflow patterns: orchestrator–workers, where a model splits a task and hands the parts to workers (article 18), and evaluator–optimizer, where one call critiques another's output (article 15). Gulli's book treats prompt chaining, routing and parallelization in its chapters 1–3 [1], and uses "agentic" for the whole spectrum. Andrew Ng's series on agentic design patterns [8] groups the field into four patterns: reflection, tool use, planning and multi-agent collaboration. These are articles 15, 14, 16 and 18 of this part.

Architectures. Two surveys cover the research literature up to 2024–2025: Wang and colleagues [5] organise agents by profile, memory, planning and action, and Xi and colleagues [6] by brain, perception and action. CoALA [4] connects language agents to earlier cognitive architectures such as Soar. Its distinction between working memory and long-term episodic, semantic and procedural memory returns in article 17.

Frameworks. As of 2026, all mainstream agent frameworks provide these workflow patterns as building blocks. LangGraph expresses them as a graph of nodes with conditional edges, so a chain is a path and a router is a conditional edge. Google's Agent Development Kit has sequential and parallel agents. The underlying ideas are the ones in this article, and the code here is small enough that a framework is optional for workflows of this size. Gulli's book gives full examples in LangChain and the ADK [1].

Structured output. Many model APIs can now constrain a reply to a JSON schema, which makes the JSON gate of section 4 fail far less often. The gate is still worth keeping, because a valid schema does not guarantee sensible content.

Problems

Problem 13.1 Classify each system as a workflow or an agent, and say who decides the next step:

  • (a) a translation service that detects the language and then calls one of five translation prompts;
  • (b) a coding assistant that reads an error, decides which file to open, edits it, reruns the tests and repeats until they pass;
  • (c) a newsletter generator that summarises ten articles in parallel and then writes an introduction.

(Answer: (a) a workflow with routing; the program fixes the paths. (b) An agent; the model chooses each action from what it observes. (c) A workflow with parallelization and a chain.)

Problem 13.2 In section 6, suppose the slowest note-taking call takes 4.00 seconds instead of 1.62, and the others are unchanged. What are the sequential and parallel times? (Answer: sequential 4.00+1.60+1.08+2.30=8.984.00 + 1.60 + 1.08 + 2.30 = 8.98 s; parallel 4.00+2.30=6.304.00 + 2.30 = 6.30 s. The parallel stage costs its slowest call.)

Problem 13.3 The citation gate of section 4 accepts the sentence Car trips fell by 25% [2]. Why, and what would catch the error? (Answer: the gate checks only that a citation is present and refers to an existing source, not that the source supports the claim. Comparing the claim with source 2, by a person or a second model call with the source in its prompt, would catch it; see article 15.)

Problem 13.4 Design a rule-based router for the three routes of section 5 using only keywords. Give one request that your rules would misroute. Why is a model-based router more robust on such requests, and what does it cost? (Example: send requests containing "how many" to fact. "How many reasons are there to doubt the 6% figure?" needs a report. A model reads the whole request rather than one phrase, at the cost of one call per request.)

Problem 13.5 Voting runs the same step nn times and takes the majority. If each run is correct independently with probability 0.7, what is the probability that the majority of three runs is correct? Why might real runs of the same model not be independent? (Answer: 0.73+3×0.72×0.3=0.7840.7^3 + 3 \times 0.7^2 \times 0.3 = 0.784. Runs share the same model and prompt, so they tend to make the same mistakes; the real gain is usually smaller.)

References