Planning: deciding the steps before taking them

The tool-using agent of article 14 decides one step at a time. After each observation it asks the model "what next?", which makes it adaptable but short-sighted. It cannot see in advance that a task needs four sources, it pays for a model call at every step, and nothing stops it from wandering. This article puts a plan in front of the work. A model first writes the steps needed to reach a goal, ordinary code carries them out, and the model is called again only when something goes wrong. Planning brings two companions that the book treats in separate chapters [1]. Goal setting and monitoring defines up front what a finished result must contain, and checks it. Exception handling and recovery decides what to do when a step fails: retry, replan or hand the problem to a person. We build all three on the running example, with one timeout and one invented URL. We then look at what research says about reasoning before acting (chain of thought, Plan-and-Solve, Tree of Thoughts) and about when a model should hand planning to a classical algorithm.

1. One step at a time, or a plan first

Gulli defines planning as formulating a sequence of actions that moves from an initial state to a goal state [1]. An agent can produce that sequence in two ways:

• Step by step (ReAct, article 14). The model chooses each action after seeing the previous result. It adapts immediately to surprises, but it makes one model call per step and never commits to an overall approach.
• Plan and execute. The model writes the whole plan in one call. Code executes it step by step, and the model is consulted again only to replan when a step fails or the situation changes. This needs fewer model calls, and the plan can be inspected, approved or tested before anything runs. The plan can also be wrong, and it stays wrong until something triggers a replan.

The book's test for whether to plan dynamically at all is a single question: does the how need to be discovered, or is it already known [1]? If it is known, as in the research chain of article 13, a fixed workflow is more predictable than any plan a model writes. Planning is for goals whose steps depend on the request.

2. Goals you can check

A plan needs a goal, and a goal is only useful if you can tell whether it has been met. Gulli's chapter on goal setting and monitoring asks for explicit, measurable objectives and for a mechanism that tracks progress against them [1]. For the research report, "a good report" is not checkable. These four requirements are:

GOALS = {
    "states the measured change in car trips": lambda r: "6%" in r,
    "mentions other changes in the same period": lambda r: "tram" in r.lower(),
    "gives the bike share's own estimated effect": lambda r: "2.5%" in r,
    "cites a source in every sentence": lambda r: all(
        "[" in s for s in r.split(". ") if s.strip()),
}

Each goal has a name, which goes into the planner's prompt so that the plan aims at it, and a test in code, which checks the result at the end. The tests are crude, since a substring is not understanding. They are deliberately cheap and deterministic, and they catch the failures that matter most here: a report that leaves out the confounding tram line or the study's estimate. Where a goal cannot be checked in code, a model judge can check it (article 15), with the caveats discussed there. What matters is that the criteria are written down before the work starts. A goal stated afterwards tends to be whatever the agent happened to produce.

3. A plan and its execution

The planner is one model call. It sees the tools, the goal and the requirements, and returns the steps as JSON:

def make_plan(goal):
    return json.loads(llm(
        "You are a planner. Tools: search(query), fetch(url), read_top(query). "
        'Return a JSON list of steps: {"tool", "args", "why"}.',
        f"Make a plan for: {goal}\nThe report must: {'; '.join(GOALS)}"))

The executor is code. It takes steps off the front of the plan, runs each, keeps the results as evidence, and finally asks a writer for the report and checks the goals:

def plan_and_execute(goal, max_replans=2):
    plan, done, evidence = make_plan(goal), [], []
    print(f"plan ({len(plan)} steps):")
    for i, step in enumerate(plan, 1):
        print(f"  {i}. {step['tool']} {json.dumps(step['args'])}  # {step['why']}")
    replans = 0
    while plan:
        step = plan.pop(0)
        print(f"run {step['tool']} {json.dumps(step['args'])}")
        try:
            result = run_step(step)
        except ToolError as error:
            print(f"    failed: {error}")
            if replans == max_replans:
                return f"Escalated to a human after {replans} replans: {error}"
            replans += 1
            plan = replan(goal, done, step, str(error)) + plan
            print(f"    replanned: {json.dumps(plan[0])}")
            continue
        print(f"    ok: {preview(result, 50)}")
        done.append(step)
        if step["tool"] != "search":
            evidence.append(result)
    sources = "\n".join(f"[{i}] {text}" for i, text in enumerate(evidence, 1))
    report = llm("You are a report writer. Cite every sentence with [n].",
                 f"Question: {goal}\nSources:\n{sources}")
    unmet = [name for name, test in GOALS.items() if not test(report)]
    return report if not unmet else f"Goals not met ({', '.join(unmet)}): {report}"

To see recovery at work, the run below has two faults. The university's server times out once (a transient failure). And the planner asks for a page about the tram at a URL it made up, a common planning error, since a model writing a plan has not yet seen the web:

plan (4 steps):
  1. search {"query": "Did Larkford's bike share reduce car trips in the city centre?"}  # find sources
  2. fetch {"url": "transport.larkford-uni.example/bike-share-car-trips"}  # the study that separates causes
  3. fetch {"url": "larkford.gov/transport/bike-share-review-2024"}  # the measured fall in car trips
  4. fetch {"url": "larkford.gov/transport/tram-line-2-opening"}  # when the tram opened
run search {"query": "Did Larkford's bike share reduce car trips in the city centre?"}
    ok: [{"url": "transport.larkford-uni.example/bike- ...
run fetch {"url": "transport.larkford-uni.example/bike-share-car-trips"}
    fetch transport.larkford-uni.example/bike-share-car-trips: timed out after 10 s; retry 1 after 1 s
    ok: Did bike share reduce car trips in central Lar ...
run fetch {"url": "larkford.gov/transport/bike-share-review-2024"}
    ok: Larkford Bike Share: 2024 annual review (2025- ...
run fetch {"url": "larkford.gov/transport/tram-line-2-opening"}
    failed: fetch larkford.gov/transport/tram-line-2-opening: 404 not found
    replanned: {"tool": "read_top", "args": {"query": "Larkford tram Line 2 opening"}, "why": "when the tram opened (the guessed URL does not exist)"}
run read_top {"query": "Larkford tram Line 2 opening"}
    ok: Tram Line 2 opens (2023-09-12) Tram Line 2 lin ...
Report: City-centre car trips fell by 6% between 2022 and 2024 [2]. Tram Line 2 opened in September 2023 [3], and a comparison with similar districts attributes about 2.5% of the fall to the bike share (95% CI 0.8% to 4.2%) [1].
  goal met: states the measured change in car trips
  goal met: mentions other changes in the same period
  goal met: gives the bike share's own estimated effect
  goal met: cites a source in every sentence
3 model calls, 664 tokens in, 215 tokens out

Read the plan before the run. It is sensible: one step per requirement, each with a reason. The step marked "the study that separates causes" exists because the planner saw the goal "gives the bike share's own estimated effect". The whole run took three model calls: the plan, one replan and the report. A step-by-step agent would have called the model before every action, at least six times here, and its transcript would have grown with each call.

4. When steps fail: exception handling and recovery

Agents meet failures that ordinary programs also meet, such as timeouts, missing pages and rate limits, plus some of their own: invented URLs, malformed tool arguments and output in the wrong format. Gulli's chapter on exception handling and recovery organizes the response into three stages [1]:

1. Detection: notice that something went wrong. Signals include error codes, timeouts, malformed output, and results that fail a check.
2. Handling: respond in the moment:
   • log the error;
   • retry transient failures;
   • fall back to an alternative;
   • degrade gracefully, delivering what is possible and saying what is missing;
   • notify someone.
3. Recovery: return to a sound state. That can mean undoing partial changes, diagnosing the cause, replanning, or escalating to a human.

The run shows three of these in miniature. The timeout is transient: the same request may well succeed a moment later. run_step retries it, and in a real system it would wait $1, 2, 4, \dots$ seconds between attempts (exponential backoff), so that a struggling server is not hammered:

def run_step(step, retries=2):
    for attempt in range(retries + 1):
        try:
            return TOOLS[step["tool"]](**step["args"])
        except ToolError as error:
            if RETRYABLE not in str(error) or attempt == retries:
                raise
            print(f"    {error}; retry {attempt + 1} after {2 ** attempt} s")
            # a real system would time.sleep(2 ** attempt) here: exponential backoff

The 404 is permanent: retrying an invented URL can never succeed. So the executor does not retry. It hands the failure, with what has been done so far, to a replanner, which is the planner prompt with the error added. The replanner replaces the bad step with a search for the tram page. Finally, a limit on replans, after which the task goes to a person, keeps a confused planner from looping indefinitely. Figure 16.1 shows the whole loop.

Figure 16.1 Plan, execute, monitor, recover. The planner writes the steps once, and code executes them. Transient errors such as timeouts are retried with growing waits. Permanent errors such as a missing page go to the replanner, which writes new remaining steps from what has been done so far. After a set number of replans, the task is escalated to a person. When the plan is finished, the report is written and checked against the goals set at the start.

One rule matters more than any other here. Only retry what is safe to repeat. Fetching a page twice is harmless. Sending an email or making a payment twice is not. Steps with side effects need either an idempotency key, so that the receiving service recognises a repeat and ignores it, or a check of whether the first attempt actually happened before trying again.

At a glance: exception handling and recovery

What: Tools time out, pages disappear, and models produce invalid steps. An agent that treats every failure alike either crashes or retries forever.

Why: Detect failures explicitly, retry transient ones with backoff, replan around permanent ones, degrade gracefully when part of the task is impossible, and escalate to a person when the limits are reached.

Rule of thumb: Classify every tool error as transient or permanent, and every tool as safe or unsafe to repeat. Put limits on retries and replans. Test recovery deliberately by injecting failures, as this example does.

5. Reasoning before acting

A plan is reasoning written down before acting. A line of research studies how prompting a model to reason first changes what it produces.

Chain of thought. Wei and colleagues showed that including worked examples with intermediate reasoning steps in the prompt makes large models write out their reasoning, and makes them far more accurate on multi-step problems [2]. On the GSM8K math word problems, PaLM-540B with eight such examples solved 56.9%, against 17.9% when the examples showed only answers.

Plan-and-Solve. Wang and colleagues replaced "Let's think step by step" with an instruction to "first understand the problem and devise a plan to solve the problem. Then, let's carry out the plan" [3]. With GPT-3 (text-davinci-003) and no examples, in the authors' analysis of sampled GSM8K problems, this cut the share with a missing reasoning step from 12% to 7%. Across six arithmetic datasets, it brought zero-shot accuracy close to that of hand-written examples (76.7% against 77.6%). The same instruction is the planner's job in miniature: write the plan, then follow it.

Tree of Thoughts. A chain explores one line of reasoning. Tree of Thoughts (ToT) [4] has the model propose several possible next steps ("thoughts"), score each one, keep the promising ones and backtrack from dead ends. It is a search over a tree of partial solutions, with the model acting both as the generator of steps and as the evaluator of states (figure 16.2b). On the Game of 24, where four numbers must be combined with $+, -, \times, \div$ to make 24, GPT-4 with chain of thought solved 4% of 100 hard games and ToT solved 74%. The search was not free: about 5,500 generated tokens per game, comparable to sampling 100 chains of thought. Taking the best of 100 independent chains solved only 49%. Search spends extra compute more effectively than independent sampling, but it still spends it.

Search with tools and reflection. Language Agent Tree Search (LATS) [5] combines these ideas with acting. It runs a tree search over ReAct-style trajectories, scores states with the model, and uses reflections on failed branches (article 15). Feedback comes from the environment. With GPT-4 on HumanEval, it reached 92.7% pass@1, at the cost of exploring many trajectories per problem.

Reasoning models. As of 2026, many models are trained to produce long internal reasoning before answering. They are trained with reinforcement learning on problems whose answers can be checked automatically, such as math and code [1]. These models plan, check and backtrack inside a single call, which absorbs some of what this section's prompting methods did by hand. The trade-off remains the one Tree of Thoughts made visible: more reasoning at inference time costs more tokens and more time, and can let a smaller model match a larger one on hard problems. Gulli calls this the scaling inference law [1]. Whether it pays depends on how hard the problem is and how much an error costs.

Figure 16.2 Three ways to reason before acting. a) Chain of thought writes one sequence of steps. b) Tree of thoughts proposes several next steps, scores them, prunes poor branches (red cross) and expands promising ones, backtracking when a branch fails. c) An LLM+P system uses the model only to translate: the model writes the problem in a formal planning language, a classical planner searches for a valid plan, and the model translates the plan back into words.

6. When the model should not plan alone

Models are fluent planners, but not reliable ones. A plan in natural language can look complete and still break a constraint: it may pick up a block that is underneath another one, or reuse a glass that was never washed. LLM+P [6] tested this on seven classical planning domains taken from the International Planning Competitions, with 20 tasks each:

• GPT-4 planning directly, in natural language, produced a plan for every task. Most of those plans could not be carried out, because the model failed to track preconditions. On Blocksworld, 15% of its plans were optimal and 30% valid at all.
• LLM+P used the model only to translate the problem into PDDL, the standard formal language for planning problems. A classical planner then searched for a plan that is guaranteed valid, and the model translated the result back into words. This found optimal plans for 90% of Blocksworld tasks, and for 85% of Storage tasks, where none of GPT-4's own plans were optimal and a quarter were valid. In one domain, Floortile, neither approach solved any task.

The lesson generalizes beyond robot planning. When a problem has a precise formal structure and a solver exists, let the model translate and let the solver solve. Examples: a SQL database for queries over records, a constraint solver for timetables, a routing engine for journeys. The model handles what it is good at, turning messy language into a formal request. The algorithm then guarantees the property the model cannot: validity.

7. Where planning fits

Research and report generation. "Deep research" products plan searches, read dozens of sources and synthesize a report, often showing the user the plan first. Gulli's planning chapter describes Google's and OpenAI's versions as of 2025 [1]. The plan is the place where a user can redirect the work cheaply, before hours of browsing.

Multi-step business processes. Employee onboarding and customer support cases that need diagnosis, action and follow-up have steps that vary with the case, but within a known space. A plan per case, checked against explicit completion criteria, makes the process auditable.

Coding tasks. A coding agent typically plans which files to read and change, executes the edits, runs the tests, and replans on failures. This combines planning with the test-driven reflection of article 15.

A poor fit. If every request follows the same steps, a workflow is cheaper and more predictable. If every step depends on the previous result, as in browsing an unfamiliar website, a fixed plan goes stale immediately and a step-by-step agent is the better design. Plans pay off in between: tasks that need several steps, whose shape can be foreseen once the request is known.

At a glance: planning

What: A goal needs several interdependent steps, which differ from request to request. Deciding one step at a time is short-sighted and calls the model at every step. A fixed workflow cannot cover the variety.

Why: A planner turns the goal into explicit steps aimed at checkable criteria. Code executes them, and the model is called again only to replan when a step fails or the situation changes.

Rule of thumb: Plan when the steps depend on the request but can be foreseen once it is known. Use a workflow when they are always the same, and step-by-step acting when each depends on the last result. Hand formally structured problems to a solver.

8. Failure modes and evaluation

Impossible steps. Plans name tools that do not exist, URLs that were never seen, or arguments of the wrong type, as in the invented tram URL. Validate a plan before executing it: do the tools exist, and do the arguments match their schemas? Prefer steps that discover things, such as searching, to steps that assume them, such as fetching a guessed URL.

Stale plans. A plan written before the first observation can become wrong without any step failing. An example: a source contradicts the premise of a later step. Monitoring only for errors misses this. Checking intermediate results against the goal, or replanning at milestones, catches more.

Replanning loops. A replanner that keeps proposing variations of a failing step burns calls without progress. Limit replans, and pass the history of failures into the replanner's prompt so that it does not repeat them.

Unsafe retries. Retrying steps with side effects can duplicate them (section 4).

Weak goal checks. Goals checked by crude tests can be met in letter and missed in spirit. A report could contain "2.5%" in the wrong sentence. Use stronger checks where the stakes justify them, and review samples by hand.

Evaluating a planning agent:

1. Task success against the goal criteria, compared with a step-by-step agent and with a fixed workflow.
2. Plan quality: the share of plans that are valid before execution, and how many replans runs need.
3. Robustness: inject failures deliberately (timeouts, missing pages, malformed tool output) and measure how often the agent recovers, degrades gracefully or escalates appropriately. This fault injection is the only way to test recovery code that rarely runs in normal use.
4. Cost: model calls, tokens and wall-clock time per task. Planning's advantage over step-by-step acting is largely here.

The planning survey of Huang and colleagues [7] organizes the research into task decomposition, plan selection, external planners, reflection and memory, and reviews the benchmarks used for each.

Key takeaways

• Plan-and-execute separates deciding from doing. One model call writes the steps, code runs them, and the model is called again only to replan. That makes runs cheaper and plans inspectable.
• Write goals as checkable criteria before the work starts, and check the result against them at the end.
• Classify failures. Retry transient ones with backoff, replan around permanent ones, limit both, and escalate. Never blindly retry steps with side effects.
• Reasoning before acting helps: chain of thought raised PaLM's GSM8K accuracy from 17.9% to 56.9%. Tree search helps more on problems that need exploration (Game of 24: 4% to 74%), at a cost in tokens.
• Natural-language plans can break constraints the model does not track. When a formal solver exists, let the model translate and the solver plan (LLM+P).
• Plan when the steps depend on the request but can be foreseen. Use workflows when they are fixed, and step-by-step agents when each step depends on the last.

Summary

Planning puts an explicit sequence of steps between a goal and its execution. On the running example, a planner wrote four steps aimed at four checkable requirements. Code executed them, retried a timeout, and handed a 404 from an invented URL to a replanner, which replaced the step with a search. The final report met every requirement, using three model calls where a step-by-step agent would have needed at least six. Exception handling turned two failures into a recovery. Goal setting made "done" checkable. Research on reasoning before acting shows the same trade-off at different scales: writing out reasoning, planning first and searching over alternatives all improve results on multi-step problems, and the more exploration they do, the more compute they cost. LLM+P shows the limit of fluent planning, and the remedy for problems with formal structure: the model translates and a solver guarantees validity.

Notes

Plan-and-execute variants. Separating a planner from an executor appears in several designs. In some, a planner writes a plan with placeholders for tool results, and a worker fills them all in without further model calls. In others, a planner revises the plan after every step, which comes close to ReAct. Gulli's chapter 6 shows a planner–writer in CrewAI, and describes Google's and OpenAI's deep research systems [1]. HuggingGPT [8] used a model to plan which specialist models to call for a multimodal task, an early large-scale example of a model planning over tools.

Goal setting and monitoring. Gulli's chapter 11 treats goals, monitoring and the decision to stop as one pattern [1]. His code example iterates on generated code until a model judges that the goals are met. Combined with article 15, the lesson is to prefer checks that do not depend on the same model's judgement where they exist.

Reasoning techniques. Gulli's chapter 17 surveys further techniques. These include program-aided reasoning, where the model writes code to compute the answer, debate-based methods, and the "scaling inference law", which trades model size against inference-time computation [1]. As of 2026, the strongest models do much of this internally, and prompting techniques matter most with smaller or older models.

Classical planning. PDDL and the planners that solve it come from decades of AI planning research, long before language models. LLM+P [6] is one of several ways to connect the two. Others use the model to propose and a verifier to check, which is the CRITIC pattern of article 15 applied to plans.

Problems

Problem 16.1 The run in section 3 used three model calls. Count the calls a step-by-step agent (article 14) would need for the same work, including the timeout and the bad URL, if it calls the model once before every action and once to answer. (Answer: search, fetch study, fetch review, fetch bad URL, then a search or read for the tram, then the answer: at least six calls. The retry after the timeout could be done in code. If it were left to the model, seven.)

Problem 16.2 Classify each error as transient or permanent, and say what the executor should do: (a) HTTP 429 "too many requests"; (b) HTTP 404; (c) a calculator error "unsupported expression"; (d) a payment API timeout after the request was sent. (Answers: (a) transient: retry with backoff. (b) permanent: replan. (c) permanent for that expression: return the error to the model or replan. (d) unknown: the payment may have gone through, so check its status, or retry with an idempotency key; never blindly retry.)

Problem 16.3 With exponential backoff starting at 1 second and doubling, how long does an executor wait in total before giving up after 4 retries? (Answer: $1 + 2 + 4 + 8 = 15$ seconds.)

Problem 16.4 In Tree of Thoughts on the Game of 24, ToT used about 5,500 generated tokens per game and solved 74%. The best of 100 chains of thought used about 6,700 and solved 49%. Compute the generated tokens per solved game for each. (Answer: ToT $5{,}500 / 0.74 \approx 7{,}400$ tokens. Best of 100 CoT $6{,}700 / 0.49 \approx 13{,}700$ tokens. Search is about twice as efficient per solved game here.)

Problem 16.5 Write two goal checks in code for a planning agent that books a meeting room, and one requirement that cannot be checked in code. (Example: the booking's start time equals the requested time; the room's capacity is at least the number of attendees. "The room is suitable for a client meeting" needs a person or a model judge.)

References

• 1: Antonio Gulli (2025). Agentic Design Patterns: A Hands-On Guide to Building Intelligent Systems. Springer. Chapters 6, 11, 12 and 17.
• 2: Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le and Denny Zhou (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35.
• 3: Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee and Ee-Peng Lim (2023). Plan-and-Solve prompting: improving zero-shot chain-of-thought reasoning by large language models. Annual Meeting of the Association for Computational Linguistics.
• 4: Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas Griffiths, Yuan Cao and Karthik Narasimhan (2023). Tree of thoughts: deliberate problem solving with large language models. Advances in Neural Information Processing Systems 36.
• 5: Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang and Yu-Xiong Wang (2024). Language agent tree search unifies reasoning, acting and planning in language models. International Conference on Machine Learning.
• 6: Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas and Peter Stone (2023). LLM+P: empowering large language models with optimal planning proficiency. arXiv:2304.11477.
• 7: Xu Huang, Weiwen Liu, Xiaolong Chen and colleagues (2024). Understanding the planning of LLM agents: a survey. arXiv:2402.02716.
• 8: Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu and Yueting Zhuang (2023). HuggingGPT: solving AI tasks with ChatGPT and its friends in Hugging Face. Advances in Neural Information Processing Systems 36.