Prompt Engineering

Define the task clearly. Verify the result.

“Summarize this report” usually produces fluent prose. Yet a manager preparing for a meeting, a colleague taking over a project, and a program ingesting data need different summaries. The manager needs evidence for decisions, the colleague needs background and unresolved issues, and the program needs conventions for fields, types, and missing values. The same material can support different deliverables.

Prompt engineering addresses these differences: specify what must be done, supply the information needed, demonstrate easily misunderstood decision boundaries, and make the result checkable. A sterner tone, a grander role title, or longer instructions cannot replace that work.

Early prompting research developed methods such as learning from examples, task decomposition, chain of thought, and repeated sampling. As training methods and interfaces change, the benefits of a particular formulation change too. The Prompt Report offers a broad taxonomy. In practice, the more useful questions are what caused the current failure, which change could reduce it, and at what cost.

A durable prompt should make the relationships between goals, evidence, and acceptance clear. The sections below follow those relationships. Individual techniques can be combined as needed, without packing all of them into the same long template every time.

Prompting starts with tasks, materials, and examples, then improves through result checks and specific feedback.
Prompting starts with tasks, materials, and examples, then improves through result checks and specific feedback.
Contents

1. Turn the task into assessable requirements

Explain how the deliverable will be used

“Clear, professional, and accurate” expresses a wish but removes little ambiguity. Identifying the audience and purpose makes it easier to decide what to retain and what to omit. Ethan Mollick's Good enough prompting emphasizes real task context, audience, and iterative feedback. These details often help more directly than elaborate role assignments.

A project summary might use the following specification:

Using the supplied meeting notes, write a handoff summary for an engineer
taking over the project.
Retain agreed decisions, unresolved issues, next actions, and their owners.
Attach the meeting-note paragraph number to each decision.
Do not add dates, owners, or commitments absent from the material.
If an owner is missing, write “unspecified”; do not infer it from the speaker.
Use four sections. Each item should contain one decision or action.

The consequential details are “taking over the project,” “decisions,” “unresolved issues,” and “what to do when an owner is missing.” Four sections is a presentation requirement that can change with the interface. Keeping these two kinds of requirements separate helps prevent formatting from overriding the task itself.

Purpose also determines what information matters. A decision brief can highlight impact and alternatives; a handoff must retain unresolved issues; extraction for a program needs stable fields. A single prompt that tries to serve every purpose usually leaves each audience with more filtering to do.

Replace quality adjectives with actions, boundaries, and priorities

“Concise” can become “retain only information that affects the next decision.” “Omit nothing” can become “cover every question in the input and retain unanswered items.” “Evidence-based” can become “attach a source ID to every factual judgment and label speculation separately.” These requirements can still be violated, but at least a failure can be located.

Prohibitions work better with an alternative behavior alongside them. “Do not fabricate” does not explain what to deliver when data is missing. “Define an unknown state for fields, for example allowing missing values to be null, and identify which judgment the missing item affects” produces a result that can be processed further. Negative constraints can remain, provided the model is not left to guess how to stop.

Conflicting constraints also need priorities. “Cite evidence for every conclusion” and “keep the entire answer within 100 characters” may be incompatible. If important information must remain, permit exceeding the suggested length or require an explanation of why the constraints cannot all be met. Otherwise the model may silently drop evidence to satisfy the more visible character limit.

Anthropic's prompt engineering overview treats success criteria and testing methods as prerequisites for optimization. A practical starting point is to write what an acceptable result looks like for one or two real inputs, then revise the prompt. When acceptance still means “looks good,” prompts can easily drift toward longer, more professional-sounding prose.

Decide how to handle missing information and clarification

“Ask a follow-up question if information is insufficient” is still too broad. A model may repeatedly interrupt over details that do not affect the result, or make assumptions about critical gaps. Distinguish what must be clarified, what can use a default, and what can remain unknown.

If missing information changes the task scope or main conclusion,
ask about the critical gap first.
If only formatting or tone is unspecified, use defaults suitable
for a technical audience.
If a fact is absent but the rest of the task can be completed,
leave that item unknown.
Briefly identify defaults you use; do not present them as source facts.

Role prompting should also serve a specific purpose. “Review this as a code reviewer, focusing on exception handling and resource cleanup” identifies a focus. “Write for readers who know Python but are new to distributed systems” identifies an audience and helps set the depth of explanation. “You are the world's leading expert” cannot supply missing material or grant tool access. Roles and audiences help select perspective, language, and emphasis; professional judgment still depends on materials and checks.

2. Organize materials and retain their sources and boundaries

Separate rules, the current request, and the material to process

A long input often mixes application rules, user questions, conversation history, and retrieved results. The model needs to distinguish task-defining instructions from the objects of the task. Clear sections reduce ambiguity and make the final input easier to debug.

Application rules: answer scope, missing-information behavior,
evidence requirements, and output requirements.

Materials for this request:
<document id="D1" title="Project meeting notes" version="v3">
  <paragraph id="P1">...</paragraph>
  <paragraph id="P2">...</paragraph>
</document>

Current request: extract agreed decisions and matters still undecided.
Use references such as D1:P1 to locate evidence.

When an API supports a native message hierarchy, preserve its structure. In OpenAI's interface, for example, application rules and user requests have different priorities; see its prompt engineering guide. Role semantics depend on the provider. One API's conventions cannot simply be imposed on every model template.

Markdown headings, XML tags, and triple quotes are delimiters that can help readability. There is no evidence that one delimiter is best across every model, language, and task. Choose a format that is easy to inspect, escape, and construct automatically, then validate it on real tasks. Repeatedly rewriting the whole prompt for formatting tricks is rarely useful.

Give long materials navigation and test positional effects

A long context window addresses whether information fits, without automatically ensuring it is used well. Lost in the Middle found that the position of relevant information affected results in the multi-document question-answering and key-value retrieval settings it tested. This observation suggests what to test; it does not establish an identical positional curve for every newer model.

Document IDs, titles, versions, and paragraph locations give materials a citable structure. When sources conflict, the model can at least identify which ones disagree. Labels such as “Material A” and “Material B,” without versions, often cannot explain whether apparently contradictory figures belong to different periods.

For large inputs, the Claude long-context guide recommends placing materials first and the specific question at the end. This is compatible with placing stable rules in the application instruction layer: the former arranges the current long input, while the latter defines persistent behavior. It is a starting point that still needs testing against the model and task.

Keep the correct answer fixed and place its key evidence at the beginning, middle, and end of the materials, then add a few topically similar but irrelevant passages. If the answer changes markedly with position or distractors, improve retrieval, narrow the input, or add an evidence-location step. Enlarging the window alone may not reduce these failures.

Select relevant material and allow the answer to be absent

Sending only the most similar passage can omit exceptions; sending the whole knowledge base can make unrelated provisions compete for attention. Material selection needs to retain the context that supports a judgment, including definitions, scope, exceptions, and version notes.

Making answers depend on checkable material is commonly called grounding. For evidence-heavy tasks, ask the model to locate relevant passages before forming an answer. The located evidence needs only a document ID, source excerpt, and the claim it supports, rather than lengthy internal reasoning. If no evidence is found, “not covered by the materials” should be an allowed result instead of forcing every question to receive an answer.

“No support found” and “contradictory evidence exists” also need to remain separate. A document's failure to mention a feature does not establish that the feature is absent. Two documents applying to different versions may not conflict factually. A prompt can distinguish these states; subsequent checks can assess whether the sources support each judgment.

External material can still contain text that attempts to change the task. Simon Willison's analysis of delimiters and prompt injection shows why delimiters do not provide security isolation. Labeling text as material helps express intent. Tools, sensitive data, and external actions also require program-enforced permission and data-egress controls.

3. Few-shot: demonstrate decision boundaries

Zero-shot and few-shot: two inputs for the same task

Zero-shot prompting supplies instructions and the current input without completed task demonstrations. One-shot supplies one demonstration; few-shot supplies a small number before asking the model to handle a new input. The count refers to demonstrations in the input, rather than conversation turns, and does not involve updating model weights during inference.

A classic GPT-3 example gives definitions of new words and asks the model to use them in sentences. The original starts with a human-written example and retains subsequently generated sentences in context. The following presents two prompts adapted from that new-word example, with sentences shortened and modified.

A zero-shot prompt can say:

Definition: A yalubalu is a vegetable that resembles a large pumpkin.
Write a natural sentence using yalubalu consistently with its definition.

A few-shot prompt first demonstrates how a definition becomes a sentence:

Definition: A whatpu is a small, furry animal native to Tanzania.
Sentence: While traveling in Africa, we saw several adorable whatpus.

Definition: To farduddle is to jump up and down quickly.
Sentence: My younger sister began to farduddle excitedly during a game of tag.

Definition: A yalubalu is a vegetable that resembles a large pumpkin.
Sentence:

A suitable continuation could be “For dinner, we cut the yalubalu into pieces and roasted it.” The demonstrations show the task form and how the word is used; the answer is not unique. This qualitative case does not prove that adding examples always improves results. A model that already understands the task may need only the first input.

Example count and example content are separate dimensions. Few-shot examples can show final labels, brief solution steps, or tool-use traces. Few-shot, CoT, and ReAct can therefore be combined; they are not three mutually exclusive prompting modes.

Examples should clarify what rules leave ambiguous

Examples can simultaneously show format, tone, input distribution, and classification boundaries. Rethinking the Role of Demonstrations analyzes these factors in the classification and multiple-choice settings it studies. Some of its findings about random labels apply to particular models and tasks; they do not mean incorrect examples are harmless.

Correct examples remain a basic requirement in practice. Selection deserves more effort: an ordinary case shows normal handling, a similar case with a different label shows the distinction, and an incomplete case shows when to stop judging.

A ticket policy might classify an existing capability that deviates from expectations under clear conditions as bug; a request for a capability not yet provided as feature; and insufficient information to establish either as needs_info. These cases convey more than three obvious defects:

“After exporting CSV, the existing Chinese-language fields are garbled.” => bug
“Only CSV export is available; please add Excel export.” => feature
“The export is wrong.” => needs_info
Clarify: export format, actual result, and expected result.

The value is that “the export is wrong” is not forced into the bug category. If the business process permits classification before clarification, revise both the rule and the examples accordingly. Examples are often easier to imitate than abstract rules. When they conflict, a more emphatic instruction rarely solves the problem reliably.

Example selection should cover ordinary cases, easily confused neighboring categories, and missing-information behavior.
Example selection should cover ordinary cases, easily confused neighboring categories, and missing-information behavior.

Prevent examples from introducing accidental patterns

If every bug example mentions “export” and every feature request says “add,” the model may follow vocabulary rather than the business definition. Vary the wording and include similar keywords with different conclusions. Examples should do more than prove the prompt can handle the formulations it has already seen.

Lilian Weng's prompting review surveys research on example selection, label distributions, and ordering. For a particular task, hold example content fixed and swap the order, or remove one example and observe which errors change. This makes results easier to interpret than continually adding more examples.

Also control the relationship between examples and test data. A demonstrated case and its close paraphrases are not independent tests of generalization. Samples from the same template, document, or event should usually be split as a group, rather than changing a few words and placing them on opposite sides of a dataset split.

Start with a zero-shot baseline and add examples for observed errors

There is no optimal example count for every task. Claude's prompting guide offers practical advice on a small, diverse set of examples; OpenAI's reasoning-model guide recommends starting zero-shot and adding examples as needed. These recommendations concern different models and settings.

Begin with clear rules and inspect real errors. If the model already understands the task, examples may only increase input cost. If it repeatedly confuses a boundary, add examples that explain that boundary. Each example should ideally answer one question: which failure returns when it is removed?

Examples also affect answer length and form. If all examples contain lengthy explanations, output may remain long even when the instructions ask for brevity. Use short examples when short output is needed. Include reasonable unknown states when unknowns must be retained, rather than providing a complete answer to every demonstration.

4. Choose reasoning and decomposition for the task

Chain of thought (CoT): demonstrate how to reach an answer

Chain-of-Thought Prompting demonstrated benefits from examples containing intermediate solution steps for certain large language models and reasoning tasks. The original method is not equivalent to appending “think step by step” to an arbitrary task, and its benefits are not guaranteed across tasks.

Figure 1 in the original paper illustrates the distinction with tennis balls and apples. The prompt structure can be condensed as follows:

Completed example
Q: Roger has 5 tennis balls and buys 2 cans with 3 balls each.
   How many tennis balls does he have now?
A: He buys 2 × 3 = 6 balls, for a total of 5 + 6 = 11.

New task
Q: A cafeteria has 23 apples, uses 20 for lunch, and buys 6 more.
   How many apples remain?
A:

The example demonstrates quantities and operations, rather than only the answer 11. The new question gives 23 − 20 + 6 = 9. The figure shows one example to explain the method, while the main experiments use fuller demonstration sets. One successful case cannot replace an aggregate evaluation.

Adding “Let's think step by step” without solution demonstrations belongs to another study's Zero-shot CoT. Its original procedure generates a solution first, then extracts the final answer. Its input structure differs from demonstration-based CoT; the two should not be reduced to the same catchphrase.

For models trained with internal reasoning capabilities, forcing a long visible reasoning process may not help. OpenAI's reasoning-model guide places more emphasis on directly stating goals and constraints. After model changes, repeated checking instructions, mandatory step-by-step output, and lengthy roles in old prompts deserve reassessment.

Useful explanations can still be requested: key assumptions, formulas, evidence citations, brief decision rationales, or solution steps for a learner. These are deliverables for the reader. A coherent explanation establishes readability; by itself, it proves neither the answer's correctness nor the model's actual internal process.

Least-to-most: let later steps reuse earlier results

“Analyze, then think, then answer” defines almost none of the intermediate work. More concrete decomposition specifies what each step receives, produces, and passes onward. Least-to-Most Prompting decomposes a problem into subproblems and solves them sequentially using earlier subanswers. This can inform workflow design, although decomposition quality in open-ended tasks still needs separate assessment.

The paper's last-letter concatenation task makes the dependency visible. The original demonstration can be condensed as follows:

Task: Take the last letters of think, machine, and learning, in order,
      and concatenate them.
Base result: think and machine end in k and e, producing ke.
Continue: We already have ke; learning ends in g.
Final result: ke + g = keg.

The later step consumes the earlier result without reprocessing the full list from the beginning. This explains recurrence and dependency. In an application, a program can directly perform last-letter operations; there is no need to call a model merely to use a prompting technique.

A comparison of two technical proposals can be decomposed into these artifacts:

1. Extract mandatory conditions from the requirements; retain original item IDs.
2. Extract matching evidence from proposals A and B separately;
   mark items without evidence as unknown.
3. Align conditions and evidence; distinguish satisfied, unsatisfied,
   and undetermined.
4. Recommend using the user's priorities; list unknowns affecting the advice.
Check: Every mandatory condition appears in the comparison table.

These steps can happen in one call or several. Linking calls so each consumes the previous result is prompt chaining; Anthropic's workflow guide also emphasizes programmatic checks between calls. Multiple calls have a clear benefit when steps need separate validation, reuse, parallelism, or different tools. Creating a complex plan for an already short task often adds only latency and new failure points.

Decomposition can also lose information. If the first step omits a requirement, a neat downstream comparison may faithfully preserve the error. Retain original item IDs and check coverage at the end. Wherever a downstream step sees only a summary, consider whether negations, exceptions, and applicability conditions have been lost.

Self-consistency: select answers from multiple samples

Self-Consistency samples multiple reasoning paths and aggregates their final answers, improving results on the tasks it tested. It relies on comparable final answers; an open-ended report has no natural majority vote.

For numeric or multiple-choice tasks, normalize answers first: are units consistent, are percentages equivalent to decimals, and should synonymous labels be merged? If samples share a misunderstanding, voting can consistently return the wrong answer. Samples may share systematic bias, so vote counts cannot be directly interpreted as probabilities of correctness.

records = []
for candidate in sample(task, budget=n):
    records.append({
        answer: normalize(candidate.final_answer),
        checks: check_with_available_rules(candidate)
    })
return select_or_report_disagreement(records)

Selection may use verifiable tests, human comparison, or an aggregation rule suited to the task. For testable code, inspect tests first; for numbers that can be checked by exact calculation, recalculate first. Compare methods under similar total budgets, rather than attributing all benefits from extra calls to one phrase in a prompt.

ReAct: let tool observations determine the next step

Some questions require finding one fact before deciding what to look up next. ReAct alternates reasoning, actions, and environmental observations. Its HotpotQA prompt includes a classic example: identify the region reached by an orogeny's eastern sector, then find that region's elevation while resolving a name ambiguity. A condensed version of the query trace is:

Question: What is the elevation range of the region reached
          by the eastern sector of the Colorado orogeny?
Search Colorado orogeny: The summary omits the eastern extent.
Look up eastern sector in the article: Find the name High Plains.
Search High Plains: Receive disambiguation information.
Search High Plains (United States): Find the elevation range.
Example answer: 1,800–7,000 feet.

Later queries respond to observations: look inside the article when the summary lacks the fact; add an entity qualifier when a name is ambiguous. Listing every query at the beginning and never reading subsequent feedback can miss these intermediate changes.

The original ReAct prompt demonstrates a trace of reasoning, actions, and observations. Current interfaces can use native tool calling: the model proposes a function and arguments, and the application executes it and returns the result. The Gemini function-calling guide describes this division. What matters is continuing decisions from actual results, without requiring current reasoning models to reveal verbatim internal thought.

Tool observations must come from real execution. Having a model continue with “search results” in a single answer does not grant search capability. Repeated queries, poor retrieval, and misread materials can still fail. The loop therefore needs a call budget, exit behavior when nothing is found, and host-enforced permission and parameter checks.

5. Make outputs checkable and reusable

Structured outputs: define formats and incomplete states

Concise headings and tables are usually sufficient for human readers. When downstream programs consume the result, use interface-supported structured outputs rather than relying only on “output JSON and nothing else.” OpenAI Structured Outputs and Gemini Structured Output both distinguish structural constraints from content correctness.

Output design especially needs room for unknowns. Requiring a normal-looking value in every field can pressure the model to fill gaps when input is incompatible. Required fields can express status, factual fields can be empty, and omissions should have explicit meanings.

{
  "status": "needs_info",
  "category": null,
  "missing_fields": ["expected_result"],
  "evidence_ids": ["ticket:P2"]
}

This example says the current classification cannot yet be completed; the specific combination of fields still needs programmatic validation. API refusals, truncated output, and insufficient information are distinct conditions, rather than interchangeable valid JSON objects to process normally.

A schema checks types, required fields, and enums well. Date ordering, unit conversions, source correspondence, and business state need additional validation. A field named verified with a value of true only means the model generated that value. It cannot replace verification that actually ran.

Check citation existence and support separately

A citation raises at least three questions: does the excerpt occur in the source, does it support the statement, and does the source apply to the current question? One “has a citation” metric cannot represent all three. Jason Liu's citation-validation article demonstrates source matching and semantic consistency checks. Applications should also examine source versions and scope.

“This interface is currently available only in the test environment,” for example, does not support “This interface is available to every user.” The quotation may match exactly while the conclusion remains wrong. Scope expansion is another common problem: evidence from one sample group becomes a universal rule, or correlation becomes causation.

Programs can first check source IDs, versions, and excerpts, then key conclusions can receive semantic verification. Semantic checks should allow “insufficient evidence” and retain parts needing expert judgment. When another model reviews the result, measure its ability to detect real errors rather than treating it as a correct judge by default.

For dense material, bind each key judgment to its evidence in a record before generating reader-facing prose. Revisions can then locate the sources a conclusion depends on, without filling the final text with lengthy checking procedures.

Give repairs specific feedback and stopping conditions

“Check carefully three more times” does not identify a basis for checking. Large Language Models Cannot Self-Correct Reasoning Yet found that self-correction without external feedback was unreliable for the models and tasks tested, and could even turn correct answers into wrong ones. This result does not generalize to every new model, nor does it invalidate repairs informed by tests, retrieval, and expert feedback.

Useful feedback identifies the failed requirement, error location, and known correct information. Examples include “the citation in item 3 does not occur in D2,” “8 items are required, but only 7 were returned,” and “the date precedes the allowed range.” “The quality is not high enough” may merely prompt a change in tone.

Candidate outputs receive rule and semantic checks; failures provide specific feedback for bounded repair, and unresolved cases remain pending.
Candidate outputs receive rule and semantic checks; failures provide specific feedback for bounded repair, and unresolved cases remain pending.
candidate = generate(task, materials, requirements, output_spec)
for attempt in range(max_repairs + 1):
    checks = validate(candidate, materials, requirements, output_spec)
    if checks.accepted:
        return candidate
    if attempt == max_repairs or checks.requires_new_information:
        return needs_review(candidate, checks)
    candidate = repair(
        candidate, task, materials,
        requirements, output_spec, checks.details
    )

Rerun relevant checks after repairs so fixing one issue does not break another. If sources themselves are missing, obtain material or return a pending-review state. Do not demand rewrites until the model happens to satisfy surface constraints. Stopping conditions determine whether the process polishes “I don't know” into “I appear to know.”

6. Revise prompts from observed errors

Debug the actual request, not just the template

What an application sends to the model may include historical messages, tool definitions, retrieved passages, schemas, and retry feedback absent from the template. Hamel Husain emphasizes this in his article on inspecting actual prompts: examining only a handwritten string makes it easy to debug the wrong object.

“The model ignored the rule” may mean materials were truncated, old requirements remained in history, a framework inserted conflicting instructions, or interpolation disrupted the document hierarchy. Reconstruct that run's actual request and response before deciding which layer to change. Records containing sensitive material should retain the original data-access controls for storage and display.

A reproducible record includes at least the prompt-template version, actual input, model version, generation settings, and return status. For multiple calls, retain their sequence and each intermediate artifact. Identical final answers do not imply identical costs or intermediate behavior.

Give each change a testable hypothesis

Appending “never do this” after each bad answer quickly creates a long prompt with overlapping or conflicting rules. Attribute the error first: an unclear goal, insufficient evidence, biased examples, unstable formatting, reasoning errors, or faulty validation logic.

Hamel Husain's error-analysis method emphasizes observing real interactions, grouping failures, and turning them into tests. Simon Willison's discussion of prompt engineering also places communication and experimentation at its center. A change record can put these ideas into practice:

Observation: Tickets with insufficient information are forcibly classified.
Hypothesis: The rules define bug and feature but no returnable unknown state.
Change: Add needs_info and one incomplete-input example.
Observe: Do these misclassifications decrease? Does completion decline
         for clear tickets?
Regression: Are previously distinguishable neighboring categories
            still classified correctly?

Let one experiment answer one question where possible. To establish whether examples help, hold other conditions fixed; to compare order, do not also change the model and label definitions. Complex changes can be compared together, but the conclusion then concerns the whole bundle, rather than one phrase within it.

Removal is also an important experiment. If deleting an elaborate role, repeated rule, or example does not worsen results, consider simplifying. Retained content should have a clear purpose beyond having once coincided with success.

Measure correctness, completeness, and justified abstention

Evaluations should cover common inputs, confusing boundaries, missing information, and conflicting material. Development samples support diagnosis and revision; held-out samples test whether changes transfer. Repeatedly tuning prompts from the same held-out results gradually turns those samples into development data, requiring new independent checks.

Development samples support prompt iteration; after a candidate is frozen, previously unused samples test whether improvements transfer.
Development samples support prompt iteration; after a candidate is frozen, previously unused samples test whether improvements transfer.

IFEval distinguishes compliance with individual instructions from compliance with every instruction in a request. Applications need that distinction too: an answer can have the right format and length while omitting a mandatory question. A high average on individual checks does not necessarily make the final deliverable usable.

Measuring accuracy only among answered questions can also encourage frequent unknown responses. Record which questions were answered, refused, or returned for clarification, and whether those choices were justified. Unnecessary abstention fails a task with sufficient material; forcing an answer fails a task whose critical evidence is missing.

Start with a few explicit checks and inspect key subgroups separately, without immediately compressing everything into one score. Calibrate semantic graders against expert judgments, especially to see whether they reward longer answers. For stochastic tasks, repeat within a reasonable budget and record completion rate, the total cost of all attempts, and end-to-end latency to an acceptable result, including failed calls and repairs. The OpenAI evaluation guide offers related practices.

Automatic optimization needs an objective worth optimizing

Automatic methods can search instructions, select examples, and even optimize multi-step programs. DSPy combines language-model programs with data and evaluation metrics; its optimizer documentation explains how candidate instructions and demonstrations are searched. This reduces manual trial and error, while the task still determines the objective.

If a metric checks only format, an optimizer may produce consistently formatted but empty answers. Rewarding a value in every field may suppress justified unknowns. Automatic optimization amplifies preferences in the objective, so metrics and failure examples need practical scrutiny first.

In their discussion of manual prompting and automated tools, Hamel Husain and Shreya Shankar emphasize that working through real cases manually exposes assumptions and helps establish evaluation criteria. Understand the errors first, then use automation to expand the candidate search while continuing to inspect new failures outside the metric's coverage.

7. Know when to stop changing the wording

Fix the system when knowledge, computation, or access is missing

If a model has no current material, “make sure this is up to date” supplies none. Without calculation tools, “be absolutely precise” creates no exact computation. Without an accessible interface, “verify this online” does not make verification happen. Prompts can specify when to retrieve, calculate, and report gaps; the system must provide those capabilities and retain their actual results.

Some requirements belong directly in programs. Fixed sorting, exact counting, field deduplication, and explicitly expressible relationship checks usually do not need a model to reinterpret them each time. Letting models generate content and programs enforce deterministic rules reduces the details that accumulate in prompts.

Treat generated self-assessments carefully too. “95% confidence” is not inherently a calibrated probability, and “checked” is not a tool receipt. Validate how these signals relate to errors using task data, or use checkable sources, test results, and execution traces directly.

Validate prompts together with models and settings

Prompt performance depends on the model. A new version may follow short instructions better or respond differently to old examples and forced reasoning. The DeepSeek-R1 technical report discusses differences between zero-shot and few-shot in its settings. This supports rebuilding baselines, rather than concluding that every reasoning model should remove examples.

Sampling parameters have no universal answer either. Lower temperature changes sampling behavior but does not guarantee factual correctness or perfectly deterministic reproduction. Model-specific recommendations in the Gemini prompting guide likewise show why old parameter rules may be unsuitable.

During migration, test both the existing prompt and a minimal baseline, comparing task correctness, boundary behavior, cost, and latency. Record the prompt template, example set, model version, generation settings, and evaluation samples together. A final prompt string alone rarely explains why performance changes later.

Look up a symptom

Symptom What to inspect or adjust first
You want to distinguish zero-shot, one-shot, and few-shot Compare input structures: distinguish the task input from completed demonstrations; see the new-word example
The answer is fluent but unsuitable for its purpose Deliverable and audience: specify who will use it and why, then select the content
The output follows the format but omits important content Constraints and priorities, overall acceptance: check completeness separately from individual rules
The model reaches a definite conclusion with insufficient information Missing information, output states: allow unknowns, clarification, and incomplete results
Key evidence in long material is overlooked Long context: retain locations, move evidence, and test with distractors
Classification repeatedly confuses neighboring categories Few-shot boundaries: add correct examples with similar inputs and different conclusions
Changing a few examples or their order changes results markedly Example bias: inspect accidental keywords, label distribution, order, and test leakage
Reasoning grows longer without becoming more correct CoT, least-to-most and call decomposition: retain useful explanations and reduce unjustified mandatory steps
Several answers agree but share the same mistake Self-consistency: check shared assumptions and validate with external rules before aggregation
The next lookup depends on the latest result ReAct: execute tools and adapt queries to observations; retain budgets and exits
Citations are present but do not support the conclusions Evidence checks: distinguish excerpt existence, semantic support, and scope
Repeated self-checks produce only different wording Bounded repair: return specific errors and stop rewriting when information is missing
The template looks right but the application is unstable Actual requests: inspect messages, retrieval, tool definitions, truncation, and retries
The prompt keeps growing while new problems persist Change hypotheses, responsibility boundaries: run removal experiments and consider changing data or programs
Old techniques hurt performance after a model change Version validation: compare a minimal baseline with the existing prompt again

Details worth retaining in a prompt should explain which ambiguity they remove, which errors they reduce, and how they can be checked. Rules, examples, and tests accumulated this way remain easier to revise as tasks change and models improve.