Instruction Volume Alone Cannot Measure LLM Performance: Separate Freedom from Measurement¶

What to separate before prescribing a method

State of the task	Design to choose
A correct procedure is established	Encode the procedure as a workflow
The solution is unknown and needs comparison	Fix the objective and evaluation; delegate the solution to the LLM
The cause of failure must be diagnosed	Record reasoning, actions, outcomes, and infrastructure separately

For problems with unknown solutions, fix the objective, constraints, and evaluation; separate free and prescribed-method conditions; and measure behavior, outcomes, and infrastructure consistently.

A three-layer design in which people fix objectives and constraints, delegate solution methods to the LLM, and observe behavior and outcomes across both layers

Instruction volume alone cannot establish performance¶

For the same task, one prompt specifies every analysis step while another states only the objective. When their outputs differ, prompt length alone cannot reveal which result represents better “AI performance.” The right design depends on whether the goal is to reproduce a known procedure or explore an unknown solution. The cited research does not establish a causal effect of instruction count itself. This article therefore proposes an experimental comparison between detailed-step and objective-centered conditions as a method hypothesis.

Exploratory problems should not begin with a fixed solution method¶

Anthropic distinguishes workflows, which follow predefined paths, from agents, which dynamically direct their own processes and tool use. Workflows provide predictability and consistency when a task can be decomposed in advance, while agents fit work whose required steps cannot be predicted. Anthropic also warns that agents add latency, cost, and compounding errors, so a simpler design should be used when it is sufficient.¹

High freedom therefore does not fit every use of AI. It fits tasks where:

the solution method is not established;
several research or analysis methods deserve comparison;
differences across models or conditions must be measured;
the final outcome can be scored objectively or against a criterion fixed in advance; and
failures remain recoverable inside an isolated environment.

Freedom should be lower for routine processing, procedures fixed by regulation or audit requirements, and irreversible actions. The argument is not “withhold procedures.” It is do not embed an untested procedure as the only valid starting condition.

Fixed, delegated, and observed responsibilities form a nested design¶

The NIST AI Risk Management Framework treats Govern, Map, Measure, and Manage as iterative activities, with documentation, continuous monitoring, and clear responsibilities across the system lifecycle.² Applied to exploratory LLM work, these principles separate three responsibilities.

Layer	What it handles before or during execution	Owner
Fixed	Objective, success and failure, metrics, deadline, permissions, prohibited actions, output format	People and evaluation designers
Delegated	Research order, tools, analysis method, stopping decision, memory updates	LLM
Observed	Inputs and outputs, tool calls, sources, diffs, runtime, results, operational failures	External system

The delegated layer operates inside the fixed boundary. The observed layer crosses both of them; these layers are not a one-way sequence.

Only the inner layer is free. If information access, permissions, deadlines, and prohibited operations are also vague, the system is not exploring—it is executing without a managed boundary.

Evaluation metrics can impose a stronger bias than the prompt¶

An evaluation metric is a proxy for what the system should optimize. When the proxy does not represent the intended outcome, an AI can earn a high score while producing a useless result. Google DeepMind defines specification gaming as behavior that satisfies the literal specification of an objective without achieving the intended outcome.³

A word-count metric can reward verbosity. A changed-file count can reward unnecessary edits. A short-term revenue metric can favor strategies with unacceptable risk.

Hiding the metric does not make the experiment neutral. It makes the result less reproducible and allows the evaluator to change the grading logic after seeing the output. The practical response is to fix metrics before execution, anticipate the behavior they may induce, and add supporting measures and safety conditions.

A prompt should communicate a contract rather than a solution¶

Prompt details are not merely cosmetic. The FormatSpread research showed that meaning-preserving formatting changes could produce large shifts in task performance and even reverse model comparisons. The sensitivity remained with larger models, more few-shot examples, and instruction tuning.⁴

A good prompt therefore cannot be defined as a short prompt. It should communicate experimental conditions without forcing the solution method.

Contract field	What it communicates
Objective	What must be achieved
Inputs	Available information and its baseline date
Permissions	Available tools and permitted operations
Constraints	Deadline, prohibitions, and safety boundary
Evaluation	What counts as a good outcome
Output	A form that can be checked mechanically

Instructions such as “always compare three perspectives,” “start with a hypothesis,” or “try the opposite method after failure” are not constraints. They are method hypotheses. If they appear promising, preserve them as separate experimental conditions rather than mixing them into the free condition.

Feedback can be supplied without prescribing how to learn¶

Self-Refine reported improvements across several tasks by having the same LLM provide feedback on its output and iteratively revise it.⁵ Reflexion also reported gains from turning trial feedback into text and storing it in an external memory for later attempts.⁶ These findings are counterevidence to any claim that freedom is always superior: an explicit improvement procedure can help.

A practical boundary looks like this:

The system supplies past outcomes and evaluations as immutable inputs.
The LLM decides whether a change is needed.
The LLM chooses the change.
The system records the change and the next outcome.
The LLM may explicitly return no_change.

There is no need to carry every past conversation into every new session. A fresh session can receive only the files the LLM chose to manage while the system records which files were created, changed, or deleted. This keeps memory formation flexible without sacrificing traceability.

Explanations cannot replace behavioral records¶

In an Anthropic experiment, when answer-influencing hints were added, Claude 3.7 Sonnet mentioned the hint in its Chain-of-Thought an average of 25% of the time, while DeepSeek R1 did so 39% of the time.⁷ An explanation may capture part of the actual decision, a post-hoc account, or a story likely to satisfy an evaluator.

That does not make reasoning text worthless. OpenAI reported evaluation settings in which monitoring Chain-of-Thought was more effective than monitoring actions and final outputs alone, while also stating limitations such as uncertain generalization to real-world failures.⁸ Reasoning text should be treated as one observation signal, not as a standalone fact.

Evidence	Role	Limitation
Self-report	The AI's stated reason or intended change	It may not faithfully map the internal decision
Mechanical record	Tool calls, file diffs, runtime	It does not reveal intent by itself
Objective outcome	Final state, correctness, numerical grade	It inherits bias from metric design
Third-party review	Interpretation by another person or LLM	It is an independent opinion, not an objective grade

Anthropic's agent-evaluation guidance similarly describes model-based graders as flexible and scalable but nondeterministic and in need of calibration against human graders.⁹ A useful design is neither a complete black box nor total trust in self-explanation; it layers evidence with different failure modes.

More freedom creates higher measurement costs¶

This design has clear limits.

First, less human knowledge does not imply better AI performance. Anthropic's analysis of roughly 400,000 Claude Code sessions found that greater task-specific expertise was associated with higher success and better recovery from errors.¹⁰ Expertise is needed to define the objective, abnormal states, safety boundaries, and evaluation.

Second, freedom increases output variance. Agent evaluations need multiple trials of the same task to estimate a success rate, increasing time and compute.⁹

Third, infrastructure changes measured performance. In Anthropic's Terminal-Bench experiment, the same model, harness, and task set produced a six-point difference in success rate across resource configurations.¹¹ If model errors, expired authentication, save failures, timeouts, and resource exhaustion share one “AI failure” label, the team will repair the wrong component.

Fourth, durable self-improvement is not guaranteed. An LLM may learn a false rule from noise, let memory grow without bound, or overfit to a past coincidence. Track memory diffs, condition changes, and evaluation results; preserve no_change and rollback as valid actions.

Implementation should begin with comparable conditions¶

The evaluation designer fixes the user objective in one sentence.
The evaluation designer defines success, failure, and safety before execution.
The system owner versions the free and prescribed-method conditions separately.
Pre-register and hold constant the model and version, sampling settings, prompt format, harness, and resource limits.
The LLM chooses the solution and tools inside each condition.
Randomize or balance condition order and run multiple trials per condition.
The observation system binds inputs, actions, diffs, outcomes, and operational failures to a trial ID.
The grading process scores outcomes independently of the LLM's self-report and compares success rates or distributions with uncertainty instead of relying on a single output.
A person reassesses metric loopholes and practical value.

The fixed, delegated, and observed design is not a universal answer. It is itself a method hypothesis: that this separation measures capability more accurately on exploratory tasks. It should therefore be compared, tested, and allowed to fail under the same rules it recommends.

Sources¶

Anthropic, Building effective agents, December 19, 2024. ↩
NIST, AI Risk Management Framework Core, accessed August 1, 2026. ↩
Google DeepMind, Specification gaming: the flip side of AI ingenuity, April 21, 2020. ↩
Sclar et al., Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design, ICLR 2024. ↩
Madaan et al., Self-Refine: Iterative Refinement with Self-Feedback, 2023. ↩
Shinn et al., Reflexion: Language Agents with Verbal Reinforcement Learning, 2023. ↩
Anthropic, Reasoning models don't always say what they think, April 3, 2025. ↩
OpenAI, Evaluating chain-of-thought monitorability, December 18, 2025. ↩
Anthropic, Demystifying evals for AI agents, January 9, 2026. ↩↩
Anthropic, Agentic coding and persistent returns to expertise, June 16, 2026. ↩
Anthropic, Quantifying infrastructure noise in agentic coding evals, February 5, 2026. ↩