Prompting & Reasoning

Context Engineering

Context engineering is the discipline of assembling everything that enters a language model's context window for a given step, including instructions, retrieved knowledge, tool outputs, memory, and conversation history, under a finite token budget. It treats the context as a state to curate rather than a single prompt to phrase.

What context engineering means

Context engineering is the practice of deciding what information occupies a language model's context window at each step of a task, and in what form. A model produces output by conditioning on the tokens it can currently see. Those tokens are not just the user's question. They include the system prompt, tool and function definitions, documents pulled from a retrieval system, the results returned by tools the model called, summaries of earlier turns, and any persistent memory the application chooses to inject. Context engineering is the work of selecting, shaping, ordering, and trimming all of that material so the model has what it needs and little that distracts it. The model can act only on what occupies its window, so output quality is bounded by what enters it.

The term moved into common use in mid-2025. Shopify chief executive Tobi Lutke described it as the art of providing all the context for a task such that it is plausibly solvable by the model, and Andrej Karpathy endorsed the phrase publicly, calling context engineering the art and science of filling the context window with just the right information for the next step. Write-ups from Anthropic and LangChain followed, framing it as the natural successor to prompt engineering for applications where the input is assembled by code rather than typed by a person.

A useful mental model, popularized by Karpathy, treats the model like a processor and the context window like its working memory. The application code plays the role of an operating system that loads the right data into that limited memory before each call. Under this view the central question shifts from how to phrase one instruction to what configuration of tokens most reliably produces the desired behavior.

Context is the full set of tokens visible to the model at inference time, not only the user's message.
Sources of context include instructions, retrieved knowledge, tool outputs, memory, and prior history.
The phrase gained traction in mid-2025 through endorsements by Tobi Lutke and Andrej Karpathy.
The framing positions the context window as working memory that application code must curate.

How context engineering differs from prompt engineering

Prompt engineering focuses on the wording of a single instruction, most often a system prompt, to steer a model toward a good answer. That framing fits a person typing into a chat box. It fits poorly for production systems where a request is constructed programmatically from many moving parts and where the model runs over many turns inside an agent loop. Context engineering generalizes the problem: the prompt is one input among several, and the harder task is managing the whole context state across an extended trajectory.

Anthropic describes context engineering as the set of strategies for curating and maintaining the optimal set of tokens during model inference, including the information that lands in the window outside of the prompts themselves. This reframing matters because models degrade as irrelevant or excessive context accumulates, a failure mode sometimes called context rot, and because every token spent on filler competes with tokens that carry signal. Good wording cannot rescue a window stuffed with the wrong documents.

Prompt engineering does not disappear under this view. Clear instructions still matter, and the system prompt is still a high-value surface. Context engineering subsumes it: phrasing the instruction is one operation within a larger pipeline that also retrieves, ranks, compresses, orders, and prunes the surrounding material before each call.

Prompt engineering optimizes one instruction; context engineering optimizes the entire window.
Production inputs are assembled by code across many turns, which prompt wording alone cannot manage.
Excess or irrelevant context degrades output, so curation often matters more than phrasing.
Prompt engineering remains a component, not a competitor, of context engineering.

The core operations: select, compress, order, prune

Most context engineering reduces to a few repeatable operations applied to candidate material before a model call. Selection chooses which items enter the window, typically by relevance to the current step, often using retrieval and reranking to score candidates. Compression reduces chosen items to the tokens that actually carry information, through summarization, deduplication, or trimming verbose tool output. Ordering places items where the model attends to them best, which matters because models show position sensitivity and can underweight content buried in the middle of a long context.

Pruning removes material that has outlived its usefulness, such as stale tool results or earlier turns that have already been summarized. Two related techniques recur in agent systems. Compaction summarizes a conversation that approaches the window limit and restarts from the condensed version, preserving coherence over long tasks. Structured note-taking, sometimes called agentic memory, writes information to an external store, for example a notes file, so it can be re-injected on demand instead of permanently occupying the window.

LangChain groups these moves into four verbs: write context to external storage, select context back into the window, compress it to fit, and isolate it across sub-agents or state fields so unrelated material does not crowd the model. The exact taxonomy varies by source, but the underlying operations of choosing, shrinking, arranging, and discarding tokens are consistent across them.

B = T_sys + T_tools + T_history + T_retrieved + T_query + R, with sum of parts ≤ C

A token budget B partitions the context window of capacity C across the system prompt, tool definitions, history, retrieved chunks, the user query, and a reserve R held back for the model's output. Allocations are chosen so the parts never exceed C, which forces explicit trade-offs about what to include.

Select: score and pull in the items most relevant to the current step, often via retrieval and reranking.
Compress: summarize, deduplicate, or trim items so they fit while keeping their signal.
Order: position high-value content where the model attends to it rather than in the lost middle.
Prune: drop stale turns and tool results, using compaction or external notes to preserve key facts.

Working under a token budget

Every context window has a fixed capacity measured in tokens, and the model's own response must also fit within that capacity. Context engineering is therefore a constrained allocation problem: given a budget, decide how many tokens to spend on instructions, on retrieved evidence, on history, and how many to reserve for the answer. When candidate material exceeds the budget, the pipeline must rank and cut rather than simply concatenate everything available.

A common pattern assigns a token allowance to each section, fills the highest-value sections first, and admits retrieved chunks in ranked order until the retrieval allowance is exhausted. Counting tokens accurately requires the same tokenizer the target model uses, since a token is a sub-word unit and word counts are only a rough proxy. The example below sketches assembly under a budget using a tokenizer to measure cost before committing each piece.

Budgeting also has a cost dimension. Tokens are billed and they add latency, so padding the window with marginal context raises price and slows responses while often lowering quality. The discipline is to spend the budget where it changes the answer.

python

import tiktoken

enc = tiktoken.get_encoding("cl100k_base")

def ntok(text):
    return len(enc.encode(text))

def assemble(system, query, history, chunks, window=8000, reserve=1500):
    budget = window - reserve            # leave room for the reply
    parts, used = [], 0
    for piece in (system, query):        # always-include sections first
        used += ntok(piece)
        parts.append(piece)
    for turn in reversed(history):       # most recent history next
        cost = ntok(turn)
        if used + cost > budget:
            break
        used += cost
        parts.insert(2, turn)
    for chunk in chunks:                 # retrieved chunks, already ranked
        cost = ntok(chunk)
        if used + cost > budget:
            continue
        used += cost
        parts.append(chunk)
    return "\n\n".join(parts), used

A minimal context assembler. It counts tokens with tiktoken, fills mandatory sections first, then admits history and ranked retrieval chunks only while the running total stays under the budget, holding back a reserve for the model's output.

The window capacity is shared between input context and the space reserved for the model's output.
When candidates exceed the budget, the system ranks and cuts rather than concatenating everything.
Token counts must come from the target model's tokenizer, since tokens are sub-word units.
More context costs money and latency, so budget belongs where it changes the output.

Relation to retrieval and memory

Retrieval-augmented generation is one of the main suppliers of context. A retrieval pipeline finds documents related to a query, usually through semantic search over a vector database, and inserts the top results into the window so the model can ground its answer in source material. Context engineering is the layer above retrieval that decides how many results to admit, how to compress them, where to place them, and when newer evidence should evict older. Retrieval finds candidates; context engineering governs how those candidates spend the budget.

Memory is the second major supplier. Short-term memory is the running conversation history, which compaction keeps within budget. Long-term memory persists facts, preferences, and prior outcomes across sessions in an external store that the application queries and injects when relevant. Both are forms of context: the model has no recall of its own beyond the current window, so any persistence must be re-supplied as tokens. This is why systems that give a model durable knowledge across sessions are, at heart, context engineering systems built around an external memory layer.

Tool use closes the loop. When a model calls a function or queries a service, the returned data becomes new context for the next step, and that output frequently needs compression or isolation before it re-enters the window. Retrieval, memory, and tools therefore all feed the same scarce resource, and context engineering is the policy that arbitrates among them.

Retrieval supplies candidate documents; context engineering decides how they spend the budget.
Short-term memory is the live conversation; long-term memory is an external store re-injected as tokens.
A model retains nothing beyond its window, so all persistence must be re-supplied as context.
Tool outputs become fresh context and often need compression before re-entering the window.

Key takeaways

Context engineering is the discipline of curating every token in the model's context window for a step, not just phrasing one prompt.
It supersedes prompt engineering by treating the context as a state to manage across an entire task, with the prompt as one component.
The recurring operations are select, compress, order, and prune, supported by compaction and external note-taking in agent systems.
Because the window has fixed capacity shared with the output, context engineering is a token-budget allocation problem.
Retrieval and memory are the main suppliers of context; context engineering is the policy that decides how they spend the budget.
The term gained wide use in mid-2025 through endorsements by Tobi Lutke and Andrej Karpathy and write-ups from Anthropic and LangChain.

Frequently asked questions

Context engineering is the practice of putting the right information in front of a language model before it answers. That information includes instructions, documents retrieved for the query, tool results, memory, and conversation history. The goal is to fill the limited context window with material that helps and to keep out material that does not, all within a fixed token budget.

Prompt engineering optimizes the wording of a single instruction, usually a system prompt. Context engineering optimizes the entire set of tokens the model sees, of which the prompt is only one part. It is the broader discipline needed when inputs are assembled by code and the model runs over many turns, where retrieving, ordering, and pruning context matter as much as phrasing.

No single person coined it, but the phrase entered common use in mid-2025. Shopify chief executive Tobi Lutke described it as providing all the context needed for a task to be solvable, and Andrej Karpathy publicly endorsed the term over prompt engineering. Engineering write-ups from Anthropic and LangChain shortly after helped standardize the meaning.

Retrieval-augmented generation is one of the main ways context is supplied. RAG finds documents relevant to a query and inserts them into the window so the model can ground its answer. Context engineering is the layer that decides how many of those documents to admit, how to compress them, where to place them, and when to evict older ones, so retrieval and context engineering work together rather than being the same thing.

A context window has a fixed token capacity that input and output must share. When candidate material exceeds that capacity, the system has to rank and cut instead of including everything. Spending tokens on marginal context also raises cost and latency and can degrade quality, so allocating the budget toward content that changes the answer is central to the discipline.

The core operations are selecting relevant items into the window, compressing them to fit, ordering them where the model attends best, and pruning stale material. Agent systems add compaction, which summarizes a long conversation and restarts from the summary, and structured note-taking, which writes information to an external store so it can be re-injected on demand instead of occupying the window permanently.

Put the idea into practice

MemX is an AI memory agent built on these ideas: store anything, skip the folders, and find it again by asking in plain English.

Try MemX Free