Context Engineering Is Governance at Inference Time

18 Aug, 2026

Context engineering is usually described as a technical discipline. Once an AI system operates against institutional policies, records, and memory, deciding what enters the context window is a governance act.

A new term has settled into AI practice over the past year. Prompt engineering, the craft of phrasing a request well, has given way to context engineering, the design of the entire information environment an AI system sees when it reasons or acts. Anthropic defines it as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference," and treats context as "a finite resource with diminishing marginal returns." Google Cloud calls it "the architecture of meaning for artificial intelligence." Nate Layman's AI4RA piece makes the same move for research administration, describing 4 competencies: intent, information, instruction, and interaction.

That framing is useful and largely correct. It is also incomplete for anyone who runs an institution. The emerging literature begins, reasonably, with an engineering problem: what to retrieve, how much, in what order, at what cost. Layman goes further than most, treating context as an organizational competency rather than a tuning exercise, but the field's center of gravity remains technical. That is the right starting point for a developer. It is only half the question for a college, an agency, or a firm. Once an AI system reasons against institutional policies, records, memory, and tools, the choice of what enters its context is not merely a retrieval decision. It is a decision about which version of the institution is true at the moment the model answers.

Context engineering is governance at inference time.

William Adolphe Bouguereau, Homer and His Guide, 1874. Milwaukee Art Museum. Public domain. Homer offers a fitting metaphor for context engineering: knowledge and judgment may reside with the thinker, but access to the surrounding world depends on the guide. AI operates in much the same way. The model may be capable, yet the information, sources, permissions, and institutional rules placed around it determine what it can see and how it can act.

Prompt Engineering Was Only the Beginning

Early generative AI practice concentrated on the prompt. Better instructions produced better output, and a genuine craft grew up around role framing, examples, and structure. That craft still matters. It simply cannot carry the weight now placed on it, because an excellent instruction cannot compensate for information that is missing, obsolete, contradictory, or unauthorized. Ask a perfectly worded question against the wrong policy version and you get a fluent, confident, well-formatted wrong answer.

Modern systems operate with far more than a prompt. The working environment of an agent can include system instructions, conversation history, persistent memory across sessions, retrieved documents, database queries, tool definitions, the results of prior actions, and whatever policies and records the organization has connected. The prompt has become one component inside a larger context architecture, and usually not the decisive one.

Layman's example makes the shift concrete. A grant administrator preparing a budget justification would not simply ask an AI to write one. The administrator would supply the funding opportunity announcement, the institution's negotiated rate agreement, the researcher's prior budgets, and the relevant sections of Uniform Guidance. As the AI4RA piece puts it, "the actual prompt becomes simpler because the hard work has already been done in setting up the context." The analogy works because organizations already understand it. We do not hand new employees a task and expect competence. We hand them the policy manual, the system access, the precedents, and a supervisor.

Selection Creates Authority

Here is where the technical framing stops being sufficient. More information does not automatically produce better answers, which is why every serious treatment of context engineering emphasizes curation. Anthropic's guiding maxim is "the smallest possible set of high-signal tokens." Context must be selected. And selection, in an institutional setting, inevitably creates hierarchy.

Suppose an AI agent working on a routine compliance question can reach all of the following: the current institutional policy, a superseded version of that policy still living in a shared drive, a departmental procedure that contradicts both, a database record, the minutes of a committee meeting where an exception was granted, a precedent from a similar case decided 3 years ago, and an email from a long-serving administrator describing how the office actually handles it.

The engineering problem is retrieval: find the relevant material and fit it in the window. The governance problem is precedence: decide which source wins.

Including one policy rather than another confers authority. Retrieving the current version rather than the archived copy enforces version control. Giving one database precedence when 2 systems disagree defines a system of record. Letting the email into the context and leaving the policy out substitutes custom for rule. None of those choices announce themselves as governance decisions. They arrive as configuration: an index, a ranking function, a memory retention setting, a connector scope. But they determine what the institution appears to believe at the moment it acts.

"How Many Students Are Enrolled?"

A question that sounds trivial shows how much institutional knowledge sits underneath an ordinary answer. Anyone who has worked in institutional research knows the count is not a fact waiting to be retrieved. It is a definition waiting to be applied.

Does the number include nondegree students? Dual enrollment? Students studying abroad, or on approved leave? Students who registered after the census date? Which census date, the state one or the federal one? Which campus, and are we reporting them separately or combined? Which academic career? And which source system, the student information system, the enrollment dashboard built on last night's extract, or the spreadsheet the dean maintains because the dashboard once disagreed with the dean?

A language model cannot resolve any of that through intelligence. Neither can a retrieval pipeline, no matter how well tuned. The institution has to define the terms and designate the authoritative source, and if it has not done so explicitly, the AI system will resolve the ambiguity implicitly, in whatever way the ranking happened to fall that day. The answer will be fast, fluent, and difficult to audit.

A well governed AI environment therefore needs more than access to data. It needs definitions, metadata, lineage, ownership, effective dates, and rules of precedence. Those are the working materials of data governance. Context engineering, done at institutional scale, is data governance executed at machine speed.

Governance Becomes Executable

Institutional governance documents tend to describe principles in the abstract. Student records must originate from the authoritative student information system. Personally identifiable information is available only to authorized users. Certain determinations require human judgment. Records are retained for a defined period and then disposed of.

Those statements have historically been enforced by people, slowly and imperfectly, at points where a human paused to check. AI systems force them to become executable. If the student information system is authoritative, the retrieval architecture has to respect that authority rather than treat all indexed sources as equivalent. If information carries access restrictions, the agent has to inherit the permissions of the person it acts for, not the permissions of the service account it runs under. If a determination requires human judgment, the system has to recognize the boundary and stop, which means it needs enough awareness of its own evidence to know when the evidence runs out. If a record has a retention limit, persistent memory cannot quietly keep it forever, and an AI system that summarizes a governed document into memory may have created a derivative record whose retention status no one has defined.

That last point deserves emphasis, because it is where most institutions are least prepared. Memory is a records management problem wearing a product feature's clothing. Provenance is an audit problem, and the obvious fix makes it worse: capturing the full context window for every interaction builds a second archive of sensitive material with its own privacy and retention exposure. What an institution actually needs to log is narrower and more durable. Which sources were consulted, at which versions, under whose permissions, with which system instructions, which tool calls executed, and what the system escalated or declined to do. That is the record that lets someone reconstruct an AI decision months later in front of an auditor, a regulator, or a faculty committee.

Who Gets to Decide

Developers build retrieval pipelines, memory systems, agent tools, and context windows. They should not, on their own, determine institutional truth, and most of them would not claim the right to. The problem is that no one else shows up to the design conversation.

Subject matter experts, data stewards, registrars, research administrators, legal counsel, information security professionals, and chief data officers each hold part of the information hierarchy. Effective context engineering requires that knowledge to be encoded alongside the technical design, which means the people who hold it have to be in the room when the connectors are scoped and the retention windows are set, not after the pilot has gone into production.

The Model Is the Commodity

That gap connects to a pattern I keep returning to on this blog. When I wrote about algorithmic monoculture, the concern was homogenization: many organizations relying on similar systems and converging on similar outcomes. Context engineering complicates that picture in an interesting way. Two institutions running the identical model can produce materially different behavior, because their information environments, source hierarchies, access rules, and definitions differ. The model is the commodity. The context is the institution. Which means governance attention spent exclusively on model selection is attention spent on the least differentiated part of the system.

The same holds for the question of when an AI system should decline to act. A system can only recognize that it lacks the authority or the evidence to proceed if its context tells it what authority and sufficient evidence look like. Recusal is not a personality trait to be prompted into a model. It is a property of the information environment we build around it.

AI governance becomes real when institutional rules shape what a system can see, retrieve, retain, do, and escalate. Policies, committees, and principles do genuine work upstream of the context window, in permissioning, tool scoping, logging, and approval gates. But the moment of action is where all of it is either enforced or quietly absent, and the context window is the largest and least governed part of that moment.

Prompt engineering taught us to ask better questions. Context engineering forces us to decide who gets to define the world in which the AI answers.

My blog