Designed Judgment Needs a Runtime

Human-in-the-loop is not a strategy.

Itamar Medeiros makes that case clearly in “Human-in-the-Loop Is Not a Strategy”. A person can remain present while control fragments around them. Review can become symbolic. Responsibility can become ambiguous. At scale, a nominal checkpoint can survive long after meaningful oversight has disappeared.

His alternative is designed judgment. Do not ask a person to monitor everything. Place human judgment at explicit points of uncertainty, risk, and consequence. Define who should decide, what they need to see, when the system must stop, and what happens next.

That is the right design move.

It creates a second question: how do we know whether the decision point actually worked?

That question needs a runtime.

A checkpoint is not a control system

An approval step looks reassuring in a workflow diagram.

The agent prepares a recommendation. The designated expert reviews it. The expert approves or rejects it. Execution continues.

But the diagram does not tell us what happened at runtime.

• Did the request reach the right person?
• Was the critical evidence visible?
• Did the person inspect it?
• Was there enough time to understand the decision?
• Did interruptions, workload, or missing context make judgment unreliable?
• Did the person have real authority to stop the system?
• What happened after the decision?

Without those answers, the checkpoint records only an artifact: approved, rejected, edited, escalated.

It does not record the decision process that produced the artifact.

Designed judgment gives the checkpoint a purpose. Runtime observability shows whether it remained a real control mechanism under operating conditions.

Authority and capability are different

Every human decision point has at least two distinct dimensions.

The first is normative. Who is entitled to decide? What mandate do they hold? Which actions are permitted? Who remains accountable? When must the decision be escalated?

The second is operational. Did the person receive the necessary context? Could they inspect the evidence? Was their intervention timely and coherent? Did it improve the outcome?

These dimensions must not be collapsed.

Authority is assigned. Capability is observed.

A job title does not guarantee that someone can exercise judgment effectively at a particular moment. A measure of attention or available capacity does not give a system permission to take authority away from that person.

Designed delegation defines the decision rights. Human Runtime supplies evidence about how the decision point operated. Policy may use that evidence to request confirmation, reduce information density, route work to an authorized alternate, or enter a safe state. The permitted response must be established in advance.

HRT informs the policy. It does not invent the policy.

The missing record is the human decision span

Agent systems increasingly record prompts, retrievals, tool calls, model decisions, errors, costs, and outcomes. When execution pauses for a person, the trace often becomes a single event: human approved.

That is too little information for evaluation, learning, or accountability.

Human Runtime extends the trace across the human decision point. A useful span can connect:

• the active role, mandate, and decision rights
• the question the system asked
• the alternatives and evidence presented
• the information opened, sampled, missed, or disputed
• relevant task and environmental conditions
• time-bound estimates of human operational state, with uncertainty and provenance
• the action, intervention, handover, or non-action that followed
• the downstream outcome

The record does not reveal private thought. It does not turn an uncertain estimate into a fact about a person. It makes observable parts of the episode available on the same timeline as the agent, tool, interface, and outcome.

The artifact says what the person selected. The trace shows the conditions under which that selection was produced.

Consider an agent approval workflow

An AI agent prepares a supplier payment that exceeds its autonomous limit. Policy requires a finance manager to approve it.

The manager receives a summary and clicks approve.

The decision right is clear. The manager is authorized. But several operational questions remain.

Was the unusual bank-account change shown prominently? Did the manager open the supporting invoice? Was the request one of fifty similar approvals delivered in a burst? Did the interface make the agent's uncertainty visible? Was an authorized alternate available if the manager could not review it properly?

A conventional audit log can prove that the manager clicked the button. An HRT-compatible trace can preserve the wider episode: what was shown, what was inspected, how the request was timed, which policy applied, what action followed, and whether the payment was later disputed.

If the system detected that required evidence had not been opened, a pre-authorized policy might require confirmation. If the request expired, policy might route it to another authorized manager. If no qualified decision-maker was available, the system might stop.

None of those actions should be improvised from a hidden score. They should be explicit parts of the delegation design.

Designed judgment needs an observability substrate

“Layer” and “substrate” describe different parts of this architecture.

An observability layer is a bounded technical component. It collects, qualifies, correlates, and exports runtime information.

An observability substrate is the shared evidence foundation used by many components. Policy engines use it to enforce intervention rules. Interfaces use it to present the right context. Evaluation systems use it to test whether checkpoints work. Domain products use it to adapt workflows. Auditors use it to reconstruct decisions.

HRT provides that observability substrate for designed judgment.

Observability is still only the starting point. The broader Human Runtime stack connects evidence to authorized policies, interfaces, agents, training systems, and outcome loops. It helps other systems act intelligently without pretending that measurement alone constitutes governance.

Test the decision structure before trusting it

The most valuable use of this trace may happen before deployment.

A team can simulate difficult cases and test the assumptions inside its delegation design:

• Does the chosen role recognize when intervention is required?
• Is the necessary evidence available and intelligible?
• Can the person act within the time allowed?
• Do escalation paths work when the first reviewer is unavailable?
• Does the policy respond safely when evidence is incomplete?
• Does the intervention improve the combined outcome?

These tests turn “a human will review it” from an assumption into an empirical claim.

After deployment, the same structure supports monitoring and learning. Teams can find where review became ceremonial, where alerts arrived too late, where decision rights were unclear, and where expert intervention produced a recovery worth preserving.

From human presence to observable judgment

Designed judgment is stronger than generic human-in-the-loop because it specifies where people matter. But a designed decision point can still degrade under scale, workload, poor interfaces, weak evidence, or broken escalation.

The system therefore needs both a contract and a trace.

The contract defines the role, authority, boundaries, required evidence, and permitted responses.

The trace records how that contract operated in a particular episode.

Designed delegation defines who may decide. Human Runtime shows whether the decision point worked.

That is the move from nominal oversight to observable judgment.

Designed judgment needs a runtime.