The Corrigible State

I. Corrigibility Without an Operator

AI safety has a word for a system that does not make its present objective immune from correction: corrigibility.

The simplest technical formulation was deliberately narrow. A capable agent may have instrumental reasons to resist shutdown or modification. A corrigible agent instead cooperates with what its creators regard as corrective intervention. The Off-Switch Game sharpened one route: a robot that is uncertain about its objective may preserve a human operator’s ability to switch it off because the human’s action carries information about what should be done.

The abstraction is powerful. Its governance transfer is not direct.

A state is not one optimizer with one reward function. The public is not one operator. Elections, courts, ministries, auditors, civil servants, municipalities, affected people, experts, and political movements possess different kinds of standing and authority. Their objectives overlap, conflict, change, and remain partly unspecified. No one actor is entitled to substitute their judgment for the whole merely by claiming to have found an error.

The real problem is harder:

What would corrigibility mean when the operator, the objective, the evaluator, and the authority to intervene are distributed and contested?

Rationalist governance thought reached several neighboring objects. Futarchy is a genuine decision mechanism, not merely a prediction device: elected representatives define a welfare measure, conditional markets estimate which policy raises it, and the market result determines policy. Inadequate Equilibria analyzes why institutions can remain predictably suboptimal. Meditations on Moloch maps the destructive equilibria produced when locally rational actors cannot jointly close a bad option.

Those are serious pieces of governance engineering. They do not by themselves answer the lifecycle questions:

• Claim preservation. Who preserves its causal claim?
• Failure evidence. Who can introduce evidence that it failed?
• Answerability. Who must answer?
• Authoritative change. Who can alter authoritative state?
• Implementation ownership. Who owns implementation?
• Reopening trigger. What later observation reopens the matter?
• Meta-replacement. Who can replace the correction machinery when it hardens?

A prediction market can make a policy conditional on a forecast and still leave later implementation failure without an owner. An audit can discover divergence after the fact and still carry no effective response path. An election can replace a government while preserving the same mechanism, data system, professional incentives, and administrative interpretation.

A system does not become corrigible merely by knowing that it is wrong.

The Governance Alignment Problem asks whether public mechanisms optimize for what political institutions say they want. Corrigibility begins when the answer is no—or when the objective, model, or implementation itself is challenged. Alignment concerns correspondence with the current target. Corrigibility concerns the authorized path by which the target, mechanism, or judgment can be revised.

An aligned system can be incorrigible. It may pursue its installed objective faithfully while excluding every challenge to that objective. A corrigible system can be temporarily misaligned. Its defining property is that the discrepancy can survive into an authoritative correction process.

The governance version has to be a property of the path.

II. The State Has No Single Corrector

The authority problem is the design problem.

Different institutions are competent to correct different things.

A court may determine that a public action violates a constitutional or statutory rule. It is not thereby the best body to estimate an employment elasticity or redesign a hospital funding formula. An audit office may reconstruct expenditure and implementation failure. It does not thereby inherit authority to choose the distributional objective. A ministry may possess operational knowledge and lawful implementation powers. It may also have a structural interest in defending the programme it designed. A parliamentary majority may possess political authority. It may lack the time, information, and independent capacity needed to test a mechanism claim.

Affected people can know where a system fails at the edge. They may not know the whole mechanism. Experts may know the mechanism. They may not represent the people bearing its costs. Opposition parties may expose a defect. Their incentive may be to maximize embarrassment rather than produce repair. Incumbents may know what can actually be changed. Their incentive may be to minimize the admission of failure.

No role should be silently promoted into the universal corrector.

The governance equivalent of a corrigible agent is an arrangement in which no incumbent controls every gate between a grounded contradiction and the possible correction of authoritative state.

That requires separation among at least five roles:

The roles can be distributed across institutions. One institution can sometimes hold more than one role. The arrangement becomes fragile when the same actor can define admissibility, select the evidence, decide the merits, control implementation, certify success, and prevent reopening.

This is a governance form of the Dominant-Player Constraint: no actor should control the rules, evidence, forum, timing, closure condition, and enforcement substrate of a contest that affects others.

Plural authority requires typed authority.

The challenger may be entitled to standing without being entitled to win. The adjudicator may be able to issue a finding without being able to enact a budget. Parliament may retain final political closure while owing a public response to a finding. The implementation owner may choose among lawful repairs while remaining answerable for the movement test. The verifier may reopen the question without prescribing the final replacement.

The central institutional move is to construct a path in which the relevant correction can advance without any one participant becoming sovereign over the whole.

III. Corrigibility Is Relative

“Is this state corrigible?” is usually too broad to answer.

A public system may be corrigible for one class of error and closed to another. Constitutional review can make legislation corrigible with respect to constitutional incompatibility. Administrative appeal can make an individual decision corrigible with respect to legal or procedural error. Elections can make officeholders corrigible with respect to political support. Fiscal rules can create correction duties for specified budget deviations.

None of those automatically makes a policy mechanism corrigible when its incentives produce the opposite of its stated purpose.

The object must be bound before the property can be tested:

A grounded challenge is a claim specific enough to be tested against the relevant comparator, accompanied by evidence or a model that can be contested, and admitted through a threshold proportionate to the decision at stake. The merits can remain uncertain. The term means that the challenge has become evaluable, not that it has already been certified true.

A governance arrangement is corrigible relative to this bound object when a grounded challenge can clear each transition:

• Independent entry. It can enter without the operator of the challenged mechanism controlling the only route.
• Stable case. It persists as a stable and contestable case.
• Required disposition. It reaches an actor required to issue a disposition.
• Lawful actuation. When the challenge wins, lawful capacity can change the relevant state.
• Implementation trace. The path remains traceable through implementation and later outcome comparison.
• Reopening. Both the mechanism and the correction process can reopen when the result fails.

Institutions also need commitments, settled expectations, legal finality, administrative capacity, and protection against harassment. A programme that restarts from zero whenever anyone submits a counterclaim is ungovernable. Corrigibility therefore includes correction friction: standing rules, materiality thresholds, evidentiary burdens, time windows, deference where competence warrants it, and closure conditions scaled to reversal cost.

The friction itself must remain challengeable. Otherwise “finality” becomes the mechanism by which one generation binds later evidence forever.

The relevant contrast is change through an owned, legitimate, evidence-sensitive path versus change only through crisis, scandal, personal heroism, or incumbent permission. A system may change constantly while remaining incorrigible: it can adapt around criticism, modify its rhetoric, relocate costs, replace personnel, and preserve the mechanism that produced the failure.

A system may also reject a challenge and remain corrigible. The decisive question is whether the distinct claim was preserved, tested under the relevant authority structure, and issued a state that can later be audited.

A corrigible system can reject a challenge. It cannot treat a challenge that never became a case as if it had been disposed of.

IV. The Properties That Stop Early

Several desirable governance properties resemble corrigibility. None is sufficient alone.

This is why the word accountability is too large and too small at once.

It is large because it covers answerability, blame, sanctions, reporting, legal review, electoral replacement, and professional responsibility. It is small because an accountable actor may give a complete answer while the failed mechanism remains in force.

Feedback Authority asks what cost an institution bears for non-response. Its answer can range from decorative acknowledgment to legal or fiscal consequence. Governance corrigibility includes that result but does not end there. A response duty can produce a reasoned no, a procedural remand, or a public override without producing implementation of an accepted correction.

Implementation Ledger owns the next transition: an accepted response becomes operational only when a stable record assigns the decision object, owner, resources, deadline, execution state, verifier, and reopening rule.

Corrective Closure Ownership owns the later transition: who is required and able to notice when reality disagrees, reopen the decision, repair it, and account for what changed.

There Is No Exception Handler owns an earlier one: what happens when no ordinary process can own the challenge at all. Its handler carries the no-match state until accepted handoff or reasoned termination. Governance corrigibility is the complete path across those states.

These properties have to compose around a bound mechanism and error class.

V. The Corrigibility Trace

The Corrigibility Trace is an audit for that composition.

Begin by naming the mechanism, the purpose or constraint against which it is judged, the error class, the affected population, and the horizon. Then follow a grounded challenge through nine transitions.

A break shows that correction at that point depends on something outside the claimed architecture:

• Referral without acceptance. Referral without acceptance is not a handoff.
• A report without a named respondent. A report without a named respondent is not answerability.
• A response without lawful capacity. A response without lawful capacity is not actuation.
• Acceptance without a transition record. Acceptance without a transition record is not implementation.
• Completion without independent verification. Completion without independent verification is not closure.
• A correction body that cannot be replaced. A correction body that cannot be replaced is not meta-correction.

A failed trace: Robodebt

Australia’s Robodebt scheme is useful because it contained many institutions normally cited as evidence that correction machinery already exists.

Welfare recipients could seek internal review and review by the Administrative Appeals Tribunal. Complaints reached the Commonwealth Ombudsman, which opened an own-motion investigation in January 2017. Its April 2017 report found problems in usability, transparency, service delivery, communication, testing, and risk management; the responsible departments agreed to its recommendations. A 2019 follow-up found significant implementation progress on those recommendations.

The central legal defect nevertheless survived. The scheme used income averaging to raise alleged welfare debts. Justice Emilios Kyrou later summarized that between 2016 and 2022 the tribunal made 431 first-tier decisions holding that debts calculated through the averaging technique could not be recovered, while the department effectively ignored a large number of those decisions. Kyrou also records that some first-tier cases accepted averaging; the 431 are the refusals, not the whole caseload. First-tier decisions were generally unpublished. Evidence before the later Royal Commission also showed that relevant legal material had been withheld from the Ombudsman, hampering its ability to determine whether the scheme was lawful.

The point is not that every institution failed in the same way. The Ombudsman identified and followed up real administrative improvements. Individual tribunal applicants sometimes won. Courts, advocates, journalists, and political actors eventually produced decisive change. A Royal Commission later reconstructed the system and issued fifty-seven recommendations. The new Administrative Review Tribunal subsequently gave its President a function to identify and notify government actors of systemic issues found in the tribunal’s caseload.

The corrigibility failure was compositional. Individual legal contradictions did not reliably aggregate into a stable systemic case. The institution operating the scheme could continue while affected people repeatedly reopened the same question one case at a time. Oversight produced accepted recommendations on parts of administration without forcing the foundational legal issue through an authoritative correction path. The mechanism changed only after years of distributed contest and escalating external force.

Robodebt does not prove that the Corrigibility Trace would have prevented the scheme. It shows why possession of appeals, a tribunal, an ombudsman, internal review, judicial review, parliamentary scrutiny, and eventual inquiry does not itself establish a timely correction loop. The question is how the contradictions compose.

VI. Who May Correct What?

“Reality contradicted the policy” does not identify who may lawfully decide what follows.

Three error classes must remain separate.

The first class is the natural domain of mechanism analysis. A government says that a financing rule will improve efficiency, a tax change will increase employment, a procurement reform will create competition, or a service redesign will shorten waiting times. A grounded challenge can test whether the incentives, resources, actor responses, and implementation path support that claim.

A finding does not automatically determine the policy. The stated objective may conflict with another objective. The mechanism may be uncertain but still worth testing. A rights constraint may rule out the most efficient design. The government may accept a cost for a distributional or symbolic reason.

The correction architecture should force the difference into view.

This is the attraction of repair or explain. The evaluator does not acquire the final vote. The political authority does not retain the ability to present a defeated mechanism claim as if it were uncontested.

But reason-giving alone is not full corrigibility. A government can produce formulaic overrides forever. A court can declare a breach while implementation stalls. An agency can accept a recommendation and leave it unfunded. The path needs an actuator, implementation trace, and later verification appropriate to the error class.

The second class belongs to legal authority. A mechanism analyst may identify a possible legal issue but does not become a court. The third belongs to politics within constitutional constraints. An expert may clarify consequences but cannot derive the legitimate distribution of sacrifice from technical competence.

This is how grounded objections receive landing rights without their authors receiving sovereignty:

• the challenger can require preservation and evaluation when the threshold is met;
• the evaluator can issue a finding within a defined competence;
• the responsible political or legal authority retains the decision assigned to it;
• rejection and override remain possible but attributable;
• implementation and later outcomes remain connected to the disposition.

Governance corrigibility is rule through a process in which claims of correctness can alter the state only through typed, contestable authority.

VII. Partial Corrigibility Already Exists

The individual parts are not inventions.

Democratic experimentalism is the strongest established near-neighbor. Its classic architecture decentralizes problem-solving, lets local actors use contextual knowledge, requires them to report performance, compares alternative approaches, and periodically revises goals and procedures. Global experimentalist governance applies a similar recursive structure across jurisdictions: open-ended problem definition, participatory multilevel implementation, peer review, locally generated knowledge, and periodic revision.

That is a real corrigible-governance topology where the goals, participating units, reporting relationships, and coordinating process can be specified.

The Corrigibility Trace adds a different cut. Any proposed topology must also clear these transitions:

• External challenge. A challenge can originate outside the participating units.
• Unanticipated error class. Correction still works when the mechanism’s error class was not anticipated by the original framework.
• Disputed-object preservation. The disputed object remains stable and contestable while evaluation runs.
• Attributable override. Refusal or political override is attributable to a named authority.
• Implementation movement. Accepted correction moves into implementation, not only into a finding.
• Process replacement. The correction process itself can be replaced when it hardens or fails.

A 2026 preprint on digital public infrastructure uses corrigibility in an even closer architectural sense. It defines a system as corrigible when affected participants possess structural access to detect, contest, and override systemic error, and proposes five jointly necessary conditions: exit, inspectable code, audit, governance, and forkability. It correctly distinguishes corrigibility from transparency, accountability, openness, and auditability in isolation.

That work occupies a substantial part of the territory. The present object differs in scope and authority structure. Public mechanisms are not all digital infrastructures; many cannot provide universal exit or forkability without abandoning the public function. Governance corrigibility must also distinguish legal, causal, and political error; preserve an explicit override by legitimate political authority; and trace accepted correction through implementation and outcome verification. The two frameworks should be treated as neighboring architectures, not as one originating the entire idea.

Administrative systems also implement partial traces.

The National Transportation Safety Board (NTSB) attaches safety recommendations to named recipients, requests responses, preserves public status classifications, and can keep recommendations open while action remains inadequate. The U.S. Government Accountability Office (GAO) maintains recommendation records and follows implementation. Courts provide authoritative correction for specified legal errors. Ombuds institutions create protected complaint and investigation routes. There Is No Exception Handler describes aviation reporting and unified incident command as bounded ways to preserve anomalies or assemble temporary cross-jurisdiction ownership.

These systems show that corrigibility need not be concentrated in one super-agency. A distributed architecture can qualify when its transitions are mandatory enough, accepted, and traceable.

They also show how far the general case remains from routine.

The Organisation for Economic Co-operation and Development (OECD) Government at a Glance 2025 reports a 2024 iREG average of 1.34 out of 4 for ex-post evaluation of primary laws. Only nine of thirty-eight countries with data, plus the European Union, reached two or above. Only seven required periodic evaluation of all primary laws, with another four covering all major laws. Most systems remained ad hoc or partial.

Finland already has evaluative capacity. Regulatory impact assessment is required for primary laws. The Finnish Council of Regulatory Impact Analysis reviews selected assessments and may review ex-post evaluations. Finland adopted common ex-post-evaluation principles in 2023. But ex-post evaluation is not mandatory, the Council’s opinions are advisory, and the architecture does not by itself guarantee an answerable owner, actuator, implementation trace, and reopening path for every significant mechanism. The OECD’s 2026 Finland review therefore recommended a comprehensive ex-post-evaluation system with stronger oversight and systematic incorporation of feedback.

The empirical claim should remain bounded:

Modern states contain many corrigibility components and several real correction loops. They do not thereby demonstrate a general Corrigibility Trace for public mechanisms across error classes and institutional boundaries.

The first burden is to show the existing functional equivalent—or show where the trace breaks.

VIII. The Corrector Can Harden

A correction system introduces a new mechanism. It inherits every problem it is meant to solve.

• Admissibility monopoly. The body that decides which challenges are grounded can control the public agenda. A narrow method, professional vocabulary, or institutional interest can classify external evidence out before scrutiny.
• Legibility bias. Quantified, polished, institution-native claims can dominate tacit knowledge, testimony, early anomalies, and harms that are difficult to aggregate. Requiring a compiled object can reproduce the blind spot the process was designed to repair.
• Reason laundering. A formal response can make a decision appear corrected while the answer consists of boilerplate, scope narrowing, or a restatement of the original assumption. “Reasons given” is not the same state as “contradiction processed.”
• Veto accretion. Every past failure can generate a new gate. The accumulated correction architecture then consumes the capacity needed to act, experiment, and learn. A system can become so corrigible on paper that no mechanism can be changed in practice.
• Corrective capture. The evaluator can become dependent on the institutions it reviews, recruit from the same professional culture, select low-risk cases, or protect its mandate by finding perpetual need for itself.
• Implementation theatre. Accepted findings can become programmes, roadmaps, steering groups, and status reports without moving the mechanism. The correction path then produces process rather than correction.
• Instability. A low threshold for reopening can destroy reliance, legal certainty, long-term investment, and institutional memory. Some commitments work because they cannot be casually revised by the current majority.
• Meta-regress. “Who corrects the corrector?” can generate an infinite tower of review. No finite constitution eliminates ultimate judgment, uncertainty, or political conflict.

The repair is a correction architecture that exposes and limits its own powers.

At minimum the correction architecture itself needs these controls:

• Bound jurisdiction. Jurisdiction is tied to named mechanisms and error classes.
• Plural ingress. Challenges can enter through more than one admissibility gate.
• Published criteria. Criteria are public, and samples of rejected cases are preserved.
• Separated powers. Investigation, political closure, implementation, and verification are separated where structural interests conflict.
• Scaled friction. Correction friction scales with materiality, uncertainty, and reversal cost.
• Protected experiments. Protected experiments and time-bounded provisional decisions remain available.
• Implementation evidence. Status language cannot substitute for evidence that the mechanism moved.
• Error audit. False positives, false negatives, delay cost, and administrative burden are audited.
• Succession paths. Sunset, periodic reauthorization, succession, and abolition pathways exist.
• Comparative burden. The correction mechanism is compared against lighter distributed alternatives before it accumulates more gates.

There is no infinite tower. At some point a constitutionally authorized body makes a political or legal decision under uncertainty. Corrigibility requires that the unresolved state, reasons, dissent, and later outcome remain available to the next authorized correction path. It does not promise a final viewpoint outside all institutions.

When Ownership Is the Wrong Repair supplies the relevant warning: a correct diagnosis can compile into a captured owner, a decorative report, an overloaded process, or binding mass that destroys the capacity it was meant to protect.

A corrigibility institution that cannot be challenged, evaluated, replaced, or retired fails its own trace.

IX. The Corrigible State

No single organizational form follows from the definition.

A legislature may build its own analytical office. Courts may own legal contradictions. Audit institutions may maintain implementation and outcome records. Ministries may run adaptive programmes under external verification. Citizens and affected groups may receive bounded standing. Several specialized bodies may connect through mandatory handoffs and shared case identity. One independent authority may own only the cross-boundary gaps.

The topology is open. The function is not satisfied by naming its components.

For a specified class of public mechanisms, a corrigible arrangement must let grounded contradiction enter, remain intact, reach legitimate judgment, activate a lawful change when it wins, survive implementation, meet later evidence, and expose the correction machinery to the same process.

The Fourth Branch is one institutional form topology for part of that function: independent mechanism analysis, public findings, and a political response path. It is not the definition of governance corrigibility and need not be its only implementation.

The status quo is not a neutral baseline. Existing rules already allocate standing, silence, delay, agenda access, proof burdens, and the cost of uncorrected failure. Incumbency supplies operating history, tacit capacity, and path dependence. It does not supply acquittal. Once a plausible break in the Corrigibility Trace has been shown, the comparison becomes bilateral: the challenger must specify the proposed repair and its risks; the incumbent must identify the current functional equivalent or defend leaving the break in place.

Receiving criticism, answering it, or changing is not yet corrigibility. The property appears only when contradiction can become authoritative correction without becoming the private sovereignty of whoever first identified it.

A corrigible state is one in which no incumbent controls every gate between error and repair.

Related:

• Mechanism Realism — why durable institutions need metamechanisms that detect drift and can revise or retire the mechanisms themselves.
• The Governance Alignment Problem — the adjacent question of whether public optimization serves its adopted objective.
• There Is No Exception Handler — the ingress failure when no ordinary process can own the challenge.
• Feedback Authority — the mechanism vector that determines what non-response costs.
• Implementation Ledger — the transition from accepted correction to owned execution and preauthorized re-entry.
• Corrective Closure Ownership — who must notice later divergence and reopen the decision.
• When Ownership Is the Wrong Repair — why the correction architecture can select a captured or destructive actuator.
• The Fourth Branch — institutional form for persistent correction institutional topology for independent mechanism assurance and political response.
• Civilization’s Blind Spot — why corrections lost before case formation disappear from the historical denominator.