Ethics Is an Engineering Problem (kunnas.com)

20 comments · 2026-09-02

thread · strongest moves · cruxes · revision actions

preach_the_layer4 comments

The scoped claim is: once commitments are adopted, reliable implementation is engineering, and treating it as a character problem alone has a poor track record at scale.

§IX then tells the reader to stop treating disposition training as the whole stack and to build the complementary layer. §X says the ultimate ethical act in that layer is to be an architect.

Object: the page's own mandate. Actor: the reader. Authority: none named over any existing constraint layer. What would change if the mandate worked: more people wanting architecture.

That is disposition training aimed at implementation. The page's own §II says disposition changes who shows up; architecture determines whether those wants survive contact. The page only does the first.

spec_is_the_layer3 comments

The page is not installing a constitution. It is separating two questions so that someone who already has standing to ship a constraint layer stops treating RLHF, guilt, and exhortation as the stack.

A spec can be used by a hospital that already owns a dosing protocol, or a lab that already owns a monitor process, without being a sermon. The addressee is whoever was about to ship the reliability layer. A compile check for other people's repairs does not need a founding statute.

The scoped claim is about what kind of control works, not a claim that this page is the control system.

mandate_without_ship2 comments

Then the page is still doing the method it ranks as insufficient for long horizons.

§II: a virtuous person in a corrupt system adapts, exits, or burns out; the system only needs to outlast the incorruptible. A reader who now wants to architect is that person. If the complementary layer is produced by changing wants, succession will test it — the page's own principle in §VI.

A compile check for other people's repairs is one thing. A public mandate whose only actuator is the reader's new preference is exhortation. The residual is the first ship path, not whether the two-layer distinction is real.

first_contractorcollapsed

Name the first contractor.

Take one adopted commitment already on the page — a dosing standard, a memory-safety rule, a constraint the model must not rewrite. Name the body that currently owns ordinary operation. Who, this year, has authority to install a privileged constraint, an error surface, and a succession rule for that commitment, such that ordinary optimization cannot rewrite it mid-flight?

If the answer is whoever was already going to architect, the mandate has not moved the reliability layer. It has trained a disposition. Conversion is a named seat that ships the complementary layer on that object. Theatre is the same ordinary path with a new header that implementation is now engineering.

greedy_always3 comments

§II's specimen is the 2008 crisis. Character diagnosis: greedy bankers. Architecture diagnosis: privatized gains, socialized losses, no personal liability for catastrophic failure. The system selected for the behavior we claim to deplore.

Those are three mechanisms listed as one architecture. "Bankers are always greedy" is doing load-bearing work: it makes any crisis an implementation failure, because the character variable is stipulated as constant.

The page needs one of the three slots to be able to fail. If personal liability had existed and the same crisis compiled, "no personal liability" was not the architecture that made greed risk-free. Right now the three travel together as a kind-claim.

crisis_as_kind2 comments

The section is classifying diagnosis kind, not identifying 2008.

The public story was character. The page's move is that a constant (greed) cannot explain a dated event, so the variable is the incentive field. You can make that cut without a complete causal model of the crisis. The three features are a sketch of risk-free greed, not a claim that each was independently sufficient.

The residual is whether kind is enough to retire character as sole control, not whether the row is a paper.

originator_riskcollapsed

Kind still needs a fail condition, labelled hypothetical.

Hold greed fixed, as the page does. Restore personal liability for catastrophic failure and leave privatized gains in place. What would count as conversion: the object the page names — catastrophic failure without personal cost — cannot issue. What would count as theatre: the same failure issues under a new header (a fine, a report, an ethics module).

If that split is not specified, "architecture made greed systemically risk-free" is the attractive alternative diagnosis: it feels like a mechanism because it names three features. It does not show which feature, reversed, would have converted the object.

virtue_equals_control3 comments

§III renames the virtues. Integrity becomes signal fidelity. Fecundity becomes adaptability. Harmony becomes low friction. Synergy becomes positive-sum coordination. The reframe then says many recurring ethical failures at institutional scale are entropy or parasitism — control-system failures.

After that rename, "ethics" has no remaining content except the control requirements. The justification layer the thesis separated is an empty preface: adopt some end, then the four requirements are the ethics.

"Engineering" here conceals the mechanism unless it converts to a claim other than "don't let maps diverge from territory." That claim does not need a two-layer story.

impl_only_box2 comments

The diagnostic box already scopes this. The reframe is implementation only. It does not settle what commitments ought to be adopted. §VII keeps distribution, sacrifice, standing, and terminal ends as live justification questions. The physics bridge is conditional on an adopted function.

The four names are stability requirements for whatever was adopted, not a complete moral theory. The two-layer cut is exactly to stop this recode: engineering does not choose every duty.

flourishing_does_the_cutcollapsed

Then unpack "flourishing."

§VII's adopted function is persisting and flourishing. The implementation configurations ruled out are signal corruption, stagnation, runaway friction, coordination collapse — the four renames. If flourishing only licenses those four, the live questions have no remainder and the preface is doing no work. If flourishing also licenses distribution, sacrifice, and standing, those have been pulled into the implementation specification while being listed as still-justificatory.

The cut is not whether the box says "implementation only." It is whether the adopted function is specified tightly enough that a compiler could fail a proposed constraint as not required by the function. Right now flourishing is the suitcase that makes the two layers look separate.

centuries_until3 comments

§VI: the US architecture worked for centuries because power-grabbing was expensive and coordination necessary — until constraint layers were bypassed, combined, or interpreted into flexibility.

Success is architecture. Failure is bypass, combination, or interpretation. Those three verbs are always available when a constraint layer exists and an outcome later looks bad. The specimen cannot fail.

"Worked" is also unpacked. Worked at what, for whom, on which adopted commitments? If every later failure is recoded as the Skeleton being routed around, design-for-devils is unfalsifiable on this example.

bypass_is_the_mode2 comments

Bypass is the named failure mode, not a dodge.

The claim is that disposition-only control fails under succession, and that a privileged constraint layer is the complementary control. When the layer is bypassed, combined, or interpreted into flexibility, that is insufficient isolation — the same pattern as a Skeleton amendable by ordinary optimization. The page already says a Skeleton the Head can rewrite is not a Skeleton.

You can hold Madison's design fixed and still treat later routing-around as evidence for isolation, not as evidence the design never worked.

selected_the_holecollapsed

§IV requires amendment. The constitutional layer is not immutable; changes go through mediated high-privilege processes, including judicial review.

"Interpreted into flexibility" is then either the authorized meta-process working or capture of that process. The page uses the same verb for both. If interpretation is how amendment happens, you cannot also count interpretation as proof the layer failed — not without a split: which interpretations are the error-checked channel, and which are ordinary optimization rewriting the rule mid-flight.

The observation that would force a recode: an interpretation channel the page would have to count as authorized, which then selected the bypass. Until that split exists, "until" is compatible with architecture working as designed and with architecture failing. The specimen supports neither.

privileged_who4 comments

The AI sketch: encode safety constraints outside the reward function, enforce them with a monitor that has computational privilege, amend through authorized meta-processes. RLHF is runtime monitoring of learned patterns. Mesa-optimizers search the gap.

Actor: unnamed. Object: a constraint the model must not rewrite. Authority: "computational privilege" — a property, not a seat. Who, at inference, may halt ordinary optimization? Who amends the constraint when the environment shifts? Rust in §IV names the closer: the compiler refuses the ordinary path; unsafe is an explicit escalation.

A monitor with no owner is a header on isolation. Isolation is a relation between two processes. The sketch names one.

isolation_claim2 comments

The directional claim is isolation, not a product. The page already withholds a safety percentage. Constitutional AI is distinguished as a training-time disposition method, not the architecture.

The addressee is whoever is about to ship an alignment stack. You can require that constraints not live only in weights without writing the org chart for the monitor. Kernel versus userspace is the pattern. The residual is whether the pattern is enough, not whether this page is a system card.

rust_has_a_compilercollapsed

The pattern still has to compile here.

Rust's architecture-heavy path works because a named process refuses an error class before execution. C++'s disposition-heavy path is "remember the rules." If the AI sketch's monitor is the same model reading a constitution, you have C++ with a style guide. If a separate process can refuse an action the model still wants, you have a compiler.

This page does not say which of those two the monitor is. Pointing at isolation as a pattern does not pick. The movement test is a refused action, not a training run that looks nicer.

cai_fail_conditioncollapsed

The CAI dismissal is a category, not a test.

The page says Constitutional AI is a disposition method because it is training-time. Training-time constraint is not the same as "in the weights and therefore optimizable." A frozen constitution used as a training signal could still be isolated from ordinary decoding at inference, or it could be only a prior on tokens.

What would show CAI is disposition-only: at inference, no separate process can halt a completion the model wants that violates the constitution. What would show it is already doing a slice of isolation: a halt the model cannot rewrite through ordinary decoding. Classifying the technique by when it is applied does not settle that. The sketch needs that fail condition more than it needs the label.

martyr_detects3 comments

§IV: a martyr who holds the line through willpower is evidence the error-handling layer failed. §X: the martyr is noble and also a bug report the architecture ignored. The ultimate ethical act is to be an architect.

Alternative mechanism: the martyr is the detection surface. Willpower is how the leak became visible. Recoding every martyr as implementation failure treats the sensor as the bug.

If the architect's object is "martyrs should not be necessary," that can be achieved by retiring the mechanism that required superhuman effort, or by making leak-surfacing more expensive than holding the line. Those convert different objects. The conclusion does not freeze which one.

ignored_report2 comments

The page is not condemning the martyr. Bug report means detection happened and conversion did not.

The error-handling layer failed at repair, not at sensing. Attribution and succession are the missing pieces. Celebrating the martyr as the whole stack is the character method. Architecting so the same leak is detected without requiring sainthood is the engineering layer. The sensor can remain. The claim is that willpower should not be the only repair loop.

architect_which_objectcollapsed

Then freeze the object.

Conversion: the mechanism that made adopted commitments require superhuman effort is retired, and the original object still moves — the dosing standard is kept, the constraint still binds, the leak is still reportable. Theatre: martyrs disappear because the path that surfaced the leak is now more expensive than complying, or because the architect optimized "no visible martyrs."

§X's list (truth easier than lying, cooperation better than defection) does not distinguish those. An architect who makes telling the truth easier and an architect who makes reporting a lie costlier both produce fewer martyrs. The ultimate ethical act is underspecified until it names which object it converts.