Governance
Evaluation
Lyotic proposes a narrow, testable hypothesis: binding independent work state, evidence, authority and outcome to a bounded representation improves human supervision. That claim has to be measured against a good alternative, not a weak one.
Boundaries
- Compare against a well-built fixed supervisory interface with the same evidence, authority, proposals and recovery. Beating a chat-only interface alone proves little.
- Keep inputs fixed. Same authoritative state, same agent trace, same allowed actions; vary only the representation. The resolver is deterministic, so this is possible.
- Separate interface from enforcement. A backend that blocks an unauthorised run is not a UI win. Measure understanding and intervention, then runtime enforcement, then both.
- Different checks, different claims. Schema checks, semantic checks, temporal checks (approval then change, late responses, stop, reconnect) and human studies each support different conclusions.
- Test adaptation and animation separately. Whether the form should change, and whether its motion helps, are two questions.
What human studies measure
State understanding · recognising judgment points · catching wrong proposals · appropriate intervention · false belief in completion · understanding of evidence · supervision load · resumption and handoff accuracy · accessibility · transfer to a second domain. Report effect sizes, uncertainty and null conditions. Faster wrong decisions are not improvements.
Synthetic users
Synthetic user ensembles (MatrAIx, UXAgent, RealUserSim, TinyTroupe) can stress scenarios. Each simulated user sees only its role’s real observations. Their agreement is not human understanding, and text-only simulators cannot judge morph continuity.
What the harness covers today
| Check | Covers | Does not cover |
|---|---|---|
checkPlan | INV-02, 03, 04, 06, 07, 11, 13, 19 on the plan | Whether the plan is the best form for a person |
checkDom | Operable controls ⊆ plan; forbidden completion words (lexical, with negation); glyph + label on every status; required disclosures present | The meaning of prose; visual salience; what motion communicates |
npm test | The reference case end to end, both paths; determinism; read-only delegation; collapse keeps pending work | Other domains until their fixtures exist |