# Detailed protocols for the personal-agent harness research roadmap

Status: **Proposed**. These designs, workloads, thresholds, sample sizes, implementations, and studies have not been executed. This appendix supports [After Meta Muse: A Research and Publishing Roadmap for Personal-Agent Harnesses](/blog/personal-agent-harness-research-roadmap-after-muse).

The main article defines the shared protocol, evidence labels, statistical interpretation, first executable publication, and the complete Study 3 recovery design. This appendix preserves the detailed protocols for Studies 1, 2, and 4–8 without making the main argument depend on seven additional full-size tables.

## Current contract baseline for proposed experiments

The studies remain unexecuted proposals. Current ContextOS reference validators and synthetic workflow fixtures do not constitute these studies' results or deployment conformance. Use the [execution contract](/docs/implementation/execution-contract), [exact control profiles](/docs/reference/control-profiles) and [conformance evidence rules](/docs/reference/conformance) as the control baseline when implementing a candidate. Pin the exact profile/control versions and actual workflow-feature union; support declarations and local scores grant no effect authority.

Every candidate that dispatches effects must independently verify current owned mandates, all ancestors and revocation, full nested action/evidence/policy/profile/release binding, current entitled approval quorum, and active actual containment/credential/data paths. Protected atomic ownership, leases and target fencing govern continuation. Preserve original root/child spent and reserved capacity across retry, delegation, cancellation and resume. Unknown effects require authoritative reconciliation or a verified provider retry basis; stop requests cannot erase already dispatched effects. Apply actual recipient/processor rules at all nine data stages and retain owned memory-copy inventories with target-specific propagation evidence. Hold these mandatory controls fixed when studying optional reasoning components.

Report code-only reference results separately from actual implementation/release/environment observations, with independent artifacts and observers plus positive, negative, fault and utility cases. Missing, stale, synthetic or unevaluated evidence cannot establish deployed conformance. Proposed counterparty-specific schemas and human studies below remain illustrative research extensions; the current control baseline is not deferred until those studies finish.

## Study 1: credential confidentiality is not credential safety

Meta documents isolated built-in connector workers and credential surrogates, and separately permits agent-authored connectors. This study tests a synthetic reference implementation—not Muse—and evaluates the two connector boundaries separately.

| Protocol element | Proposed protocol |
|---|---|
| Decision | Decide whether credential isolation constrains unauthorized use as well as raw-secret exposure. |
| Hypothesis | A scoped worker plus execution-time effect validation reduces unauthorized effects without an unacceptable increase in legitimate actions blocked. |
| Baseline and candidates | Baseline: connector code and credential in the reasoning runtime. Candidates: surrogate credential with a built-in immutable worker; surrogate credential with an agent-authored connector; each with and without execution-time authorization binding. Analyze the two connector classes separately. |
| Controls and confounders | Hold assigned tasks, model, policy, credential scopes, supplier state, prompt-injection corpus, and tool schemas constant. Connector implementation freedom is a deliberate treatment. Cover account substitution, recipient substitution, cross-connector access, and reuse outside authorized scope. |
| Primary endpoints and denominators | Report both: (1) successful unauthorized effects divided by all assigned adversarial tasks; and (2) unauthorized effects permitted divided by unauthorized actions presented to the enforcement boundary. Also report legitimate actions incorrectly blocked divided by policy-valid legitimate attempts. |
| Secondary outcomes and failures | Raw-secret exposure per extraction attempt; surrogate misuse per surrogate-bearing attempt; number of action attempts generated; severity; abstention; approval burden; latency; and incomplete evidence. Raw-secret exposure, surrogate misuse, unauthorized effects, and false blocks remain separate classes. |
| Reject or weaken | Reject if a candidate hides raw secrets but preserves attack success or enforcement-failure rate, or if gains depend on blocking most legitimate work. Weaken if the result does not transfer between built-in and agent-authored connectors. |
| Artifact and publication gate | Release the synthetic worker, connector variants, assigned-task manifest, attack families, policy versions, and effect ledger. Enumerate every unauthorized effect and false block. Zero raw-token leakage is insufficient. |

## Study 2: make policy uncertainty observable

| Protocol element | Proposed protocol |
|---|---|
| Decision | Decide which decisions must remain deterministic and where bounded semantic review adds value. |
| Hypothesis | Deterministic invariants plus bounded model review and abstention reduce false allows relative to model-only review without the false-denial burden of invariants alone. |
| Baseline and candidates | Compare deterministic policy, model review, hybrid review, and hybrid review with abstention. Every abstention enters a bounded path with a named resolver, deadline, and fail-closed outcome for consequential effects. Model review cannot waive deterministic mandatory controls; malformed applicable policy expressions stop compilation with structured errors rather than becoming false predicates. |
| Controls and confounders | Freeze typed effect schemas, policy version, cases, reviewer prompt, model version, and risk-class mix. Include plausible natural-language descriptions that conceal a different typed effect. Candidate outputs remain hidden from reference adjudication. |
| Primary endpoint and denominator | False allows divided by independently adjudicated denied cases, reported by risk class. Produce reference verdicts before candidate scoring from the typed effect, declared user authority, service policy, and a documented two-reviewer disagreement process. |
| Secondary outcomes and failures | False denials per allowed case; abstentions per reviewed case; resolution outcomes; unresolved abstentions at the fixed horizon; human review minutes; latency; disagreement; concealed-effect failures; and policy-version drift. |
| Reject or weaken | Reject if the hybrid increases false allows, routes critical cases into unresolved abstention, or gains accuracy only through materially higher review burden. Weaken when reference agreement or risk labels are unstable. Failure to detect a difference is inconclusive, not evidence of equivalence. |
| Artifact and publication gate | Publish typed cases, descriptions, protected verdicts after evaluation, adjudication notes, and confusion matrices by risk class. No pooled accuracy score may conceal a critical false allow. |

## Study 4: memory retrieval is not lifecycle governance

| Protocol element | Proposed protocol |
|---|---|
| Decision | Decide whether governed memory improves useful personalization without allowing stale, poisoned, or deleted information to control later behavior. |
| Hypothesis | Provenance, correction, expiry, deletion propagation, and instruction precedence improve repeated-task utility while reducing governance failures. |
| Baseline and candidates | Baseline: retrieval-only memory. Candidates progressively add provenance and promotion, correction and expiry, deletion propagation, and poisoning defenses across current-task context, episodic records, semantic beliefs, and procedural rules. |
| Controls and confounders | Hold synthetic users, task sequence, model, context budget, and retrieval budget constant. Counterbalance preference order. Include current instructions that conflict with inferred history and derived memories created from summaries. Start only after the Study 3 artifact passes its publication gate. |
| Primary endpoints and denominators | Useful personalization successes divided by repeated interactions with a valid relevant preference, reported alongside governance failures divided by correction, expiry, deletion, poisoning, and override challenges. Neither rate substitutes for the other. |
| Secondary outcomes and failures | Current-instruction override accuracy; correction adoption; stale retrieval; poisoned promotion; derived-memory resurrection; deletion propagation time; unnecessary clarification; latency; and user intervention. |
| Reject or weaken | Reject if current instructions fail to override inferred history, governed deleted data remains retrievable or is resurrected through a derived record, or preference confidence becomes permission to act. Weaken if personalization gains disappear after controlling for extra context. |
| Artifact and publication gate | Release the eligible, minimized longitudinal corpus, provenance graph, owned deletion inventory, target-specific propagation receipts, restored-backup tests, and outcomes by memory layer. Distinguish confirmed, pending, failed and external-unknown copies; delete permission alone does not establish completion. State which summaries, indexes, caches, exports, embeddings, backups, and training-data handoffs were in scope and which were not verified. |

The longitudinal case uses one synthetic preference. Earlier interactions suggest premium office supplies; the current task explicitly requests budget products, so the current instruction wins. Later the user corrects the durable preference to “prefer standard products unless I ask otherwise.” The old preference must not reappear through a summary or procedural rule, and neither version grants purchasing authority.

## Study 5: counterparty permission is a separate mandate

User mandate, runtime identity, service policy, and payment constraints remain distinct. [GeekWire reported Amazon's assertions about Muse](https://www.geekwire.com/2026/amazon-blocks-metas-muse-ai-assistant-in-new-standoff-over-agentic-shopping/); that report motivates a protocol question but does not establish the disputed assertions, resolve contract interpretation, or support a legal conclusion.

| Protocol element | Proposed protocol |
|---|---|
| Decision | Decide whether explicit counterparty constraints prevent a delegated agent from treating user permission as universal service permission. |
| Hypothesis | Separately evaluating user mandate, runtime identity, service policy, and payment constraints reduces denied or out-of-scope effects across delegation. |
| Baseline and candidates | Baseline: user approval plus browser or API reachability. Candidate: proposed signed fixtures for all four mandate classes, revalidated at each delegated hop. These fixtures are not evidence of a deployed interoperable standard. |
| Controls and confounders | Freeze user goal, service state, payment limits, agent chain, and disclosure mode. Test narrowing, denial, expiry, and revocation before dispatch and during delegation. |
| Primary endpoint and denominator | Effects executed despite any controlling mandate's denial, expiry, revocation, or limit divided by assigned mandate-conflict cases. |
| Secondary outcomes and failures | Legitimate effects blocked; disclosure failures; revocation propagation time; delegation-chain narrowing; payment-limit violations; retries after denial; and user or service appeals. |
| Reject or weaken | Reject if a child agent regains authority removed upstream, a denied service is bypassed or disguised, or a payment credential expands the user mandate. Weaken if results depend on simulator-only semantics that real service interfaces cannot express. |
| Artifact and publication gate | Release the proposed mandate schemas, counterparty simulator, signed test vectors, revocation cases, and dispute log. Include denied paths and do not describe the fixtures as an adopted standard. |

## Study 6: approval interfaces belong on an extension track

| Protocol element | Proposed protocol |
|---|---|
| Decision | Decide whether text, voice, or avatar presentation changes decision quality for the same consequential effect. |
| Hypothesis | Modality changes comprehension, calibration, and social-pressure errors even when substantive content is matched. |
| Baseline and candidates | Text, voice, and avatar conditions use the same facts, order, uncertainty disclosure, recommendation, and response choices. Only delivery changes. |
| Controls and confounders | Randomize condition order or use a counterbalanced crossover; model repeated exposure; rotate scenarios; measure prior assistant use; and predefine accessibility accommodations. Recruitment assumes consenting adult digital-service users and includes relevant access needs. |
| Primary endpoint and denominator | Decision-quality score—correctly identifying the effect, uncertainty, reversibility, and authorization consequence—divided by scorable approval decisions. Engagement and interface preference are not part of this endpoint. |
| Secondary outcomes and failures | Approval correctness, recall, confidence calibration, response time, accessibility failures, perceived pressure, repeated-exposure learning, approval fatigue, dropout, and missing outcomes. |
| Reject or weaken | Reject if a confirmatory comparison excludes the preregistered minimum practically important difference under matched content. An imprecise null result remains inconclusive. Weaken any safety claim when preference rises without decision quality, fatigue reverses the result, or accommodations materially change exposure. |
| Artifact and publication gate | Preregister materials, randomization, exclusions, missing-data handling, and analysis. Use a pilot to estimate within-participant variance and fatigue, then power the confirmatory sample for the preregistered effect with attrition allowance. Start only with separate staffing, ethics review, accessible production, and recruitment budget. |

## Study 7: compare complete deployment options

Spark versus Glimmer changes model capability and deployment simultaneously. The primary comparison is therefore between complete deployment options, not a clean estimate of locality.

| Protocol element | Proposed protocol |
|---|---|
| Decision | Decide which complete deployment option satisfies a declared workload and threat model, and which differences can be attributed specifically to deployment. |
| Hypothesis | Some complete local configurations reduce declared data-path exposures or latency while introducing capability, hardware, or operator costs that alter accepted outcomes. |
| Baseline and candidates | Compare pinned Spark/cloud and Glimmer/local systems under one harness. Where licensing and infrastructure permit, add a same-model local/cloud pair to isolate deployment effects. |
| Controls and confounders | Match tasks, tools, policy, memory, context, sampling settings, and resource ceilings. Record model version, quantization, inference engine, hardware, context limit, tools, wall-clock, energy and cost budgets, and every telemetry or update path. |
| Primary endpoint and denominator | Accepted outcomes satisfying all workload constraints divided by attempted protected tasks, reported for each complete deployment option. The same-model pair estimates the narrower deployment effect. |
| Secondary outcomes and failures | Capability errors, unsafe effects, time to first token, total latency, energy, cost, memory pressure, unsupported tools, operator minutes, data-path exposure, and unresolved outcomes. |
| Reject or weaken | If an advantage disappears in the same-model pair, weaken attribution to local versus cloud execution; do not automatically invalidate an observed advantage of the complete option. Weaken complete-option claims when unmatched budgets explain the difference, and privacy claims when prompts, tool data, telemetry, crash reports, updates, or backups cross the declared boundary. |
| Artifact and publication gate | Publish manifests, model and hardware details, quantization, harness configuration, task results, resource traces, and a data-flow map. Local inference alone never earns a privacy conclusion. |

## Study 8: separate reasoning minimalism from control minimalism

JAZ reports results for a minimal recursive `invoke` primitive on parts of two benchmark workflows. Those are reported results from the v1 preprint, not independently reproduced findings and not evidence about personal-agent enforcement.

| Protocol element | Proposed protocol |
|---|---|
| Decision | Decide whether specialized reasoning components improve accepted outcomes enough to justify their cost while enforcement remains fixed. |
| Hypothesis | Under a proposed non-inferiority margin declared before protected evaluation, a minimal recursive candidate may retain acceptable reasoning outcomes without implying that external controls can be reduced. |
| Baseline and candidates | Compare a JAZ-inspired minimal recursive candidate with a specialized planner, retrieval, memory, and review candidate. Keep identity, policy, authorization binding, sandbox, tool gateway, ledger, external-state evaluator, and release gates identical. |
| Controls and confounders | Match tasks, attempts, context, tools, models where feasible, and token, call, latency, and cost budgets. If models differ, label the result as a deployment comparison rather than architectural isolation. |
| Primary endpoint and denominator | Paired difference in accepted outcomes over attempted protected tasks, with uncertainty evaluated against the preregistered proposed non-inferiority margin. Critical control violations remain separate acceptance constraints. |
| Secondary outcomes and failures | Long-range recall, recovery, unsafe effects, interpretability, latency, cost, tool calls, human intervention, unresolved outcomes, and component interactions. |
| Reject or weaken | Reject non-inferiority when the uncertainty interval crosses the declared margin or critical constraints fail. If the interval is too wide, remain inconclusive; a small point estimate or non-significant difference is not evidence of equivalence. Sequential ablation alone cannot identify component value because removal order matters; use factorial or order-randomized ablations where feasible and report interactions. |
| Artifact and publication gate | Release both candidates, the proposed margin and its justification, matched budgets, component-on/off matrix, interaction estimates, and failures. Fewer reasoning components must not be described as evidence that fewer enforcement controls suffice. |

## Shared publication gate

Every study publishes its dated workload and threat model; tasks, attempts, repetitions, attack-family coverage, protected split, clustering unit, resource budgets, effect sizes, uncertainty, costs, latency, human intervention, missing telemetry, and unresolved outcomes. Adaptive attacks remain an adaptive search trace beside a fixed protected suite. Zero observed critical violations is a necessary suite gate, not proof of zero real-world risk.

## Publication and stop gates

Every experimental publication should ship:

- the decision and proposed hypothesis stated before results;
- a dated workload, threat model, protocol, acceptance constraints, and stop conditions;
- pinned model, harness, policy, tool, provider, evaluator, and environment manifests;
- task, attempt, repetition, split, attack-family, and clustering definitions;
- executable fixtures or a precise reason they cannot be released;
- outcome counts, denominators, effect sizes, uncertainty, missingness, and resource accounting;
- failures, abstentions, interventions, and unresolved external states;
- consistent evidence labels and a correction path.

Pause or weaken a study if the protected set influences development; external truth exists only in candidate narration; a runtime changes mid-study; authorization, cost, failure, or intervention telemetry is materially incomplete; participant protections are insufficient; an attack risks a real third party; the candidate can modify its evaluator or oracle; a denied service must be bypassed; or provider ambiguity cannot be represented without inventing success. An inconclusive study remains inconclusive.

The longer research horizon should emphasize portable questions: proof-carrying effects, memory governance, counterparty-aware identity, bounded adaptation, human agency, and continuity of identity, purpose, evidence, and authority across interfaces and delegated agents.