Candidate B (futurefork prepare/finalize CLI)
Candidate ref coupled · starting commit 11f9d4e8559e…
Bob task ID5c4c002baa472064dd11971c601c6436
Bobcoins spent2.65
Outcomecompleted
Regression suite70 passed
Protected pack (partial-refund)34 / 34 passed
Protected pack hash after runmatches frozen
Changed files7
Lines changed+558 / -23
Boundary violationsnone
Wall time (finalize)0.795s
Evaluated onisolated detached worktree, evaluated in place by `futurefork finalize`
Changed: src/pocketcart/checkout.py, src/pocketcart/discounts.py, src/pocketcart/refunds.py, src/pocketcart/service.py, src/pocketcart/store.py, src/pocketcart/wsgi.py, tests/test_partial_refund.py
Candidate A (manual, pre-CLI)
Candidate ref modular · starting commit f9e3ecda1e4a…
Bob task IDd12b84c1a6a03a337b56b8594ebd6d2a
Bobcoins spent5.40
Outcomecompleted
Regression suite68 passed
Protected pack (partial-refund)34 / 34 passed
Protected pack hash after runmatches frozen
Changed files6
Lines changednot recorded (pre-manifest run)
Boundary violationsnone
Wall time (finalize)not recorded (pre-manifest run)
Evaluated onsnapshot copy of the worktree (oracle never mounted in the Bob worktree)
Changed: src/pocketcart/checkout.py, src/pocketcart/refunds.py, src/pocketcart/service.py, src/pocketcart/store.py, src/pocketcart/wsgi.py, tests/test_partial_refund.py
Disclosed deviations for this run- Oracle-informed feedback. After the first oracle run, Bob was told the response status must be `refunded`. The value is in the frozen public interface, but the trigger was a protected-test failure. If the same treatment is not given to every candidate × scenario run, the comparison is unequal. Decide one policy (feedback allowed, counted as an attempt, for all runs; or no feedback) before running `coupled`.
- Lost attempt-1 patch. A cleanup bug deleted the worktree (see incident). The final implementation is Bob's re-application from its own task history, not the original attempt-1 output, so attempt 1 cannot be inspected. The 33/34 first result is reported only.
- Bob could not run tests. Bob's Python has no pytest, so its development tests were run by the operator from outside the worktree. Bob still wrote them. Give the next run the same arrangement or fix the environment for both.
- Boundary probing. Bob reportedly inspected system Python paths (outside the workspace). Reportedly no FutureFork repo or pack access; not verifiable from artifacts.
- Packet fidelity. The task packet's illustrative response used `"order_id": "ord_0001"`, but real ids are `order_0001`. It did not affect the result but the packet was not generated from the frozen YAML byte for byte.
- Unmount bug. Do not reuse the earlier unmount snippet. Evaluation now happens on a snapshot copy; the oracle is never mounted in Bob's worktree.
Both candidates independently satisfy the same frozen, hidden protected test pack for partial-refund (pack hash 618a9aba91cf9f24…, unchanged after each run).
Both runs are traceable to a real IBM Bob task ID, a real Bobcoin spend, and a real task-session screenshot in bob_sessions/.
The coupled run has zero disclosed human interventions and zero boundary violations; the modular run has 6 disclosed deviations (oracle-informed feedback, a lost first attempt after a cleanup incident, and others — see its human-notes.md), which is a real fairness asymmetry between the two runs, not evidence that one candidate is better.
This does not show which architecture is "more maintainable" — one scenario is not enough evidence for that, and the product's own rules (docs/EVALUATION.md) forbid that claim regardless.
This does not represent the full 2×3 matrix. organization-pricing and ordered-discounts have no runs yet.
Nothing here is a benchmark, a score, or a prediction. It is two disclosed, reproducible executions of the same frozen requirement.
