Research plans across their lifecycle

Write for the people who will design, execute, and revisit the work

The plan's authors may be different from its reviewers, executors, or later auditors. Keep the record in the notebook's version history so each can recover the design that applied at the time. The plan can be revised; its earlier versions remain traceable.

Use harness planning for the execution task

An agent harness is the application that runs the agent. Its planning tools can save, revise, approve, and resume plans for assigned tasks. A notebook plan preserves the research undertaking across its full lifecycle, including evidence and decisions needed after any one task ends. Both kinds of plan can survive a session.

Both can guide the same pilot. Several agents might critique the notebook design first. An executing agent can then use its harness plan to implement the approved scoring script and run the pilot. The experiment owns the measurements; the notebook plan records their consequence for the next phase. Pass the plan path and authorized phase to the executor, rather than maintaining a second copy of the research design in its task plan.

A small code repair may need only a harness plan. Use a notebook plan when the research design, decisions, and resulting work need a shared, lasting account.

Choose the decision the plan will organize

The same plan structure can serve a research line, a paper revision, or a proposed scope expansion. The purpose determines the objective and the evidence needed to finish. Work is divided into packets: pieces with defined limits that someone can take responsibility for. Each purpose uses the same schema and status values. State it in the objective; no plan-type field is required.

Follow a line of research

Organize packets around tests that change what the project believes or does next. In the synthetic retrieval project, one explanation might attribute a difference to retrieval quality and another to a scoring artifact. A scoring-validation packet precedes an interpreted comparison; later packets distinguish explanations that remain viable. A gate specifies a condition that must be met before dependent work can proceed.

Define how much uncertainty the campaign will try to resolve and what stops it. A valid negative result can complete the plan. A failed instrument may instead leave the question unanswered and require a reviewed repair or abandonment. Keep the answer in the experiments and findings, with the plan preserving the sequence of decisions that led to it.

Strengthen a paper or respond to review

Start from named claim weaknesses or reviewer comments. Link each concern to the affected claim, manuscript passage, and existing evidence. A packet might clarify an explanation, check an analysis on retained observations, or run a newly justified control. A request for stronger evidence does not automatically require new data, and every comment need not become a separate experiment.

If a reviewer questions whether the retrieval result depends on one scorer, first identify what the existing scoring checks establish. Gate any additional measurement on the gap that remains and its cost. Close the concern with a reviewed disposition: new evidence, narrower wording, a documented limitation, or an evidenced explanation for retaining the claim. Update the manuscript and submission packet through the paper-revision workflow; retain the response-to-evidence mapping in the plan.

Decide which scope expansions to pursue

Inventory plausible extensions before committing to them. In the retrieval project, candidates might include another language, another document population, or additional diagnostic observations during an already planned run. For each, record the question it could answer, which claim it might change, what data and instruments it needs, and its incremental cost. Separate a cheap recording change from a new analysis or confirmatory test.

Review those options against the project's objective and budget. Define work packets and appropriate design or confirmation gates for adopted extensions. Record why other options were deferred or rejected. The plan can close after selection if that was its objective. If it also includes execution, closure must account for the authorized packets.

These uses can lead into one another. A paper-review plan may reveal a new research question; a scope-expansion plan may commission a successor investigation. Link the successor and its rationale instead of silently enlarging the original objective. The evidence network connects those plans to the experiments, claims, and papers they concern.

Bound the objective before dividing the work

Use a plan when several experiments or decision-gated phases serve one objective, or when execution and review must survive a handoff. A single self-contained measurement usually belongs in an experiment. A reminder to consider a direction belongs in PRIORITIES.md until it needs an objective, work phases, and review decisions.

Start with the question and its existing evidence. Link the experiments, findings, and claims that make the next decision worth taking. “Improve retrieval” leaves every result open to another attempt. “Decide whether this candidate merits adoption on the named benchmark under the registered pilot and confirmation rules” provides an exit. Exclude model changes, other corpora, and production deployment if those are outside the decision. An exclusion prevents an interesting side result from silently expanding the plan.

Create a dated file in plans/draft/. Name an owner who maintains it and a reviewer who decides whether phases proceed. Give the body its objective, existing evidence, phases, risks and controls, and terminal conditions. Use the record reference for the exact contract.

Define each work packet's inputs, output, and limits

Give each packet inputs, an outcome to produce, dependencies, and a resource limit. It should produce something another packet or reviewer can use: a validated scorer, a processed comparison, or a reproducible analysis. “Investigate retrieval” is too open to hand off; “validate the scorer against these known-answer cases and record its failures” has an exit.

In the retrieval project, one packet could validate scoring and another could perform the paired pilot. The pilot requires validation evidence that passes the scoring gate; finishing the scoring implementation alone is insufficient. Packets may be independent when they do not share an unresolved input or decision.

A phase groups packets whose results support the same review decision. The development phase might contain both scoring validation and the pilot; its outcome determines whether to design confirmation. Implementation work may support one experiment, while a comparison packet may coordinate several related tests. Link the scientific records to the packets that use them; the mapping need not be one-to-one.

Use stable packet labels within the plan so reviews and handoffs can name the same piece of work. The work-packet reference gives a body-record pattern; packet names and fields are not a separate scheduler protocol.

Make state, budget, and review independently visible

These records answer different questions. A plan marked active can contain one running packet, another blocked on evidence, and a future phase awaiting review. A reviewer being named does not mean the design has been approved.

For a ten-CPU-minute development phase, allocating two minutes to scoring checks leaves eight for the pilot; it does not authorize ten minutes for every retry. When the work threatens its limit, stop for a budget or design decision and retain the partial evidence. The plan records the limit; runner controls and execution supervision enforce it.

During collaborative design, new critiques may change a method, threshold, or resource requirement. Preserve the review trail and seek approval for the revised scope. An earlier approval applies only to the work it covered. The metadata reference distinguishes parsed plan frontmatter from budget and review information recorded in the body.

Use gates to connect phases to evidence

Each phase needs available inputs, bounded work, an acceptance rule, stop branches, and expected notebook updates. Specify the dataset and code identities, required controls, and missing dependencies. Put cost and execution limits beside the work so an executor sees them before launching it.

Name the evidence that decides each gate and what follows if it passes, fails, or cannot be evaluated. A validity gate asks whether the measurement can be interpreted. A futility gate asks whether this campaign should continue. A human review gate records the decision about the evidence and the scope authorized next.

### Phase 1: Development pilot

- Inputs: frozen baseline and candidate; development sample A;
  registered scoring check in the pilot experiment.
- Bound: one paired comparison, at most 10 local CPU minutes.
  No paid compute, reserve access, or configuration search.
- Invalid scoring or missing inputs: stop, record the problem,
  and request review; this does not answer the research question.
- Valid pilot, candidate improvement at or below zero:
  recommend closure under the registered futility rule.
- Valid pilot, improvement above zero: propose Phase 2 design
  for review. Do not open the confirmation reserve.
- Record: inputs, job identity, artifacts, checks, and outcomes
  in the pilot experiment; link its disposition here.

The futility branch settles whether this campaign continues under its chosen rule. It does not establish that the candidate can never help. Define that limit in the terminal conditions. Describe the decision that would justify a later phase; create its full measurement design only after the pilot passes its gate.

Reserve confirmation evidence when creating the plan

A plan intended to support a confirmatory claim needs a Confirmation reserve before exploration starts: evidence held back from the choices made during exploration. Name that evidence and its decision rule together, then commit the declaration. An empty reserve or a rule written after seeing outcomes cannot preserve that separation.

## Confirmation reserve

**Held back:** independently sampled benchmark questions,
collected under the sampling specification after design freeze.
The reviewer controls release. Executors may not inspect questions
or outputs during piloting or use them to choose configurations,
metrics, exclusions, or sample size.

**Decision rule:** accept adoption only if the lower endpoint
of the prespecified paired 95% interval for accuracy improvement
exceeds 2 percentage points. Otherwise decline adoption.
Freeze the interval procedure and sample-size justification
before collection; an outcome-informed repair spends this reserve.

Locate the sampling specification and name the access custodian in the actual plan. Implement the restriction with permissions or a separate collection process; Markdown cannot enforce it. Reserve enough independent units for the intended inference. Fresh run IDs or seeds alone do not remove earlier selection on the same questions. The evidence workflow explains access history and what an outcome-informed revision consumes.

Authorize one phase and its execution limits

Pause for human review before moving a draft to active. Record the approved phase and its limits, move the file directly into plans/, and point next_action at that work. Plan approval does not authorize paid compute, destructive changes, publication, external writes, or an unattended loop.

Execute the phase through its experiment records. Put job IDs, output locations, validity checks, and results there, following the experiment workflow. Keep dated decisions and design changes visible. Record any deviation discovered during execution, its reason, and its effect on interpretation. Preserve the original design so readers can distinguish it from the work that occurred. Once the evidence is stored for later recovery, update the plan with links, the gate outcome, and a proposed disposition. Pause again for review. Passing a pilot does not authorize the larger study.

Leave the next executor a current decision

Hand off the plan path and phase name instead of copying its instructions into a prompt. The recipient reads the current requirements, evidence, and approval. Keep current_phase current and give next_action a specific limit, such as “Review the pilot disposition and decide whether to design Phase 2.” Update them only after the phase's evidence and disposition are stored for later recovery. Process outputs before the handoff.

“Benchmark access approved by its custodian” is testable. “The user decides to run it” is ordinary scheduling, not an external gate. A satisfied gate brings the plan back for explicit review before execution. Move nonactive plans to the matching status directory and repair their pointers whenever their location changes.

Close an answered question, including a negative one

A valid negative gate can complete a plan. In the synthetic pilot, a futility stop may close the campaign without spending the confirmation reserve. Record that it remained unopened because the planned branch made confirmation unnecessary. This supports the scoped stop decision, not a confirmatory claim about the untested reserve population.

Use abandoned when the question remains unanswered because its premise changed or the work no longer merits doing. Record abandoned_because. If another plan replaces it, use superseded and name superseded_by.

Before setting completed, write ## Completion report and ## Evidence. State the terminal disposition, link each goal's met, unmet, or void outcome, and record limitations, successor work, and job and artifact status. Explain unused phases and reserves. Keep the authoritative numbers in experiments or findings. Abandoned and superseded plans instead need ## Disposition and ## Evidence.

Empty the terminal next_action, update the date, and move the file to its status directory. Repair incoming links and relative links inside the moved file, update status and priority pointers, regenerate the plan index with --write-plan-index, and validate before committing. Remove the plan from active queues.

Later corrections still reach closed plans. When a cited result is corrected, preserve the original passage and add a dated retraction notice directly beneath it, linking the correcting experiment and identifying the superseded values. Apply verified corrections, then search for the experiment ID and old values elsewhere. The retraction convention preserves what was believed without leaving it unqualified.

Audit or reproduce the completed undertaking

Start from the completion report and follow each outcome to its experiment or finding. Compare the executed phases with the design and approvals in version history. Check which branches were taken, which assumptions failed, and whether later choices used evidence that the original plan intended to hold back.

For reproduction, follow those records to the actual scripts, code and environment versions, data identities, parameters, and retained artifacts. The plan locates this material; its existence does not establish reproducibility. Record any missing dataset or unrecoverable code revision explicitly.

Give a new reproduction attempt its own experiment record and link the original work under Informed by. Preserve the original plan's outcome, then record whether the reproduction agrees and what differs. Re-running an analysis on the same data and independently confirming a claim answer different questions; use the data-reuse workflow to keep that distinction visible.