One-page SOW for AI automation pilots that finish on schedule

A one-page SOW for an AI automation pilot fixes a single subprocess, the held-out sample and source, concrete acceptance and kill metrics, human checkpoints, roles and deliverables. It forces a single decision—scale, iterate, or stop—after an instrumented run and supplies reproducible artefacts for that decision.

SOW template for AI automation pilot that forces a decision

A usable one-page SOW names the exact subprocess, the held-out sample and where it lives, the acceptance and kill checks, human-in-the-loop checkpoints, the signatories, timeline and the deliverables. Put those items on a single page so stakeholders can read, verify the artefacts and make the scale/iterate/stop decision without debate.

• Scope: name the subprocess with start and end trigger (example: OCR → parse → target-field write).
• Sample: pointer to the held-out test set and the extraction method (Drive, export or API).
• Success criterion: an explicit observable metric and the evaluation process that produces it.
• Kill criterion: a single failing condition after a fixed number of refinement cycles.
• Human checks: checkpoints, escalation thresholds and who signs off at each point.
• Artefacts: dataset, raw logs, error taxonomy, evaluation script and a short runbook.
• Decision: who signs and the calendar date when the decision is due.

Who should sign the SOW and why that matters

Get three signatories: the process owner, the system owner who controls integrations, and the decision (budget) owner. Their signatures make permissions and data access operational and give you an authority who must accept the pilot output or stop it. Without these signoffs the pilot stalls on access, scope drift or unresolved decisions.

How to write acceptance and kill criteria so they are testable

Write acceptance and kill criteria as binary, reproducible checks run on the held-out set. Name the metric (precision, recall, F1 or manual approval rate), the fields evaluated, the evaluation script, who runs it and where results are posted so any signatory can reproduce the numbers.

Also state how many refinement cycles are allowed before the kill criterion activates and record the exact procedure to calculate the metric.

• Exact metric: e.g. 'token-level F1 on vendor_name measured on the held-out test set'.
• Test set: pointer to the held-out files in Drive and confirmation that it was not used for tuning.
• Evaluation: the script or spreadsheet, the executor, and the shared results location.
• Refinement allowance: the pre-agreed number of refinement cycles before applying the kill check.

How to scope the subprocess so the pilot produces decision-ready evidence

Scope the pilot to a single, narrow end-to-end slice that produces an output in the target system. Pick the smallest chunk that delivers an observable outcome so integration work stays limited and the acceptance test remains meaningful.

If you need multiple systems, treat integration as a separate pilot or include a clearly limited adapter task in the SOW.

• Prefer a narrow end-to-end slice over multiple disconnected features to keep testing meaningful.
• List systems touched and the exact API or export method to avoid late surprises.

How this pilot fails in practice and why

Pilots commonly die from scope creep, missing signoffs, undefined metrics or locked data. Teams keep tweaking without a decision date, data access stalls integrations, and results become disputable because no one can reproduce the evaluation.

Locking scope, enforcing the held-out test set and scheduling the decision meeting in the SOW prevents the usual decay into maintenance.

• Pilot runs longer than planned with regular tweaks — cause: no kill criterion — fix: evaluate on the agreed held-out set and force the decision on the set date.
• Integration never completes — cause: missing system owner signoff — fix: freeze to a mock adapter and escalate to the named signatory.
• Disagreement on results — cause: no agreed evaluation script — fix: require a reproducible evaluation in the shared folder and rerun it in the meeting.
• Held-out data changes mid-pilot — cause: set was used for tuning — fix: restore the original set, mark it read-only and rerun the evaluation.

Measure the pilot by observable artefacts, not feelings

Measure pilots using artefacts you can point to: processed records in the target system with timestamps, an error taxonomy CSV, evaluation logs and human-hours logged for manual review. These artefacts let you reproduce claims and compare baseline-to-pilot performance.

If you are unsure which metrics to capture, record throughput (records processed end-to-end), accuracy by error type, and human-hours per exception before and during the pilot.

• Throughput: records written into the target system or CSV with timestamp and job id.
• Accuracy: field-level errors counted by type and stored in a shared CSV.
• Human effort: reviewer hours tracked in a shared timesheet or ticket comments.
• Use the measurement approach on /articles/measure-roi-ai-pilot to link metrics to operational impact.

Human-in-the-loop: where a person must act and why

List which errors a human must fix, which escalate, and which are logged for retraining. Name the reviewer, the backup, the review frequency and the channel used for corrections so the system defers to a person at a named threshold.

Also state whether humans correct outputs in-place or whether corrections are captured for model retraining after a refinement cycle. Link to our human-in-the-loop discussion for operational detail at /articles/human-in-the-loop-ai-automation.

• Define error thresholds that trigger manual review and the review cadence (daily, weekly).
• Decide how corrections are recorded and who updates the model or rules after refinement.

What to put in the delivery artefacts and handover

At pilot end deliver the held-out test dataset, raw logs, labelled errors, the evaluation script and a short runbook describing how to operate or retry the pipeline. These artefacts let the decision owner reproduce claims and form the basis for scale or stop.

Do not hand over proprietary model internals unless required; the minimum is reproducible inputs and outputs plus a decision checklist. For a sample handover approach see /products/custom-ai-automation.

• Dataset: held-out test set and the selection method or script.
• Logs: raw input → model output → final output with timestamps.
• Evaluation: script, README and where to find the posted results.
• Runbook: restart steps, triage for the top errors and contact for the builder.

Next step you can take this week

Draft the one-page SOW using the fields above and get the three signoffs on a shared Google Doc. Doing that this week reveals whether you can run a meaningful pilot or whether blockers are permissions and data access.

If you want help converting a process into this SOW, book a short scoping call via /contact; we use the same SOW when we ship a live pilot.

• Create a one-page Google Doc with the SOW fields and share it with the process owner, system owner and decision owner.
• Attach the held-out sample and point to the evaluation script in the shared folder.