Visual verification.
A run can tell you the tests pass and still hand you a screen nobody looked at. Visual verification builds what the run produced, drives it, screenshots it, and judges the captures against an acceptance statement, then files the verdict back onto the run. It is off by default.
01What visual verification is
One check is one request. A request carries what was tested, the criteria it was judged against, the images it captured, and a verdict.
Every check runs against a clean detached checkout of one commit, in an isolated worktree of its own.
02Turning it on
Two checkboxes, both at Settings → AI → Feature controls → Visual Verification.
- Enable visual verification is the master switch, and it is off. While it is off, nothing is captured unless a per-run override or a Verify Setup run turns it on.
- Let runs set up verification themselves is on, and stays grayed out until the master switch is on. It covers the run-book bootstrap, not the checks themselves.
A flow launch can override the global switch for that one run. The launch wizard’s Advanced section carries a Visual verification control with three choices: Use global setting, On, and Off.
A flow run’s verification posture is stamped once when the run is created and never changes, so changing the setting mid-run does not touch runs that are already going. A quick chat session works the other way: it re-resolves the posture on every request, so your change takes effect on that session’s next one.
03Run-books
A run-book is how Cyboflow knows to build and serve your project. It has two halves, kept apart on purpose.
A run-book is either proven or an unproven draft. An unproven draft behaves exactly like having none. Every check that has to build or serve your project is skipped, and a non-blocking finding records that this project has no proven run-book. You triage it in Insights, not on the review queue.
Proven is re-checked on every read rather than stored as a flag. Change your build scripts, your lockfile, or the host, and every read answers unproven for as long as the change stands. The proof comes back when the inputs do, or when you prove it again.
A check that only points at an already-running URL needs no run-book at all.
04The Verify Setup flow
Verify Setup is the per-project flow that writes the run-book: it inspects the project, derives the run-book, asks you to approve it, proves it, and hands the result to human review. Five steps, one phase.
It is kept out of the launch wizard’s flow list. You start it from the verify queue health block, where every project row carries Set up or Re-run setup, and from the Workflows gallery or the project overview’s Set up verification recommendation.
It stops for you twice. On the review queue the two gates read Human gate: Approve runbook and Human gate: Human review. On an orchestrated run the in-chat gate is headed Approve setup.
- Before anything touches the repo
Approve all, Pick subset, or Reject over the proposal. On a programmatic run the gate is all-or-nothing: approve, reject, or send it back with a note.
- At the end
Approve or Reject over the committed diff and the proof outcomes.
The flow cannot declare its own run-book proven. Only a proof run that builds, serves, identifies the deliverable, and passes causes the engine to stamp it. After three failed rounds for a modality it leaves an unproven draft committed instead of a green badge.
Its proof runs drain at lower priority than live Sprint work. The flow’s place among the others is on the workflows page.
05Run-book bootstrap
With Let runs set up verification themselves on, a Sprint or Ship lane that would be skipped for want of a run-book writes one instead.
Only one lane per run derives it. The other checks in that run skip, and the next run verifies normally. If an earlier attempt on this project and host concluded no run-book could be derived, it is not retried.
It drafts the run-book and commits it by itself, so a parallel lane’s staged work is never swept in. A configuration change it needs to stand the project up is applied and committed separately. Then it registers the run-book and proves it, trying at most two drafting rounds.
Pass or fail, it leaves an artifact describing what it derived and what it proved, and any config file it touched is named in a non-blocking finding you triage in Insights.
Set CYBOFLOW_DISABLE_RUNBOOK_BOOTSTRAP=1 to force it off on that machine, whatever the setting says.
06The verify queue
Verify Queue sits in the left rail under the sub-label Visual verification. It opens a full-width panel headed Verify Queue over Visual-verification requests · captures & verdicts, with a Project dropdown. The queue is scoped to one project at a time and has no all-projects option.
- In flight lists still-running work oldest first, which is the order the scheduler drains it.
- History lists finished work newest first. Both carry a count.
- A row leads with the request id, the verification type, the originating session’s name and a status pill, then the task summary, the backend and the attempt number.
Click a row for the Verification detail dialog: What was tested, Criteria, Captured, Verdict, Build log, Error, Failure evidence, and Provenance. A failed request also carries its class there, as a chip reading failure class: env, failure class: deliverable, or failure class: ambiguous.
07Host health and probes
Inside the verify queue sits a Health block, sub-labeled live probes · setup state. It lists every project with one status label, Set up, Set up (derived by a run), Not proven, or Not set up, beside the button that opens the launch wizard on Verify Setup.
The same block probes this machine once per panel open, never on a timer, and shows three rows: Playwright browser control, Screen recording, and Computer control (accessibility). Each reads Healthy, Pending action, Unhealthy, Unknown, or N/A.
A row that can be fixed shows the fix in place, then re-probes: Install provisions Chromium for you, Open settings opens the macOS Screen Recording pane, and Grant access triggers the macOS accessibility prompt.
08How a Sprint lane uses it
The lane’s Visual check step is optional. It runs only when verification is on for that run, which in a default install means it does not run at all until you turn it on.
When it does run, the check is fire-and-continue. The lane parks on a state the run rail labels Awaiting visual verify, the verification runs elsewhere, and the verdict drives the lane afterwards.
- A pass marks the lane integrated.
- On a failure the lane goes back to implement with the verdict’s feedback attached. After three failed attempts it is marked failed, and the other lanes keep going. On a programmatic run the run’s monitor gets one chance to rescue such a lane first, once per lane and four times per run (see Sprint).
- Low confidence never loops the lane. It advances as advisory and raises a non-blocking needs human visual review item.
- Skipped and timed-out checks advance the lane too, with a non-blocking finding naming the reason, so a check that did not run is never read as a pass.
- One build break across several lanes is filed as a single grouped finding.
Screenshots, the verdict, and a Behaviors tested table land on the run’s Screenshots artifact tab, one row per behavior. However many lanes were verified, they merge into one artifact per run.
A quick chat session can request a verification too. It returns at once, the chat carries on, and the session reads the outcome back later.
09What it cannot do
Three things the check cannot do.
- Mobile verification does not exist. The mobile modality is deferred, and a run stamped mobile is declared unverifiable outright.
- Real OS windows are observe-only. The verifier can screenshot a native surface but cannot click or type on it, and a step that needs a click is reported not testable rather than attempted.
- Native desktop checks are macOS-only. The only backend that can see a real running app ships for macOS alone.
Design sessions and visual verification always run on Claude. Turning the Claude provider off at Settings → Integrations takes both with it.