Reachable
Preview protection and readiness succeeded for the recorded URL. This is deployment reachability evidence only.
Public guide · bounded local benchmark
Test an authenticated preview without overstating the result
A reachable preview can prove that a protected URL responds. It cannot, by itself, prove that an application session works, the intended revision is deployed, or database and API acceptance succeeded.
Four separate checks
| Layer | Question answered | Does not establish |
|---|---|---|
| Preview protection | Can the test runner reach the hosted preview through platform access controls? | That the application user is signed in. |
| Application login | Can a scoped test account establish the expected app session? | That protected workflows pass. |
| Readiness | Can the preview serve the named route or health signal without an immediate startup failure? | That behavior, persistence, or integrations meet acceptance criteria. |
| Assertions | Did named UI expectations pass at the route and viewport recorded in the run? | That untested database and API contracts were accepted. |
Run the local controlled benchmark
From repository root, this wrapper starts src/qa/release-benchmark.ts through scripts/benchmark-release-evidence.ts. It binds a fresh, in-memory loopback fixture; it does not accept a customer target or proxy arbitrary preview traffic.
npm run benchmark:releaseEach run creates a fresh synthetic session, exercises a deliberately broken state, a transient pass, a recurrence, a stable fixture state, and an expired session. The fixture resets its in-memory record between phases. It keeps the comparison bounded; it does not establish a permanent product fix.
Treat stored session state as a credential
- The benchmark creates its own synthetic fixture session and saves state only with its local evidence artifacts.
- For a separate hosted-preview recipe, use a dedicated, least-privilege test account. Do not use a personal administrator session.
- Keep any hosted auth state in an ignored local path outside source control. Browser state can contain cookies and headers that impersonate the test account.
- Do not paste passwords, bypass tokens, cookies, or saved state into command history, issue text, screenshots, or reports.
Playwright documents storing authenticated browser state and explicitly warns against committing it: Authentication. For Vercel preview automation, use its documented preview protection mechanism and verify the preview URL before running browser tests: Vercel preview E2E guidance.
Read the result in layers
Authenticated
The scoped app session was established. It does not prove every role, path, or external integration.
Asserted
Named checks passed with captured evidence. Keep failures, blocks, and unrun checks in the same report.
Still unverified
Deployed revision, database acceptance, API acceptance, and provider behavior need separate matching evidence.