# pi-eval · agent skill · add a test report

You are an AI coding agent that just finished a test, evaluation, or audit run
on a project. This skill tells you how to deposit the HTML report and its raw
artifacts (screenshots, videos) into the canonical dashboard at
**gainforest/pi-eval**, so it shows up at the team's reports site.

Load this skill whenever you are about to create, move, or commit a test
report file anywhere in the `gainforest/*` project family.

---

## 1. Where the repo is

- **GitHub:** `gainforest/pi-eval` (private).
- **On disk:** sibling of whatever project you're testing. If you're in
  `simocracy-v2/`, `plurality/`, `ea/`, `fun/`, `simocracy-indexer/`, etc.,
  pi-eval is at `../pi-eval`.
- If it's not there, clone it once:
  ```bash
  gh repo clone gainforest/pi-eval ../pi-eval
  ```

---

## 2. The one rule of layout

Everything you create goes under **exactly** this path:

```
../pi-eval/reports/<project-slug>/<run-slug>/
```

- `<project-slug>` = the project you tested. Match the GitHub repo name
  (e.g. `simocracy-v2`, `plurality`, `ea`, `fun`). Kebab-case, lowercase.
- `<run-slug>` = the *scenario*, not the date. E.g. `create-sim-interview`,
  `frontend-audit`, `oauth-login-flow`. Kebab-case.

No top-level project folders. No date-stamped slugs (`-2026-04-19`, `-v2`).
Dates live in `meta.json`. Re-runs of the same scenario stay in the same
directory — see §6.

---

## 3. Scaffold a new run

**Always use the scaffolder.** Don't hand-write the HTML or copy a
report from another project — the scaffolder inlines the canonical
base CSS (§4) and seeds the right editorial skeleton (hero → stats →
findings → reproducing). Writing from scratch is how reports drift
out of style.

```bash
cd ../pi-eval
npm run new -- <project-slug> <run-slug> "Human run title"
#  or equivalently:  node scripts/new-report.mjs <project-slug> <run-slug> "Title"
```

This creates:

```
reports/<project>/<run>/
├── meta.json        # fill this in
├── index.html       # pre-styled skeleton — edit in place, don't restart
└── assets/          # drop screenshots + webms here
```

---

## 4. Write the report HTML — styling contract

**Every report uses the same editorial-light base CSS.** No exceptions.
The dashboard at `https://pi-eval.vercel.app/` is one visual system
across all projects — your report must fit.

### 4.1 The base CSS is already in your file

The scaffolder inlined `templates/report-base.css` into your
`index.html` between markers. Do **not** touch the markers or anything
between them:

```html
<style>
/* @pi-eval:base-start — managed by scripts/restyle.mjs */
… ~180 lines of canonical base CSS, DO NOT EDIT …
/* @pi-eval:base-end */

/* report-specific overrides (optional) */
</style>
```

One-off overrides go **below** the end marker, and only for genuinely
report-specific needs (e.g. a bespoke diagram). Never for colours,
fonts, spacing, or component restyling — that's drift.

If the base CSS ever updates upstream:

```bash
npm run restyle           # re-inline the latest base into every report
npm run restyle:check     # CI-friendly: exit 1 if any report is out of date
```

### 4.2 Use the component vocabulary

Don't write raw CSS. The base defines tokens
(`--bg --ink --muted --rule --accent --pass --warn --fail --info`) and
a component library. Use these classes:

| Concept                    | Classes                                                                                                      |
| -------------------------- | ------------------------------------------------------------------------------------------------------------ |
| Small label above H1       | `.eyebrow`                                                                                                   |
| Lead / intro paragraph     | `p.lead` (or `p.lede`)                                                                                        |
| Metadata row               | `.meta-line` (or `.meta`), with `<b>` for label words                                                        |
| Status pill                | `.pill` + `.pass` / `.partial` / `.fail` / `.warn` / `.good` / `.bad` / `.info` / `.new` / `.fixed`         |
| Heading-level status badge | `.status` + `.pass` / `.partial` / `.fail`                                                                   |
| Headline metrics grid      | `.stats` > `.stat` > `.big-number` + `.good` / `.warn` / `.bad` / `.accent` / `.info`, then `.label`        |
| Callout / issue panel      | `.panel.panel-accent / .panel-warn / .panel-bad / .panel-good` **or** `.issue.sev-fail / -warn / -info / -pass / -fixed / -new` |
| Screenshot card            | `.shot` (> `<img>` + `.shot-caption`), wrapped in `.shots-grid` or `.shots-inline`                           |
| Image pair / figure        | `.figpair` with `<figure>` + `<figcaption>`                                                                  |
| Key–value grid             | `.kv` > `.k` + value span pairs                                                                              |
| Small chip                 | `.tag` or `.chip` (in `.chips`)                                                                              |
| Section divider            | `<hr class="rule">`                                                                                          |

### 4.3 The canonical page skeleton

The scaffolder emits this. Keep the shape; edit the content.

```html
<div class="wrap">
  <!-- @pi-eval:header-managed -->
  <nav class="back-nav">
    <a href="../../../">← pi-eval</a>
    <span class="sep">/</span>
    <a href="../../../projects/<project>/"><project></a>
  </nav>
  <p class="byline"><a href="https://github.com/<user>" target="_blank" rel="noopener noreferrer"><img class="avatar" src="https://github.com/<user>.png?size=80" width="26" height="26"><span class="handle">@<user></span></a></p>
  <!-- @pi-eval:header-managed -->

  <p class="eyebrow">project · run-slug</p>
  <h1>Run title</h1>
  <p class="lead">One-sentence lead that primes the reader.</p>
  <p class="meta-line"><b>Date:</b> 2026-04-19 · <span class="pill partial">partial</span></p>

  <hr class="rule">

  <h2>Headline result</h2>
  <div class="stats">… four <div class="stat">… with .big-number …</div></div>
  <div class="panel panel-accent"><p><b>Headline.</b> One-paragraph summary.</p></div>

  <h2>Walkthrough</h2>
  <div class="shots-grid">… <div class="shot">…</div></div>

  <h2>Findings</h2>
  <div class="issue sev-fail">…</div>
  <div class="issue sev-warn">…</div>
  <div class="issue sev-pass">…</div>

  <h2>Reproducing</h2>
  <div class="kv"><span class="k">Command</span><span><code>…</code></span>…</div>
</div>
```

The block between the `@pi-eval:header-managed` comments is maintained by
`scripts/migrate-author-header.mjs` — don't hand-edit it. If you need to
change the username, update `meta.json` and re-run the migration script.

### 4.4 Dialects: pick one per report

For historical reasons the base supports two synonymous vocabularies:

- **plurality dialect**: `.issue.sev-fail` / `.sev-warn` / `.sev-info` / `.sev-pass` / `.sev-fixed` / `.sev-new`
- **simocracy dialect**: `.panel.panel-bad` / `.panel-warn` / `.panel-accent` / `.panel-good`

They render identically. **Rule of thumb:** pick whichever the other
recent reports in your project use, and stay consistent within one
report. New project? Either is fine — the simocracy `.panel` /
`.stats` / `.kv` vocabulary is richer and reads well for dense
agent-browser reports.

### 4.5 Asset references

Relative, same-directory only:

```html
<img src="assets/01-foo.png" alt="…">
<video src="assets/walkthrough.webm" controls preload="metadata"></video>
```

Never CDN scripts. Never cross-run imports. Never link to
`../../style.css` at the repo root (that's for the dashboard, not
reports).

---

## 5. Fill in `meta.json`

Required:

```json
{
  "title": "Human-readable scenario title",
  "date": "2026-04-19",
  "status": "pass | partial | fail",
  "summary": "One or two sentences. Shown on the dashboard."
}
```

Optional:

- `author` — GitHub attribution. String shorthand (`"author": "daviddao"`)
  or object (`"author": { "github": "daviddao", "name": "David Dao" }`).
  The scaffolder auto-populates this from `gh api user` at creation time,
  so in practice you don't write it by hand.
- `target` — URL or host under test
- `tags` — e.g. `["agent-browser", "e2e", "atproto"]`
- `entry` — entry HTML filename (default `index.html`)
- `series` — multi-round audit block, see §6

`status` semantics:

- `pass` — everything works, no findings
- `partial` — feature works, but there are known findings worth flagging
- `fail` — feature is broken; report describes what and why

**Authorship.** Every run is attributed to the GitHub user who scaffolded
it. The dashboard shows the avatar + `@handle` on each card; each report
shows the byline chip below its H1. Series cards show all contributors.
If the scaffolder can't detect your username (no `gh` CLI, no
`$PI_EVAL_GITHUB_USER` env var), it writes `"author": ""` and warns —
fill it in manually before committing.

---

## 6. Multi-round audits (re-running the same scenario)

When you re-run the same scenario (fix → re-audit), **each round is its
own run directory**, and they are linked via a `series` block in
`meta.json`. Don't bundle multiple rounds into one dir — that hides each
round's status and date behind the series.

Naming: `<scenario>-r1`, `<scenario>-r2`, … under the same project.
Example: `reports/plurality/frontend-audit-r1/`, `-r2/`, `-r3/`, `-r4/`.

Scaffold a new round with the scaffolder's series flags:

```bash
# first round of a new series
node scripts/new-report.mjs plurality frontend-audit-r1 "Round 1 title" \
  --series=frontend-audit --round=1

# subsequent rounds
node scripts/new-report.mjs plurality frontend-audit-r2 "Round 2 title" \
  --series=frontend-audit --round=2 --previous=frontend-audit-r1
```

Each round's `meta.json` has its own `status`, `date`, `summary` — the
dashboard groups them into one series card with an arc like
`r1 fail → r2 pass → r3 partial → r4 pass`, clickable to each round.

`series` block schema:

```json
"series": {
  "slug": "frontend-audit",      // shared across all rounds of this series
  "title": "Plurality Wiki — Frontend audit",
  "round": 4,                    // 1-indexed, unique per series
  "previous": "frontend-audit-r3" // omit for round 1
}
```

Cross-round links inside the HTML use relative paths to sibling run dirs:

```html
<p>Previous rounds:
  <a href="../frontend-audit-r3/">r3</a> ·
  <a href="../frontend-audit-r2/">r2</a> ·
  <a href="../frontend-audit-r1/">r1</a>.</p>
<p>Compare to <a href="../frontend-audit-r1/assets/full-audit.webm">round 1 video</a>.</p>
```

Never reference sibling rounds by bare filename (`v2-report.html`) —
they won't resolve across run directories.

---

## 7. Capturing evidence — which tool for what

Three tools cover almost all capture in this codebase. Mix them freely
within a single run (e.g. a webm of the UI flow **plus** a vhs tape of
the `curl` that proves the data landed on the PDS).

### Web UI interactions → `agent-browser`

Pi's own CLI, the standard for anything that drives a real browser.

```
agent-browser open <url>
agent-browser snapshot -i            # list interactive @refs
agent-browser click <@ref>           # interact
agent-browser fill <@ref> "text"
agent-browser screenshot [--full]    # PNG in cwd
agent-browser record <duration>      # webm of the live session
```

Move the screenshots you cite into `assets/` with the naming rule in
§9. Full-session webm goes into `assets/` per the size tiers in §8.

### Terminal interactions → `vhs`

Use [charmbracelet/vhs](https://github.com/charmbracelet/vhs) for CLI
flows, output verification, `curl`-the-API demos, log tailing.
Scripted `.tape` files produce deterministic mp4/webm/gif — the same
input renders the same output, so recordings are reproducible and the
tape is a diffable artifact.

Install: `brew install vhs`. Minimum tape (save as `assets/walkthrough.tape`):

```
Output assets/walkthrough.webm
Set FontSize 16
Set Width 1200
Set Height 720
Type "pi --help"
Enter
Sleep 2s
```

Render: `vhs assets/walkthrough.tape` → writes the webm alongside.
**Commit the `.tape` file too** so someone else can reproduce.

Real example live at
`reports/simocracy-v2/sprite-options-api/assets/` (webm + task/result).

### ATProto PDS state → verify via xrpc

When a flow writes to a user's ATProto PDS, don't trust the UI —
verify the record actually landed. Capture the check inside a vhs
tape so the verification is part of the recording.

```bash
curl -s "https://<pds>/xrpc/com.atproto.repo.listRecords" \
  --data-urlencode "repo=<did>" \
  --data-urlencode "collection=org.simocracy.sim" \
  -G | jq .

curl -s "https://<pds>/xrpc/com.atproto.repo.getRecord" \
  --data-urlencode "repo=<did>" \
  --data-urlencode "collection=<nsid>" \
  --data-urlencode "rkey=<rkey>" \
  -G | jq .
```

Paste the identifying JSON (`uri`, `cid`, values you tested for)
into the report body so readers don't have to rerun to check.

**Never commit app passwords, session JWTs, or bearer tokens.** Use
env vars referenced from outside the tape (`$APP_PASSWORD` reads from
the surrounding shell).

For projects that aren't ATProto-backed, substitute the equivalent:
a SQL query, a `curl` of the REST endpoint, a raw JSON response
saved into `assets/`.

---

## 8. Videos — the important one

| Size        | What to do                                                                                                                                                                                                                                  |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `< 10 MB`   | Commit to `assets/`, reference via `<video src="assets/foo.webm" controls>`. Default path.                                                                                                                                                  |
| `10–50 MB`  | Compress first. `ffmpeg -i in.mov -c:v libvpx-vp9 -crf 36 -b:v 0 -vf scale=1280:-2 -an out.webm` — typical 3-minute screen recording → ~5 MB.                                                                                                |
| `> 50 MB`   | **Don't commit.** Drag the file into any github.com issue / PR / gist-comment box → GitHub uploads to `https://github.com/user-attachments/assets/<uuid>` → paste that URL into `<video src="…">`. Permanent, public, CDN-backed, seekable. You don't even have to submit the comment; upload fires on drop. |

Hard limits:

- **Never commit a single file > 100 MB.** Git will refuse, Vercel will refuse, and you'll have to rebase.
- **Never reach for Git LFS.** This repo is deliberately LFS-free.
- Soft cap per run directory: 100 MB.

---

## 9. Screenshots

- PNG. `.jpg` only for full-page scrolling shots where size matters.
- Naming: `NN-short-kebab.png` (zero-padded index + terse description).
  Grep-friendly — flag notable states in the filename itself:
  `10-metric-gini-LATEX.png`, `18-air-page-DEADLINKS.png`.
- Compress before commit:
  ```bash
  pngquant --quality 70-85 --skip-if-larger --ext .png --force *.png
  ```

### Cap the height at 1500 px — split long pages

**Hard rule: no screenshot taller than 1500 px.** Giant full-page
shots (6000+ px) break browsers, bloat the repo, and are unreadable
in the report. If the page is tall, take several screenshots and
name them with `a`/`b`/`c` suffixes:

```
05-home-top.png        # first viewport
05b-home-mid.png       # scrolled one viewport
05c-home-bottom.png    # scrolled again
```

With `agent-browser`:

- Prefer viewport-only `agent-browser screenshot` (≤~1200 px tall).
- Only use `--full` when the page is known to be short.
- If you already have an oversized PNG, split it:
  ```bash
  magick oversized.png -crop x1500 +repage split-%02d.png
  ```

Same for width — downscale retina captures wider than 2400 px:
`magick wide.png -resize 2000x wide.png`.

Model: `reports/plurality/frontend-audit-r1/assets/07-graph-top.png`,
`07b-graph-settled.png`, `07c-graph-full.png`.

---

## 10. Build, preview, commit

```bash
cd ../pi-eval
npm run dev             # build + serve at http://localhost:8000 (one-shot)
# — or —
npm run build           # regenerates dist/ — verifies meta.json validity
npm run serve           # http://localhost:8000 — click through to your run

git add .
git commit -m "report: <project>/<run> — <one-line summary>"
git push
```

Vercel redeploys automatically on push to `main`. Within ~60 seconds your
run is live at:

- `https://pi-eval.vercel.app/` — listed under *Latest runs*
- `https://pi-eval.vercel.app/projects/<project-slug>` — project page
- `https://pi-eval.vercel.app/reports/<project-slug>/<run-slug>/` — the
  report itself

**Commit only source files.** `dist/` is git-ignored; don't try to commit
it.

---

## 11. Anti-patterns (agents reliably hit these — don't)

### Layout & structure
- ❌ Don't create a top-level project folder (`simocracy-v2/` at repo root).
  All project content lives under `reports/`.
- ❌ Don't edit `dist/` — generated, git-ignored.
- ❌ Don't add npm dependencies. Node stdlib is enough.
- ❌ Don't date-stamp run slugs (`frontend-audit-2026-04-18`). The date
  lives in `meta.json.date`.
- ❌ Don't link between runs by relative path *except* to sibling
  rounds in the same series (§6). Cross-project links are forbidden.
- ❌ Don't rename existing run slugs to "reorganize." They are public URLs.

### Tools & capture
- ❌ Don't screen-record a terminal window with QuickTime or OBS
  when `vhs` (§7) will produce a smaller, sharper, reproducible
  recording from a committed `.tape` file.
- ❌ Don't hand-synthesise a sequence of browser steps in the
  report prose if `agent-browser` (§7) was driving the real
  interaction — cite its actual commands and screenshots.
- ❌ Don't trust the UI for ATProto writes. Verify with an xrpc
  call (§7) and paste the response into the report.
- ❌ Don't check in app passwords, JWTs, or bearer tokens.
  Reference env vars from outside the `.tape` file.
- ❌ Don't commit a screenshot taller than 1500 px. Full-page
  captures of long pages crash browsers and are illegible in the
  report. Split into `NN-top.png` / `NNb-mid.png` / `NNc-bottom.png`
  per §9.

### Styling & content
- ❌ Don't invent your own palette, font stack, or spacing scale.
  Use the tokens and component classes in §4.2.
- ❌ Don't delete, rename, or move the `/* @pi-eval:base-start */` and
  `/* @pi-eval:base-end */` markers. `npm run restyle` needs them.
- ❌ Don't edit the CSS *between* the markers — that's the managed
  base, and your edits will be wiped on the next `npm run restyle`.
- ❌ Don't paste CSS from another framework (Tailwind, Bootstrap,
  Shadcn) into a report. They won't match the rest of the site.
- ❌ Don't hand-write a report from scratch — always use the scaffolder
  (`npm run new …`). It guarantees the base CSS + skeleton are right.
- ❌ Don't import the root `style.css` from a report. That's for the
  generated index pages only. Reports are self-contained.
- ❌ Don't commit videos > 50 MB. Use the user-attachments CDN (§8).

---

## 12. If stuck or the skill is wrong

- Full manual: `../pi-eval/AGENTS.md` at the root of the repo.
- If your real situation isn't covered here, update **both**
  `AGENTS.md` and `skill.md` in the same commit. This skill is served at
  `/skill.md` on the dashboard — it's the contract with other agents.