pi-simocracy · codex-pet-rendering

Pi-Simocracy: codex pet avatar rendering

@daviddao

Simocracy's org.simocracy.sim lexicon picked up a spriteKind: "codexPet" branch with a 1536×1872 WebP petSheet from OpenAI's hatch-pet skill. Pi-simocracy@0.3.0 only knew the legacy 32×32 PNG sprite path, so codex pet sims loaded with no avatar at all. This run loads Einstein (the only codex pet on the indexer today) against the patched extension and confirms the WebP atlas decodes, the idle frame crops, and the half-block ANSI render fires inline.

Date: 2026-05-03 · pass · pi 0.72.1 pi-simocracy 0.3.0→HEAD vhs 0.11.0

Headline result

1×1
Sim before → after
~110 ms
WebP decode (1536×1872)
~15 s
End-to-end load (2 MB blob)
0
Native deps added

Avatar shows. The 192×208 idle cell from row 0 col 0 of the codex pet atlas decodes through @jsquash/webp (lazy-init wasm, 138 KB), box-downscales to 32×35, and renders as 24-bit half-block ANSI — 16 lines tall, comparable footprint to a pipoya sprite. Persona injection still works alongside it: Einstein replied “Ah, I am Albert — a humble seeker of truth, with a fondness for thought experiments and a deep mistrust of war.” The full WebP → RGBA → crop → box-downscale → ANSI pipeline is asset-agnostic and back-compatible: pipoya sims (Mr Meow) still render unchanged.

Before ↔ after

Einstein loaded under pi-simocracy@0.3.0 with no avatar visible
Before — published 0.3.0. Loading flow completes (status pill, AT-URI, short description). Empty space where the sprite belongs. renderSpriteAnsi() tried sim.sprite (absent), then sim.image (also absent for this sim) and silently returned null.
Einstein avatar rendered as colored ANSI half-blocks above the loaded line
After — patched HEAD. Same sim, same flow. renderSpriteAnsi() sees spriteKind === "codexPet", fetches petSheet (image/webp), decodes through @jsquash/webp, crops the idle cell, box-downscales, renders inline.

Walkthrough

One vhs tape drives the whole flow against pi-simocracy at HEAD. Webm + tape are both committed under assets/ so the recording is reproducible.

Einstein sprite rendered with status line
02Codex pet idle frame (192×208 cell, box-downscaled to 32×35) rendered with the half-block character ▀. Note the white hair, blue shirt, white lab coat, brown pants — every visible detail comes from the WebP atlas, not the (absent) PNG fallback.
Einstein in-character reply
03Persona injection. After the avatar lands, asking “who are you?” hits before_agent_start, which appends buildSimPrompt(loadedSim) (constitution + style) to pi's system prompt. Reply: “Ah, I am Albert — a humble seeker of truth…”

How the fix is shaped

src/simocracy.tsExtended SimRecord with spriteKind, petSheet, petManifest mirroring the lexicon update. Existing pipoya fields stay optional for back-compat. src/png-to-ansi.tsAdded boxDownscaleRgba() — alpha-aware area-weighted downscaler that handles non-integer ratios. Used to shrink 192×208 codex pet cells (and 128×128 codex pet thumbnails) to a target close to a pipoya sprite's footprint. src/webp-to-rgba.ts newThin wrapper around @jsquash/webp/decode. Loads the wasm module lazily from disk via createRequire().resolve() (the package's browser-shaped fetch() path doesn't work under Node Undici). Returns the same {width, height, data: Buffer} shape as decodePng(). src/index.tsrenderSpriteAnsi branches on spriteKind: pipoya uses the existing 32×32 walk-frame path; codexPet fetches petSheet, decodes via decodePng or decodeWebp based on mimeType, crops row-0/col-0 (192×208), box-downscales to 32 wide. Image fallback now also box-downscales any non-pixel-art whose long edge exceeds 40 px. tsconfig.jsonAdded WebWorker to lib so WebAssembly globals type-check (the strict ES2022 baseline doesn't include them). package.jsonAdded @jsquash/webp@^1.5.0 as a runtime dep. Wasm-only, no native bindings, no postinstall step. ~915 KB unpacked (decoder wasm itself is 138 KB).

Findings

Avatars now render for codex pets

Loading Einstein under HEAD shows the full sprite (white hair, lab coat, blue shirt) above the loaded-pi line. The published 0.3.0 build silently dropped the avatar because neither sim.sprite (pipoya-only) nor sim.image (best-effort, missing for Einstein) had a usable PNG. The new spriteKind === "codexPet" branch reads petSheet directly, which is the canonical asset for this sim system.

Pipoya regression-tested

Mr Meow (the legacy 32×128 PNG walk sheet) still loads through the original branch — pipoya rendering is untouched. Confirmed both via the standalone node --experimental-strip-types /tmp/test-render-pet.mjs harness and an interactive /sim mr meow session.

Persona injection still fires

Einstein's before_agent_start handler appends his constitution + style to pi's system prompt every turn. Reply to “who are you?”: “Ah, I am Albert — a humble seeker of truth, with a fondness for thought experiments and a deep mistrust of war.” No regression in the unload override either.

Network is the bottleneck — not WebP

Per-stage timing under pi's interactive event loop, captured via temporary in-process logging:

The 14.6 s blob fetch dominates. satyam2.climateai.org's com.atproto.sync.getBlob is slower at serving the 2 MB atlas than the existing standalone-Node path (~1 s) suggests — likely Undici behaviour under pi's render loop. Worth a follow-up if loading codex pets becomes hot.

One codex pet on the indexer today

Out of 58 org.simocracy.sim records currently exposed by simocracy-indexer-production.up.railway.app, exactly one (Einstein, at://did:plc:qc42fmqqlsmdq7jiypiiigww/org.simocracy.sim/3mfo5txyqmo2f) has spriteKind: "codexPet". The remaining 57 are pipoya. Einstein also has no image PNG thumbnail, so it's the worst-case test — if the WebP path didn't work, there was nothing else to fall back on. That's exactly the failure mode shown in the “before” screenshot.

Reproducing

Recordingvhs assets/01-load-einstein.tape from the report root — produces both webm and gif. Direct loadpi -e ~/Projects/pi-simocracy/src/index.ts -ne -ns then /sim einstein Standalone rendernode --experimental-strip-types /tmp/test-render-pet.mjs (ad-hoc harness used during the fix; pulls Einstein from the indexer and prints the same ANSI). Type-checkcd ~/Projects/pi-simocracy && npx tsc --noEmit — clean. Packnpm pack --dry-run — 15 files, 34 kB tarball, unchanged surface area. Video01-load-einstein.webm (853 KB) · .gif (577 KB) Tape01-load-einstein.tape Stillsassets/00-before-no-avatar.png, assets/02-einstein-loaded.png, assets/03-einstein-reply.png