Rendering bench audit (2026-10-18, read-only)

Goal: a precise, reliable, readable record of CPU and GPU rendering performance and rendering problems, per backend and per device. This lists what exists, what is partial, what is missing, with citations (paths relative to the repo root).

1. Exists

1.1 Measuring in the runtime (env switches)

1.2 Tools (tools/)

1.3 Device facts that exist (the plumbing to reuse)

• Mac: tools/bench-bridge mac, headless (tools/bench-bridge:65).
• Nexus 7: zinc export --target android, Gradle, adb, results from logcat; render env passes via am start --es zinc.env "ZINC_PERF_HUD=1 ZINC_PROFILE=1" (src/app/shell_sdl.cpp:158-177; release builds only keep ZINC_PRESENT_STATS / ZINC_GL_STATS from zinc.env in the app’s external files).
• Frame pacing without the engine: dumpsys SurfaceFlinger --latency (docs/reports/zinc-next-android.md:289-300).
• TouchPad: env through /tmp/zinc/env (plugins/display-gl/src/webos.cpp:166); zinc-helper sets the governor (docs/reports/hardware/hp-touchpad-webos.md:356-368).
• iOS: pymobiledevice3 + os_log (tools/bench-bridge:136-175; docs/reports/zinc-next-ios.md:69).
• Pi: tools/device pi@host profile (tools/device:210-227).

1.4 Scenes that exist

• bench/render/b1,b2,b3,b5,b8,b10 (synthetic, 1000x640 growDrawCommands, seeded RNG bench/render/b1-quads.ts:2-3), scene-dump deterministic.
• Deterministic UI/app corpus: tests/visual/{clock.ts,overlays.tsx,shapes.ts,ui.tsx}, examples/hero (ZINC_DEMO=home|gallery|detail|playground|tasks|settings|palette|dialog|dark|bench scripted tour), examples/maps/navigation (ZINC_DEMO=navdrive per docs/reports/zinc-next-perf.md:764-775), lottie/svg galleries, breakout (framehash test), examples/boards/*/*/framehash.golden, rn-showcase (scripted flings via ZINC_INPUT, docs/reports/zinc-next-perf.md:792-808), 64 proto goldens, examples/{gl-cube,webgl-cube,webgl-studio,three,js-three,marble,museum-table,pinball,pinball3d,platformer,video,3d,canvas,bouncing-ball}. Frame hash available for any of them via ZINC_FRAMEHASH.

2. Backends and where timing hooks sit

Vsync vs CPU: not separated in ZINC_PROFILE (PRESENT swallows the vsync wait). ZINC_PRESENT_STATS has a separate swap and interval column (hal_sdl.cpp:1127-1129), but only on the SDL HAL. Driver-specific fb drivers print their own vsync column.

3. What previous sessions recorded on hardware

Numbers by kind (all in docs/reports/zinc-next-perf.md unless noted):

• fps + present p50/p99 per path (software vs GLES 2): Nexus 7 :776-791 (rn-showcase 60 fps rest, work p50 1.38 / p99 4.3, present p50 15.0 / p99 17-20; GLES 2 13.7 fps then 60 fps at rest, tab switch p99 17.4 ms, first switch 147 ms).
• Step split of present (ZINC_PRESENT_STATS), flings via ZINC_INPUT, “late frames of 720” counts and interval p99: Nexus 7 :792-821; iOS simulator :822-835.
• Interval p50/p99 on SurfaceFlinger, vsync drops: docs/reports/zinc-next-android.md:289-300.
• Pi 3B+: present p50 per screen, draw calls per frame, CPU % per frame with temperature, GPU ms from ZINC_GL_TIME (:697-712); fps, present, convert, vsync, CPU a frame (:764-775); film on the Pi (:722-736).
• TouchPad: software 25-45 fps, “16 ms draw + 19-25 ms SDL show” (docs/reports/hardware/hp-touchpad-webos.md:179-182); GPU draw call cost 0.2-0.9 ms CPU, glClear 0.6 ms (:277-279); frames a second over a scripted session table (:283-, method docs/targets/adapting-to-hardware.md).
• Mac GL fill rate: Radeon Pro 555X (:421-440, :504-517).
• Console p50: PS Vita and emulators (:840-858).
• CPU %, peak RSS, energy: tools/resmon (tools/resmon), Pi ps CPU (:764-775).
• Per-machine JSON artifact (script bridge only): docs/reports/bench/script-bridge-*.json.

Pitfalls recorded:

1. Any Magisk su during a run makes every other present about 105 ms for the following 18 s; no root command during a measurement (docs/reports/zinc-next-android.md:180-182; zinc-next-perf.md:788).
2. CPU governor/hotplug: TouchPad ondemandtcl sits at 192 MHz, a burst stalls hundreds of ms, a touch lifts to 1.188 GHz for 3 s (docs/reports/hardware/hp-touchpad-webos.md:356-362); Nexus 7 cpuquiet governors and PM QoS cores; any app launch boosts all four cores about 1.5 s (docs/reports/zinc-next-android.md:191-213).
3. Thermal: Pi runs ended at 1200 MHz were dropped, temperature checked under 55 C before each run (zinc-next-perf.md:699,775); Intel Mac held at 33-60% by heat with several builds (AGENTS.md “development machine”, zinc-next-perf.md:441-470 “indicative”).
4. Machine load of other sessions makes numbers indicative; sections state load average per run (zinc-next-perf.md:9-12,410,441).
5. Vsync locked to 60 Hz: frame interval p50 16.6 ms regardless of work; “swap” of an idle frame is the display wait, so those frames are excluded (hal_sdl.cpp:1166). ZINC_VSYNC=0 works on webOS EGL, SDL ignores the swap-control attribute (hp-touchpad-webos.md:277).
6. Software vs GPU paths give different “present” meanings (replay inside present); glFinish measures serialise the pipeline and over-count (display_gl.cpp:214-215, zinc-next-perf.md:709).
7. ZINC_GL_FINISH=1 needed for per-step GPU honesty (hal_sdl.cpp:1519).
8. A tablet must be given to the measuring agent alone: runs with another app in front are discarded (zinc-next-perf.md:794).
9. Debug vs release APK: panel on vs off changes the p99 (zinc-next-perf.md:812-813).
10. webOS: a program started by hand registers as the app, later launches go to a dead card (hp-touchpad-webos.md:373-375); the card needs /tmp/zinc/env for env vars.
11. Software dumps lose the process (webOS stops programs): reports every 120 frames (display_gl.cpp:219).
12. GL driver differences: conformance known-failure lists per GPU family (tools/webgl-conformance:11-12); gl_renderer.sh skips on virtual GPUs (tests/t1/gl_renderer.sh:8).

4. Corpus candidates and determinism

• Deterministic, scene-dump-able, frame hashed: bench/render B1-B10 (hash in bench/render.csv), hero (ZINC_DEMO tour, ZINC_FRAMES), navigation, tests/visual four programs, galleries, breakout, boards (framehash.golden), rn-showcase (needs ZINC_INPUT script, already used for flings).
• Window-only, non-deterministic by nature: video, camera, webcam, museum-table (video, ZINC_DEMO not present), three/marble/pinball3d (physics and timers), music (network), remote/webrtc, webview.
• ZINC_DETERMINISTIC=1 forces headless (src/main.cpp:254), so a deterministic run does not exercise the window/GL path; on a device the GL path needs ZINC_INPUT + ZINC_FRAMES with a window (hal_dispatch.cpp:305, hal_sdl.cpp:445).

5. Correctness and rendering problems

• Policy (TESTING.md, “Pixel goldens”): exact (tol 0, max_diff 0) by default against the prototype’s PNGs; tolerance per entry only where a library replaces the prototype renderer; known rows (over count + reason) may not grow. Example row: ulp-level float differences on arm64 (manifest.json, boards/esp32-2432s022/src/main.tsx).
• Software raster vs GL: threshold metrics (<=5% pixels over 24 per channel, MAE < 3, wider for LINE/POLY) in tests/t1/gl_renderer.sh:2-6, glcompare and pixel-compare classes (edge 24, text 3, interior 2, share_gt8 0.01; tools/pixel-compare:11-12).
• Cross-backend: SSIM/PSNR/MAE only in pngdiff and three-compare (SSIM >= 0.90 / 0.95).
• Hash only: ZINC_FRAMEHASH, bench-render hash.
• Canary: proto-capture --canary (4 entries; tools/proto-capture:7,85-94), full = --all (T2).
• Known per-GPU or per-device failures: webGL known lists per vendor family; hero “stacked translucent glows need the raster’s integer blend in the shader” (tests/t1/gl_renderer.sh:6-7); GL replay keeps rounded CLIPs above 32 device px square (backlog/later.md:88); fp16-only GPUs (Mali-400): ZINC_GL_MEDIUMP=10 (later.md:298); SHADER under a matrix or rounded clip on the Android GPU layer (later.md:252); Android window cannot switch to GL after first frame (later.md:253); EGL restore after background untested (later.md:338); LINE/POLY skipped until ZN-181 (older); GL replay on retina Mac (later.md:30). RESUME.md:12 records that GPU 60 fps scrolling on Tegra 3 is not reached (D126).

6. Partial

1. GPU time: GL timer query exists but only on Apple core GL (runtime/gl_replay.cpp:106, display_gl.cpp:216); no EXT_disjoint_timer_query path for GLES 2 (Tegra 3, Adreno 220, VideoCore IV, SGX; most do not expose it, so glFinish stays the fallback).
2. The profiler produces CPU phases and a Chrome trace but the numbers are text on stderr/logcat, not structured; each target parses its own (bench-bridge parse, device profile grep).
3. Device plumbing: bench-bridge handles four targets for one JSON schema (microseconds per unit of a bench program); device profile handles Pi text; none carries a render artifact.
4. bench-render replays on the software raster only; no GL replay mode of --scene --bench, no device run.
5. Pixel comparison has 5+ different tools and tolerances (golden tol, classes, SSIM, glcompare, hash); no single “diff report” per device and backend.
6. Frame hash is of the software frame (gfx.cpp:935); the GPU’s displayed frame is only compared through ZINC_SHOT BMP/PNG + glcompare on the Mac.
7. Present split is in the SDL HAL only (ZINC_PRESENT_STATS), separate from ZINC_PROFILE.
8. Frame interval/jank counts (“late of 720”) are computed by hand from ZINC_PRESENT_STATS or SurfaceFlinger; there is no built-in “frames late” counter.
9. Energy and thermal: resmon --energy on the host only; Pi temperature read by hand; no Android/iOS/webOS energy/thermal logging.
10. Env passing to release apps is limited to a whitelist on Android (shell_sdl.cpp:177).
11. The corpus is split between bench/render, tests/visual, examples and per-task scripts; only B1-B10 + two apps are listed in one place.

7. Missing

• A rendering bench artifact format (JSON per machine and backend) next to docs/reports/bench/script-bridge-*.json.
• A render bench program/runner that runs a fixed list of scenes on a device (window, GL, scripted input, N frames) and collects CPU phases, present split, GPU time, interval distribution, and a frame hash/PNG.
• Metal GPU timing on iOS/macOS (MTLCommandBuffer.GPUStartTime/GPUEndTime, completion handler, presented time) and a Metal renderer of command lists to time.
• Vulkan: no backend, no design doc.
• GPU timing on GLES 2 devices (disjoint timer query when available, else documented glFinish fallback), recorded with a flag saying which.
• Frame-pacing statistics in the engine: frame interval p50/p99, count of frames over 1.5x vsync, dropped frames (only measured by hand).
• Per-device pixel baselines: goldens exist only from the prototype (host) and per-feature; no device-rendered frame is stored to compare an A9X, Tegra 3, Adreno 220 frame with the Mac’s. No SSIM/PSNR on device frames.
• A “rendering problems” log: structured list of per-device known differences (today spread over later.md, T1 comments, manifest known rows, webgl known lists).
• Environment capture in the artifact: governor, frequency, temperature, battery, display refresh rate, GL vendor/renderer strings (only ZINC_GL_INFO / glGetString at hal_sdl.cpp:230), build type, engine version/commit (bench-bridge writes a machine block, to reuse).
• Pi, ESP32 and TouchPad run of bench-render: the Pi does have device profile but no artifact.
• Thermal/sustained runs (long loops with throttle logging).
• Regression gate for GPU paths (bench-render --check-regressions covers software only).

8. Summary (10 lines)

1. The CPU side is well instrumented: ZINC_PROFILE phases with p50/p99/max (runtime/gfx.cpp:368-431), Chrome trace, perf HUD, command and damage stats.
2. ZINC_PRESENT_STATS splits raster / upload / draw / swap / interval, only in the SDL HAL (hal_sdl.cpp:1126).
3. GPU time exists only through GL_TIME_ELAPSED on Apple core GL (gl_replay.cpp:106); GLES 2 devices fall back to glFinish and inference.
4. iOS “Metal” is a blit of the software raster (hal_metal.mm); there is no Metal renderer and no GPUStart/End, completion or presented-time code; Vulkan has no code or design.
5. bench-render + bench/render/*.ts + render.csv (median, p99, hash, 1 and 4 threads, regression gate) is the best base for a corpus but runs the software raster on the Mac only.
6. bench-bridge already gives build, install, run, collect and JSON per machine on Mac, Nexus 7, TouchPad, iPad; the Pi has tools/device profile.
7. Real-hardware history is rich but in prose tables in zinc-next-perf.md, not in comparable artifacts.
8. Known pitfalls are well documented (su, governors, thermal, vsync, glFinish, load) but not captured in the artifact.
9. Pixel comparison is plentiful (pngdiff, pixel-compare, glcompare, proto-capture, three-compare) but host-only and inconsistent in metrics and tolerances.
10. Determinism hooks (ZINC_INPUT, ZINC_FRAMES, ZINC_DEMO, ZINC_FRAMEHASH, ZINC_SHOT) exist and work on real windows, so a scripted corpus is feasible.

9. Eight biggest gaps

1. No structured per-machine rendering artifact (JSON with environment, backend, scene, phases, present split, GPU, interval, hash).
2. No device-capable render runner: bench-render is host and software only.
3. GPU time on GLES 2 and Metal (and Vulkan later): absent except Apple GL.
4. No frame-pacing/jank metric (interval p99, late frames, drops) in the engine; vsync wait is mixed into PRESENT under ZINC_PROFILE.
5. No pixel baseline from device frames; cross-device and cross-backend correctness is checked by ad hoc thresholds and only on the host.
6. No single rendering-problem registry (known differences are scattered across later.md, manifest known, webgl known lists, test comments).
7. Environment/thermal/governor/energy not recorded with the numbers although they explain most past surprises.
8. No Metal command-list backend or Vulkan backend design yet to hang timing on; the scene dump (ZINC_SCENE_DUMP) is the ready common input.

10. What to reuse

• Plumbing: tools/bench-bridge (build, install, run, collect for mac / android / webos / ios, show table, machine block, docs/reports/bench/ layout); tools/device for Pi.
• Corpus and gate: bench/render/*.ts, tools/bench-render (hash + 15% gate), src/host/render_bench.cpp, ZINC_SCENE_DUMP format (include/zn/scene.h).
• Phase data: ZINC_PROFILE ring and stat_line (runtime/gfx.cpp:405), Chrome-trace writer, ZINC_PRESENT_STATS columns, ZINC_GL_STATS counters.
• GPU timer: the 4-deep query ring in runtime/gl_replay.cpp:106-120 (extend with EXT_disjoint_timer_query and a Metal equivalent in hal_metal.mm).
• Determinism: ZINC_INPUT, ZINC_FRAMES, ZINC_DEMO=bench (hero), ZINC_FRAMEHASH, ZINC_SHOT.
• Pixels: pngdiff (SSIM/PSNR/MAE), pixel-compare classes, proto-capture manifest with known rows, webgl-conformance per-family known lists as the model for per-device known-problems.
• Method notes: docs/targets/adapting-to-hardware.md:106-115, pitfalls in docs/reports/zinc-next-android.md:177-213, docs/reports/hardware/hp-touchpad-webos.md:356-368.

Edit this page on GitHub ↗