openvisengine/worklog.md

# Worklog — AVA (Advanced Visualization Agent)

Project: Audio-reactive visualization agent (AVS-style scripting, 3-band spectrum detection,
predictive beat tracking). Built as a Next.js 16 web app (reference implementation) plus a
Rust/Vulkan scaffold for the future native cross-platform export.

User requirements:
- 3-band detection: Bass 20–100 Hz, Mid 700 Hz–2 kHz, High 7 kHz–16 kHz (user-adjustable)
- Beat detection from bass + beat PREDICTION (keeps beat when bass drops out)
- Green dot beat indicator + "Set Beat" tap-tempo button
- AVS-style custom visualization script editor: edit, save as default, save to file, load from file
- Help section with variables/functions reference
- 5 preset scripts (user provided real Winamp AVS presets on Google Drive)
- Rust + Vulkan target for Linux/Windows/macOS export (scaffold + blueprint delivered here)

---
Task ID: 1
Agent: main (Z.ai Code)
Task: Project inspection + Google Drive preset retrieval

Work Log:
- Verified dev server running on port 3000 (Next.js 16.1.3, dev.log clean)
- Fetched Google Drive folder (public) — extracted 5 file IDs and downloaded all files:
  - "Synth-C - Gap The Mind.avs" (12.5 KB)
  - "zamuz - wonder 'bout.avs" (34 KB)
  - "Synth-C - Quantization.avs" (14.8 KB)
  - "zamuz - keep tryin'.avs" (19 KB)
  - "zamuz - vorticity (brain activity remix).avs" (17 KB)
- Parsed binary AVS format (null-terminated strings): extracted Superscope code, Texer II
  configs, movement expressions. These are scene classics from visbot.net.
- Decided engine design: AVS-eel-flavored expression language (if/above/below/equal/rand,
  getspec/getosc-equivalents sv/v, reg-free flat variables) so the presets port naturally.

Stage Summary:
- All 5 original presets secured and understood; will be adapted to the AVA script dialect
  (originals preserved as downloadable files in public/avs-originals/).
- Core engine architecture defined: expression compiler, script parser, BeatTracker (PLL-style
  predictor + tap tempo), AudioEngine (WebAudio FFT, mic/file/demo synth), canvas renderer.

---
Task ID: 2
Agent: main (Z.ai Code)
Task: Project setup — deps, Prisma schema, DB, original preset files

Work Log:
- Installed jszip (for the Rust scaffold zip download API)
- Prisma schema rewritten: `Preset` (name/author/description/content/isBuiltIn, unique name)
  and `Settings` (defaultScript + bandConfig + beatConfig JSON) — ran `bun run db:push` OK
- Copied the 5 original Winamp .avs binaries to public/avs-originals/ (downloadable from UI)

Stage Summary:
- DB ready; jszip available; originals served at /avs-originals/*.avs

---
Task ID: 3
Agent: main (Z.ai Code)
Task: AVA core engine libraries (src/lib/ava/*)

Work Log:
- types.ts — BandConfig / BeatConfig / BandLevels / BeatFrame + defaults
  (bass 20–100 Hz, mid 700–2000 Hz, high 7000–16000 Hz as requested)
- expression.ts — full expression compiler: tokenizer → recursive-descent parser → closure
  tree. AVS-eel flavored: if() is LAZY, && / || short-circuit, ternary, ^ power (right-assoc),
  assignment valid inside call args + ternary branches (if(cond, x=1, x=2) works),
  newline & ';' statement separators (newlines inside parens ignored), '#'/'//' comments,
  deterministic hash rand(seed). 30+ functions incl. above/below/equal/band/bor/bnot/frac.
- script.ts — text format parser: header keys (name/author/description/mode/blend/persist/
  linewidth) + [init]/[frame]/[beat]/[point] sections, error objects with section+line,
  validateScript() never throws.
- beat.ts — BeatTracker: rolling-average energy detection w/ threshold+minInterval+floor,
  interval learning (median + EMA period lock, spread gating), PLL-style phase predictor that
  keeps firing beats when bass drops (confidence-gated, slowly decays), tap() tempo learning.
- audio.ts — AudioEngine: WebAudio AnalyserNode FFT 8192 (~5.4 Hz bins), byte-spectrum band
  averaging with attack 35ms / release 220ms smoothing; sources: demo (synthesized techno:
  kick 160→43Hz sweep, snare bandpass 1.8k, hats HP 8.2k, saw bassline — audible + analyzable),
  mic (never routed to speakers), file (loops, audible). getWave/getSpectrum for scripts.
- presets.ts — 5 built-in presets ADAPTED from the user's real AVS files:
  Gap The Mind (radial warp rings), wonder 'bout (bass-spawned expanding rings),
  Quantization (grid-snapped dot rings), keep tryin' (bass burst dots + swirl),
  vorticity (twisting spiral arms) + STARTER_TEMPLATE.
- engine.ts — AvaEngine: rAF loop (bands → beat → env → frame/beat progs → point prog ×N →
  draw), canvas 2d renderer with additive blending, per-point color, 2-pass glow,
  persistence trails, lines/dots/solid modes, DPR-aware resize via ResizeObserver,
  throttled stats callback for the UI. Singleton via getEngine().
- SMOKE TESTED via bun script: all 5 presets parse & execute 120 frames; lazy-if/ternary/pow
  OK; beat sim locks 124.1 BPM from 124 BPM kick train; predictor sustains beats through 10s
  of silence; tap tempo locks 120 BPM from 500ms taps. (2 parser bugs found & fixed: assignment
  inside call args; newline statement separators with paren-awareness.)

Stage Summary:
- Core engine complete and verified. Env variables exposed to scripts: n, t, time, dt, frame,
  bass/mid/treb (+ _att), vol (+ _att), beat, bpm, beatphase, beatconf, beats, w, h, aspect,
  persist, i, v, sv, x, y, red, green, blue, skip, size, alpha.

---
Task ID: 4
Agent: main (Z.ai Code)
Task: API routes

Work Log:
- GET/POST /api/presets — GET seeds built-ins on first run and lists all; POST upserts user
  presets by name (built-ins protected, 409 on name clash with built-in)
- PUT/DELETE /api/presets/[id] — user presets only
- GET/PUT /api/settings — defaultScript + bandConfig + beatConfig (upsert, JSON strings in DB)
- Created src/lib/ava/blueprint.ts STUB (interface Blueprint + placeholder) for Task 3-b

Stage Summary:
- Backend complete. Frontend contract: GET /api/presets → { presets: [...] };
  POST /api/presets {name, content, author?, description?} → {preset};
  GET /api/settings → { defaultScript, bandConfig, beatConfig };
  PUT /api/settings { defaultScript? , bandConfig?, beatConfig? }.


---
Task ID: 3-a
Agent: full-stack-developer (UI)
Task: Complete UI for AVA — single-page app (dark club/VJ aesthetic) with header
(audio source controls, BPM readout, green beat dot + Set Beat pairing), visual
stage (canvas + engine loop), 5-tab control panel (Presets / Script / Detect /
Beat / Native) and sticky footer status bar.

Work Log:
- Read worklog + all engine APIs (types/engine/script/presets/blueprint/audio/beat)
  and the existing API routes before writing any code.
- src/lib/store.ts — zustand store: throttled stats (AvaStats = EngineStats +
  beatCount), beatFlash pulses, bandConfig/beatConfig mirrors that write the
  engine live + persist (localStorage instant, PUT /api/settings debounced 600 ms
  with shape guards), preset list + refreshPresets(), script text, active tab.
- src/components/ava/audio-source.ts — tapBeat() (engine.beat.tap + dot flash +
  lastTap), applyScriptText() (validate → setScript, engine.setScript), and
  useAudioSource() hook wrapping startDemo/startMic/startFile/stop with error
  mapping (mic permission / no device / decode failure → destructive toast) and
  store sync from engine.audio.kind (authoritative).
- src/components/ava/beat-dot.tsx — the green indicator: SVG phase ring
  (stroke-dashoffset follows beat.phase, 150 ms linear transition), emerald dot
  that scales 1.9× + glows 180 ms on audio/tap beats, and for predicted beats an
  expanding outline ring (framer-motion, 450 ms) + "PRED" badge. Flash state is
  derived (seq vs clearedSeq) — no setState-in-effect.
- src/app/page.tsx — useInitialLoad (presets + settings; server → localStorage →
  first preset → STARTER_TEMPLATE fallback chain; applies configs + script),
  useBeatWatcher (see decisions below), useSpaceToTap (Space outside
  inputs/textarea/select/button → tap, preventDefault). Root layout:
  min-h-screen flex-col, lg:h-screen with internal panel scroll (footer always
  at viewport bottom; mobile pushes footer naturally).
- src/components/ava/header.tsx — sticky, backdrop-blur; emerald logo square +
  AVA title + subtitle (hidden on mobile); segmented Demo/Mic/File/Stop control
  (44 px targets, active state emerald, hidden file input); demo BPM slider
  (60–200, only while demo active); right cluster = BPM readout (mono, emerald,
  '—' when unknown) + BeatDot + Set Beat button, always visible, wraps to row 1
  on mobile (source controls wrap to row 2).
- src/components/ava/visual-stage.tsx — owns engine lifecycle (engine.start/
  stop + onStats → store), idle glass overlay with 3 big source buttons
  (AnimatePresence fade), top-left FPS/PTS readout (pointer-events-none), top-
  right fullscreen toggle (requestFullscreen on container). Canvas absolute
  fill; engine's ResizeObserver handles DPR.
- src/components/ava/presets-tab.tsx — preset cards (name/author/description
  line-clamp-3, mode + built-in/custom badges), Load (applies + toasts, jumps
  to Script tab on compile error), ".avs original" download link for built-ins
  via originalFileUrl(name), two-step Delete (3 s confirm timeout) for customs,
  "New Script" (STARTER_TEMPLATE + switch tab), save-as-preset section (POST,
  409 handled, refresh + toast). List: max-h-96 + thin styled scrollbar.
- src/components/ava/script-tab.tsx — toolbar (Apply ▶ / Save as Default 💾 /
  Save to File ⬇ Blob download <meta.name>.ava / Load from File 📂 (.ava,.avs,
  .txt, auto-applies when valid) / New / Help ?), gutter+textarea editor
  (synced scroll, Tab → 2 spaces, min-h-320, wrap=off), rose gutter line on the
  error line, live debounced validation (250 ms), status line "✔ Compiled OK —
  mode: lines, n=480" (n from running [init] on a scratch env) / compile error
  / stats.error runtime errors.
- src/components/ava/help-panel.tsx — Quick Start, full variables table (frame
  + point-only), functions table, operators, 5 example formulas, .ava file
  format doc.
- src/components/ava/detect-tab.tsx — Bass/Mid/High cards: Low/High Hz sliders
  (bass 10–300/20–500, mid 100–5000/300–8000, high 1000–12000/4000–20000; low <
  high enforced with min gaps 10/50/200 in the store), Gain 0.1–4, mono Hz
  readouts, live VU meters (instant fill emerald/amber/rose + _att peak-hold
  tick), per-band reset.
- src/components/ava/beat-tab.tsx — 80 px BeatDot + source badge (AUDIO/PRED/
  TAP), big BPM readout, large SET BEAT button (framer-motion whileTap), tap
  feedback ("Tap 3… keep tapping" → "Locked: 128.0 BPM"), confidence bar, beat
  count, sensitivity (threshold 1.1–2.5, "lower = more sensitive"), min
  interval 150–600 ms, prediction switch ("Keeps the beat alive when the bass
  drops out"), how-it-works explainer.
- src/components/ava/native-tab.tsx — renders BLUEPRINT (title/intro/sections/
  bullets/code blocks in scrollable pre), emerald "Download Rust Scaffold
  (.zip)" <a href="/api/rust-scaffold" download>, identical-language note.
- src/components/ava/footer-bar.tsx — sticky mt-auto status bar: SRC/FPS/PTS/
  BPM/SCRIPT micro-stats (mono), green "engine ok" / pulsing red "script error".
- src/app/layout.tsx — dark class on <html>, title "AVA — Advanced
  Visualization Agent", body bg-zinc-950.
- FIXED BUGS during browser testing: (1) Detect tab crashed — levels key for
  the high band is `treb` not `high` (added levelKey/attKey mapping to BandSpec);
  (2) lint: setState-in-effect in BeatDot (derived flash state instead);
  (3) lint: // text node in help panel.
- Tuned src/hooks/use-toast.ts: TOAST_LIMIT 1→3, TOAST_REMOVE_DELAY 100 s→5 s
  (lingering toasts were covering header/panel buttons during testing).
- VERIFIED in headless browser: initial load seeds presets + applies Gap The
  Mind (480 pts @60 fps); demo start/stop + DEMO slider; beat dot flashing live
  (scale 1↔1.9); tap tempo lock ("Locked: 119.9 BPM" from 4×500 ms taps, Space
  key too); after Stop the PRED badge/outline ring shows prediction carrying
  the beat; preset load/save/delete (two-step) round-trip via API; Save as
  Default → PUT /api/settings + localStorage; band slider → debounced PUT;
  .avs originals serve 200; script upload (auto-apply, n=300) + export to
  Upload Test.ava; mic error toast ("No microphone found" in this headless
  env); no blue/indigo anywhere; lint clean; tsc clean for owned files.

Stage Summary:
- Files created: src/lib/store.ts; src/components/ava/{shared.ts, audio-source.ts,
  beat-dot.tsx, header.tsx, visual-stage.tsx, presets-tab.tsx, script-tab.tsx,
  help-panel.tsx, detect-tab.tsx, beat-tab.tsx, native-tab.tsx, footer-bar.tsx,
  control-panel.tsx}. Modified: src/app/page.tsx (replaced), src/app/layout.tsx,
  src/hooks/use-toast.ts (toast duration/limit only).
- KEY DECISION — beat flash sampling: engine stats are throttled (~8 Hz) but
  beat=true lasts one frame, so most beats are invisible in stats. The watcher
  in page.tsx ALSO fires on beat.phase running backwards (real beats reset
  phase to 0; the predictor grid wraps at 1). Classification: stats.beat.beat
  sampled → exact source; phase drop + prediction on + confidence > 0.35 +
  bassAtt < 0.06 → 'predicted'; phase drop + bassAtt > 0.05 → 'audio'; silent
  no-lock → no flash (idle dot doesn't pulse). Tap flashes suppressed for
  300 ms to avoid double pulses. Verified working in-browser.
- KEY DECISION — desktop uses lg:h-screen + internal panel scroll so the footer
  stays at the viewport bottom and the canvas fills the height; mobile uses
  natural flow (min-h-screen + mt-auto footer) so the footer is pushed down.
- Store is the only place configs touch the engine (applyBandConfig/
  setBandValue etc.), keeping live-update + persistence consistent everywhere.
- NOTE for Task 5 (integration): /api/rust-scaffold is linked from the Native
  tab but the route does not exist yet (Task 3-b owns it — expect 404 until
  then). BLUEPRINT is still the stub; the Native tab renders whatever 3-b puts
  in it, no UI changes needed. Settings row 'main' currently holds
  defaultScript="Gap The Mind" + default bandConfig from testing (both equal
  defaults — safe to leave or clear). Pre-existing tsc errors in
  src/lib/ava/{audio,script}.ts and examples/ are NOT from this task
  (ignoreBuildErrors is set).

---
Task ID: 3-b
Agent: general-purpose (Rust scaffold + blueprint) — [timed out at the very end;
        deliverables verified complete by the main agent, which appends this entry]

Task: Rust/Vulkan native-export scaffold + architecture blueprint + zip download API

Work Log:
- Built rust-agent/ cargo crate (ava-agent, edition 2021): winit 0.30 (ApplicationHandler
  API), wgpu 22 (Vulkan-first; `vulkan-force` feature for MoltenVK on macOS), cpal 0.15,
  rustfft 6.2, pollster, anyhow
- src/audio.rs — cpal default-input capture, ring buffer, AudioProcessor::poll() per frame
- src/spectrum.rs — Hann-window 8192 FFT, band levels w/ same attack(35ms)/release(220ms)
  smoothing as the web engine
- src/beat.rs — 1:1 port of beat.ts (same constants, median+EMA lock, PLL predictor, taps)
- src/script/{lexer,parser,env,mod}.rs — expression engine + .ava format parser with the
  same lazy-if / assignment-in-args / deterministic-rand semantics
- src/engine.rs — script runtime host (init/frame/beat/point stepping, shared state)
- src/render/{mod.rs,shader.wgsl} — wgpu pipelines: audio-reactive fullscreen background +
  additive point cloud, #[repr(C)] uniforms, persistence via TODO'd feedback texture
- presets/*.ava — all six scripts (5 classics + starter) portable across engines
- Rewrote src/lib/ava/blueprint.ts with 8 real sections (architecture map, wgpu/Vulkan
  strategy, audio pipeline, beat engine, script engine, rendering, per-platform builds,
  roadmap)
- Built src/app/api/rust-scaffold/route.ts (jszip walk, skip target/.git/node_modules)

Stage Summary:
- Scaffold ~1,700 lines, all files present; zip route verified 200 in dev.log; lint clean.
- NOT compiled (no cargo in sandbox) — scaffold quality, clearly TODO-marked where partial.

---
Task ID: 5
Agent: main (Z.ai Code)
Task: Integration, lint, agent-browser E2E verification & fixes

Work Log:
- bun run lint → zero errors across the whole project
- dev.log clean: / 200, /api/presets 200, /api/settings 200, /api/rust-scaffold 200,
  no runtime errors, no hydration warnings
- Agent-browser E2E golden path, all VERIFIED:
  * initial load: idle overlay, 5 preset cards + .avs original links, tabs render
  * Demo Track → canvas lit (30.7% pixels @60fps, 480 pts), footer SRC/FPS/PTS/BPM live
  * footer shows detected 120.0 BPM from the demo kick pattern
  * Set Beat ×4 taps → "Locked: 99.8 BPM" (matches real ~600ms tap intervals), confidence 65%
  * Beat tab: AUDIO badge, beat count, sensitivity ×1.40, min interval 250ms, prediction switch
  * Stop audio → badge switches to PRED, beat count 20→35 (predictor keeps the beat),
    confidence decays 65%→35% — core requirement verified
  * Script editor: broken script → "✘ [point] line 8: Unknown function 'badfunc'";
    fixed script → "✔ Compiled OK — mode: lines, n=300"
  * Help panel: variables table, functions, operators, examples
  * Save as preset → "Smoke Test Ring" listed + toast; two-step delete verified
  * Save as Default → PUT /api/settings verified via curl; reload auto-loads the default
  * Detect tab: Bass 20–100 Hz / Mid 700–2000 Hz / High 7k–16 kHz cards, gains, live
    meters reading real audio (0.78/0.60/0.32)
  * Native tab: blueprint sections render + zip download (116 KB, 25 files, valid zip)
  * Mobile 390×844: responsive, footer pushed naturally, sticks to viewport bottom when scrolled
- Cleanup: deleted the test preset, cleared test defaultScript (DB + localStorage) so a
  fresh visit loads "Gap The Mind"

Stage Summary:
- ALL acceptance criteria verified in-browser. Zero console errors. The application is
  fully interactive: 3-band detection, beat detection + prediction + tap learning, AVS-style
  scripting with editor/help/save/load, 5 authentic presets (adapted), Rust scaffold export.