openvisengine/rust-agent

ava-agent — AVA native scaffold (Rust + wgpu)

Native counterpart of the AVA — Advanced Visualization Agent web edition (the Next.js reference implementation lives at the repository root). One cargo crate that runs the same .ava visualization scripts: cpal audio capture → rustfft band analysis → the ported beat predictor → the AVS-eel expression engine → a wgpu (Vulkan-first) renderer.

The portability contract: every script under presets/ runs unmodified on both engines.

Architecture

┌──────────────────────────── ava-agent (binary) ───────────────────────────┐
│                                                                          │
│  cpal input thread ──► RingBuffer ──► spectrum.rs (rustfft 8192, Hann)   │
│                                          │                               │
│                                          ▼                               │
│   audio.rs  BandLevels (bass/mid/treb + _att, attack 35 ms / release     │
│              220 ms — identical constants to the web engine)             │
│                                          │                               │
│                                          ▼                               │
│   beat.rs   BeatTracker: detect → learn (median+EMA) → predict (PLL)     │
│                                          ▲      └── tap tempo (Space)   │
│                                          │                               │
│                                          ▼                               │
│   script/   .ava parser + expression engine                              │
│             [init] once · [frame] ×1 · [beat] on beats · [point] ×n      │
│                                          │                               │
│                                          ▼                               │
│   render/   wgpu: fade → audio-reactive background → additive points     │
└──────────────────────────────────────────────────────────────────────────┘

Web ↔ native module map:

web (src/lib/ava/)	native (src/)
audio.ts (WebAudio Analyser)	audio.rs + spectrum.rs
beat.ts	beat.rs (statement-for-statement)
expression.ts / script.ts	script/{lexer,parser,env,mod}.rs
engine.ts (rAF loop + Canvas2D)	engine.rs + render/{mod.rs,shader.wgsl}
presets.ts	presets/*.ava

Building

Linux (Debian / Ubuntu)

sudo apt install libasound2-dev libvulkan-dev   # or: libvulkan1 vulkan-loader
cargo run --release -- --preset presets/gap-the-mind.ava

wgpu speaks native Vulkan on Linux; if no Vulkan loader is present the agent falls back to other primary backends with a warning.

Windows

1. Install Visual Studio Build Tools (C++ workload) and Rust with the x86_64-pc-windows-msvc toolchain.
2. cargo run --release -- --preset presets/gap-the-mind.ava

wgpu picks the best available backend — Vulkan preferred (per the project's Vulkan requirement), DX12 as the practical fallback on machines without a Vulkan driver.

macOS

cargo run --release -- --preset presets/gap-the-mind.ava

By default this uses Metal via wgpu's automatic backend selection (macOS has no native Vulkan). For strict Vulkan install the Vulkan SDK (MoltenVK) and build with:

cargo run --release --features vulkan-force -- --preset presets/gap-the-mind.ava

One codebase, three platforms, Vulkan-first.

Controls & CLI

input	action
Space	beat tap tempo ("Set Beat") — the planned --bpm-tap key
Esc	quit

ava-agent [--preset <path.ava>] [--input <device-index|default>] [--list-devices]

• --preset path.ava — load a visualization script (see presets/)
• --input <index> — pick an audio input device by index (--list-devices enumerates them); default uses the system default input
• Audio capture failing? The agent exits with the full device list; feed line-in or a loopback device via --input.

Port status

area	status
audio capture (cpal)	✓
FFT / 3-band analysis	✓ (bass 20–100 Hz · mid 700–2000 Hz · high 7000–16000 Hz, tunable via BandConfig)
beat tracker + predictor	✓ (1:1 port of beat.ts)
script engine	✓ (subset: same grammar & function set; no reg[] buffers yet)
renderer	✓ (basic reactive fullscreen shader + additive point cloud)
AVS compatibility	partial (the 5 adapted presets run identically; raw Winamp AVS features like Texer sprites are roadmap items)

Known v0 simplifications (search TODO(avb) in the source): persistence trails use a fade pass instead of an offscreen feedback texture, solid render mode falls back to points, and point size is not yet honored (wgpu point-list draws single pixels).