115 lines
5.7 KiB
Markdown
115 lines
5.7 KiB
Markdown
# ava-agent — AVA native scaffold (Rust + wgpu)
|
||
|
||
Native counterpart of the **AVA — Advanced Visualization Agent** web edition
|
||
(the Next.js reference implementation lives at the repository root). One cargo
|
||
crate that runs the *same* `.ava` visualization scripts: cpal audio capture →
|
||
rustfft band analysis → the ported beat predictor → the AVS-eel expression
|
||
engine → a wgpu (Vulkan-first) renderer.
|
||
|
||
The portability contract: every script under `presets/` runs **unmodified** on
|
||
both engines.
|
||
|
||
## Architecture
|
||
|
||
```text
|
||
┌──────────────────────────── ava-agent (binary) ───────────────────────────┐
|
||
│ │
|
||
│ cpal input thread ──► RingBuffer ──► spectrum.rs (rustfft 8192, Hann) │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ audio.rs BandLevels (bass/mid/treb + _att, attack 35 ms / release │
|
||
│ 220 ms — identical constants to the web engine) │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ beat.rs BeatTracker: detect → learn (median+EMA) → predict (PLL) │
|
||
│ ▲ └── tap tempo (Space) │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ script/ .ava parser + expression engine │
|
||
│ [init] once · [frame] ×1 · [beat] on beats · [point] ×n │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ render/ wgpu: fade → audio-reactive background → additive points │
|
||
└──────────────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
Web ↔ native module map:
|
||
|
||
| web (`src/lib/ava/`) | native (`src/`) |
|
||
| ------------------------------- | -------------------------------- |
|
||
| `audio.ts` (WebAudio Analyser) | `audio.rs` + `spectrum.rs` |
|
||
| `beat.ts` | `beat.rs` (statement-for-statement) |
|
||
| `expression.ts` / `script.ts` | `script/{lexer,parser,env,mod}.rs` |
|
||
| `engine.ts` (rAF loop + Canvas2D) | `engine.rs` + `render/{mod.rs,shader.wgsl}` |
|
||
| `presets.ts` | `presets/*.ava` |
|
||
|
||
## Building
|
||
|
||
### Linux (Debian / Ubuntu)
|
||
|
||
```sh
|
||
sudo apt install libasound2-dev libvulkan-dev # or: libvulkan1 vulkan-loader
|
||
cargo run --release -- --preset presets/gap-the-mind.ava
|
||
```
|
||
|
||
wgpu speaks native Vulkan on Linux; if no Vulkan loader is present the agent
|
||
falls back to other primary backends with a warning.
|
||
|
||
### Windows
|
||
|
||
1. Install [Visual Studio Build Tools](https://visualstudio.microsoft.com/downloads/)
|
||
(C++ workload) and Rust with the `x86_64-pc-windows-msvc` toolchain.
|
||
2. `cargo run --release -- --preset presets/gap-the-mind.ava`
|
||
|
||
wgpu picks the best available backend — Vulkan preferred (per the project's
|
||
Vulkan requirement), DX12 as the practical fallback on machines without a
|
||
Vulkan driver.
|
||
|
||
### macOS
|
||
|
||
```sh
|
||
cargo run --release -- --preset presets/gap-the-mind.ava
|
||
```
|
||
|
||
By default this uses **Metal** via wgpu's automatic backend selection (macOS
|
||
has no native Vulkan). For **strict Vulkan** install the
|
||
[Vulkan SDK](https://vulkan.lunarg.com/sdk/home) (MoltenVK) and build with:
|
||
|
||
```sh
|
||
cargo run --release --features vulkan-force -- --preset presets/gap-the-mind.ava
|
||
```
|
||
|
||
One codebase, three platforms, Vulkan-first.
|
||
|
||
## Controls & CLI
|
||
|
||
| input | action |
|
||
| ---------------- | --------------------------------------------------------- |
|
||
| `Space` | beat tap tempo ("Set Beat") — the planned `--bpm-tap` key |
|
||
| `Esc` | quit |
|
||
|
||
```text
|
||
ava-agent [--preset <path.ava>] [--input <device-index|default>] [--list-devices]
|
||
```
|
||
|
||
- `--preset path.ava` — load a visualization script (see `presets/`)
|
||
- `--input <index>` — pick an audio input device by index (`--list-devices`
|
||
enumerates them); `default` uses the system default input
|
||
- Audio capture failing? The agent exits with the full device list; feed line-in
|
||
or a loopback device via `--input`.
|
||
|
||
## Port status
|
||
|
||
| area | status |
|
||
| ----------------------------- | ------ |
|
||
| audio capture (cpal) | ✓ |
|
||
| FFT / 3-band analysis | ✓ (bass 20–100 Hz · mid 700–2000 Hz · high 7000–16000 Hz, tunable via `BandConfig`) |
|
||
| beat tracker + predictor | ✓ (1:1 port of `beat.ts`) |
|
||
| script engine | ✓ (subset: same grammar & function set; no `reg[]` buffers yet) |
|
||
| renderer | ✓ (basic reactive fullscreen shader + additive point cloud) |
|
||
| AVS compatibility | partial (the 5 adapted presets run identically; raw Winamp AVS features like Texer sprites are roadmap items) |
|
||
|
||
Known v0 simplifications (search `TODO(avb)` in the source): persistence
|
||
trails use a fade pass instead of an offscreen feedback texture, `solid`
|
||
render mode falls back to points, and point `size` is not yet honored
|
||
(wgpu point-list draws single pixels).
|