GPU audio

A device's gpuAudio.ts runs on the node's audio output as a GPU compute post-stage, dispatched once per block, pipelined and delay-compensated. Msh transpiles the TGSL to WGSL (typegpu), validates it with naga, and the engine dispatches it every block. Import everything you use from @msh/device-sdk/device/gpu/audio. See the how-to: Process audio on the GPU.

defineGpuAudio

Two forms. Single-pass:

export default defineGpuAudio(fn, opts?);

fn(gid, info, p) runs once per gid.x, a sample index. The body must begin with the "use gpu" directive. Guard if (gid.x >= info.numSamples) return;. opts.params is a list of param ids exposed on p.

Multi-pass:

export default defineGpuAudio({ params, passes });

passes is up to 8 pass objects { fn, dispatch?, workgroupSize? }, dispatched in order each block. See Multi-pass.

GpuAudioInfo

The info argument:

• numSamples : samples in this block.
• channels : channel count.
• sampleRate : f32 Hz.
• bpm : host tempo, or 0 when the host reports none.
• ppq : pulses per quarter for the block start.
• playing : 1 while the transport runs, else 0.
• frameLo, frameHi : low and high 32 bits of a running sample counter for the block's first sample. Use for free-running phase without transport, e.g. f32(info.frameLo % 4194304) + f32(i) over info.sampleRate.

Params

p holds the ids in opts.params, each an f32 in the param's declared min..max units. These are block-rate: one value for the whole block. For audio-rate (per-sample smoothed and modulated) values, use the ramps buffer instead.

Bound buffers

Audio is planar. The index for channel c, sample i is c * info.numSamples + i. Import each buffer you use from @msh/device-sdk/device/gpu/audio.

input

This block's input samples, the wasm process output, planar, read-only. Read input.$[idx].

output

This block's output samples, planar. Write output.$[idx]. Write every sample you own.

state

65536 persistent f32s (grow with m.gpuState), zeroed at build and every rebuild, carried across blocks in order: oscillator phases, filter memories, delay lines. Give each slot exactly one writing invocation, or gate the shared write on gid.x === 0.

data

The per-block payload the wasm wrote with ctx.writeGpuAudioData(i, v): note freqs, gates, envelopes, modulated values. Read-only on the GPU, fresh every block. Declare capacity with m.gpuAudioData(floats).

analysis

GPU to CPU output. Write non-audio results here: a spectrum, levels, a detected tempo or key. The host reads it back each block, and it also lands in the wasm via ctx.gpuAnalysis(i). Declare with m.gpuAnalysis(floats).

staticData

Bulk read-only data the wasm filled with writeGpuStatic: wavetables, IRs, LUTs. Declare with m.gpuStatic(floats). Uploaded once at build and on commitGpuStatic(), then GPU-resident.

ramps

Per-sample param values from m.gpuParamRamps([ids]), up to 16. Slot order is manifest order. Read ramps.$[slot * info.numSamples + i] for the smoothed, modulated value of param slot at sample i.

Multi-pass

defineGpuAudio({
  params: ["mix"],
  passes: [
    { fn: analyze, dispatch: "samples", workgroupSize: 128 },
    { fn: synth, dispatch: 64 * 128 }, // voices x partials
    { fn: mixdown, dispatch: { dynamic: 512 } },
  ],
});

Up to 8 passes, dispatched in order each block. A pass boundary is a GLOBAL sync point: every write in pass N is visible everywhere in pass N+1. This is stronger than workgroupBarrier(), which only synchronizes within one workgroup. Use passes for FFT stages, per-voice synth then mix-down, and reductions.

dispatch modes:

• "samples" (default) : one thread per sample.
• a fixed number : e.g. 64 * 128 for voices times partials.
• { dynamic: max } : the wasm sets the per-block thread count with ctx.setGpuDispatch(pass, threads). Unset reads as max; 0 skips the pass.

workgroupSize defaults to 64, clamped to 1..256.

Helpers

Exported from @msh/device-sdk/device/gpu/audio:

• workgroupVar(arrayOf(f32, 256)) : workgroup-shared memory. Synchronize with workgroupBarrier(). Does not persist across dispatches; use state for that.
• hash(u32) -> u32 : PCG hash for stateless noise.
• rand01(u32seed) -> f32 : uniform [0, 1). White noise per sample: rand01(info.frameLo + gid.x); bipolar with * 2 - 1.

The module also re-exports the typegpu data constructors (f32, i32, u32, vec2f, vec3f, vec4f, arrayOf) and all of typegpu/std math (sin, cos, floor, min, max, ...), so a program imports only from @msh/device-sdk/device/gpu/audio, never typegpu.

Manifest builders

Called in describe on the Manifest:

• m.gpuAudioData(floats) : capacity of the per-block data payload.
• m.gpuAnalysis(floats) : capacity of the analysis readback.
• m.gpuState(floats) : grow the persistent state buffer.
• m.gpuStatic(floats) : capacity of staticData.
• m.gpuParamRamps([ids]) : which params get per-sample ramps (up to 16).
• m.gpuPipelineDepth(1..4) : pipeline depth for the stage.

Ctx methods

Called in process:

• ctx.writeGpuAudioData(i, v) : write float v at index i of the data payload.
• ctx.gpuAnalysis(i) : read back float i the GPU wrote to analysis.
• ctx.gpuAnalysisCount() : number of analysis floats.
• ctx.setGpuDispatch(pass, threads) : set the thread count for a { dynamic } pass.
• ctx.gpuActive : true when the GPU stage is running this session. Fall back to CPU DSP when false.

Raw WGSL

Prefer hand-written WGSL? Author gpuAudio.wgsl instead of gpuAudio.ts. The same bound buffers apply. Msh validates it with naga and dispatches it as the same post-stage. Use it as an escape hatch when you need WGSL features the TGSL transpiler does not model.