⌘K

Process audio on the GPU

Run a device's DSP as a GPU compute post-stage.

For DSP that is massively parallel, hundreds of voices, big FFTs, long convolution, per-partial additive synthesis, run it on the GPU. Author a gpuAudio.ts next to your device's index.ts. It is the DSP sibling of the shader canvas: Msh runs it on the node's audio output as a compute post-stage, pipelined and delay-compensated, dispatched once per block.

The split is two files. The wasm brain (index.ts) does the sequential work: MIDI, voice allocation, parameter smoothing. Each block it ships the GPU a small payload (per-voice frequencies, gates, envelopes) and then silences its own output. The GPU program (gpuAudio.ts) does the wide arithmetic: synthesize every voice, sum them, write the block.

Warning

Your CPU process output is the GPU stage's input. If your wasm both computes audio and lets the GPU write the same block, you double the signal. When the GPU owns the sound, silence the CPU output (write zeros) and let gpuAudio.ts produce everything.

A gain stage

The simplest program reads input, scales it, writes output. Audio is planar: for channel c and sample i, the index is c * info.numSamples + i.

gpuAudio.ts
import {
  defineGpuAudio,
  input,
  output,
} from "@msh/device-sdk/device/gpu/audio";

export default defineGpuAudio(
  (gid, info, p) => {
    "use gpu";
    const i = gid.x;
    if (i >= info.numSamples) return;
    for (let c = 0; c < info.channels; c++) {
      const idx = c * info.numSamples + i;
      output.$[idx] = input.$[idx] * p.gain;
    }
  },
  { params: ["gain"] },
);

fn(gid, info, p) runs once per gid.x, a sample index. Always guard if (gid.x >= info.numSamples) return;. info carries the block: numSamples, channels, sampleRate, bpm, ppq, playing. p is your declared params, each a live f32 in its declared units.

The additive-synth pattern

For a polyphonic synth the wasm is the brain and the GPU is the muscle. In index.ts you allocate voices and, each block, write [freq, gate] per voice plus a couple of globals with ctx.writeGpuAudioData(i, v). Declare the payload size in describe:

index.ts
m.gpuAudioData(2 + 2 * MAX_VOICES); // [partials, waveform] + [freq, gate] per voice

In gpuAudio.ts you read that payload from the data buffer and synthesize. Give each voice its running phase a slot in state, a block of 65536 persistent f32s carried across blocks, so oscillators stay continuous:

gpuAudio.ts
import {
  defineGpuAudio,
  output,
  state,
  data,
  sin,
} from "@msh/device-sdk/device/gpu/audio";

export default defineGpuAudio((gid, info) => {
  "use gpu";
  const i = gid.x;
  if (i >= info.numSamples) return;
  const t = f32(i) / info.sampleRate;
  let sum = f32(0);
  for (let v = 0; v < MAX_VOICES; v++) {
    const freq = data.$[2 + v * 2];
    const gate = data.$[2 + v * 2 + 1];
    sum = sum + sin(6.2831853 * freq * t) * gate;
  }
  for (let c = 0; c < info.channels; c++) {
    output.$[c * info.numSamples + i] = sum * 0.2;
  }
}, {});

Note

state is zeroed at build and on every rebuild. Give each slot exactly one writing invocation, or gate the shared write on gid.x === 0. Two invocations writing the same slot race.

Free-running time

Without transport you cannot rely on bpm or ppq. info.frameLo and info.frameHi are the low and high 32 bits of a running sample counter for the block's first sample. Build a phase clock from them: f32(info.frameLo % 4194304) + f32(i) over info.sampleRate.

CPU fallback

The GPU stage may not be running: no device, an unsupported machine, an offline render path. Check ctx.gpuActive in process. When it is false, do the DSP on the CPU as usual and write real samples. When it is true, ship the payload and silence your output.

index.ts
if (ctx.gpuActive) {
  for (let v = 0; v < voices; v++) {
    ctx.writeGpuAudioData(2 + v * 2, voice[v].freq);
    ctx.writeGpuAudioData(2 + v * 2 + 1, voice[v].gate);
  }
  // leave the output buffer silent; the GPU fills it
} else {
  renderVoicesOnCpu(out);
}

Note

The GPU stage costs one buffer of latency, which Msh delay-compensates automatically. Your audio arrives on time relative to the rest of the graph. You do not compensate for it yourself.

Inspecting it

msh-helper instance <id> gpu-stats --node <n> prints the round-trip timings and the analysis readback for a running GPU node, so you can confirm the stage is live and see where the time goes. See the CLI reference.

Every buffer, the multi-pass form, the dispatch modes, and the raw WGSL escape hatch are in the GPU audio reference.

PreviousCompute pass for a canvasNextAuthor from the CLI