Skip to content
FEE
Frontend Engineering Essentials

[FEE-420] Web Audio API — The Browser's Audio Graph ​

INFO

Playing a sound file is <audio>'s job. The Web Audio API is a modular synthesis and processing engine: sources, effects, analysers, and destinations are AudioNodes wired into an audio routing graph inside an AudioContext, and everything renders on a dedicated real-time audio thread with sample-accurate timing. The API has been available in every engine since April 2021 (Baseline Widely Available since late 2023) and is the substrate for browser games, DAWs (digital audio workstations), meeting products with level meters and noise gates, and data sonification. In production, three things bite: the autoplay policy that starts contexts suspended, the two-clock scheduling problem, and writing custom DSP (digital signal processing) in an AudioWorklet without blowing its ~2.7 ms budget.

Context ​

The Web Audio API grew out of the death of Flash and the inadequacy of <audio> for anything interactive: an element can play, pause, and seek, but it cannot mix twenty overlapping sound effects with sample accuracy, apply a low-pass filter that follows a game character behind a wall, or synthesize a tone from scratch. The spec (largely designed at Google, shipped in Chrome in 2011, a W3C Recommendation since June 2021, with a 1.1 revision in progress) chose a graph model borrowed from modular synthesizers and audio frameworks like CoreAudio rather than a callback model. Native nodes (GainNode, BiquadFilterNode, ConvolverNode, DynamicsCompressorNode, PannerNode, and friends) run as optimized C++ on the audio rendering thread. The deprecated main-thread ScriptProcessorNode escape hatch was replaced by AudioWorklet, which runs developer JavaScript (or WebAssembly) on that same real-time thread.

Visual ​

Example ​

Everything starts with a context, and because of the autoplay policy the context is only reliably usable after a user gesture:

js
const ctx = new AudioContext({ latencyHint: "interactive" });

playButton.addEventListener("click", async () => {
  // Without prior user interaction the context starts (or gets) suspended.
  if (ctx.state === "suspended") await ctx.resume();
  startEngine();
});

A synthesized note demonstrates the two ideas that make the API musical: nodes are cheap, disposable one-shots, and AudioParams are scheduled, not set. Jumping gain.value directly produces an audible click; ramps are how you avoid it:

js
function playNote(freq, at = ctx.currentTime) {
  const osc = new OscillatorNode(ctx, { type: "sawtooth", frequency: freq });
  const amp = new GainNode(ctx, { gain: 0 });

  // Attack/release envelope (ADSR-style: attack-decay-sustain-release),
  // scheduled sample-accurately on the audio clock.
  amp.gain.setValueAtTime(0, at);
  amp.gain.linearRampToValueAtTime(0.8, at + 0.02);           // attack
  amp.gain.setTargetAtTime(0.0001, at + 0.25, 0.08);          // release (exponential decay)

  osc.connect(amp).connect(ctx.destination);
  osc.start(at);
  osc.stop(at + 1);
  osc.onended = () => { osc.disconnect(); amp.disconnect(); }; // let GC reclaim
}

Playing decoded assets uses the same shape: fetch, decode once, then create a fresh AudioBufferSourceNode per playback (a source node can be started exactly once):

js
const buffer = await ctx.decodeAudioData(
  await (await fetch("/sfx/hit.wav")).arrayBuffer(),
);
function playHit() {
  const src = new AudioBufferSourceNode(ctx, { buffer });
  src.connect(ctx.destination);
  src.start();
}

Custom DSP lives in an AudioWorkletProcessor, a class that runs on the audio rendering thread and receives audio in render quanta (128 frames per call today; read the array length, don't hardcode it):

js
// noise-gate-processor.js -- loaded into the AudioWorkletGlobalScope
class NoiseGateProcessor extends AudioWorkletProcessor {
  static get parameterDescriptors() {
    return [{ name: "threshold", defaultValue: 0.01, automationRate: "k-rate" }];
  }
  process(inputs, outputs, parameters) {
    const input = inputs[0], output = outputs[0];
    const threshold = parameters.threshold[0];
    for (let ch = 0; ch < input.length; ch++) {
      const inCh = input[ch], outCh = output[ch];
      for (let i = 0; i < inCh.length; i++) {
        outCh[i] = Math.abs(inCh[i]) < threshold ? 0 : inCh[i];
      }
    }
    return true; // keep the processor alive
  }
}
registerProcessor("noise-gate", NoiseGateProcessor);
js
// Main thread: load the module once, then instantiate like any node.
await ctx.audioWorklet.addModule("noise-gate-processor.js");
const mic = ctx.createMediaStreamSource(
  await navigator.mediaDevices.getUserMedia({ audio: true }),
);
const gate = new AudioWorkletNode(ctx, "noise-gate");
mic.connect(gate).connect(ctx.destination);
gate.parameters.get("threshold").setValueAtTime(0.02, ctx.currentTime);

Run this last one with headphones: a live microphone routed straight to the speakers feeds back.

Best Practices ​

  • MUST create or resume() the AudioContext from a user gesture and handle the "suspended" state; the autoplay policy uses sticky activation. Where supported (Firefox only as of 2026), navigator.getAutoplayPolicy("audiocontext") reports "allowed" or "disallowed" up front.
  • MUST use AudioWorklet for custom DSP. ScriptProcessorNode is deprecated, runs on the main thread, and turns every jank into an audible glitch.
  • MUST keep process() allocation-free: no object creation, no closures, no postMessage per quantum. A garbage-collection pause on the audio thread is a dropout; at 48 kHz the budget for 128 frames is roughly 2.7 ms.
  • MUST schedule musical time on the audio clock (ctx.currentTime plus AudioParam automation and start(when)), never with setTimeout/Date.now() alone; the main-thread clock jitters by tens of milliseconds under load.
  • SHOULD use one AudioContext per page and suspend it when idle; contexts own OS audio resources, and some browsers (notably iOS Safari) limit how many can exist.
  • SHOULD ramp every user-audible parameter change (linearRampToValueAtTime, setTargetAtTime) instead of assigning param.value; instantaneous jumps produce clicks and zipper noise (the staircase distortion of many small value steps).
  • SHOULD render non-real-time work (mixdown, export, waveform pre-computation) with OfflineAudioContext, which processes the same graph faster than real time into an AudioBuffer.
  • SHOULD feed visualizations from AnalyserNode (getByteFrequencyData / getFloatTimeDomainData per animation frame) rather than tapping samples through a worklet.
  • MAY pass latencyHint: "playback" for non-interactive media to let the browser buffer more aggressively and save power, and MAY request a specific sampleRate at construction.
  • MAY bridge graphs into the rest of the platform via MediaStreamAudioDestinationNode; the resulting MediaStream plugs into RTCPeerConnection or MediaRecorder.

Design Thinking ​

Graph, not callbacks. The Web Audio API could have been a single "give me a buffer callback" hook; that is what the audio APIs of most operating systems expose, and what ScriptProcessorNode tried. The graph model costs a learning curve and some flexibility, but it buys the browser the right to run the entire native-node pipeline off the main thread, fuse and optimize it, and keep audio glitch-free while the page stutters. The design bet: most applications compose from stock nodes, and the rest get AudioWorklet on the engine's terms (fixed quantum, real-time constraints) rather than the main thread's.

Two clocks, one seam. ctx.currentTime advances on the audio hardware clock; performance.now() and setTimeout live on the main thread. The API's answer is Chris Wilson's lookahead pattern: a sloppy main-thread timer wakes up every ~25 ms and schedules, on the precise audio clock, everything that must happen in the next ~100 ms window. The main thread stays in control (tempo changes, user input) while the audio thread executes with sample accuracy, and a blocked main thread merely delays new scheduling instead of breaking what is already queued.

Parameters are signals. An AudioParam is a timeline rather than a plain number, and it can itself be driven by another node's output: connect an LFO (low-frequency oscillator) into gain.gain and you have tremolo. That single decision (parameters are audio-rate inputs) makes the API a synthesizer rather than a mixer, at the cost of the value property being the API's classic trap.

Deep Dive ​

The rendering loop. The audio thread pulls the graph once per render quantum: 128 frames in the 1.0 spec (the 1.1 draft makes the quantum configurable). At 48 kHz that is one process() call every 2.67 ms per worklet node. Inside a quantum, native nodes process in topological order; cycles are only legal through a DelayNode. AudioParam automation distinguishes a-rate parameters (evaluated per sample; the parameter array has one value per frame) from k-rate (one value per quantum). Worklet authors declare the rate in parameterDescriptors and must handle both array shapes, since an a-rate array with no scheduled changes arrives with length 1.

Worklet lifetime and messaging. process() returning true pins the processor alive; returning false lets the engine reclaim it once it is no longer sourcing or receiving audio. (Chrome does not yet honor the return value correctly, so MDN advises returning true unconditionally for cross-browser behavior.) Each AudioWorkletNode/AudioWorkletProcessor pair shares a MessagePort: fine for control messages, wrong for audio data. The established pattern for streaming samples between a Worker and the audio thread is a SharedArrayBuffer ring buffer with Atomics for coordination, which is also the bridge for running existing C/C++ DSP compiled to WebAssembly inside the worklet.

Decoding and memory. decodeAudioData() inflates compressed audio to 32-bit float PCM: a 3-minute stereo MP3 at 48 kHz becomes roughly 69 MB of AudioBuffer (48,000 samples × 2 channels × 4 bytes × 180 s). Decode once and share buffers across source nodes, prefer streaming via MediaElementAudioSourceNode for long-form material, and treat AudioBuffers as the dominant memory cost of any sample-based app.

State machine. A context is "suspended", "running", or "closed"; an "interrupted" state covering OS-level preemption such as phone calls exists in the editor's draft but not yet in the published spec. statechange fires on every transition. suspend() releases the audio hardware without tearing down the graph: the right idle behavior for apps that only sometimes make noise.

Scheduling Reference ​

The automation methods on AudioParam are the vocabulary of the two-clock pattern; all take absolute times in seconds on the context's clock.

MethodBehaviorTypical use
setValueAtTime(v, t)Instant step at tAnchoring a ramp's start point
linearRampToValueAtTime(v, t)Linear glide from previous event to tAttack envelopes, crossfades
exponentialRampToValueAtTime(v, t)Exponential glide (values must be non-zero, same sign)Perceptually smooth volume/pitch
setTargetAtTime(v, t, timeConstant)Asymptotic approach starting at tReleases, smoothing a live control
setValueCurveAtTime(curve, t, dur)Follow an arbitrary Float32Array curveCustom fades, ducking shapes
cancelScheduledValues(t)Drop automation at/after tRescheduling on tempo change
cancelAndHoldAtTime(t)Cancel but freeze at the in-flight value (no Firefox support)Interrupting a ramp without a jump

The lookahead scheduler that drives them:

js
const LOOKAHEAD_MS = 25;     // how often the main thread wakes
const HORIZON_S = 0.1;       // how far ahead it schedules on the audio clock
const tempo = 120;
let nextNoteTime = 0;

function startScheduler() {
  nextNoteTime = ctx.currentTime; // never schedule from t=0 on a running clock
  scheduler();
}

function scheduler() {
  while (nextNoteTime < ctx.currentTime + HORIZON_S) {
    playNote(440, nextNoteTime);          // scheduled sample-accurately
    nextNoteTime += 60 / tempo / 4;       // sixteenth notes
  }
  setTimeout(scheduler, LOOKAHEAD_MS);    // jitter here is harmless
}

Shrinking HORIZON_S toward zero increases responsiveness to tempo and input changes but risks underruns when the main thread stalls; growing it makes playback bulletproof but laggy to control. 25 ms/100 ms is the classic starting point, and this is the same pattern Tone.js packages as its Transport if you would rather not maintain the scheduler yourself.

  • [WebRTC — Peer-to-Peer Media and Data Channels](/en/Browser APIs and Web Platform/webrtc)
  • [Web Workers & Concurrency](/en/Browser APIs and Web Platform/405)
  • [WebGL & WebGPU](/en/Browser APIs and Web Platform/408)
  • [requestAnimationFrame & Animation Timing](/en/Browser APIs and Web Platform/412)
  • [Web Speech API](/en/Browser APIs and Web Platform/416)

References ​