Last time, I used a naive approach to implemented real time audio synthesis in the browser. It was good as a stepping stone to understand the general mechanics of a loop that repeatedly fills a buffer with audio samples, which I played back using regular AudioBufferSourceNode‘s. As already mentioned, this is not the proper way to do it. Why?

• It all runs on the main thread, so performance is weak
• The scheduling is a mess

AudioWorklet

The AudioWorklet node is a web audio feature that solves all of this: It gives as a dedicated audio thread to fill buffers as we like + some neat abstractions. As in the last post, here is a live codable text input, but this time it runs inside an AudioWorklet:

Of course, this also works with bytebeat:

Here’s a minimal example of how to create an AudioWorklet:

document.addEventListener("click", async function handleClick() {
  const name = "my-processor";
  const expression = `Math.sin(t / 20) / 4`;
  const workletCode = `class MyProcessor extends AudioWorkletProcessor {
  constructor() {
    super();
    this.t = 0;
  }
  process(inputs, outputs, parameters) {
    const output = outputs[0];
    for (let i = 0; i < output[0].length; i++) {
      const out = ((t) => ${expression})(this.t);
      output.forEach((channel) => {
        channel[i] = out;
      });
      this.t++;
    }
    return true;
  }
}
registerProcessor('${name}', MyProcessor);`;
  const ac = new AudioContext();
  await ac.resume();
  const dataURL = `data:text/javascript;base64,${btoa(workletCode)}`;
  await ac.audioWorklet.addModule(dataURL);
  const node = new AudioWorkletNode(ac, name);
  node.connect(ac.destination);
});

You can enter that snippet into your browser console and click into the document to hear a beatiful, infinite sine tone.

Let me go a bit into the details of why this works..

Faking URLs

Because an AudioWorklet runs in a separate thread, like a Worker, it needs to be loaded from a URL. I could’ve created a new file with the workletCode in it and passed its URL to ac.audioWorklet.addModule. But the fact that I want to dynamically insert an expression to evaluate on every sample, and also the fact that I want to have a self contained code snippet that can be copy pasted, I need to dynamically create a “fake” url:

document.addEventListener("click", () => {
  const base64String = btoa("console.log('hello world')");
  const dataURL = `data:text/javascript;base64,${base64String}`;
  console.log(dataURL);
});

This will create a Data URL from a string. Try copy pasting it into the address bar! It will behave just like a regular URL.

AudioWorkletProcessor

With that out of the way, let’s look at the actual AudioWorklet code:

// my-processor.js
class MyProcessor extends AudioWorkletProcessor {
  constructor() {
    super();
    this.t = 0;
  }
  process(inputs, outputs, parameters) {
    const output = outputs[0];
    const getSample = (t) => Math.sin(t / 20) / 4;
    for (let i = 0; i < output[0].length; i++) {
      output.forEach((channel) => {
        channel[i] = getSample(this.t);
      });
      this.t++;
    }
    return true;
  }
}
registerProcessor("my-processor", MyProcessor);

The process method is the interface to the computers audio processor, giving us access to input and output buffers. Inside, we just loop over all output buffer samples and fill them with a sine wave.

This is the code that we would need to run from a separate URL like this:

const ac = new AudioContext();
await ac.audioWorklet.addModule("./my-processor.js");
const node = new AudioWorkletNode(ac, name);
node.connect(ac.destination);

Stopping a Worklet

We still need a way to stop the worklet… We need 2 pieces of knowledge to do that:

1. As long as we return true from process, we signal the browser that our worklet is still active.
2. Both AudioWorkletNode and AudioWorkletProcessor have a port property, which can be used to send messages around.

// my-processor.js
class MyProcessor extends AudioWorkletProcessor {
  constructor() {
    super();
    this.stopped = false;
    this.port.onmessage = (e) => {
      if (e.data === "stop") {
        this.stopped = true;
      }
    };
  }

  process(inputs, outputs, parameters) {
    /*....*/
    return !this.stopped;
  }
}
// index.js or whatever
const ac = new AudioContext();
await ac.audioWorklet.addModule("./my-processor.js");
const node = new AudioWorkletNode(ac, name);
node.connect(ac.destination);
const stop = () => node.port.postMessage("stop");

I am not 100% sure if this is the most elegant way to stop a worklet.. If you know a better approach, let me know.

Conclusion

AudioWorklets are cool! This architecture should be a good foundation for writing custom DSP! It is still not the best you can get, as it still runs on JS, but it can get you quite far. You could crank out more performance by compiling a systems level language to WASM and run that inside the AudioWorkletProcessor, which is what I want to try out in another post.

Bonus: Minimal ByteBeat

Here’s a little snippet that will play bytebeat on click in just 585 bytes:

document.addEventListener("click",async function e(){let t="my-processor",o=`class MyProcessor extends AudioWorkletProcessor{constructor(){super(),this.t=0}process(s,t,r){let e=t[0];for(let _=0;_<e[0].length;_++){var o;let c=((((o=this.t)*(15&"36364689"[o>>13&7])/12&128)+(((o>>12^(o>>12)-2)%11*o/4|o>>13)&127)&255)/127.5-1)/4;e.forEach(s=>{s[_]=c}),this.t++}return!0}}registerProcessor("${t}",MyProcessor);`,s=new AudioContext;await s.resume();let r=`data:text/javascript;base64,${btoa(o)}`;await s.audioWorklet.addModule(r);let c=new AudioWorkletNode(s,t);c.connect(s.destination)});