Let’s talk about how you can make music with a bunch of numbers! So far, I’ve mostly used the builtin Web Audio API nodes, which abstract sound synthesis into an object graph. Today I want to go a little deeper and use only the lowest level digital audio synthesis: numbers in a buffer.

A little teaser of what’s to come:

This little player uses a single mathematical expression to render a little melody.

Digital Audio in a nutshell

Sound travels to your ear through air pressure waves. A loudspeaker is just a very precise pressure wave generator. If you feed a voltage to a loudspeaker, it will move to a certain position. When that voltage changes quickly enough, you can make a sound. In an analog music system, those voltage changes are created directly by something like an oscillator circuit.

In the digital world, the voltage levels for the speaker are generated by the Digital-to-Analog Converter (DAC). The DAC expects a stream of numbers, which it turns into appropriate voltage levels. Those numbers are typically floating point values between -1 (speaker in), 0 (speaker rests) and 1 (speaker out).

The DAC is eating those numbers at a constant rate, which is called the sample rate. It is typically around 44100Hz (or 48000Hz), which means a list of 44100 numbers is consumed per second!

Each of those numbers is also called a sample, not to be confused with a sample meaning recorded clip of audio.

Playing Buffers on the web

It is fairly easy to create and play a buffer on the web, thanks to the Web Audio API:

// we need to wrap everything in a click because of web audio autoplay policy
document.addEventListener("click", () => {
  const ac = new AudioContext();
  // now let's generate a list of numbers
  const seconds = 1;
  let samples = new Float32Array(ac.sampleRate * seconds);

  for (let i = 0; i < samples.length; i++) {
    samples[i] = Math.sin(i / 20) / 5;
  }

  const buffer = ac.createBuffer(1, samples.length, ac.sampleRate);
  const source = ac.createBufferSource();

  buffer.getChannelData(0).set(samples);
  source.buffer = buffer;

  source.connect(ac.destination);
  source.start();
});

You can copy the above code and paste it into the browser console, then click somewhere on the page to hear a beatiful sine tone! You should probably reload the page now, as it might be annoying to always hear that tone on click…

The above snippet is mostly boiler plate, the most interesting part is what happens inside the for loop.

To simplify things, here is a text field where you can enter an expression that is evaluated for every sample t:

You can also play with ctrl+enter!

Pitches

To get an accurate frequency, we have to calculate it like this:

To spare some characters, let’s define s = t/sampleRate and c = s * 2 * pi:

We can play a major triad like this:

Waveforms

Sawtooth wave:

Triangle Wave:

Square Wave:

Variable Curve:

We can abstract the shapes into functions like this:

const sine = (f) => Math.sin(f * c);
const saw = (f) => 1 - 2 * ((f * s) % 1);
const square = (f) => Math.sign(1 - 2 * ((f * s) % 1));
const tri = (f) => 1 - 4 * Math.abs(Math.round(s * f) - s * f);

Envelopes

We can add a linear envelope by multiplying with a linear function:

Attack:

With some conditional logic, we can create a whole ADSR envelope:

Let’s abstract this into a function:

function adsr(a = 0.001, d = 0.001, sl = 1, st = 0.1, r = 0.001) {
  let ramp = (a, b, n) => n * (b - a) + a;
  let s = t / sr;
  let gain = 0;
  if (s < a) {
    gain = s / a;
  } else if (s < a + d) {
    gain = ramp(1, sl, (s - a) / d);
  } else if (s < a + d + st) {
    gain = sl;
  } else if (s < a + d + st + r) {
    gain = ramp(sl, 0, (s - a - d - st) / r);
  }
  return gain;
}

… and use it:

We can abstract it a little more into a variable length line segmentation function:

// api idea from csound
function linsegs(...args) {
  let [v, ...rest] = args;
  let a = 0;
  let lerp = (a, b, n) => n * (b - a) + a;
  while (a < rest.length) {
    const dur = rest[a];
    const next = rest[a + 1];
    if (s < dur) {
      return lerp(v, next, s / dur);
    }
    s -= dur;
    v = next;
    a += 2;
  }
  return args[args.length - 1];
}

A simple AD envelope could be written like this:

Vibrato & FM

FM is like vibrato but faster!

more bell-like:

Tremolo

Writing a tune

We could define multiple notes like this:

Let’s write a simple helper for that:

function note(freq, time, duration) {
  return (
    oscillator("triangle", freq) *
    linseg(0, time, 0, 0.01, 1, duration, 1, 0.05, 0)
  );
}

At this point, we could already synthesize something like a midi track with just a bunch of mathematical functions!

I want to make a cut here, but I’ll probably dive deeper into this topic. So far, we’ve only been pregenerating all the things. It would be very handy if all of this could run in real time, using AudioWorklets. I am also curious if I can reimplement ZZFX as an AudioWorklet. Until next time!