Introduction to the Web Audio API

For the meesterproef I had to learn how to work with the Web Audio API and some libraries

What is the Web Audio API

the Web Audio API allows you to use the browser to play sounds. You can add audio sources, add effects to the audio and visualize audio. According to MDN audio is handled inside a audio context. This allows the audio to be used in modular routing. Audio operations are performed with audio nodes, which are linked together to form an audio routing graph.

The cycle of the the Web Audio API goed something like the following. First an audio context is created. Inside the context the audio is loaded as sample or oscillator or stream. Then you can create effect nodes which changes the way the audio sounds. You can then choose a destination (we'll see this term appear once more in Tone.js) which most of the time are just your speakers or headphones of your computer. And finally you can connect the sources up to the effects which will then be connected to the destination.

All the interfaces and functions here

How does it work

I'm going over the vanilla implementation of the Web Audio API now.

You can create and audio context like this

const audioContext = new AudioContext();

The vanilla way to load in sounds or samples would be through an <audio> tag

<audio src="sample.wav"></audio>

Then you select in in JavaScript like you normaly would with other elements.

const audioElement = document.querySelector('audio');
const audioContext = new AudioContext();
let track = audioCtx.createMediaElementSource(audioElement);

const gainNode = audioContext.createGain();
const waveShaperNode = audioContext.reateWaveShaper()();

track.connect(gainNode).connect(WaveShaperNode).connect(audioContext.destination);

In the code snippet above you can see how the cycle works in code. You select the audio and create a new audio context. Then you apply the effects, in this case gain and waveShaper. These have their own properties and parameters that can be customized just like every other effect. And at the end you link the effects to the destination

When working with the web audio API during the meesterproef the same warning kept recurring.

error

This warning warns developers to only have audio trigger once a user interacts with the page. It seems quite logical to have it work like this as you don't want to be blasted by some song once you come on a page for the first time. The also applies to the libraries I'm about the discribe now since their an extention and build upon the vanilla Web Audio API.

Libraries

Both Blip. and Tone.js can be installed via npm or imported with a <script> tag in your html page.

$ npm install blip

$ npm install tone.js

<script src="https://cdn.rawgit.com/jshanley/blip/master/blip.min.js"></script>
<script type="module" src="https://unpkg.com/[email protected]/build/Tone.js"></script>

Blip.

Blip simplifies the way the default Web audio API handles audio. Blip has a sample, a clip and a loop functionality.

With the use of some code snippets from my project I will demonstrate how to use this incredibly simple library.

First you need to load in the samples. Blip does this asynchronously. You specify the name you want to give to the sample. You will use this name to create a clip which will be inside another variable. After the name comes the path to where the sample is located in your file tree.

blip.sampleLoader()
        .samples({
            'kick': `./sounds/kick.wav`,
            'snare': `./sounds/snare.wav`,
            'hihat': `./sounds/hihat.wav`,
        })
        .done(callback)
        .load();

Once the samples have loaded a callback function is triggerd. Inside this callback you are able to make clips from the samples and have them loop to create a beat or music. In my case I used it to make a simple beat.

var kick = blip.clip().sample('kick')
var snare = blip.clip().sample('snare')
var hihat = blip.clip().sample('hihat')

I will put the samples in to variables or clips in Blip terms so I can use it to make loops.

function loaded() {

   var kick = blip.clip().sample('kick')

   kickBeat = blip.loop()
            .tempo(150)
            .data([0, 1, 0, 0, 0, 1, 0, 0])
            .tick(function (t, d) {
                if (d) {
                    kick.play(t)
                }
               });

   kickBeat.start()
   kickBeat.stop()
}

Now comes the fun part. A Blip loop consists out of three parts. .tempo(), which represents the beats per minute the loop should be played in. .data() allows you to time when a sample is played. It has to be an array of numbers for it to work. The numbers can either be 0's or 1's or decimal numbers between 0 and 1. Then .tick() allows you to use which sample is used for the loop. If you provided a .data() you can chech for it if it's there and then play the sample with .play(). And to get the actual sound out of the browser you have to call .start(). And as you might think you make the loop stop with .stop().

One downside of this easy library is that it is so barebones that indivitual instruments such as a kick, snare and hihat need to have their own loop. You can't have them all three in one. But to make sure the intruments play in sync you can start them at the exact same moment and use the same bpm for each loop.

I used this sampler instead of Tone.js' one because the looping was a lot easier to figure out. It turns out Tone.js has a sampler as well and can be used to make beats but I didn't know that from the beginning plus I think Blip is a lot easier to understand for starters like me who have never used a Web Audio Library.

Tone.js

Tone.js is interesting because it is a really versatile librabry. It's basicly music producing software without an interface and instead has code. A lot of the terminology also is derived from music producing software. Terms like oscillator, attack, release, sine shaped tones, sawtooth shaped tones, square shaped tones.

The API documentation is quite extensive which is nice but it also doesn't make a whole lot of sense. So a lot of the code I will show now may not be the best as I had to figure it all out by myself with a lot of trial and error. In my project I mostly use it to make synths with and apply weird and cool effects to the melodies. Tone.js has all kinds of synths and effects with unique sounds to them.
But the main reason I chose Tone.js is because you don't need samples to make music.

I will explain the code I used in my project.

const melody = ["C3", ["E3", "G3"], "D3", ["C3", "A3"], "B2", "C2", ["E3", "A2"], "G2", "C4"];

let synth = new Tone.Synth({
        oscillator: {
            type: "fatsawtooth",
            volume: 0.2
        },
        envelope: {
            attack: 0.05,
            decay: 0.5,
            sustain: 0.5,
            release: 1
        }
    }).toDestination();

    sequenceOne = new Tone.Sequence(function(time, note) {
        synth.triggerAttackRelease(note, 0.5)
        console.log(note)
    }, melody, '4n')

    Tone.Transport.bpm.value = 150
    Tone.Transport.start()
    sequenceOne.start()

To begin making music I add a synth to play the notes. Inside it I set up the synth which determins how the synth will sound. In the oscilator part you can set a shape for you tone, this wil greatly affect how it's going to sound. In the envelope you can have minor changes to the sound. After the setup I send it to the destination as is the same with the vanilla Web Audio API.

Something that's really cool about Tone.js is you can have an array of notes be loaded in by a sequence. A sequence is basicly a loop and will play the melody continuously. The melody is the first line of code. The sequence will also be responsible for the attack and release of the notewhich means how long the fade in and fade out of a note will be. I also specify what araay to use for the melody and after that I set the time scale with 4n.

Then I start the audio by Tone.Transport.start() and I also have to start the sequence with sequenceOne.start(). To stop the audio I have it hooked up to a button which will call Tone.Transport.stop() to stop the melody from playing.

Then I also add an effect to the synth as follows:

const autoPanner = new Tone.AutoPanner("4n").toDestination().start();
const oscillator = new Tone.Oscillator().connect(autoPanner).start();

This method seems to have similarities to the way the vanilla Web Audio API applies effects to the music. You also need an oscilator to add an effect and the .connect() part seems the same aswell. All effects in Tone.js are also able to be customized further if necessary

Wrapping up

I enjoyed learning about sound in the browser. There were a few hickups as the documentation can be quite daunting in the beginning. It also doesn't help I don't know a enough about music production so the cycle of working wasn't always the most logical. But it's really cool to see what the browser can do with sound. And if used to it's full potential it can be a powerfull tool to make music with. When researching for the meesterproef and for this article I can across an article How to create music with Tone.js where Ričardas Faturovas explains how to code a quite complex song. It is truely a work of art, here is the end result.