Storing audio locally - abudaan/heartbeat GitHub Wiki

#Storing parsed audio locally in an indexedDB

Loading and parsing the samples of multiple instruments can take quite a while. Tim Venison suggested to store samples as parsed audio data locally in an indexedDB. Ideally, we then only have to retrieve the parsed audio data as AudioBuffer instance from the local database and add it as buffer to an AudioBufferSourceNode instance.

Unfortunately that isn't possible because we can't store an AudioBuffer instance in the indexedDB; we have to convert it to something that we can store in the database. I have tested two options:

store only the channel data of the AudioBuffer instance as Float32Array in the database
convert the AudioBuffer instance to a wav file and store it as Blob in the database

Both options work in all browsers that heartbeat supports, except Chrome for iOS because indexedDB has not yet been implemented in Chrome for iOS.

###Option 1

First we have to get the channel data from the AudioBuffer by using getChannelData(). This method returns a Float32Array instance per channel containing the samples in float values between -1.0 and 1.0. We can store theses arrays directly in the database.

If we want to playback a sample that is stored this way in the database, we query the channel data from the database and add it as buffer to an AudioBufferSourceNode instance.

In Firefox we can use AudioBuffer.copyToChannel() which is very fast. Other browsers don't have this method implemented so we have to loop over the channel data and add the samples one by one:

var buffer = context.createBuffer(2, numFrames, context.sampleRate);
var samplesLeftChannel; // a Float32Array that we retrieved from the database
var samplesRightChannel; // a Float32Array that we retrieved from the database

if(buffer.copyToChannel !== undefined){
    // Firefox -> about 50% faster than decoding
    buffer.copyToChannel(samplesLeftChannel, 0, 0);
    buffer.copyToChannel(samplesRightChannel, 1, 0);
}else{
    // Other browsers -> about 20 - 70% slower than decoding
    var leftChannel = buffer.getChannelData(0);
    var rightChannel = buffer.getChannelData(1);
    var i;
    for(i = 0; i < numFrames; i++){
        leftChannel[i] = samplesLeftChannel[i];
        rightChannel[i] = samplesRightChannel[i];
    }
}

It isn't possible to add the channel data in a web worker because the Float32Array instances that hold the channel data get copied to a new instance in the web worker. Also when using transferable objects you loose reference to the original channel data arrays.

You can test this example and read the code here.

###Option 2

For this option we have to convert the AudioBuffer to a wav file. We have to loop over every sample to remap the sample value from a -1.0 to 1.0 range to a -32,768 and 32,767 range, and we have to interleave the samples.

Fortunately we can do this in a web worker. Because it isn't possible to access the database from a web worker, the web worker returns the generated binary wav file as an ArrayBuffer to the main thread where it is stored in the database as a Blob.

Some browsers don't yet support Blobs to be stored in the database. For these browsers we use FileReader to store the Blob as DataURL, which makes storing the Blob a bit slower.

To playback the stored wav, we have to convert the Blob back to an ArrayBuffer using the FileReader. If the wav file was stored as DataURL we have to convert the DataURL back to a Blob before we can pass it to the FileReader.

In browsers that support storing Blobs in the indexedDB (Chrome, Firefox) this method is faster that decoding, and even a bit faster than option 1.

For browsers that don't support storing Blobs (Safari iOS, Chromium) this method is a slower than decoding and slower than option 1.

You can test this example and read the code here.

###Conclusion

Both options work but dependent on your target audience you can choose one or the other.

On my iPad 4th generation option 1 is sometimes faster than decoding and sometimes about 30% slower. Option 2 is 200 - 250% slower than decoding. So if your target audience is iOS users, you should choose option 1. Option 1 is also less and simpler code, and uses slightly less database storage.

Option 2 on the other hand is the fastest on Firefox and Chrome, both on desktop and Android. So if your target audience is desktop user and/or Android users, option 2 is the better choice.

Note that probably all browsers will eventually support storing Blobs in an indexedDB and AudioBuffer.copyToChannel, so both options will become faster in browsers that currently don't support these functions.