It is now possible to inject new samples to use to create new instruments in MMR. (1.13 introduced this feature)

Right now this only works with ripped OOT instruments, because we rely on the audiobank already being made for the instruments, custom samples has not been fully explored yet, and I'm not convinced ripping samples from games like sm64/mk64 is working perfectly yet.

75-95% of the effort involved with sample injection is making the samples MM/MMR compatible, once that is done, the process isn't that much harder than making a custom bank to begin with.

Already ripped from OOT instruments ready to inject without ripping: https://mega.nz/folder/XWgWTa4S#YjNEtP_Uoe4OuAr1IskdPg

I still advise you attempt to read the whole guide as it will explain what to do with these samples and xml files.

Summary of using previously ripped instruments:

Once these instruments are ripped and converted to MM/MMR they should exist as a pair of files:

• The Sound sample files that will be injected into the game (binary .zsound)
• The Instrument data necessary to turn the sound sample into an instrument (instrument set .xml)

The Sound sample file is a aifc that has been stripped of its header, we call it zsound because I didn't know what else to call it when I first got it, N64Tools/Soundtool calls it "Raw" but that's what seq64 calls a zseq file, Raw is too ambiguous. To use in a MMRS we place this file as-is in our MMRS file next to all our other files.

The instrument data can exist in three formats:

1. The instrument can exist as a seq64 bank xml file, that contains all the instrument data as part of an instrument in the bank, to use this data we have to load the xml into seq64 and add to a custom instrument set. We also have to make sure the instrument address matches the data in the zsound filename, scroll down to learn more about this (although if sharing your instruments with others, this should already be done).

2. The instrument can already exist in a premade instrument set xml file, which contains not just the one instrument but an entire bank, you can modify this bank as needed to create the bank you want to use.

3. The instrument can already exist as a pre-compiled zbank file, which to use we place in the MMRS file like our custom banks, and tweak the zseq to use the instruments used in that bank. You'll have to use songtest to test this song, as seq64 does NOT support importing raw bank files. Don't forget to include the bank meta.

Summary of creating new instruments:

To create a new instrument you need sound files to turn into samples, and you need instrument meta data used to turn the samples into instruments.

Any sample needs to be in a format that the n64 hardware can read, N64-Tools Soundtool can convert them from wav, but in OOT's case we can just rip them from the game and re-inject as-is without modification. Creating new samples from scratch is kinda tricky right now, so this guide assumes you are ripping instruments from other N64 games only.

Each sample has predictors which help the hardware render the sound, these have to be created or ripped (in the instrument set data).

We need to modify (or rip if already exists in a game) an audiobank metadata for the instrument that will use this sample. Sometimes this is as easy as ripping straight from the game (OOT) but often modifications are required. We need to inject our predictors into this bank for the instrument to use, for instance.

Then the instruments have to added to the final bank for any music you want, and that bank ripped to make a mmrs.

Sample ripping from N64

For OOT and other N64 games, to get our sample we rip sound file as raw from OOT with N64-Tool/Sound Tool. In Soundtool, open the game file, find the bank that has your instrument, find the instrument in that bank, you should be able to hit the play button to play the sample to know you're on the right one.

Hit Export-> 16-Bit Raw button in the bottom right to extract the sample used by the instrument.

images/soundtool_sample_extraction.jpg

However some Instruments have multiple samples, be sure to extract every sample for the instrument you want.

images/soundtool_instrument_select.jpg

This version of the sample is already in the same format we need to use it in MM, or at least that's true for OOT/SM64 so far, although I suspect its true for most N64 games.

Building a test rom that has our sample

We'll want to inject our samples and banks into a testing rom so we can work with it to get finishing touches from our songs that need to use the instrument. Do this BEFORE you open the rom in seq64 or it will get overwritten when you save your rom again.

To do this, I open both the rom and the sample in HxD, select the whole raw hex contents of the sample with ctrl-a and copy with ctrl-c, then find a nice spot in my testing rom that is blank for long enough to put the sample, I use 0x03000000. Select the first byte there, and use ctrl-v to inject the sample into the rom, save.

Keep in mind some instruments need multiple samples, like MM strings uses 3 different samples, and OOT Organ uses 2. These samples are isolated files, they need to find similar locations in your rom to the first.

Getting the rest of the N64 instrument data

We need more than just the sample, each instrument has a envelope that describes how the sample is manipulated by the soundchip which we want if we want our instrument to sound exactly the same as OOT, and these raw samples have ADPCM book/loops that help the chip read them, predictors, which if we don't match will make the sample sound really bad. For OOT this is already provided for us since both MM and OOT use the same audiobank instrument set format, we just need to rip it.

We want the audiobank data from those games saved as xml data. Open the original rom of the game you extracted from, open a/multiple audiobank instrument sets that use that instrument you want, and save them as xml for later. You can save them as-is, with all instruments as a backup, but eventually you'll want to remove all other instruments in that bank, all other samples/envelopes/adpcmbook/adpcmloops because of an annoying bug in seq64 having a bank with just that one instrument will be valuable.

The bug is: if you take any bank modified, save over a current MM bank, and then try to load an instrument from this newly saved bank into another bank, seq64 will sometimes read the wrong information from the bank, EG if you have three instruments (flute/accordion/bongo) and save it over a MM bank, then try to load one of them (accordion) seq64 might load the data from bongo instead, your accordion becomes a bongo. Having only one in the bank fixes this bug. Removing all of the other instruments can be done last second, but doing it once, then saving as a xml, saves you tedious re-work in the future.

Pointing your instrument sample at the right spot in your testing rom

Change the instrument address to match your sample in the test rom. This address is an offset from the soundbank (where the file was supposed to be, but the address allows for a long distance). To modify the address, go to your bank tab and in the top right is your bank file system, there is a folder called "samples" open it and you should see the sample(s) your new instrument uses, open it as a file and you should see a field called "sampleaddr" that is your address.

images/seq64_change_instrument_pointer.jpg

To calculate the offset of your address, take the rom location of your file and subtract the rom location of the soundbank. If you put your file in 0x03000000, then 0x03000000 - 0x97F70 = 0x2F68090, you want to write 2F68090 to your sampleaddr value in your bank's sample data.

Save this instrument/bank for the future when you need to use that instrument.

Switch back to MM, reload the instrument set if you want to use it. The bank loaded into memory from the other game will have a bug and if you try to save it to a different game rom. Instead, save the bank to xml, switch games, then load the saved bank. IE: From OOT, save bank, switch to MM, re-load bank from file.

Now that you have your new instrument set loaded you can inject it into your set bank slot and load a test sequence to make sure it's working. (I have several sequences that only need one instrument, load one, point it at the instrument, inject like you would any custom instrument set). If you hear noise/static then the game is trying to load the instrument from the wrong spot and is reading garbage.

To use your instrument in a custom bank that needs a percussion kit, Save your single instrument bank to the test slot for temporary storage, then build the bank you want as normal until you need the instrument, then you can load that one instrument from the storage slot just like you would a MM instrument from a vanilla slot. Don't try this if there is more than one instrument, it sometimes breaks, I'm not sure why, single instrument is the safest way. If you need more than one of these special instruments, save to multiple storage banks or save the bank you are building to a XML and load a new instrument bank and store before restoring your building bank from XML, which is tedious but should work.

Change the address to a Unique Marker

Now before you export the bank with the bankripper script, you need to change the address again. Currently your address should be pointed at the sample in your test rom, but in MMR that sample could be anywhere on the rom (it moves based on what needs to be in the rom). Because there can be multiple custom samples, we need a way to tell MMR which sample in the song is which, and for that we use markers. Markers are unique numbers that do not already appear in the bank, so MMR can identify which bytes to change to match the sample filename.

Rename your sample to Filename_MarkerIn32bitHex.zsound where the filename is optional but helps us remember what the sample is called (imagine a song that needs not one but 10 or 20 samples) and the marker is a hex representation of a 32 bit unsigned number. Example: WoodChimes_51525300.zsound the filename tells us this is the OOT Water Temple wood chimes sample, 51525300 is a unique number that does not show up in the bank already, so when we search for it we'll find the only spot that is used in the bank for the sample address, and MMR can then change the marker to match the exact location in the rom that the woodchimes sample was placed after MMR shifts the rom around.

Change the sample address pointer in your audiobank to match your marker, so 0x51525300 in this example but any unique number can be used as a marker. Remember to set the seq64 field into hex mode to make that easier. Save the bank again and to the rom, and rip.

Note: This is actually sorta optional, if you placed your samples at a location in your test rom where the address pointed to a unique value, you accidentally created a unique marker, and you can leave the address to your test machine and match your sample without making up a new marker, just realize it might not always be unique.

Merge files together into MMRS

Rip the bank, bankmeta, and seq from the rom, and put them in the same zip file with the categories and your sample(s) and rename to a MMRS file, where the only difference here from a regular custom instrument set is that this one has sound sample file(s), and your bank has marker-pointed instruments.

Our MMRS doesn't look much different, only the new samples are visibly added. The bank is modified to have a new instrument pointed at empty space (our marker) and would make it play garbage if not for the feature, but otherwise the MMRS file system is the same:

MY-MUSIC.zip

• categories.txt
• 18.zseq
• 18.zbank
• 18.bankmeta
• OOT_WoodChimes_51525300.zsound

Other ZSEQ compatible games

The tools offered in n64 sdk converted aiff files to aifc files, suggesting almost all sound files for all games on the n64 use these files. Other games by nintendo that used a similar system to Zelda64 can have their instrument metadata ripped so we can find the predictor data for the samples, heck, it's already in an audiobank like format for us to use in seq64, other games I think the challenge is finding that metadata, without the right predictors aifc sounds terrible.

New Custom samples from scratch

edit: NVM, Soundtool lies, and doesn't actually create real predictors, just uses a vague aproximation, and that's why some sounds sound like garbage in soundtool. Great.

edit: I did find that someone de-compiled the irix programs that were originally used to make the samples, vadpcm_enc and tabledesign, but the files made from this process seem to be different from the files we rip from the game, however we might be able to feed more accurate predictors to soundtool

Creating brand new samples for MMR is an experimental process, I only just got it to work when writing this.

The process starts with a wav file of the sample you want to convert. If you want to make an instrument that has multiple samples you will need multiple wavs each, and do the steps for each sample.

Tool introduction

We will NOT be using N64-Tools/Soundtool, because last I checked (okay it has been years now...) soundtool always uses static predictors which can make the file sound terrible. We want to generate new predictors for our soundfile for it to sound as good on n64 hardware (emulated or not) as possible.

There are two software tools which can make the zsound+predictor pair we need: [z64audio] and the original re-compiled [vadpcm+tabledesign] pair of programs that came with the n64 sdk.

[z64audio] is a new program that does the same thing as the old sdk and hopes to one day offer a clean GUI to ease the process, although it is still a work in progress. We can still use as is, though. To use this program, I made a bat script that allows you use it without needing to know anything more about the program in its WIP state, you only need to download the file and include it in the folder with my script.

[vadpcm+tabledesign], the old SDK programs, once re-compiled to run on modern machines (they were originally compiled for a mips irix machine used to develop n64 games, and would not work on x86 hardware. The explosion in reverse-engineering tools has allowed people to re-compile them to work on modern machines now, so you can use them, although they are still CLI based only, and I don't think anyone is improving them after the fact, so they are abandonware. For now I leave this program instructions unfilled, because I can't remember the exact steps, I think you ffmpeg to turn your wav into a aifc then pass it into table design and then vadpcm but its been years forgot the exact steps.

Using z64audio

We use z64audio to make two files: a prediction table and a cut-down aifc file we call zsound, you need both or the sample doesn't play properly.

First download the latest release of z64audio:

z64audio download (you want the exe): https://github.com/z64tools/z64audio/releases/

To use it in it's current state (Aug 2022) you need to use the CLI, assuming you don't know how to, I have made a basic script, you put the script in the same folder as your z64audio.exe, and drag-and-drop your .wav file you want to convert on top of the DropYourWavHere.bat script.

bat download (found in the "SampleCreationTools" folder): https://mega.nz/folder/XWgWTa4S#YjNEtP_Uoe4OuAr1IskdPg

If you wanted to use the CLI itself, you use the following command to convert your wav to predictor+zsound:

z64audio.exe --i <path to your wav file.wav> --o <new zsoundname.bin>

Both the command and the bat will do the same thing: create two new files, a *.vadpcm.bin which is your new zsound and a *.book.bin which is your binary predictors.

Predictor converting

These predictors could be injected into the game as-is... except for MMR we need to make a bank with them in seq64 and seq64 doesn't know what to do with these, so we need to convert them to a format Seq64 will understand which is the instrument set xml (or bank xml).

I created a script that turns the binary file into a text file that contains the xml boilerplate that allows us to copy-paste on top of a new bank that will contain our new instrument.

download the bin-to-xml converter (in the "SampleCreationTools folder)": https://mega.nz/folder/XWgWTa4S#YjNEtP_Uoe4OuAr1IskdPg

The bin-to-xml converter requires python to be installed, then you can drag-and-drop your *.book.bin on top of the converter and it will create a new text file called *book.bin.txt which will contain 4 sections of 16 lines of xml, you copy paste these onto your new bank.

Bank manipulation (needs work)

I don't actually have any good advise on where to start here, I think it depends on your sample. We are going to jerryrig a new bank instrument from an old one rather than create one by hand since that has never worked for me.

If you want to create a new sound effect like instrument, I recommend starting with a new empty bank, then ripping a single instrument from bank 00 or bank 01, almost all of those instruments work as sound effects so that should work as an acceptable starting point.

If you wanted a looping instrument to behave like violin strings... I haven't done this yet, I have no idea.

// TODO figure out looping

To merge your predictors from the .txt file into your new instrument bank, save the bank as a xml somewhere on your computer, open the file in a text editor and copy paste the new data on top of the old predictor data (for the old instrument, invalid for our case). There should be a section in each bank for aladpcmbooks that will contain the predictors for any instrument samples (in this case, it should be one-3 depending on your bank 00/01 instrument you ripped)

images/predictor-txt-copy-paste.png

Then save, re-open your your bank XML in Seq64. I recommend renaming all of the data in the bank and re-saving (the instrument name, the sample name, the adpcm book/loop name) just so later when you make a bank you know where this data came from if you have to tweak that bank for any reason.

We still have a few things to change in the bank: navigate to your bank's /samples folder and open the sample metadata. You need to set the samplesize data for your sample to the exact length of your new sample zsound (probably still named vadpcm.bin) and while you are here you should create a test rom and inject your sample into the test rom like above and modify the sampleaddr so this bank can be used with that rom.

Assume you are working with sfx-like samples, that don't loop, you now need to navigate to this bank's /adpcmloops folder and find your samples loop data and set the end data to be somewhere after your sample's total length. Even if the sample does not loop this data is still used, and if the end is too short it will cut off your data.

You should also modify the sample pitch in the /instruments/<yourinstrument>/splits/AbSound/tuning to match what you think your sample tuning is (TODO: learn the math on this)

Soundfont extraction

Now that you have a working bank and sample you should extract a soundfont so you can work on it in your MIDI player.

// TODO figure this out

Finish

At this point you have the finished components to inject samples, rename your vadpcm.bin to .zsound like above, and your xml bank has all the details you need to use it in a new bank.

Go to this and continue from there: https://github.com/isghj5/mm-rando/wiki/Instrument-Sample-Injection#pointing-your-instrument-sample-at-the-right-spot-in-your-testing-rom