FAQ - atsushieno/aap-core GitHub Wiki

AAP (Audio Plugins For Android) FAQ

This is an AAP documentation, explained in FAQ style.


What is AAP, briefly?

AAP is to realize an audio plugin format as well as its audio plugins ecosystem on Android platform.

What are audio plugins, and what is an audio plugin format?

People use DAWs (Digital Audio Workstation) to compose music, or record live audio performances, and so on. Audio plugins are plugins for DAWs to work as either instruments or effects (or sometimes otherwise). As long as an audio plugin software conforms to an audio plugin format, which is usually some sort of API, it can be used by any DAW that supports the format.

There are private audio plugins that are only for specific DAW(s), and publicly shared audio plugins that can be used by any DAWs that support it. AAP is latter.

What audio plugin formats are there in the world and what are available on Android, other than AAP?

On desktop PC world, there are some popular audio plugin formats:

  • VST(2 or 3) by Steinberg
  • AudioUnit (AU, v2 or v3) by Apple
  • Some community driven plugin formats like LV2 (popular on Linux), WebAudioModules (Web), and CLAP

AAP is the only audio plugin format on Android (I'd say). The reason why Android does not have any audio plugin format is that those traditional ones (VST2/3, AUv2, LV2, CLAP, ...) are designed to work only on desktop OSes (we mention more about it at the "Rationale" section). Apple's AUv3 is the only exception; it is designed to work on iOS. AAP stands at similar position to AUv3.

(There used to be a similar project to AAP called AudioRoute, and it has a supported DAW called n-Track Studio. But the development activity on GitHub has been stopped since 2021.)

What does AAP aim as a project?

There is no audio plugin format that can be used on Android. That means, all instrument plugins and effect plugins on Android are specific to certain DAWs. We want to change that, and have Android as a viable platform for audio and music creatives.

AAP does not aim to just be a plugin format in-theory, but with the actual plugins ecosystem to use, to avoid typical "chicken and egg" problem on a plugin format. If your DAW or other audio app supports AAP, there will be already 20+ plugin apps with >100 plugins that are ported from cross-platform audio plugins written in JUCE or LV2.

What can we do with AAP right now?

AAP is still under heavy development, and also lacks viable products as an audio plugin format. However, things are partially working.

Most notable one would be AudioPluginMidiDeviceService. You can turn any AAP instrument plugin into Android MidiDeviceService using AAP. It would be impressive as you can use instruments like Sfizz, Fluidsynth, OB-Xd, Dexed, ADLplug, Odin2, Vitaloid (my Android port of Vital OSS) etc. as MIDI instruments. Install those AAP ports, and launch your favorite MIDI keyboard app or DAW that supports MIDI output. You can find new MIDI output devices for those instrument plugins. (Do not try JUCE based Android apps, like ZenBeats. JUCE Android MIDI support is completely broken.)


Who are using AAP right now?

There is no one I believe. Note that AAP is not practically usable yet.

Where are AAP community people around?

Expecting community while there is no user...? Well, there could be developer community. Though again, no there isn't yet. @atsushieno is the only developer so far. The GitHub repositories are the official places that handles bug reports (issues), development (pull requests), and discussions. We always welcome contibution, helps and joining development.

However there are handful of people who told @atsushieno that they are interested in AAP. @atsushieno is around some audio dev. community and would answer to questions there too. See the user profile to find those.

Why are you working on it? Are you making money?

One of my dream is to have generative music platform, where audio plugins work on any environment and platform. Audio plugins are essential to music authoring. Missing audio plugin format on Android is definitely no-go.

I should be making money and make AAP ecosystem sustainable, but so far everything is under development and the ecosystem is not at very usable state.

AAP Availability

are there available audio plugins that support AAP?

There are many !

Note that ports do not mean they are usable on all features the original projects (LV2 or JUCE) offer (most notably, there is no GUI support).

Are there available DAWs that support AAP?

It is not practically usable, but you can experience my AAP integration to Helio Workstation, a JUCE-based DAW: aap-juce-helio

Tracktion Engine is also known to build on Android, so there are more potential.

Are AAP audio plugins and DAWs available on Google Play?

Not yet. I don't plan to publish useless apps in general and collect a lot of 1-star reviews.

If they are not available on Google Play, how can I try out AAP?

We have a dedicated installer application for AAP: AAP APK Installer

It downloads those plugin APKs from either a GitHub release or a build artifact. It is developed as an OSS, and the build artifacts are available on GitHub Actions. Everything is automated and safe as much as you can read the source code. See the GitHub repo for details.

Can't those audio plugins distributed in a single application?

Technically we can, but that would not meet the ideal state of the plugin ecosystem: everyone can publish their plugin or DAW by themselves, not dominated by us. It is just like AUv3 plugins on iOS App Store (or Mac App Store).


Is AAP an application?

The entire AAP ecosystem is not a single application, but a bunch of applications.

One or more AAP plugins can be packaged and distributed as an application (apk/aab). For example, AAP MDA-LV2 is distributed as an apk and contains 35-ish plugins.

Isn't AAP just a specification i.e. an API definition? Why does AAP have to be offered as the implementation?

Before giving the actual answer, we would like to warn you - you should probably skip learning about the API in AAP, if you are just a plugin developer, or a DAW developer even. We don't intend to let you learn about it yet.

AAP is certainly a specification on one hand, as it defines a set of API in C header files. At the same time, the plugins has to exist as an AudioPluginService which is an Android Service.

We offer androidaudioplugin.aar that contains the AudioPluginService class as well as the hosting implementation, along with C/C++ headers and implementation.

The plugin hosts (DAWs) have to function as Service clients. In theory they could be implemented totally independent of androidaudioplugin.aar, but both are bundled in this single aar so far.

There could be better isolation and we continuously explore that.

Is AAP a JUCE application?

Sometimes people guess it is (as we mention JUCE a lot). No. we support JUCE apps (by providing custom audio plugin format implementation for juce_audio_plugin_client as well as plugin wrapper implementation for juce_audio_processors), but AAP itself is not a JUCE application. AAP must be available under the MIT license so it will not be a JUCE application (unless their licensing model changes).

Having said that, AAP had started as an experimental JUCE project and it owes a lot of API design from it.

Our JUCE support is split into separate GitHub repository called aap-juce.

Doesn't JUCE itself support Android?

JUCE supports Android in general. And it would support any audio plugin format that exists. There is no such a format on Android yet, thus there is no plugin support. Note that JUCE itself is not an audio plugin format.

A JUCE audio plugin application could be built as a Standalone module, and we make use of it on Android.

Is AAP an LV2 application?

AAP supports LV2 in that LV2 plugins could be imported to AAP ecosystem. AAP is NOT an LV2 application in that we do not use LV2 as the underlying plugin format. aap-lv2 is certainly an LV2 application in that we use lilv (a modified version by us) for wrapping LV2 plugins built for Android.

When we want to "port" some existing audio plugin to another plugin format, we use "wrapper" technique (lv2vst, yabridge, Ildaeil, clap-wrapper); lots of the plugin APIs overlap their features, so they could be translated to work as if the wrapped plugin were implemented in the wrapping plugin API. aap-lv2 offers such a wrapper foundation. It is not a perfect solution, but it covers basic features.

Why AAP-LV2 does not support hosting?

We just haven't investigated it. That means, we are not sure how much we could offer for LV2 hosting that wraps AAP, and gain any benefit. We are not aware of any DAW that supports LV2 and runs on Android (unlike JUCE apps like Helio Workstation). It does not seem easy to port plugin hosts like Carla to Android; @atsushieno had a quick look and quickly gave up.

Is AAP an extensible plugin format?

AAP does support extensions, and it should offer extension features to preserve API compatibility. Though we keep breaking the core API and it would take a while until we finalize the API.

Also, due to the nature of host-plugin relationship in isolated processes, our extension API actually involves some implementation code. See Technical Details section to learn more about "AAPXS".

Does AAP use Oboe?

AAP as an audio plugin format does not depend on Oboe. It is quite common to any audio plugin format, even including their implementation SDKs. It is because audio I/O and MIDI I/O are what DAWs deal with. Audio plugin SDKs usually deal only with binary memory stream for both audio and MIDI data. That's why typical audio plugin format API is cross platform (until they deal with GUI).

AAP MidiDeviceService is different. It works as a MIDI output device, which generates audio outputs. Thus we make full use of Oboe there.


Does AAP support complicated audio buses other than stereo and mono?

AAP intends to offer audio bus configuration extension, but so far it is not implemented yet.

Does AAP support MIDI events?

Yes, probably more than what you expect - AAP deals with MIDI events in MIDI 2.0 UMPs.

MIDI 1.0 stream could be translated to UMPs according to MIDI 2.0 UMP specification. You can use atsushieno/cmidi2 library that supports UMP-MIDI1 bytestream translation. JUCE also provides UMP translators.

Do you mean, AAP "maps" and consumes MIDI events for its own? Should we worry about some messages swallowed by AAP?

UMP is the event message structure. Parameter changes are sent as Universal SysEx8 messages, or optionally Control Changes, or Assignable Controllers (NRPNs). Presets can be selected by Program Changes too.

Unlike VST3 MIDI CCs and program changes, those mappings can be disabled by each plugin so that it can receive raw UMP message as is.

Does AAP support realtime audio processing?

The answer is going to be complicated but in short: not now. Because it is impossible. See Technical Details section on why.

What kind of GUI does AAP support?

AAP does NOT support GUI integration yet. It has been on our stack for multiple years (almost since the beginning in 2019) and we will eventually support it. Probably in both local in-process GUI and remote out-process GUI.

Usually, audio plugin GUIs developed for desktop are awful on mobile platforms. Their UI should be optimal for different form factors.


Why do we have to work with another audio plugin format? There are already too many!

People are tempted to discuss like that. But none of those who state like that actually provides solution for mobile and web platforms. How can VST3, LV2, CLAP work on Android as is? You cannot load plugin's dynamic library in the host process as it is prohibited. Then your only solution with those existing formats is to publish both plugins and DAWs within one single application. It is as if only Steinberg shiped ALL audio plugins under the name of Cubasis. No one would be able to ship their plugin without approval from Steinberg. That's not a diverse ecosystem.

Apple came up with their complete answer: AUv3. It is a different plugin format than AUv2. They are incompatible. AUv3 is not a privileged solution on iOS. It has to deal with isolated processes from the ground, at the cost from isolated processes. People often find it awkward, but that had to be done. It's like we cannot go back to HTTP without S. AAP is at a similar position.

Likewise, WebAudioModules would never happened if any of the existing plugin formats worked.

In practice, when people say "we should not invent another audio plugin format", there would be three kinds of people. Here are the answers to each:

  • "We don't want to build for multiple plugin formats unnecessarily" => it does not apply if there is no alternative format. AAP is so far the only one solution for Android. You build AAP, or you don't support Android.
  • "We don't want to write code for another plugin format" => people already don't. They use multi-plugin-format exporters like JUCE, DPF, FAUST, MATLAB etc. See how the emerging CLAP plugins have been built. We offer entry slot for JUCE and LV2 (thus indirectly DPF too). You can use them to program your plugin, then build the AAP version using aap-juce or aap-lv2.
  • "We don't want to depend on a plugin format that may vanish and the plugin binaries became useless" => true, though plugins with sources lived longer, and binaries would not die that soon as long as the DAWs and the plugins remain on the platform. On mobile OSes you should probably rather worry about apps vanishing from the platform due to the platform updates. arm32 is already dead, and Android 14 will be arm64-only on Arm architectures.

Isn't Google supposed to provide such a format?

Considering that AAP is at similar position to AUv3, maybe? But note that AU is the only plugin format that the platformer designed the API. (Well, Microsoft also had DirectX plugins in the past, in case you care). Essentially, anyone could work on such a project.

Why are JUCE and LV2 supported? How about VST3, or CLAP?

Because they are "the top two". As far as I have searched around GitHub repositories, JUCE is the most used audio plugin SDK by the audio plugin projects out there. LV2 is next. Of course they live in different tiers (JUCE is not a plugin format and LV2 specification is not an SDK), but from our (plugin porters') point of view, these two are the most beneficial.

DPF probably comes next. But since we already support LV2, DPF is already covered via LV2. We do have aap-lv2-dragonfly-reverb and aap-lv2-string-machine.

There are much less audio plugin projects that are implemented in direct VST3 API. So as CLAP. While there are many people who find that CLAP is a future, they do not really implement CLAP plugins in CLAP API.

Having said that, we are still seeking for CLAP integration on occasion, for host developers' sake. CLAP is one of the best suited plugin solution for embedding to host apps.

I figured that everything is so hacky. I don't think it is a good approach. Why don't you build a solid foundation first?

@atsushieno did not start the project as a perfect C/C++ audio developer and did not know much about modern C++. Probably still not. It should also be noted that it was not started with any solid plan. It was a hobby bonsai project which grew randomly.

But that doesn't matter. What is the most important in our perspective is to show a blueprint of the entire plugin ecosystem that looks feasible, which is a grand ecosystem with a lot of plugins. Those plugins were ported to explore many different porting setup (JUCE plugins are quite diverse; they use either CMake or Projucer. Plugin can be an effect or an instrument. JUCE version difference matters) which helped designing the entire ecosystem.

Crappy implementation internals can be changed at any time while the iron is hot. Designing portable plugin ecosystem takes another significant steps. The AAP fundamentals have been reflecting the porting experiences.

Technical details

What do you mean by process isolation, and how does it relate to AAP?

One of the latest trends on certain operating systems (like Android, iOS, or macOS), often along with the application markets, is that each application is "contained" and has explicitly limited access to the computer resources. They cannot infect on the platform e.g. they have no access to system file resources or files on other apps' space, or they cannot dynamically load (dlopen(), LoadLibrary()) other app's native executable code.

On desktop platforms, a plugin host (DAW) "instantiates" an audio plugin by loading its executable code (dynaic library). It is prohibited on modern OSes as explained above. Therefore, every interactive operation needs to be performed through some kind of IPC (inter-process communication).

AAP depends on Binder IPC and shared memory. The audio processing IPC performs without copying audio and MIDI buffers (because they are on the shared memory) so if the IPC stack works in realtime manner it could work in realtime.

What is Binder, and what is AIDL?

Binder is a low level IPC mechanism that resides partly in Linux Kernel internals. It is special in that the messaging involves shared memory space to send and receive arbitrary data structure (to some extent) within a machine, and involves native threads, often realtime ready, that can switch tasks seamlessly.

AIDL is the interface definition language. We have our AIDL for AudioPluginService. A host and a plugin interact each other based on this protocol.

Whenever our AIDL changes, the entire ecosystem gets broken and the world gets diverged between old and new. We should avoid that at all cost (but things happen until we finalize the API). We do not intend to let DAW developers and plugin developers to deal with the binder messaging directly. They are capsuled in the API in androidaudioplugin module (both C/C++ and Kotlin).

Isn't Binder and AIDL obsolete? I heard that WorkManager is the modern solution

If you are an Android app developer, you would have heard like "Use WorkManager for most cases". It is probably true for Java/Kotlin app developers. We are different. We are native C/C++ developers who worry about realtime safety and performance.

Audio processing has to be done in very short cycle like <10 milliseconds for professionals. Our requirement would not be that tight, but it should not involve extraneous latency caused by the Dalvik bytecode VM (ART, for Java/Kotlin code).

As explained in the next answer to the Q, binder is capable of realtime processing at its core. Binder support is still actively developed in Android platform, and it has public native API since API Level 29, which is still somewhat new.

Is Binder good to use for audio processing?

In short: Binder itself is, but we cannot benefit from it, at least yet.

More sophisticated answer:

Binder after Android 8.0 is implemented to work in realtime manner. However, Binder does not support realtime processing for general application, while it is technically possible(!). There is a bit more detailed description on AOSP website. Their audio related services could benefit from it, but our apps that depend on framework /dev/binder cann't.

When a Binder IPC is not realtime safe, it involves a lot of mutexes, which means no-go for realtime audio.

We could discuss with Google to open some opportunity to it. But we would need more PoC product first.

Why does AAP choose MIDI 2.0 as the event protocol?

One of the implicit principles in AAP is "whenever a standard exists and is useful, prefer using it". MIDI 1.0 was too rough for modern audio plugin processing, but MIDI 2.0 UMP (Universal MIDI Packet) is fairly good. Many UMP message comes with 32-bit data, namely in Control Changes, Registered and Assignable Controllers (RPNs and NRPNs in MIDI 1.0), key pressures (both chanel and poly), and Pitch Bend. Ultimately we use System Exclusive messages for controlling parameters, but being capable of processing UMPs directly, we would not even need MIDI mappings that each DAW somehow manages (sometimes even plugin format involves; CLAP).

What is AAPXS, and why do we need them?

AAPXS stands for "AAP eXtension Service". Unlike typical desktop audio plugin formats, we have to deal with some sort of mediator between host and plugin. AudioPluginService is one part of it.

Audio plugin extensibility is usually achieved in the form of dynamically loadable extension functions. But as we have discussed earlier, Android host cannot load plugin code dynamically, as well as plugins are unable to invoke host functions by pointer. Therefore, as explained earlier, we use IPC instead.

Extensions have to be implemented like binder messages. It is based on extension() service method in our AudioPluginService AIDL, and it only passes shared memory for request body and response body, indicated by the "opcode". We serialize and deserialize the request and the response on both sides.

AAPXS involves some implementation code, unlike extension API in other plugin formats, to implement these serialization parts that every plugin developer and host developer should not deal with by themselves.

Why don't you design and implement AAPXS based on Protocol Buffers, or AIDL?

It feels like we are inventing another wheel for serialization format in AAPXS. It is true in a sense, but since it is not that complicated, it should not be of a lot of problem. It is rather problematic to incorporate protocol buffer library into libandroidaudioplugin.so.

AIDL is good for protocols where all plugins and hosts implement everything, or it is okay if we can implement different Services for each AIDL. Apparently the latter is not true for AAP - there should not be too many IPC connections. If we define all extensibility in AIDL, that would mean every DAW and plugin needs to update the implementation whenever a new extension API appears (and not all extensions are "official"). They would work without missing service methods support, but the AIDL will be kept updated frequently and it will mess implementing service or your binder proxy.

Why do we have to export plugin metadata as in XML? Couldn't you choose better data format like JSON?

The answer is twofolds. We provide plugin metadata as Android Service metadata. It can be queried by PackageManager.queryIntentServices(). It returns the XML metadata content that is referenced from AndroidManifest.xml without instantiating the plugin.

If we decided to let each AAP plugin to return metadata by code, it would have been required to instantiate each plugin (or plugin factory, like what CLAP does) and that prevents fast plugin scanning. VST3 had similar slow scanning problem until 3.7.6 that started to support metadata in JSON (opt-in feature).

And interestingly XML is the only format Android system allows to return the metadata content. We cannot choose JSON etc. for that reason. The acceptable content size seems to keep shrinking, so at some stage we might have to remove some parts from aap_metadata.xml (e.g. <parameters> are already optional and can be dynamically populated in code), but the basica policy to provide metadata with XML won't change.

⚠️ **GitHub.com Fallback** ⚠️