BMPAPI - joric/qmk GitHub Wiki

Do not use that. That isn't even a proof of concept, it simply doesn't work at the moment.

This API is specific to the closed source bootloader for the Japanese board, "BLE-Micro-Pro" by sekigon. If you're not interested in reverse-engineering this board and its bootloader just skip this chapter, use ZMK or QMK instead.

BMPAPI stands for BLE-Micro-Pro API. My open source version doesn't work for now, see ZMK or QMK for the working stuff.

There's a QMK dev/ble_micro_pro branch (based on the abandoned nrf52 branch) that can import JSON keyboard layouts from the USB drive. It needs a binary blob that exposes a set of nRF52-based API (BMPAPI) functions at address 0xFDE00 (see apidef.h), this is an attempt to weasel out of the Nordic/QMK licensing issues (are there any?), so the BMPAPI-based QMK branch could be eventially merged into the QMK upstream. Looks like it's totally one way to do it.

The binary blob is distributed as UF2 and it's called "BLE Micro Pro Bootloader". Unfortunately it was closed source last time I checked and it's impossible to proxy because it only gives access to a fixed set of BLE-Micro-Pro pins, so you'd need to rewrite it from scratch. If you want it opensourced, bother @_gonnok here: https://discord.gg/MqufYtS.

As of May 2020 I have no idea if he is going to opensource this closed source blob. I can't even tell if he has slightest intention to do so or not. Closed source aside, there's a duplication of so much stuff so it ends up with its own matrix, encoder, etc. implementation rather than implementing the current GPIO interface, so it still won't get merged in its current form.

Another possible workaround to redefine pins using the standard BMP bootloader would be using nrf_gpio_pin_read() instead of read_pin() in the QMK (see matrix_basic.c file). You can start from my "Older version" setup below (dev/ble_micro_pro QMK branch). It breaks licensing, but you get useful BMP features, such as JSON config files.

BMPAPI-Related repositories

BMPAPI Intro

I got API working at much higher address (0xA0000 instead of 0xFDE00). Looks like Adafruit bootloader is refusing to rewrite higher address which is of course reserved for bootloader itself. So it's either change the API address to a lower one in the QMK firmware or ditch Adafruit bootloader altogether which is kind of a major nuisance.

It's possible to update bootloader from uf2 with rewriting fuses from the userspace app and proper bootstrapping. The easiest way to make it work would be adding BMPAPI to the Adafruit nRF52 Bootloader and rebuilding the bootloader, but it needs a lot of SWD flashing at least at the initial stage.

The way I'm doing it now is just two userspace apps, a hardware layer (bmpapi.uf2), and a QMK app (ble_micro_pro_default.uf2) that I flash next to each other in arbitrary order (it's also really easy to merge them into a single uf2 file).

BMPAPI Status

The main issue for now is that USB CDC doesn't work as two apps, it only works as a single app. I have no idea why it's happening, but USB only works in a monolithic firmware built with origin 0x26000. Tried to change the upload order but it doesn't help. Probably linker script issues.

The latest version is in the nRF5x folder, it's currently has a built in launcher to test USB code. if you want to run it from the another launcher app you just have to set bmpapi.ld back to default (+0x16000 from start):

-  FLASH (rx) : ORIGIN = 0x26000, LENGTH = 0xD2000
+  FLASH (rx) : ORIGIN = 0x26000+0x16000, LENGTH = 0xD2000-0x16000

That would need a separate launcher app at 0x26000 (max size 0x16000), either my test launcher from the launcher folder or the BMPAPI QMK version. I haven't even tried calling BLE softdevice functions from the blob, probably there would be the same problems as with USB CDC (init calls missing or something, not sure why it doesn't work, blinker app works just fine).

  • 2020-06-25 Trying to run USB CDC in nrf5x (hangs in app_usbd_power_events_enable, only works as a single app)
  • 2020-06-24 Created launcher, QMK, nrf5x and minimal configurations (exported at 0xA0000), calls API, blinks.

Uploading BMPAPI

  • Burn the Adafruit bootloader with the SWD programmer (skip this if you already have a bootloader)
  • Flash QMK as ble_micro_pro_default.uf2 (or use launcher app for testing)
  • Flash BMPAPI as bmpapi.uf2

To flash the .uf2 file, press reset twice withing 500 ms interval, then drag and drop the file to the appeared USB drive.

It works just fine with QMK firmware at 0x26000 with length 0x16000 (90kB) and BMPAPI exported at 0xA0000 (BMPAPI firmware starts from 0x26000+0x16000 with length 0xD2000-0x16000, so it occupies the rest of the FLASH memory).

TL;DR: just flash those two .uf2 files (ble_micro_pro_default.uf2 and bmpapi.uf2) in any order:

Building BMPAPI

Repository: https://github.com/joric/bmpapi

The nRF5 SDK have to be changed in components/toolchain/gcc/Makefile.posix, the line: GNU_INSTALL_ROOT := /usr/local/gcc-arm-none-eabi-4_9-2015q1 replaced with: GNU_INSTALL_ROOT := /usr/bin/ (same as with mitosis firmware).

git clone https://github.com/joric/bmpapi && cd bmpapi
cd minimal
export NRFSDK15_ROOT=~/nRF5_SDK_15.0.0_a53641a && make
python uf2conv.py .build/nrf52840_xxaa.hex -c -f 0xADA52840 -o bmpapi.uf2

The "minimal" version just blinks for now. The nrf5x version is in progress (it's just a matter of porting the nrf52 branch).

Building QMK for BMPAPI

Repository: https://github.com/sekigon-gonnoc/qmk_firmware

It should be only done once. There are "default" and "no_msc" keymaps, the latter disables "Mass Storage Class" (USB drive). The whole point of BMPAPI is that QMK branch (dev/ble_micro_pro) doesn't have any Nordic dependencies, only generic ARM.

git clone https://github.com/sekigon-gonnoc/qmk_firmware && cd qmk_firmware
git checkout dev/ble_micro_pro

Fix keyboards/ble_micro_pro/ld/nrf52840_ao.ld:

  FLASH (rx) : ORIGIN = 0x26000, LENGTH = 0x16000

Fix tmk_core/protocol/nrf/sdk15/apidef.h:

#define BMPAPI ((bmp_api_t*)0xA0000)

You can also use my branch that's already fixed: https://github.com/joric/qmk/tree/dev/ble_micro_pro

Build the rest of QMK:

make git-submodule
make ble_micro_pro:default
python uf2conv.py .build/ble_micro_pro_default.hex -c -f 0xADA52840 -o ble_micro_pro_default.uf2

You can get uf2conv.py here: https://github.com/microsoft/uf2/blob/master/utils/uf2conv.py

Older version

There's also another experimental repository (obsolete):

This firmware is based on the dev/ble_micro_pro branch and my version of the BLE-Micro-Pro "Default Firmware" binary blob (BMPAPI) ported from nrf52 branch, but instead of using a separate binary blob I'm building everything as a single uf2 file, it's a little bit easier to test and debug on initial stages. What it currently does:

  • adds bmpapi folder with all the Nordic SDK-dependent code
  • modifies nrf.mk to add Nordic SDK files to the compilation
  • modifies apidef.h so BMPAPI is an extern symbol and not 0xFDE00
  • modifies bmp.c to add bmpapi_init() call, filling the BMPAPI structure

It does just a few very basic things (will be eventually deleted and merged into bmpapi repository):

  • Pointers to functions are filled, firmware compiles and runs, doesn't crash
  • ws2812-based RGB strip, built in nRFMicro status LED
  • virtual COM port via USB, microshell and dfu handler

Figures the monolithic version can actually make USB work (can't make it work in split version so far).

Misc

GPIO Access

BMPAPI uses its own set of pins (PIN1 ... PIN21) there's no access to arbirary nRF5x pins. See bmp_api_gpio_t structure:

typedef struct {
    bmp_error_t (*set_mode)(uint8_t pin_num, bmp_api_gpio_mode_t const* const);
    uint32_t (*read_pin)(uint8_t pin_num);
    bmp_error_t (*set_pin)(uint8_t pin_num);
    bmp_error_t (*clear_pin)(uint8_t pin_num);
} bmp_api_gpio_t;

That means stock BMPAPI couldn't run on boards other than BLE Micro Pro (pin mapping is almost always different on different boards, there are hardware limitations). This is the main reason for reimplementing this closed source blob from scratch, patching it inplace seems a much harder task than rewriting it (pin references are pretty much scattered across the address space). Also I haven't managed to run this binary blob on the nRFMicro just yet, it just hangs (probably stock Adafruit bootloader issues, but it won't work as intended anyway).

Exporting BMPAPI

First, add this to the linker script:

SECTIONS
{
    .api_table 0xFDE00:
    {
    KEEP(*(.api_table))
    } > FLASH
}

Then prepare the content in C like this:

void bootloader_jump(void) { }
void reset(uint32_t _) { }
void enter_sleep_mode(void) { }

typedef struct
{
//  int32_t (*init)(bmp_api_config_t const * const);
  void (*reset)(uint32_t);
  void (*enter_sleep_mode)(void);
  void (*main_task_start)(void(*main_task)(void*), uint8_t interval_ms);
/*
  void (*process_task)(void);
  bmp_error_t (*push_keystate_change)(bmp_api_key_event_t const *  const key);
  uint32_t (*pop_keystate_change)(bmp_api_key_event_t* key, uint32_t len, uint8_t burst_threshold);
  uint16_t (*keymap_key_to_keycode)(uint8_t layer, bmp_api_keypos_t const * const key);
  bmp_error_t (*set_keymap)(const uint16_t* keymap, uint16_t len, const char * layout_name);
  bmp_error_t (*set_config)(bmp_api_config_t const * const);
  bmp_error_t (*get_keymap_info)(bmp_api_keymap_info_t* const keymap_info);
  const bmp_api_config_t* (*get_config)(void);
  bmp_error_t (*save_file)(uint8_t file_id);
  bmp_error_t (*delete_file)(uint8_t file_id);
  bmp_error_t (*get_file)(uint8_t file_id, uint8_t **buf, uint32_t * const len);
  uint16_t (*get_vcc_mv)(void);
  bmp_error_t (*set_state_change_cb)(bmp_api_state_change_cb_t);
*/
} bmp_api_app_t;

typedef struct
{
  //////DO NOT CHANGE///////
  uint32_t api_version;
  void (*bootloader_jump)(void);
  /////////////////////////
  const char* (*get_bootloader_info)(void);
  bmp_api_app_t app;
/*
  bmp_api_usb_t usb;
  bmp_api_ble_t ble;
  bmp_api_gpio_t gpio;
  bmp_api_i2c_master_t i2cm;
  bmp_api_i2c_slave_t i2cs;
  bmp_api_spi_master_t spim;
  bmp_api_ws2812_t ws2812;
  bmp_api_logger_t logger;
  bmp_api_web_config_t web_config;
  bmp_api_encoder_t encoder;
  bmp_api_adc_t adc;
*/
} bmp_api_t;

__attribute__((section(".api_table")))
bmp_api_t API_TABLE = {
    .api_version = 0x01020304,
    .bootloader_jump = bootloader_jump,
    .app = { reset, enter_sleep_mode, /* ... */ },
    // ...
};

You can export the bmp_api_t structure at 0xFDE00 with the define in the linker ld file, no issues here. All pointers to functions should be filled somehow, either in compile time or dynamically. Mind that there's no init call in the QMK. it just starts from BMPAPI->api_version and then BMPAPI->bootloader_jump() and then right away it's BMPAPI->logger.init() as the next call. It's either linker can set them properly at the compile time or some kind of the init call (similar to dllmain function) should fill up the structures. I'm not sure if everything could be settled up in compile time or I still would need an init call.

You can examine the original bootloader dump:

od -Ax -tx1 ble_micro_pro_bootloader_0_5_1.uf2 | less

  • bootloader starts at 0xE0000 (00 00 0e 00)
  • api table starts from 00 de 0e 00
  • UICR region starts from 0x10001000

Address 0x10001014 (13 UICR — User information configuration registers) is called NRFFW[0] and it is where MBR looks to know the location of the main application. It this case bootloader is the main application. and qmk is sort of child application launched from bootloader.

Use objdump to make sure if it's properly compiled:

arm-none-eabi-objdump -d -D -C _build/nrf52840_xxaa.out | less

Example BMPAPI builds fine with this .ld but doesn't call anything (apparently I also need to modify the UICR region from the bootloader).

MEMORY
{
  FLASH (rx) : ORIGIN = 0xE0000, LENGTH = 0x1E000
  RAM (rwx) :  ORIGIN = 0x20018000, LENGTH = 0x28000
}

SECTIONS
{
    .api_table 0xFDE00:
    {
    KEEP(*(.api_table))
    } > FLASH
}

For ORIGIN = 0x26000, you should make LENGTH = 0xD2000 or something like that so it could compile without overflow.

Memory placement

Default FLASH memory placement for nRF52840-QIAA (see Nordic infocenter):

Usage Start End Size Size (kB)
Master Boot Record (MBR) 0x0 0x1000 0x1000 4 kB
SoftDevice (S140 v6.1.x) 0x1000 0x26000 0x25000 148 kB
Application area (incl. free space) 0x26000 0xF8000 0xD2000 840 kB
Bootloader 0xF8000 0xFE000 0x6000 24 kB
MBR parameter storage 0xFE000 0xFF000 0x1000 4 kB
Bootloader settings 0xFF000 0x100000 0x1000 4 kB
  • Example Nordic App: 0x26000 ... 0xF8000 (length 0xD2000)
  • Original Adafruit Bootloader: 0xF4000 ... 0xFD800 (length 0x9800)
  • Original dev/ble_micro_pro QMK: 0x26000 ... 0xE0000 (length 0xBA000)
  • Original BMPAPI offset: 0xFDE00

There is a separate launcher app (https://github.com/joric/bmpapi/tree/master/launcher) for testing.

Launcher .ld script:

  FLASH (rx) : ORIGIN = 0x26000, LENGTH = 0x16000

BMPAPI .ld script:

  FLASH (rx) : ORIGIN = 0x26000+0x16000, LENGTH = 0xD2000-0x16000

BMPAPI only works starting from 0xA0000 for some reason (0xFDE00 is out of reach for the userspace app, it's a bootloader area). I tried overlapping addresses first but it crashed in the very middle (there still was a subtle LED flash). This is because functions were overwritten by the launcher so I needed to move all the functions out of the launcher memory space as well. I did it with the setup above and two apps now work together just fine.

How to align a specific function

GCC linker description file can force symbol to be at specific address, see:

The best read about this is probably https://github.com/adafruit/Adafruit_nRF52_Bootloader it has a lot of stuff aligned at specific addresses.

C file:

#define LOCATE_FUNC  __attribute__((__section__(".mysection")))
...
void LOCATE_FUNC Delay(uint32_t dlyTicks)
{
    ...
)

Linker .ld file:

MEMORY
{
	FLASH (rx) : ORIGIN = 0x0, LENGTH = 128K
	RAM (rwx) : ORIGIN = 0x20000000, LENGTH = 32k
	MY_MEMORY (rx) : ORIGIN = 0x20000, LENGTH = 128k
}
...
SECTIONS
{
 .mysection :
  {
    . = ALIGN(4);
    __mysection_start__ = .;
	*(.mysection*)
    __mysection_end__ = .;
  } > MY_MEMORY
}

Clean and build the project again to use the updated linker script, open the .map file to verify that the Delay() function is indeed located at 0x20000.