Quick Start - andy3471/GBD GitHub Wiki

The material in this section assumes:

• An economy GBD setup as presented in GBD Architecture with the source of the PCM being an audio player executing locally on the PC/laptop.

• A GNU/Linux host for the GBD client (i.e. the GBD PCM plugin). Any standard GNU/Linux distro should do. Ubuntu 18.04 LTS AMD64 was used.

It is recommended that an effort is made to first get GBD working with the setup and utilities described on this page before proceeding with any form of application development with the GBD framework on a different host OS/Platform.

Three sets of the GBD server executable (gbdserver) and GBD DSP library (gbd.so) binaries are included in the repository. The armv6l directory contains binaries for RPi Zero. The binaries in armv71 have been tested on RPi 3B. The PC versions are in x86-64-ubuntu-18.04.

For the ARM versions, the development platform was the official 32-bit Raspbian/Debian 9.4 (Stretch) OS on Raspberry Pi Model 3B -- with cross-compilation for armv6l. Refer to section ABI Compatibility for a detailed discussion of possible ABI issues if you intend to use a different Raspberry platform.

• The GBD DSP library is a LADSPA module and must be installed under /usr/lib/ladspa/, e.g.:

  $ sudo mkdir /usr/lib/ladspa
  $ sudo install -m 644 gbd.so /usr/lib/ladspa/
  

• The GBD server executable can be run from any location on the file system. The current GBD release removes the cloud authentication requirement and includes the "cracked" rtbeats-keys keys in directory usr. These keys must be installed in the following location for the GBD server to work:

  $ sudo mkdir -p /usr/local/share/rtbeats
  $ cd ./GBD
  $ sudo install -m 644 ./usr/local/share/rtbeats/rtbeats-keys /usr/local/share/rtbeats/
  

The gbdclient/gbdclient.c file defines an ALSA external PCM filter plugin for a GNU/Linux-based GBD setup. ALSA is the official/core sound system for GNU/Linux. This PCM plugin works transparently for both an economy or standalone RPi GBD model. To build and install this PCM plugin on an x86_64 host (i.e. economy GBD setup):

  $ sudo apt-get install libasound2-dev libasound2-plugins ## ALSA Dependencies
  $ cd gbdclient
  $ mv Makefile-x86_64 Makefile
  $ make 
  $ sudo make install

The make command should build a libasound_module_pcm_gbdclient.so PCM plugin library. On x86_64, the make install should result in an installation under /usr/lib/x86_64-linux-gnu/alsa-lib/. There also exists a Makefile-arm-rpi for builds in a standalone RPi GBD setup. If running on a non-x86_64 GNU/Linux (or non-RPi) system, edit the install target of gbdclient/Makefile accordingly to point to /usr/lib/$ARCH/alsa-lib.

Chapter ALSA Configuration discusses advanced ALSA system configuration (i.e. ALSA virtual PCM device definitions) for GBD use in real world scenarios. Chapter Configuring Applications For GBD presents examples of configuring various audio applications for use with those ALSA virtual PCM devices.

This section provides an entry-point ALSA PCM device definition and test setup for the gbdclient/gbdclient.c PCM plugin.

• Signal Length and Channel Count

  The beat detection algorithm assumes input signals of 1024 samples per channel. The gbdclient/gbdclient.c PCM plugin currently supports stereo signals only. It expects interleaved L/R channel samples in the ALSA frames.

• Sample format

  The GBD library runtime mechanism borrows from LADSPA. Its prototyping platform was the ALSA/LADSPA framework. As a result, the GBD DSP algorithms assume PCM samples encoded in the ALSA FLOAT format. This conversion from PCM integer to ALSA FLOAT is either performed automatically in an ALSA/LADSPA setup (the default setup assumed in this guide), or will performed by the PCM plugin writer developing for another platform. See Notes for GBD Client PCM Plugin Writers. The recommended PCM integer encoding before the conversion to ALSA FLOAT is S16.

• Sample rate

  The beat detection algorithm assumes a sample (or sampling) rate of 44100Hz.

• Launch the GBD server. No root privileges required. A numeric port value must be specified though, e.g.:

  $ ./gbdserver -p 7777
  

  At this juncture, the GBD server will attempt to authenticate itself using the cracked /usr/local/share/rtbeats/rtbeats-keys keys (or, for older GBD versions, with a remote server on the Internet) before starting to listen for incoming connections from a GBD client. If it prints

  >> Authenticating... (Success)
  >> Listening for incoming gbdclient connections on port: 7777
  

  then it means that authentication was successful and that an audio application (via the GBD client) may establish a connection and begin streaming audio signals. Otherwise, if authentication failed, the gbdserver instance will print the reason for "denial of service" and quit. Also see A Known GBD Server IPC Issue that may occur during application startup.

• Inspect the state of the GBD server socket, e.g.:

  $ sudo netstat -tapn | egrep '(^Proto|gbdserver)'
  Proto Recv-Q Send-Q Local Address  Foreign Address State   PID/Program name
  tcp        0      0 0.0.0.0:7777   0.0.0.0:*       LISTEN  29122/gbdserver 
  

  or

  $ sudo ss -tapn | egrep '(^State|gbdserver)' 
  State    Recv-Q   Send-Q   Local Address:Port   Peer Address:Port                                                                                   
  LISTEN   0        1        0.0.0.0:7777         0.0.0.0:*          users:(("gbdserver",pid=29122,fd=4))
  

  At this point, the GBD server is up and running; ready to receive PCM signals from the GBD client and to pass them to the GBD library for realtime beat analysis. What is now required is to configure the GBD client's host system (i.e. ALSA in this case) to deliver signals with the expected PCM parameters to the GBD PCM plugin.

• Configure the following basic ALSA virtual PCM device definition

  $ cat ~/.asoundrc
  ...
  pcm.gbd {
    type plug
    slave {
      pcm plugGbd
      rate 44100 
    }
  }
  
  pcm.plugGbd {
    type gbdclient
    slave.pcm plughw
    ipaddr "192.168.0.19" ## your RPi IP Address here
    port "7777"
  }
  

  NOTE: The GBD client PCM plugin should always be configured for ALSA PCM plugins such as hw, plug, and dmix. Using ALSA PCM plugins is generally incompatible with sound server (notably, PulseAudio) services -- especially on systems that lack H/W stream mixing capabilities.

  So before proceeding with the GBD test below, first check whether some other resource has claimed the playback device, e.g.

  $ aplay -D plughw foo.wav
  aplay: main:722: audio open error: Device or resource busy
  

  On hardware that does not support PCM stream mixing, likely causes of this error include active instances of sound server clients or some other application using, say, hw. Stop all other programs currently performing playback. If this problem persists, see section ALSA PCM Plugins vs. Sound Servers for a more detailed discussion on how to identify the audio application contending for the playback device.

• Now test the GBD PCM plugin configuration using the aplay(1) commandline shown below. The (very important) --period-size 1024 parameter specifies signals of 1024 ALSA frames per ALSA period and the -v switch is used to print out the software conversions and PCM parameters at the various stages of the ALSA PCM plugin pipeline. The uncompressed 16-bit PCM gbd-house-mix.wav may be used for this test:

    $ sha1sum gbd-house-mix.wav 
    26dd9dbb0ea435d1018ca72a68c9bb2dbb2a0a19  gbd-house-mix.wav
  
    $ aplay -D gbd gbd-house-mix.wav --period-size 1024 -v
    Playing WAVE 'gbd-house-mix.wav' : Signed 16 bit Little Endian, Rate 44100 Hz, Stereo
    Plug PCM: Linear Integer <-> Linear Float conversion PCM (FLOAT_LE)
    Its setup is:
      stream       : PLAYBACK
      access       : RW_INTERLEAVED
      format       : S16_LE
      subformat    : STD
      channels     : 2
      rate         : 44100
      ...
      period_size  : 1024
      ...
    Slave: gbdclient
    Its setup is:
      stream       : PLAYBACK
      access       : MMAP_INTERLEAVED
      format       : FLOAT_LE
      subformat    : STD
      channels     : 2
      rate         : 44100
      ...
      period_size  : 1024
      ...
    Slave: Plug PCM: Linear Integer <-> Linear Float conversion PCM (S32_LE)
    Its setup is:
      stream       : PLAYBACK
      access       : MMAP_INTERLEAVED
      format       : FLOAT_LE
      subformat    : STD
      channels     : 2
      rate         : 44100
      exact rate   : 44100 (44100/1)
      ...
      period_size  : 1024
      ...
    Slave: Hardware PCM card 0 'HDA Intel PCH' device 0 subdevice 0
    Its setup is:
      stream       : PLAYBACK
      access       : MMAP_INTERLEAVED
      format       : S32_LE
      subformat    : STD
      channels     : 2
      rate         : 44100
      ...
      period_size  : 1024
      ...
  

  At this point, the GBD client (PCM plugin) should be routing the audio player's signals to the GBD server instance. For instance, on the RPi machine, the following command can be used to inspect the state of the network connections:

  $ sudo netstat -tapn | egrep '(^Proto|gbdserver)'
  Proto Recv-Q Send-Q Local Address  Foreign Address    State       PID/Program name
  tcp        0      0 0.0.0.0:7777   0.0.0.0:*          LISTEN      29122/gbdserver 
  tcp        0      0 127.0.0.1:7777 192.168.0.6:52746  ESTABLISHED 29442/gbdserver 
  

  Recall that the PCM plugin then routes the audio signal to the host computer's audio interface after transmitting a copy to the GBD server. In an economy GBD setup, sound should be audible via the PC/laptop's audio interface (e.g. speakers/line-out).

  • NOTE: The very first audio player connection (i.e. via the GBD client) after GBD server launch usually takes a second or two for the GBD server to dynamically allocate resources for the GBD library. But once these resources have been allocated, they are maintained for the lifetime of the server instance. So, after this first time connection, future GBD client connection requests will complete without noticeable latency.

  Thoroughly inspect the aplay(1) output above and compare it to your execution instances. At the very least, the period_size and exact rate values at the Slave: gbdclient stage should be identical to those displayed in the snippet above. DO NOT proceed with GBD until you obtain this output.

On the RPi machine:

• The demo programs in the maker-templates directory can be used to view GBD beat detection:

  • The console mode gbd-text-display.c prints beat counts to a terminal interface. It illustrates the essentials of the GBD Linux shared memory IPC. Compile and execute with:

     pi@raspberrypi:~ $ gcc -Wall -O2 gbd-text-display.c -o gbd-text-display -lrt
     pi@raspberrypi:~ $ ./gbd-text-display
    

  • The (ridiculously amateur and hacked-up) gbd-gl.c OpenGL program emulates a simple LED configuration. Feel free to improve it, e.g. porting to OpenGL ES (or something). This version hogs the CPU and should only be used for verifying that GBD actually works as advertised. The sequence of "light patterns" in the emulation is kept deliberately simple to allow for a scrupulous assessment of the accuracy of the beat detection algorithm. Compile and execute with:

    pi@raspberrypi:~ $ gcc -Wall -O2 gbd-gl.c -o gbd-gl -lglut -lGLU -lGL -lrt -lm
    pi@raspberrypi:~ $ ./gbd-gl 
    

  • If you have a WS281x-type LED strip at your disposal, check GBD rpi_ws281x out.

On the GBD client (PCM plugin) host:

• Re-run the aplay(1) command. If testing with gbd-house-mix.wav, then you should witness beat detection output as similar to Youtube GBD OpenGL Demo video (which was performed on GBD v0.20).

The GBD test instructions outlined in this section have been tested and re-tested. Until one succeeds with this most basic GBD setup and test, please refrain from raising or opening an issue.

For instance, run:

$ PIDLIST=$(for i in `pidof gbdserver` ; do echo -n " $i,"; done ; echo " 0")
$ top -d .4 -p $PIDLIST

Recall that the gbd.so library (which performs the actual signal processing) is loaded by gbdsever instance.

The beat detection (and the soon-to-be-added music DSP analysis algorithms) rely heavily on the Fast Fourier Transform (FFT). Beat detection in particular not only executes several instances of the FFT per PCM signal, but also has to compute them in a serialized fashion; the output of one FFT instance is used as the input of the next FFT instance. This means that the algorithm cannot take advantage of all the cores of the RPi SMP. However, it is quite impressive that what would have been prohibitively heavy realtime FFT usage for regular desktop machines around 20 years ago consumes only an average of less than 5% on a single core of the RPi.

For example:

$ sudo apt-get install nethogs
$ sudo nethogs wlan0

Section GBD Architecture explained the reasons behind using TCP sockets in economy GBD setups. As discussed, please make an effort to test GBD (preferably) with either a "dedicated" WiFi setup or Ethernet, and for 4-to-8 hours non-stop; noting the occurrences and the duration of any (unlikely) instances of underruns.

Standalone RPi GBD setups are not subject to this problem.

The GBD music DSP analysis library uses Linux POSIX shared memory (SHM) for beat count IPC with the IoT maker's Light/LED control engine. An abrupt RPi system shutdown (e.g. sudden power loss without graceful OS poweroff) may result in a stale /dev/shm/gbd file that persists across reboot. This file will be unusable and the system might even set file ownership to root:root upon reboot. Should this occur, the GBD server will fail to start. If the first instance of the server after system (re)boot authenticates successfully but then exits with an error, e.g.

init:1210:: Permission denied

check for the presence of a stale and possibly root owned /dev/shm/gbd file. Delete it if it exists then restart the GBD server.

As a temporary solution, the following systemd service could be installed:

$ cat gbd.service

  [Unit]
  Description=GBD: A Generic and Realtime Music Beat Detector

  [Service]
  Type=simple
  ExecStart=/bin/rm -f /dev/shm/gbd

  [Install]
  WantedBy=multi-user.target

$ systemd-analyze verify gbd.service 
$ sudo cp -a gbd.service /lib/systemd/system/
$ sudo systemctl enable gbd.service 
Created symlink from /etc/systemd/system/multi-user.target.wants/gbd.service to 
/lib/systemd/system/gbd.service