2.2. SORACOM Beam (Advanced) MQTT bidirectional - soracom/handson GitHub Wiki

Introduction

You can learn how to control your device remotely and safely by using SORACOM Beam and MQTT.

Preparation

  • RPi connected with SORACOM Air
  • Solderless Breadboard
  • LED
  • Resistor
  • 2 x Jumper wires

What is MQTT

quoted from MQTT

MQTT is a machine-to-machine (M2M)/"Internet of Things" connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium. For example, it has been used in sensors communicating to a broker via satellite link, over occasional dial-up connections with healthcare providers, and in a range of home automation and small device scenarios. It is also ideal for mobile applications because of its small size, low power usage, minimised data packets, and efficient distribution of information to one or many receivers.

Basics

You need to know following keywords to understand hwo MQTT works.

  • Message Data on MQTT is called "Message".

  • Broker MQTT server is called as "Broker". It authenticates and accepts connections from MQTT clients and exchange data called "Message"s between.

  • Topic MQTT messages are sent to a group called "Topic".

MQTT clients can communicate MQTT Broker in two ways. One is "Publish" and the other is "Subscribe".

  • Publish Clients can "Publish" data to specific "Topic".

  • Subscribe Clients can listen to a "Topic" or multiple "Topic"s by using wildcard meta character "#". For example, If a client listen to "foo/#", it can get message from "Topic" "foo/bar", "foo/baz" at the same time.

Install and test client (MQTT.fx) on your PC

We will use GUI MQTT client "MQTT.fx" to send/receive messages.

Download and Install

open http://mqttfx.jensd.de/index.php/download and find the link to the latest version (1.5.0 as of August 2017), download installer file for your PC (Windows/Mac/Linux).

Once you download and open the app, you will see a window like this. mqtt.fx

For Mac, you need to approve to open it in System Preferences / Security and Privacy.

Configure server profile

We have a server to test with. To connect the server, click on the gear mark next to "Connect" button.

open config

You need to click on "+" in the bottom/left corner and fill in information of test server.

add config

Connetion Profile

  • Profile Name: beamtest
  • Broker Address: beamtest.soracom.io
  • Broker Port: 8883
  • Client ID: click on Generate to generate your unique ID

profile

User Credentials

  • User Name: beamuser
  • Password: passwd

credentials

SSL/TLS

  • Enable SSL/TLS: ☑️ checked, select "CA signed server certificate"
  • Protocol: TLS v1.2

ssl/tls

Then save setting by clicking on OK

Connect to test server

Now you can connect to the server by clicking "Connect" button.

When the indicator in upper/right corner turns to green, it could connect to the servrer.

Subscribe a topic

Click on "Subscribe" button and fill in some topic name like "test"(it can be anything). Now it is ready to receive messages.

Subscribe

Publish a message

Click on "Publish" button, input topic name you choose(i.e. "test") to input box, input something like "THIS IS TEST!" in the textarea, then click "Publish".

Publish

Click on "Subscribe" and you will see the message you just sent.

Receive

It may take a while to receive.

Now you understand how MQTT is working!

What is SORACOM Beam

SORACOM Beam (“Beam”) is a service that enables you to offload the high-load processes resulting from device encryptions into the cloud and to set up their destination. By using Beam, you can easily manage your IoT devices anytime and anywhere through the cloud. You do not need to directly set up large numbers of devices.

Beam Features

Secure uploads and downloads using encryption

Sending and receiving sensitive information through IoT devices need to go through an encryption process, but sometimes it is difficult to do so with the limited resources on IoT devices. In these situations, the encryption can be done using SORACOM resources. By using Beam, it will do what your device has difficulty doing, i.e., encrypt communication between your devices and your server for you.

With Beam, when uploading data from your IoT device, the data will be sent to the end point in Beam through a closed 3G/LTE network. The data emerging from Beam will be encrypted so that you can deliver it to your server safely and securely.

More specifically, TLS (Transport Layer Security) is used between Beam and your server, and the data is encrypted during both data upload and data download (data communication between Beam endpoint and your server). Through TLS, besides encryption of information, peer connection (to check whether your server has been spoofed by a malicious third party) and detection of information tampering (to check whether the data has been rewritten intentionally by a malicious third party) are also possible.

In this way, you can encrypt any data communication of IoT devices that do not support data encryption due to CPU and power limits. Complicated processes like certificate management and responses to vulnerabilities can be offloaded to the cloud and the encrypted data can be easily used.

Currently, Beam supports the following encryption offloading.

Device Protocol Server Protocol
HTTP HTTPS
MQTT MQTTS
TCP TCPS

In this tutorial, we will use MQTT to MQTTS offloading.

Connect RPi to MQTT using Beam

In this section, we will try MQTT ovet Beam togther with command line MQTT client called mosquitto.

Configure Soracom metadata and Beam setting for MQTT

Go to Soracom console, on the top left menu select Group.

You can then select the group you created previously.
In the basic setttings, open up Soracom Air for Cellular and switch on Metadata Service. This will expose a http interface on http://metadata.soracom.io/v1/subscriber which allows you to get various metadata items attached to your SIM card.

Metadata configuration

Once you have enabled metadata, scroll down to Soracom Beam and add a new endpoint, click on the + sign and select "MQTT entry point", create an entry point with the following settings:

  • Protocol: MQTTs
  • Hostname: beamtest.soracom.io
  • Port Number: 8883
  • Username: beamuser
  • Password: passwd

In the Options, make sure you enable "Append IMSI to topic"

For this tutorial, we will leave client cert OFF

Beam configuration

install mosquitto-clients

Before starting, make sure you unplug your USB Dongle, that way you'll use WiFi rather than 3G data to install packages Now that you have configured your Beam MQTT entry point, we will install command line MQTT client to test and make sure everything has been configured correctly:

command

sudo apt-get update && sudo apt-get install mosquitto-clients

use mosquitto_pub/sub commands

Now that your sofware has been installed, you can connect your USB dongle again

You need to have 2 terminals, one is for publish, the other is for subscribe. Please open another ssh terminal. We will call terminal 1 and 2.

Find out your RPi IMSI:

command

on terminal 1

curl http://metadata.soracom.io/v1/subscriber.imsi
imsi=$(curl http://metadata.soracom.io/v1/subscriber.imsi)
echo $imsi

result

on terminal 1

pi@raspberrypi:~ $ curl http://metadata.soracom.io/v1/subscriber.imsi
29505xxxxxxxxxx
pi@raspberrypi:~ $ imsi=$(curl http://metadata.soracom.io/v1/subscriber.imsi)
pi@raspberrypi:~ $ echo $imsi
29505xxxxxxxxxx

Subscribe to a topic:

command

on terminal 1

mosquitto_sub -h beam.soracom.io -p 1883 -t "beamtest/$imsi"

result

on terminal 1

pi@raspberrypi:~ $ mosquitto_sub -h beam.soracom.io -p 1883 -t "beamtest/$imsi"

Publish to a topic (As you configured it in Beam, IMSI will be appended to topci when you send a message):

command

on terminal 2

mosquitto_pub -h beam.soracom.io -p 1883 -t 'beamtest'  -m "hello Beam MQTT world"

result

on terminal 2

pi@raspberrypi:~ $ mosquitto_pub -h beam.soracom.io -p 1883 -t 'beamtest' -m "hello Beam MQTT world"
pi@raspberrypi:~ $

on terminal 1

pi@raspberrypi:~ $ mosquitto_sub -h beam.soracom.io -p 1883 -t "beamtest/$imsi"
hello Beam MQTT world

Test LED with RPi

Now, let's test LED with RPi.

You will use GPIO (General Purpose Input/Output) pins to control LED. Please refer following picture for pin layout.

GPIO

Turn LED on (using 3.3v and GND pin)

You can use a RPi like a battery or power supply. It has plus (5v and 3.3v) and minus (GND) pins. Plug a LED and resistor like below.

  • Attach LED to breadboard. Longer pin is for plus, shorter is minus.
  • Bend feet of resistor and attach it to the breadboard, one is to LED minus (shorter) pin.
  • Connect black cable to the other foot of resistor and RPi pin 6 (GND).
  • Connect green cable to the longer foot of LED and RPi pin 3 (3.3V)

LED1

If LED does not turn on, you may have something wrong. Maybe you can try to make LED feet reverse.

Turn LED on/off by command

Next, we will control LED from RPi.

Login to RPi using ssh, then type following command.

command

echo 4 > /sys/class/gpio/export
echo out > /sys/class/gpio/gpio4/direction
echo 1 > /sys/class/gpio/gpio4/value

result

pi@raspberrypi:~ $ echo 4 > /sys/class/gpio/export              # initialize GPIO4
pi@raspberrypi:~ $ echo out > /sys/class/gpio/gpio4/direction   # use GPIO4 as Output
pi@raspberrypi:~ $ echo 1 > /sys/class/gpio/gpio4/value         # turn it to HIGH(1)

Then, change pin of green cable from Pin 1 (3.3v) to Pin 7 (GPIO4).

LED2

The LED should be ON. If not, please check the commands and retry.

Type following command to turn LED off.

command

echo 0 > /sys/class/gpio/gpio4/value

result

pi@raspberrypi:~ $ echo 0 > /sys/class/gpio/gpio4/value         # turn it to LOW(0)

Blink LED

So, if you put 0 and 1 in turn, it should be blinking. Let's try to do this with following command.

command

while [ 1 ] ; do echo 1 > /sys/class/gpio/gpio4/value ; sleep 1 ; echo 0 > /sys/class/gpio/gpio4/value ; sleep 1 ; done

result

pi@raspberrypi:~ $ while [ 1 ] ; do echo 1 > /sys/class/gpio/gpio4/value ; sleep 1 ; echo 0 > /sys/class/gpio/gpio4/value ; sleep 1 ; done
(Ctrl+C to stop)

Controlling LED via Beam MQTT

Install node.js and MQTT test app

Before starting, make sure you unplug your USB Dongle, that way you'll use WiFi rather than 3G data to install packages

cd ~
wget http://nodejs.org/dist/v8.4.0/node-v8.4.0-linux-armv6l.tar.gz
cd /usr/local
sudo tar xzvf ~/node-v8.4.0-linux-armv6l.tar.gz --strip=1
sudo apt-get remove --purge npm node nodejs
sudo npm install -g npm
cd ~
rm node-v8.4.0-linux-armv6l.tar.gz

mkdir beam_mqtt
cd beam_mqtt
curl -O https://soracom-files.s3.amazonaws.com/handson/beam_mqtt/app.js
curl -O https://soracom-files.s3.amazonaws.com/handson/beam_mqtt/package.json
npm i

now that you've installed the software, you can plugin your USB dongle again

Start the app to control LED

Start node application to control LED.

command

node app.js

result

pi@raspberrypi:~/beam_mqtt $ node app.js
[DEV] Loading MQTT client

              ..;;ttLLCCCCCCLLtt;;..              
          ..11CCCCCCCCCCCCCCCCCCCCCC11..          
        ::LLCCCCCCttii::,,::iittCCCCCCLL::        
      ::CCCCCC11..              ..11CCCCCC::      
    ::CCCCCCCCttii::..              ::LLCCCC::    
  ..LLCCCCCCCCCCCCCCCCffii::..        ,,LLCCLL..  
  11CCCC::,,;;ttLLCCCCCCCCCCCCff11::..  ::CCCC11  
..CCCC11          ,,;;11LLCCCCCCCCCCCC..  11CCCC..
iiCCCC,,                  ..::11LLCCCC..  ,,CCCCii
ttCCff                          ;;CCCC..    ffCCff
LLCCii                          ;;CCCC..    iiCCLL
CCCC;;                        ,,11CCCC..    ;;CCCC
CCCC::                ,,iittLLCCCCCCCC..    ::CCCC
CCCC;;      ..::iittCCCCCCCCCCCCCCffii      ;;CCCC
LLCCii    ;;CCCCCCCCCCCCLLttii,,            iiCCLL
ttCCff    ..LLCCCCtt;;,,          ::        ffCCff
iiCCCC,,    iiCCCC,,          ,,::tt,,..  ,,CCCCii
..CCCC11    ..LLCCtt          ;;LLCCtt..  11CCCC..
  11CCCC::    iiCCCC,,          LLff;;  ::CCCC11  
  ..LLCCLL,,  ..LLCCtt  ..tt11..,,  ::,,LLCCLL..  
    ::CCCCLL::  iiCCCC::ffCCCC;;    ::LLCCCC::    
      ::CCCCCC11,,LLCCCCCCCC11  ..11CCCCCC::      
        ,,LLCCCCCCLLCCCCCCffiittCCCCCCLL::        
          ..11LLCCCCCCCCCCCCCCCCCCLL11..          
              ..;;ttLLCCCCCCLLtt;;..              

  ___  ___  _ __ __ _  ___ ___  _ __ ___  
 / __|/ _ \| '__/ _` |/ __/ _ \| '_ ` _ \
 \__ \ (_) | | | (_| | (_| (_) | | | | | |
 |___/\___/|_|  \__,_|\___\___/|_| |_| |_|

[DEV] successfully sent data to Soracom Harvest
[DEV] Device IMSI is: 29505xxxxxxxxxx

[DEV] Connected to Beam MQTT
[DEV] Subscribe to MQTT topic: imsi_in/29505xxxxxxxxxx

The app will periodically post RPi performance data to MQTT topic "imsi_out/" and also subscribe to messages on "imsi_in/" to control your LED

On you computer, you can now remotely control your LED

Turn LED on

Send "on" message to your device with topic 'imsi_in/' to switch on the LED

LED ON

[DEV] Received MQTT message on topic: imsi_in/295057330042739 with message on
[DEV] switching LED on

Turn LED off

Send "off" message to your device with topic 'imsi_in/' to switch off the LED

LED OFF

[DEV] Received MQTT message on topic: imsi_in/295057330042739 with message off
[DEV] switching LED off

Blink LED using

Send "blink" message to your device with topic 'imsi_in/' to let the LED blink

LED BLINK

[DEV] Received MQTT message on topic: imsi_in/295057330042739 with message blink
[DEV] switching LED blink mode

Cleanup

Since Beam configuration itself will not cause any charge, you can keep them as is for free. But if you still want to remove resources, you can delete entry point configuration or group itself.

Appendix

How to connect device to AWS IoT message broker

This section shows how to connect your device to AWS IoT message broker in easy and secure way.

AWS IoT requires devices to connect with TLS client certificate. You usually install that certificate directly to your device, but with Beam, devices do not need to have certificate files because Beam will handle TLS authentication instead. It means, even if your device were stolen, credentials for AWS IoT is not there. Also, your device can talk to Beam endpoint without TLS encryption. It helps even tiny MCUs with low cpu/memory capability can connect to AWS IoT.

Create policy

To use AWS IoT via MQTT, you need a policy to allow connection.

  1. Open AWS IoT Console and click to open "Secure" in left menu, click "Policies". Click "Create a policy" button (if you don't have any policies yet) or click "Create" button (if you already have some policies).

  2. Input following fields and click "Create"

    • Name: MQTT (or anything you like)
    • Action: iot:*
    • Resrouce ARN: *
    • Check "Allow"

AWS IoT: create a policy

Register a device

Now you are ready to register a device.

  1. Click "Manage" in the left menu. Click "Register a thing" button (if you don't have any Things yet) or click "Create" button (if you already have some Things).

  2. Click 'Create a single thing'. Input "Name" of the device (i.e. raspi) and push "Next" AWS IoT: register a device

  3. Click "Create certificate" AWS IoT: create certificate

  4. Download ceritificate files (do not forget to download "root CA for AWS IoT from Symantec", in the bottom). Then, click "Activate" and click "Atach a policy". AWS IoT: download certificate

  5. Check the policy you created and click "Register a Thing" to finish device registration. AWS IoT: attach policy

Configure Beam setting

  1. Check your AWS IoT custom endpoint FQDN. Click "Settings" in left menu and copy Custom endpoint FQDN. AWS IoT: Custom endpoint

  2. Open SORACOM user console and click "Groups" in upper/left menu. Click the group you want to configure (if you do not have any group, click "+ Add" to create a new group).

  3. Open "SORACOM Beam" setting and click "+" button, then select "MQTT entry point". Beam: add entry point

  4. Input fields like below.

    • Name: AWS IoT (or anything you like)
    • Protocol: MQTTS
    • Host name: your custom endpoint
    • Port number: 8883
    • Client cert: ON
    • Append IMSI to topic: ON Beam: add entry point
  5. Click '+' button next to "Credentials Set" and register a credentials set.

    • Credentials set ID: raspberrypi (or anything you like)
    • Key: copy/paste from (random)-private.pem.key file
    • Cert: copy/paste from (random)-certificate.pem.crt file (not public.pem.key file!)
    • CA: copy/paste from 'VeriSign-Class 3-Public-Primary-Certification-Authority-G5.key' file Beam: register credentials
  6. Click 'Save' to finish setting.

Test MQTT connection

  1. Open AWS IoT Console and click "Test" in left menu, type # as "Subscription topic" and click "Subscribe to topic". AWS IoT: Subscribe

  2. Use mosquitto_pub command to send data from Raspberry Pi.

Command

mosquitto_pub -d -h beam.soracom.io -t 'beam' -m '{"foo":"bar"}'

Result

pi@raspberrypi:~ $ mosquitto_pub -d -h beam.soracom.io -t 'beam' -m '{"foo":"bar"}'
Client mosqpub/1553-raspberryp sending CONNECT
Client mosqpub/1553-raspberryp received CONNACK
Client mosqpub/1553-raspberryp sending PUBLISH (d0, q0, r0, m1, 'beam', ... (13 bytes))
Client mosqpub/1553-raspberryp sending DISCONNECT

You will see message from device like this. If you this message, configuration is completed. AWS IoT: Data

You can use this MQTT client test tool instead of mqtt.fx.

⚠️ **GitHub.com Fallback** ⚠️