AWS IoT - seurat-atreides/Sonoff-Tasmota GitHub Wiki

Connecting Tasmota to AWS IoT

This feature is still experimental, and undergoing thorough testing to check stability (i.e. that memory pressure does not cause a crash).

Benefits

AWS IoT provides secure, bi-directional communication between Internet-connected devices such as sensors, actuators, embedded micro-controllers. This basically allows to communicate in both direction from the cloud using MQTT over secure channels using TLS.

More Alexa controls

It's actually easy to develop smarthome Alexa skills, so that you can control your whole house. Currently you can only use the local Philips Hue/Wemo emulation - limited to lights and switches. You can imagine for instance controlling your Sonoff RF Bridge and send IR codes to your TV.

Alexa skills need to communicate back to your devices, which is easy using MQTT and AWS IoT

No need for a local gateway

Of course you can do it with a local gateway like Raspberry PI using many of the open-source solutions (Domoticz...).

You can also do it entirely from the cloud without the hassle of managing and updating a local gateway.

On top of it, AWS IoT provides tools to collect and archive your data, automate (AWS IoT things).

Maximum security

Keep in mind that AWS IoT is based with 'security first' in mind. All the data in AWS IoT is your data and is not shared with anyone else.

Communication is done over TLS 1.2 tunnels, using client certificates to authenticate each device. Up to now it was challenging to enable TLS on ESP8266 because of the high memory requirements of TLS.

Thanks to the switch of Arduino to BearSSL and aggressive optimization, the amount of memory needed is as low as 6.0k during normal operation, and an additional 6.6k during connection (TLS handshake). This makes it totally doable with standard 'Tasmota' firmware with Web and Hue emulation activated. You should see more than 20k of memory still available.

Caveats

As mentioned earlier, this is still experimental and we are happy to get your feedback.

AWS IoT requires each Tasmota device to have its own distinct Private Key and Certificate (~800 bytes). Although you could imagine to use the same Private Key in all your devices, this is considered as a very bad practice. You are warned!

Currently the only way to onboard your private key is to custom compile your own firmware, each one with its own private key embedded in C code. We explore later ways to store the keys in EEPROM or somewhere that wouldn't need to burn it into the firmware and that would survive OTA update.

During TLS handshake, a secondary stack of 5.1k is allocated on the heap to allow BearSSL to have enough stack room. Memory is freed at the end of the handshake. Allocating such big chunks of memory can cause issues when heap fragmentation gets too high. During the first testing campaign, I didn't see any crash due to lack of memory - but this is something we need to keep on monitoring. The alternative would be to allocate this memory once and for all, meaning less memory for Tasmota but no possible crash due to fragmentation.

Cost

AWS provides a Free Tier that allows you to use some services for free up to a specific level. For example, it allows you to have 50 devices connecting 24 hours a day exchanging 300 messages per day. For a typical house, there is a good chance the service costs you nothing (the first year).

How to configure?

AWS IoT requires a distinct Private Key and Certificate per Tasmota device. Currently you need to custom compile your Tasmota firmware and burn the Key and Certificate in your firmware. We will later explore how to configure them separately.

Here is a simple guide.

Step 0. Open an AWS Account

If you don't have already one, just follow the guide: https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/

Step 1. Prerequisites

You will need to install/compile the following:

To compile bearssl, do the following:

  • Clone the BearSSL repo: git clone https://www.bearssl.org/git/BearSSL

  • Compile bearssl:

    cd BearSSL
    make
    

Step 2. Configure AWS IoT (to be done once)

Open the AWS Console.

Click on "Services" and select "IoT Core".

Select the AWS Region where you want to locate your data, for ex: "(EU) Frankfurt".

Now we need to create a security policy to allow your Tasmota devices to connect to AWS IoT, publish and subscribe to topics.

On the left panel, click on "Secure" > "Policies". Click on the "Create" button in the upper right corner.

Enter in the "Name" field, enter the name of your policy, for ex: "TasmotaMqttPolicy". Then click on "Advanced mode" Cut and paste the policy below. Click on "Create" in the lower right corner.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "iot:Connect",
        "iot:Publish",
        "iot:Subscribe",
        "iot:Receive"
      ],
      "Resource": "*"
    }
  ]
}

Step 3. Create a Private Key and Certificate (once per Tasmota device)

Do not use the default AWS IoT feature to generate your private key online. It creates a 2048 bits RSA key. Instead we are using elliptic curves keys - they are much smaller in memory than RSA keys (this saves ~1k of memory).

First create an ECC private key for your device (as described in this Blog). Keep you private key in a safe place.

$ openssl ecparam -out tasmota-01.key -name prime256v1 -genkey

Next, using this private key, create a certificate signing request (CSR). When asked enter teh certificate details. This is not really used later, you can just enter a 2 letters country code like "EU" and leave all other fields blank (type 8 times enter).

$ openssl req -new -sha256 -key tasmota-01.key -nodes -out tasmota-01.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) []:EU <enter>
State or Province Name (full name) []: <enter>
Locality Name (eg, city) []: <enter>
Organization Name (eg, company) []: <enter>
Organizational Unit Name (eg, section) []: <enter>
Common Name (eg, fully qualified host name) []: <enter>
Email Address []: <enter>

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []: <enter>

Next ask AWS IoT to sign your key with its certificate. On the left panel, click on "Secure" > "Certificates". Click on "Create a certificate".

Then choose the "Create with CSR" button, locate your CSR file from above. Click on "Upload file". Download the certificate file, click on "Download", save the file as "tasmota-01.cert.pem".

Important: don't forget to click on the "Activate" to activate the certificate.

Your Private Key and Certificates are ready to use. Now we need to register the Tasmota Device.

Step 4. Write down your AWS IoT endpoint (same for all devices)

Click on "Settings" in the left panel. You should see a field called "Custom endpoint". Write down the endpoint domain name. It should look like this (if you have chosen the Frankfurt region:

<xxxxxxxxxxxxxx>-ats.iot.eu-central-1.amazonaws.com

This is your MQTT endpoint, the port is 8883 - MQTT over TLS.

Step 5. Register the device in AWS IoT (once per Tasmota device)

Now on the left pane, click on "Manage" > "Things". Click on "Register a thing", then "Create a single thing".

Give your device a name like "Tasmota-01". Scroll down and click "Next" at the botton right. Then click on "Create thing without a certificate".

Now we need to associate the certificate created earlier to your device. In the left panel, click back on "Secure" > "Certificates". Select the certificate created earlier. In the next pane, click on "Actions" in the upper right part. First select "Attach policy", check "TasmotaMqttPolicy" and "Attach". Click again on "Actions" and select "Attach thing", check "Tasmota-01" and "Attach".

Your setup is done in AWS IoT. Let's proceed to the custom firmware.

Step 6. Enable AWS IoT in Tasmota

Using your favorite IDE, create sonoff/user_config_override.h, and add #define USE_MQTT_AWS_IOT. You will need to #undef #define USE_DISCOVERY because mDNS will add too much code size.

Note: TLS handshake takes ~1.2s on ESP8266 @80MHz. You may choose to switch to 160MHz if the power supply of your device supports it. If you do so, handshake time should be ~0.7s.

Try to compile the firmware to make sure everything is good.

This step is only to check compilation goes well. Your firmware is still not usable since it does not contain the Private Key + Certificate.

Step 7. Embed your Private Key + Certificate

You will now need to copy/paste your credentials in the file sonoff/sonoff_aws_iot.cpp. This is where you need the bearssl binary compiled in step 1.

Copy/paste the Private Key first, into lines 46-55:

$ BearSSL/build/brssl skey tasmota-01.key -C
File 'tasmota-01.key': decoding as PEM
(skipping 'EC PARAMETERS')
EC key (curve = 23: secp256r1)

static const unsigned char EC_X[] = {
	0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX,
	0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX,
	0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX
};

static const br_ec_private_key EC = {
	23,
	(unsigned char *)EC_X, sizeof EC_X
};

Now copy/paste your certificate, into lines 69-139:

$ BearSSL/build/brssl chain tasmota-01.cert.pem
Reading file 'tasmota-01.cert.pem': 1 certificate

static const unsigned char CERT0[] = {
	0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX,
	0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX,

[.../...]

	0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX,
	0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX
};

static const br_x509_certificate CHAIN[] = {
	{ (unsigned char *)CERT0, sizeof CERT0 }
};

#define CHAIN_LEN   1

Now compile the Tasmota firmware. Your firmware is now ready.

Step 8. Flash your device

Flash your device the normal way; either through serial or OTA. If you use OTA, first flash a sonoff-minial firmware, then your target firmware.

Step 9. Configure Tasmota device

This is the last step, you need to configure the MQTT parameters. The easiest way is through the web console.

Here is a command per command description. Most commands will trigger a device reboot. You can use backlog to launch all commands at once.

Deactivate MQTT: SetOption3 0

Enter the AWS IoT endpoint. Unfortunately it will not fit in MqttHost field, so you'll need to split into two parts.

First enter the first part of the domain: MqttUser <xxxxxxxxxxxxxx>-ats

Enter the remaining part of the endpoint in MqttHost: MqttHost iot.eu-central-1.amazonaws.com

Set the MQTT port: MqttPort 8883

Optional, change the topic to distinguish the devices from each others: Topic sonoff/Tasmota-01

Normally Tasmota will check the fingerprint of the public key of the server. To ease configuration you are advised to activate the 'learn on first connect feature'. Tasmota will learn the fingerprint during the first connection. To do so use: MqttFingerprint1 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Alternatively you can completely disable fingerprint validation and accept any server. Keep in mind that this allows Man-in-the-Middle interception of your data. To do so use: MqttFingerprint1 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

Finally reactivate MQTT: SetOption3 1

Here is the wrap-up of commands:

SetOption3 0
MqttUser <xxxxxxxxxxxxxx>-ats
MqttHost iot.eu-central-1.amazonaws.com
MqttPort 8883
Topic sonoff/Tasmota-01
MqttFingerprint1 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
SetOption3 1

Keep in mind that AWS IoT does not support 'retained' messages. Whatever the 'retained' configuration in Tasmota, messages are always published as 'retained=false'.

Here is an example of output you should see:

00:00:04 HTP: Web server active on sonoff-4585 with IP address 192.168.1.59
00:00:04 UPP: Multicast (re)joined
21:28:25 MQT: Attempting connection...
21:28:25 MQT: AWS IoT endpoint: xxxxxxxxxxxxx-ats.iot.eu-central-1.amazonaws.com
21:28:26 MQT: AWS IoT connected in 1279 ms
21:28:26 MQT: Connected
21:28:26 MQT: tele/sonoff/LWT = Online
21:28:26 MQT: cmnd/sonoff/POWER =
21:28:26 MQT: tele/sonoff/INFO1 = {"Module":"Sonoff Basic","Version":"6.5.0.14(sonoff)","FallbackTopic":
"cmnd/DVES_67B1E9_fb/","GroupTopic":"sonoffs"}

Step 10. Check end-to-end communication

In the AWS IoT console, click on "Test" in the left panel.

In the "Subscription topic" field, type +/sonoff/# then click on "Subscribe to topic". This will display all MQTT messages received. Type a command in the Web Tasmota console, you should see MQTT message flow.

Enjoy!


Implementation notes

To be completed...

Memory usage

TLS on Tasmota has been aggresively optimised to use as little memory (heap) as possible. It was also optimized to limit code size.

Memory consumption (nominal):

  • BearSSL lib: 1424 bytes (or 1024 bytes for TLS alone)
  • BearSSL ClientContext: 3440 bytes
  • Buffers (1024 bytes in + 1024 bytes out + overhead): 2528 bytes
  • Total = 7.4k (or 7.0k with letsencrypt or regular TLS)

Note: if you use USE_WEBSERVER, your impact is lowered by 2k since the Web log buffer is reduced from 4k to 2k. Overall when activating USE_WEBSERVER, you just see a memory impact of 5.4k.

Memory needed during connection (TLS handshake - fingerprint validation):

  • ThunkStack = 4808 bytes
  • DecoderContext = 1152 bytes
  • PrivateKey + Certificate = 800
  • Total for connection = 6.7k

Memory needed during connection (TLS handshake - full CA validation):

  • ThunkStack = 4808 bytes
  • CA Certificate in RAM = 328 bytes (AWS CA) or 344 bytes (Letsencrypt CA)
  • DecoderContext = 3072 bytes
  • PrivateKey + Certificate = 800
  • Total for connection = 9.0k

Connection Time

ESP8266 is quite slow compared to modern processors when it comes to SSL handshake. Here are the observed performance to connect to an SSL/TLS server, depending on frequency (80MHz or 160MHz):

AWS IoT Connection, with EC Private Key, simple fingerprint validation:

  • 0.7s at 160MHz
  • 1.3s at 80 MHz

AWS IoT Connection, with EC Private Key, full CA validation (easier to configure than fingerprints):

  • 1.0s at 160MHz
  • 1.8s at 80 MHz

Letsencrypt based server (Mosquitto for ex), simple fingerprint validation:

  • 0.3s at 160MHz
  • 0.4s at 80MHz

Letsencrypt based server (Mosquitto for ex), with full CA validation (easier to configure than fingerprint):

  • 0.4s at 160MHz
  • 0.7s at 80MHz

TLS Troubleshooting

Here are the common TLS error:

Error code Description
-1002 Cannot connect to TCP port
-1000  Out of memory error
1 Bad fingerprint
62 X509 not trusted, the server certificate is not signed by the CA (AWS IoT or Letsencrypt)
298 missing client private key
⚠️ **GitHub.com Fallback** ⚠️