Including support for a new sensor - mitra42/frugal-iot GitHub Wiki

LOTS MORE TO GO HERE - STILL A DRAFT - FOR NOW ADD A NEW ISSUE AND WE'LL ENGAGE THERE

Frugal IoT - including support for a new sensor

This page is to show you how to add support for a new sensor. If these instructions don't work, then I am happy to assist.

Prerequisites

I'm going to assume

  • You've used FrugalIoT in some form
  • You can code in C++, and Javascript - at least at beginner level
  • You know how to use an IDE - preferably PlatformIO plugin on Visual Studio, but the repositories work with Arduino IDE as well.
  • You have a Github account

Setup your development environment

This needs expanding

  • Fork the Frugal-IoT Repo
    • On https://github.com/mitra42/frugal-iot
      • in the top right click on Fork
      • The owner should be your account
    • On the page of the fork e.g. https://github.com/yourname123/frugal-iot
      • Click the Code
      • Copy the https URL
    • Your personal workflow for editing this might vary.
      • In PlatformIO
        • New Window
        • Clone Git Repository
        • Paste the URL copied above.

Steps to add support

Lets assume you want to add support for the Xyz sensor,

The main support is added in two files

- src
  - sensor_xyz.cpp
  - sensor_xyz.h

Knowledge of your sensor is added to a few places

  • src
    • _settings.h
    • frugal_iot.h

An example gets built that is just your sensor, talking to the cloud

- examples
  - xyz
    - xyz.ino - simple top level 
    - platformio.ini - construction instructions and constants

And then its added to the UX - more to come here

The first decision is finding a base class you can look at some of the other sensor_*.cpp files at examples, but it usually comes down to a choice between:

  • Sensor_Analog - if you are reading an A2D value from a pin
  • Sensor_Float - if you are returning a single float value
  • Sensor_HT - if you are adding support for a sensor with just Temperature and Humidity (e.g. SHT, or DHT series)
  • Sensor_Uint16 - if you are returning a single, unsigned 16-bit value

Each of these base classes already does most of the work, so you only need to add functions where the behavior differs. Lets go through the ones that are typically overridden

The second decision is which driver to use, there are a bunch of drivers that cover pretty much any sensor. The ones by Rob Tillart are usually good, I tend to avoid the ones from Adafruit which are perfectly good drivers but have a pretty unusual complicated interface - there are currently no examples in the library using them.

You can also write your own driver - in which case there is some functionality that makes it easier to work with I2C, sensor_ensaht and sensor_ms5803 are examples of using this.

Most of the work is in sensor.cpp so we'll start there.

sensor_xyz.cpp

Initialization

The constructor - e.g. Sensor_Xyz::Sensor_Xyz(const char* const id, const char * const name, const uint8_t addr ...)

By convention the first two parameters are the id (which should always stay the same and be a short, lowercase version of your sensor e.g. xyz) and the name, which can be more descriptive, and be edited in the UX.

Sensor_Xyz::setup()

Typically this calls any initialization functions of the library, and the superclass e.g.

void Sensor_BH1750::setup() {
  Sensor_Float::setup(); // Call the superclass - this will also read any persistent configuration
  wire->begin(I2C_SDA, I2C_SCL); // initialize I2C 
  if (!lightmeter.begin(BH1750::Mode::CONTINUOUS_HIGH_RES_MODE, addr, wire)) {  // Initialize the library, and pass it the I2C
    Serial.println(F("Warning: Failed to initialize BH1750 light sensor!")); delay(5000);
  }
}

reading the sensor and setting variables

There is a functionreadValidateCalibrateSet() which follows the typical pattern, and if your sensor is simple you can override just the things unique to it - specifically

float Sensor_Xyz::readFloat() or int Sensor_Xyz::readInt() or uint16_t Sensor_Xyz::readUint16()

Read is overridden for almost all sensors, it should (usually) read the sensor and return a value of the same type (float or uint16_t) as your sensor will return. Here's an example from sensor_bh1750.

float Sensor_BH1750::readFloat() {
  return lightmeter.readLightLevel(); // Actual level (in Lux I think)
}

Here we are calling a function readlightLevel provided by the BH1750 library. Typically the function includes some a debugging statement.

bool Sensor_Xyz::validate(float v) or (int v) or (uint16_t v)

This should return false if the value should be ignored, for example the BH1750 library returns NaN for an invalid value and we can test for this.

// actual value, so convert() not needed
bool Sensor_BH1750::validate(float v) {
  return (!isnan(v));
}

float Sensor_Xyz::convert(float v) .... or float Sensor_Xyz::convert(int v) (for analog reads) or uint16_t Sensor_Xyz::convert(uint16_t)

Some sensors return the result you want, others need a scaling, offseting etc function - Sensor_Analog, and its subclasses, for example do a transformation and return (v - offset) * scale

Its not always as simple as that .... here are some of the functions typically overridden.

void Sensor_Xyz::set(float v) or void Sensor_Xyz::set(uint16_t v) or void Sensor_Xyz::set(int v)

Rarely needs changing - as the default just sets the output's value and sends to the MQTT broker.

void Sensor_Xyz::readValidateConvertSet() for more complex cases

If your sensor is more complex, for example if it reads multiple values from a library then you'll need to override readValidateConvertSet there are examples of this in Sensor_HT or Sensor_MS5803.

Miscellaneous

Captive Portal void Sensor_Xyz::captiveLines(AsyncResponseStream* response)

The sensors can add a small amount of information to the captive portal, so that local users can get a reading by connecting to the device's WiFi. For example Sensors which subclass Sensor_HT (e.g. Sensor_SHT and Sensor_DHT) add the temperature and humidity to the screen.

void Sensor_HT::captiveLines(AsyncResponseStream* response) {
  response->print(String(F("<p><label>")) + name
     + "<br>Temperature: " + temperature->StringValue() 
     + " C<br>Humidity: " + humidity->StringValue() 
     + " %</label>");
}

There are set of functions in sensor_captive.cpp that can be used for this.

Tare and Calibrate

Analog sensors need calibrating - two functions are provided for this: tare() and calibrate(float). By default: tare() presumes the output should be zero, so it takes a reading and saves the value as an offset.
calibrate(float) presumes the output should be a certain value, and calculates a scale to apply againt future readings.

This functionality is activated by set/xyz/output=0 and set/xyz/output=nn.nn, and it is important that they are called in this order, or the scale will be set wrongly.

As a sensor developer, you can provide alternatives to the default tare and calibrate, for example sensor_loadcell.cpp uses the builtin tare and calibrate functions in the HX711 library.

`void Sensor_Xyz::dispatchTwig(const String &topicSensorId, const String &topicTwig, const String &payload, bool isSet)

Some sensors neeed to implement dispatchTwig. This function will be called for each incoming message, and it is the sensor's job to decide if it is going to handle the message or pass it to the superclass.

See the example in sensor_analog.cpp for a sensor which catches an "output=xx" message to do calibration

sensor_xyz.h

Most of sensor_xyz.h is just the signatures of the functions in the .cpp file, and can be filled in when you know what functions you want to overrid, but importantly this is also where you define any local variables, and the outputs - the variables sent to the cloud.

The example below is illustrative, here you'll see it include the superclass (Sensor_Float), define an address for the sensor for different, known situations, if the project doesn't define it. Define some variables, and that is pretty much all that is required in most cases.

#ifndef SENSOR_BH1750_H
#define SENSOR_BH1750_H

#include <Arduino.h>

// Include the superclass
#include "sensor_float.h"

// Include the library we are building on
#include <BH1750.h>

// Define a default for the sensor address that can be used in the main.cpp or *.ino file
#ifndef SENSOR_BH1750_ADDRESS
  #ifdef LILYGOHIGROW
    #define SENSOR_BH1750_ADDRESS (0x23)  // This may be a generally useful default ?
  #else
    #define SENSOR_BH1750_ADDRESS 0x23  // This is the default also in the BH1750 library
  #endif
#endif

// Define the class, in terms of a parent class
class Sensor_BH1750 : public Sensor_Float {
  public:
    // Define its constructor - used in main.cpp or *.ino
    Sensor_BH1750(const char* const id, const char * const name, const uint8_t addr, TwoWire* wire, const bool retain);
  protected:
    // Define some variables
    const uint8_t addr; // I2C address
    TwoWire* wire; // The I2C interface used
    BH1750 lightmeter; // The data from the library

    // Define any functions we override
    void setup() override;
    float readFloat() override;
    bool validate(const float v) override;
};
#endif // SENSOR_BH1750_H

_settings.h

The only part of _settings.h that needs editing, is where it #defines the debugging on the superclasses. e.g.

#if defined(SENSOR_BATTERY_DEBUG) || defined(SENSOR_SOIL_DEBUG)
  #define SENSOR_ANALOG_DEBUG
#endif

This makes sure that when you are testing, it will also output relevant debugging from superclasses.

frugal_iot.h

Add a line that includes your .h file e.g.

#include sensor_xyz.h

example

Finally you should build a simple example, look at examples/sht30 for a template.

example/xyz/xyz.ino

In most cases you'll just swap out the line that adds an SHT sensor with one that adds your sensor by calling its constructor, and the line that gives your sensor a name.

example/xyz/platform.ini

In the [platformio] section, change the name:anddescription: In[common]changebuild_flags_library = to include any specific defines you need, for example your sensor address or pin if it can change. Typically you will also add-D SENSOR_XYZ_DEBUG` to get debugging info for your sensor.

The lower part of this file is specific to each board, usually you'll just change the name of each section e.g. from env:sht30_lolin_c3_pico to env:xyz_lolin_c3_pico and the '-D SYSTEM_OTA_KEY="sht30_c3_pico"' to '-D SYSTEM_OTA_KEY="xyz_c3_pico"'

Library files

library.json

Add a line like { "name": "HX711", "owner": "robtillaart"}, for any external library you call

library.properties

Edit depends to add the libraries you use.

Documentation

In README.md note any libraries you use (search for TO_ADD_SENSOR)

Submitting a PR

(This specific part is untested, please contact us if you have difficulties).

This will be different in Arduino IDE.

  • Push it back to Github
    • On PlatformIO this is the on the bottom row
    • In the terminal on most systems it will just be git push
  • In your repo page on github (e.g. https://github.com/yourname123/frugal-iot)
    • Click Sync fork to make sure your branch includes any new changes since it was forked.
    • Click contribute to submit a PR ("Pull Request")
    • which should notify me that there is code to incorporate.

Adding to the UX

The UX will sometimes, but not always, do a good job of displaying a new sensor, but much better results will come from configuring it to recognize the sensor.

The UI is defined in the [frugal-iot-client](https://github.com/mitra42/frugal-iot-client) repository in webcomponents.js. Follow the same process as above to clone the repo onto your machine, or you can edit it in place on github (where it will prompt you to fork it to your own workspace)

Look for the definition of discover_mod and you'll see each sensor and its topics. The simplest way is to define each topic here, e.g.

lux:
 name: Light meter
 topics:
  - leaf:     lux
    name:     Lux
    type:     exponential
    display:  bar
    min:      0
    max:      65000
    color:    yellow
    rw:       r

or a slightly more complex but abbreviated form can be found below, where for example

discover_mod["sht"] = { name: "SHT", topics: [ d_io_v("temperature"), d_io_v("humidity")]};

the SHT sensor is defined as having two topics temperature and humidity which are define in discover_io.

Once this is working you can submit a PR in a similar manner as for frugal-iot

More to add on testing

I need to write up how to setup a full test environment. Its a little non-trivial and partly depends on the software and system you are running.

⚠️ **GitHub.com Fallback** ⚠️