esp8266 as a zephyr network interface - linino/zephyr GitHub Wiki

Overview

There are 3 processors on the primo and primo mini boards: an esp8266 (providing wifi connectivity), a nordic nrf52832 or nrf52840 cortex-m4 (main processor) and an stm32 (debug interface, IR led). Zephyr will run on the nrf52. The goal is providing zephyr with a wireless network interface. Physical communication medium is an spi bus where the esp8266 acts as a master and the nrf52 is a slave. Besides the standard spi wires (clock, miso, mosi, cs), an extra line allows the nrf52 to request attention from the esp8266 (an spi slave cannot initiate a transaction).

Signals map (primo mini)

  • SPI CS: NRF52 P0.12 <-> NSS_nRF <-> ESP_GPIO15 <-> ESP8266 MTD0 (pin 13)
  • SPI MOSI: NRF52 P1.09 <-> MOSI_nRF <-> ESP_GPIO12 <-> ESP8266 MTDI (pin 10)
  • SPI MISO: NRF52 P0.11 <-> MISO_nRF <-> ESP_GPIO13 <-> ESP8266 MTCK (pin 12)
  • SPI CLK: NRF52 P1.08 <-> SCK_nRF <-> ESP_GPIO14 <-> ESP8266 MTMS (pin 9)
  • SLAVE READY: NRF52 P0.19 <-> SLAVE_READY <-> ESP_GPIO5 <-> ESP8266 GPIO5 (pin 24)

Wifi drivers in zephyr

All existing wifi drivers in zephyr rely on an external device for socket or network context offloading (see https://events.linuxfoundation.org/wp-content/uploads/2017/12/WiFi-and-Secure-Socket-Offload-in-Zephyr-Gil-Pitney-Texas-Instruments.pdf).

Our goal

Since our board(s) have both bluetooth and wifi, we could potentially have multiple network interfaces to be supported by Zephyr, which would not be possible with sockets offloading. Plus, the esp8266 firmware is under our control (we're writing it), and a simple esp8266 (ieee80211_output_pbuf()) is available for sending raw ethernet packets. So it was decided, as a first draft, to implement a simple l2 network interface which just sends and receives level 2 packets. We're also interested in supporting network management functions (scan for networks, connect, disconnect, ....).

The main issue

At the moment, wifi network interfaces are bound to zephyr devices providing a net_wifi_mgmt_offload API. This can be easily verified in subsys/net/l2/wifi/wifi_mgmt.c. For instance, let's take the wifi_scan() function:

static int wifi_scan(u32_t mgmt_request, struct net_if *iface,
		          void *data, size_t len)
{

        struct device *dev = net_if_get_device(iface);
        struct net_wifi_mgmt_offload *off_api =
	    (struct net_wifi_mgmt_offload *) dev->driver_api;

    if (off_api == NULL || off_api->scan == NULL) {
	    return -ENOTSUP;
    }

    return off_api->scan(dev, scan_result_cb);
}

As can be seen, dev->driver_api is casted to a pointer to a truct net_wifi_mgmt_offload * . On the other hand, since wifi l2 is very similar to ethernet l2, we'd like to use the ethernet l2 infrastructure (also implementing arp), but the l2 ethernet code assumes that network interfaces are bound to devices providing an ethernet API. For instance (subsys/net/l2/ethernet/ethernet.c):

static int ethernet_send(struct net_if *iface, struct net_pkt *pkt)
{
        const struct ethernet_api *api = net_if_get_device(iface)->driver_api;
        struct ethernet_context *ctx = net_if_l2_data(iface);
        ....
}

Our solution

We introduce a Kconfig item named CONFIG_NET_L2_WIFI, which will be enabled by non offloaded wifi drivers. CONFIG_NET_L2_WIFI selects CONFIG_NET_L2_ETHERNET, thus enabling the building of the l2 ethernet infrastructure. The wifi management code shall be patched to expect ethernet like devices when CONFIG_NET_L2_WIFI is defined, and we'll add some function pointers for wifi management to struct ethernet_api. So for instance, the function taking care of wifi connection will become:

#ifndef CONFIG_NET_L2_WIFI
static int do_wifi_connect(u32_t mgmt_request, struct device *dev,
			   struct wifi_connect_req_params *params)
{
    struct net_wifi_mgmt_offload *off_api =
     	    (struct net_wifi_mgmt_offload *) dev->driver_api;

    if (off_api == NULL || off_api->connect == NULL) {
	    return -ENOTSUP;
    }

    return off_api->connect(dev, params);
}
#else /* CONFIG_NET_L2_WIFI */
static int do_wifi_connect(u32_t mgmt_request, struct device *dev,
			       struct wifi_connect_req_params *params)
{
    const struct ethernet_api *eth_api = dev->driver_api;

        if (eth_api == NULL || eth_api->wifi_connect == NULL) {
            return -ENOTSUP;
    }

        return eth_api->wifi_connect(dev, params);
 }
 #endif

static int wifi_connect(u32_t mgmt_request, struct net_if *iface,
		    void *data, size_t len)
{
        struct wifi_connect_req_params *params =
	        (struct wifi_connect_req_params *)data;
        struct device *dev = net_if_get_device(iface);

        NET_DBG("%s %u %u %u %s %u",
	    params->ssid, params->ssid_length,
	    params->channel, params->security,
	    params->psk, params->psk_length);

        if ((params->security > WIFI_SECURITY_TYPE_PSK) ||
            (params->ssid_length > WIFI_SSID_MAX_LEN) ||
            (params->ssid_length == 0U) ||
            ((params->security == WIFI_SECURITY_TYPE_PSK) &&
            ((params->psk_length < 8) || (params->psk_length > 64) ||
            (params->psk_length == 0U) || !params->psk)) ||
            ((params->channel != WIFI_CHANNEL_ANY) &&
            (params->channel > WIFI_CHANNEL_MAX)) ||
            !params->ssid) {
	            return -EINVAL;
        }

        return do_wifi_connect(mgmt_request, dev, params);
    }

    NET_MGMT_REGISTER_REQUEST_HANDLER(NET_REQUEST_WIFI_CONNECT, wifi_connect);

The SPI protocol

Goals:

  • Supporting both network management commands and exchange of regular network packets.
  • Full duplex (i.e.: sending and receiving network packets at the same time).
  • Flexibility to allow for new functionalities in the future.
  • Easy to implement both under zephyr and with no o.s.
  • Reasonable performance.

Messages are split into 32 bytes sub-frames (32 is the size of the spi fifo on the esp8266). A single spi message is divided into two main sections: a header, which is contained in the first 32 bytes sub-frame, and some data:

Frame format:

 <------ 32 ----->|<----- 32 ----->|            |<----- 32 ----->|
 +----------------+----------------+            +----------------+
 |                |                |            |                |
 |    HEADER      |    DATA 0..31  |  ......... | DATA 32*n ...  |
 |                |                |            |                |
 +----------------+----------------+            +----------------+

Since spi is full-duplex, two 32 bytes subframes can be running on the bus in opposite directions at the same time:

            +----------+----------+----------+
MISO --->   |   HDR    |   DAT    |   DAT    |
            +----------+----------+----------+
                       +----------+----------+---------+
MOSI <---              |   HDR    |   DAT    |   DAT   |
                       +----------+----------+---------+

Note that sub-frames are always 32 bytes long, even though a message does not always contain a multiple of 32 bytes.

Frames are divided into requests and replies (see R bit in header description below). A reply can have zero, one or multiple replies (for example one could ask for a periodic update of some measurement and receive periodic replies without issuing more requests, thus saving bandwidth). A request and its replies are all part of the same transaction, which is identified by a 16 bits number (see TRANS# field below)

A header is defined as follows:

   3    2   1   0     
 +----+---+---+---+   0x00
 |                |
 |      MAGIC     |              MAGIC = 0xdeadbeef
 |                |
 +-------+-+------+   0x04
 |       | |      |
 |PROTO  |R| CODE |              PROTO = 16 bits protocol id
 |       | |      |              R = request bit (1->request,0->reply/notif)
 +-------+-+------+   0x08       CODE = 15 bits request/reply code
 |       |        |
 |TRANS# |DATA LEN|              RSVD = reserved 16 bits field
 |       |        |              DATA LEN = number of data bytes
 +-----+-+--------+   0x0c       TRANS# = transaction number
 |     | |        |
 |RSVD |L|  ERROR |              ERROR = ERROR STATUS (16bits, 0 no error)
 |     | |        |		     L = Last reply in transaction.
 +-----+-+--------+   0x10
 |                |
 |     RSVD       |              RSVD: reserved
 |                |
 +----------------+   0x1f

PROTOCOL ENDIANNESS is LITTLE.

Known protocols at the moment:

  • 0x0001 : spi-ipc low level management (probing, alive packet, ....)
  • 0x0002 : wifi management (network scan, connect, disconnect, ...)
  • 0x0003 : network interface (ethernet like).

Examples of future protocols:

  • Socket offloading
  • Generic RPC
  • LininoIO like
  • .....

Existing SPI protocols

spi-ipc management

The following frames are defined at the moment:

ALIVE : send an alive notification to the other end

This frame must be sent both by the master and by the slave every configurable period of time.

     3   2   1   0
  +----+---+---+---+   0x00
  |                |
  |   0xdeadbeef   |
  |                |
  +-------+-+------+   0x04
  |       | |      |
  |0x0001 |0| 0x001|
  |       | |      |
  +-------+-+------+   0x08
  |       |        |
  |TRANS# |    0   |
  |       |        |
  +-----+-+--------+   0x0c       TRANS# = transaction number
  |     | |        |
  |     |0|    0   |
  |     | |        |
  +-----+-+--------+   0x10
  |                |
  |     VERSION    |              Version of spi-ipc-protocol
  |                |
  +----------------+   0x14
  |                |
  |                |
  |                |
  |     RSVD       |
  |                |
  |                |
  +----------------+   0x1f

Network interface (ethernet)

The following frames are defined at the moment:

MAC_ADDR : read interface mac address.

Request message contains no data. Reply contains MAC address, which is 6 bytes long. Request:

    3   2   1   0
  +----+---+---+---+   0x00
  |                |
  |   0xdeadbeef   |
  |                |
  +-------+-+------+   0x04
  |       | |      |
  |0x0003 |1| 0x001|
  |       | |      |
  +-------+-+------+   0x08
  |       |        |
  |TRANS# |    0   |
  |       |        |
  +-----+-+--------+   0x0c       TRANS# = transaction number
  |     | |        |
  |RSVD |0|    0   |
  |     | |        |
  +-----+-+--------+   0x10
  |                |
  |  PROTO RSVD    |              PROTO RSVD: reserved for specific
  |                |              protocols
  +----+---+---+---+   0x20

Reply:

    3   2   1   0
  +----+---+---+---+   0x00
  |                |
  |   0xdeadbeef   |
  |                |
  +-------+-+------+   0x04
  |       | |      |
  |0x0003 |0| 0x001|
  |       | |      |
  +-------+-+------+   0x08
  |       |        |
  |TRANS# |    6   |
  |       |        |
  +-----+-+--------+   0x0c       TRANS# = transaction number
  |     | |        |
  |     |L| ERROR  |              ERROR = error status 
  |     | |        |              L = Last reply in transaction
  +-----+-+--------+   0x10       Should be 0 on first reply.
  |                |
  |      RSVD      |              RSVD: reserved
  |                |
  +----+---+---+---+   0x20
  |    |   |   |   |
  | M3 |M2 | M1| M0|              M0 : lsb of mac
  |    |   |   |   |              M5 : MSB of mac
  +----+---+---+---+   0x24
  |        |   |   |
  |  RSVD  | M5| M4|
  |        |   |   |
  +--------+---+---+   0x28
  |                |
  |                |
  |     RSVD       |
  |                |
  |                |
  +----------------+   0x3f

NET_PACKET: transport an ethernet network packet.

There's no request for this message, packets are unsolicited.

     3   2   1   0
  +----+---+---+---+   0x00
  |                |
  |   0xdeadbeef   |
  |                |
  +-------+-+------+   0x04
  |       | |      |
  |0x0003 |0| 0x002|
  |       | |      |
  +-------+-+------+   0x08
  |       |        |
  |TRANS# |   len  |              len = network packet length in bytes
  |       |        |
  +-----+-+--------+   0x0c       TRANS# = transaction number
  |     | |        |
  |     |0|   0    |
  |     | |        |
  +-----+-+--------+   0x10
  |                |
  |     RSVD       |              RSVD: reserved
  |                |
  +----+---+---+---+   0x20
  |    |   |   |   |
  | B3 | B2| B1| B0|              Bn: nth byte of data packet
  |    |   |   |   |              M5 : MSB of mac
  +----+---+---+---+   0x24
  |        |   |   |
  |  ..... | B5| B4|
  |        |   |   |
  +--------+---+---+   0x28
  |                |              n = len / 32 + (len % 32 ? 1 : 0)
  |                |              32 bytes data sub frames, with padding
  |    [PADDING]   |              if needed (i.e. if packet length is not
  |                |              an integer multiple of 32)
  |                |
  +----------------+   (0x20 + 0x20 * n) - 1

START/STOP: start/stop interface

Both requests and replies have no data.

    3   2   1   0
  +----+---+---+---+   0x00
  |                |
  |   0xdeadbeef   |
  |                |
  +-------+-+------+   0x04
  |       | |      |
  |0x0003 |R|  CMD |              CMD = 3 -> start
  |       | |      |              CMD = 4 -> stop
  +-------+-+------+   0x08       R = 1 for request, 0 for reply
  |       |        |
  |TRANS# |    0   |
  |       |        |
  +-----+-+--------+   0x0c       TRANS# = transaction number
  |     | |        |
  |     |L| ERROR  |              ERROR = error status (replies only)
  |     | |        |	      L = Last reply in transaction
  +-----+-+--------+   0x10	      Should be 1 on first reply
  |                |
  |      RSVD      |              RSVD: reserved
  |                |
  +----+---+---+---+   0x20

Wifi management.

SCAN_NETWORK: scan for available networks

Request:

    3   2   1   0
  +----+---+---+---+   0x00
  |                |
  |   0xdeadbeef   |
  |                |
  +-------+-+------+   0x04
  |       | |      |
  |0x0002 |1| 0x001|
  |       | |      |
  +-------+-+------+   0x08
  |       |        |
  |TRANS# |    0   |              len = network packet length in bytes
  |       |        |
  +-----+-+--------+   0x0c       TRANS# = transaction number
  |     | |        |
  |     |0|    0   |
  |     | |        |
  +-----+-+--------+   0x10
  |                |
  |     RSVD       |              RSVD: reserved
  |                |
  +----+---+---+---+   0x20

Reply[ies]: one for each found network:

    3   2   1   0
  +----+---+---+---+   0x00
  |                |
  |   0xdeadbeef   |
  |                |
  +-------+-+------+   0x04
  |       | |      |
  |0x0002 |0| 0x001|
  |       | |      |
  +-------+-+------+   0x08
  |       |        |
  |TRANS# |   42   |
  |       |        |
  +-----+-+--------+   0x0c       TRANS# = transaction number
  |     | |        |
  |RSVD |L| ERROR  |              ERROR = error status
  |     | |        |	          L = last reply bit (1 on last reply)
  +-----+-+--------+   0x10
  |                |
  |     RSVD       |              RSVD: reserved
  |                |
  +----+---+---+---+   0x20
  |                |
  |      SSID      |
  | [ + padding ]  |              0x0 padding needed only if SSID
  +----+---+---+---+   0x40       shorter than 32
  |    |   |   |   |
  |rssi|sec|ch |ssl|              ssl  -> ssid length in bytes
  |    |   |   |   |              ch   -> channel #
  +----+---+---+---+   0x44       sec  -> security
  |    |   |   |   |              rssi -> signal strength
  | B3 |B2 |B1 |B0 |
  |    |   |   |   |              B0 ... B5 -> BSSID
  +----+---+---+---+   0x48       RSVD: reserved
  |		   |   |   |
  | RSVD   |B5 |B4 |
  |        |   |   |
  +--------+---+---+   0x4c
  |                |
  |                |
  |      RSVD      |              RSVD: reserved
  |                |
  |                |
  +----------------+  0x5f

Security (sec) field encoding:

  +-----+--------------+
  |  0  | OPEN         |
  +-----+--------------+
  |  1  | WEP          |
  +-----+--------------+
  |  2  | WPA_PSK      |
  +-----+--------------+
  |  3  | WPA2_PSK     |
  +-----+--------------+
  |  4  | WPA_WPA2_PSK | 
  +-----+--------------+

CONNECT: connect to network

Request:

     3   2   1   0
  +----+---+---+---+   0x00
  |                |
  |   0xdeadbeef   |
  |                |
  +-------+-+------+   0x04
  |       | |      |
  |0x0002 |1| 0x002|
  |       | |      |
  +-------+-+------+   0x08
  |       |        |
  |TRANS# |   len  |              data length: 64 is psk no longer than 32
  |       |        |                           bytes, 96 otherwise.
  +-----+-+--------+   0x0c       TRANS# = transaction number
  |     | |        |
  |RSVD |0|   0    |
  |     | |        |
  +---+-+-+---+----+   0x10
  |   |   |   |    |             ssl -> ssid length in bytes
  |psl|sec|ch |ssl |             ch  -> channel #
  |   |   |   |    |             sec -> security (see previous paragraph for details on encoding).
  +---+---+---+----+   0x14      psl -> password length (8 to 64)
  |   |   |   |    |
  |B3 |B2 |B1 |B0  |
  |   |   |   |    |             B0 ... B5 -> BSSID (don't care if all 0xff)
  +---+---+---+----+   0x18
  |       |   |    |
  | RSVD  |B5 |B4  |             RSVD: reserved
  |       |   |    |
  +---+---+---+----+   0x20
  |                |
  |                |
  |      SSID      |
  | [ + padding ]  |
  |                |
  +----------------+   0x40
  |                |
  |                |
  |      PSK       |
  |                |
  | [ + padding ]  |
  +----------------+   0x60/0x80

Reply:

     3   2   1   0
  +----+---+---+---+   0x00
  |                |
  |   0xdeadbeef   |
  |                |
  +-------+-+------+   0x04
  |       | |      |
  |0x0002 |0| 0x002|
  |       | |      |
  +-------+-+------+   0x08
  |       |        |
  |TRANS# |   0    |              data length: 64 is psk no longer than 32
  |       |        |                           bytes, 96 otherwise.
  +-----+-+--------+   0x0c       TRANS# = transaction number
  |     | |        |
  |RSVD |1| ERROR  |              ERROR: error status (0 -> no error)
  |     | |        |
  +---+-+-+---+----+   0x10
  |                |
  |      RSVD      |
  |                |
  +----------------+   0x1f

DISCONNECT: disconnect from currently connected network

Request/reply:

     3   2   1   0
  +----+---+---+---+   0x00
  |                |
  |   0xdeadbeef   |
  |                |
  +-------+-+------+   0x04
  |       | |      |
  |0x0002 |R| 0x003|              R: 1 for request, 0 for reply
  |       | |      |
  +-------+-+------+   0x08
  |       |        |
  |TRANS# |   0    |              TRANS# = transaction number
  |       |        |
  +-----+-+--------+   0x0c
  |     | |        |
  |RSVD |L| ERROR  |              L: last reply bit (0 for request, 1 for
  |     | |        |              reply)
  +---+-+-+---+----+   0x10       ERROR: error statis (0 for request)
  |                |
  |                |
  |     RSVD       |
  |                |
  |                |
  +----------------+   0x1f

Zephyr side implementation

A modular implementation was preferred. Architecture goes as follows:

+---------------+    +-------------------+
|               |    |                   |
| spi-ipc-ether |    | spi-ipc-wifi-mgmt |        more protocols
|               |    |                   |
+---------------+    +-------------------+
       ^  |                   ^  |               .........
rx_cb()|  | open(),    rx_cb()|  | open()
       |  V submit_buf()      |  V submit_buf()
+----------------------------------------------------------+
|                                                          |
|                 spi ipc device driver                    |
|                                                          |
+----------------------------------------------------------+
                            |
                            |
                            V
                   +----------------+
                   |                |
                   |   spis driver  |
                   |                |
                   +----------------+

A virtual spi-ipc device can be used by specific protocol modules (such as spi-ipc-ether) and provides the following api:

/* Driver API */
struct spi_ipc_driver_api {
	/*
     * Does initialization for a given protocol.
	 */
	 int (*open)(struct device *, const struct spi_ipc_proto *,
                 void *proto_data);
    /*
     * Sends a message, blocks until relevant reply has been received, if
     * applicable
     */
     int (*submit_buf)(struct device *dev, struct net_buf *request,
		               struct net_buf **reply);
};

The open() method registers a new protocol described by a struct spi_ipc_proto:

/*
 * High level protocol descriptor
 */
struct spi_ipc_proto {
    const char *name;
#define HAVE_SPLIT_TRANSACTIONS	
    int flags;
    uint16_t proto_id;
    /* Rx notifications callback (unsolicited messaeges) */
    void (*rx_cb)(const struct spi_ipc_proto *, struct net_buf *in,
	              void *proto_data);
    /* Pointer to protocol private data */
    void *priv;
};

where proto_id is the protocol identifier (PROTO field in the spi ipc header), rx_cb() is a pointer to a callback invoked on data reception and priv is a protocol specific pointer which passed to rx_cb() as last parameter.

There can be multiple instances of spi_ipc virtual devices, each using a different spi slave driver, but each protocol can be instantiated just once per spi_ipc device.

Low level implementation

A workhorse thread specifically takes care of spi transactions. Such thread is awakened in two cases:

  • When a 32 bytes sub-frame must be transmitted.
  • When a slave spi transaction is completed.

Tx subframe are enqueued using a zephyr k-fifo

                       SPI IPC user

         |                    rx_cb
         |                       ^
=========|=======================|========================================
         V                       |
     submit_buf()                |
         |                       |
         V                       |
     +---------+                 |
     |         |                 |          SPI IPC device driver
     |   tx    |                 |
     |         |                 |
     |  kfifo  |                 |
     |         |                 |
     |         |                 |
     |         |                 |
     +---------+                 |
          |                      |
           \                     |
            \                    |
             |                   |
             V                   |
     +-------------------------------+
     |                               |
     |         spi thread            |
     |                               |
     |                               |
     +-------------------------------+
                      ^
                      |
                      |
               +-------------+
               |             |
               | spis driver |
               |             |
               +-------------+

Two sources of events must be monitored. For this reason, the spi driver is used in asynchronous (i.e. non blocking) mode, and polling is performed on the tx kfifo and on the k_poll_signal used by the spi slave driver to notify clients about transactions completion. We have one kfifo for each spi ipc protocol, and just one spi slave polling signal per spi slave driver. The spi thread is instantiated just once.

Peripheral drivers

Device tree bindings

An "spi-ipc" compatible node lives under an spi bus as a device and contains a node for each virtual device. For instance:

&spi0 {
        status = "ok";
        csn-pin = <12>;
        mosi-pin = <41>;
        miso-pin = <11>;
        sck-pin = <40>;

        spi_ipc0: esp8266-primo@0 {
                status = "ok";
                compatible = "spi-ipc";
                spi-max-frequency = <2000000>;
	            reg = <0>;
                poll-request-gpios = <&gpio0 5 0>;
	            label = "ESP8266-PRIMO";

	            spi-ipc-wifi {
		                compatible = "spi-ipc-wifi";
                };
		};		
};
⚠️ **GitHub.com Fallback** ⚠️