Pairing Devices - dresden-elektronik/deconz-rest-plugin GitHub Wiki

Pairing devices to deCONZ typically involves the following steps:

Joining the ZigBee network;
Reading the device config;
Creating the REST API resources;
Configuring the device;
Setting up the device in the application(s): Phoscon, the old web app, or a plugin for homebridge, Home Assistant, Domoticz, ...

Steps 1 and 2 are done by the deCONZ core programme; steps 3 and 4 are done by the deCONZ REST API plugin; step 5 is done by the application.

1. Joining the ZigBee network

How a device connects to the ZigBee network depends on the type of device:

ZigBee routers (typically mains-powered devices) connect to multiple neighbouring routers, forming the mesh network. The coordinator (the RaspBee or ConBee) is a router (with additional functionality). Routers are coloured yellow in the deCONZ GUI, the coordinator is blue. The radio of a router is always on;
ZigBee end devices (typically battery-powered devices) connect to a single parent router. End-devices are coloured grey in the deCONZ GUI. Typically, the radio of an end-device is off most of the time, so it cannot receive requests. Requests to the device are sent to its parent, who caches the request (for like 7 seconds). When the device wakes, it needs to poll its parent to receive any pending requests. Some devices (e.g. the Hue motion sensor) poll their parent very often, appearing to be reachable all the time; others (e.g. most Xiaomi end devices) only poll once an hour. Some devices poll shortly after sending a command or report, some (again, most Xiaomi end devices) only poll at fixed intervals. See End Device Polling for more details. Because the parent needs to cache the requests for its end devices, a router only supports a limited number of end devices.

Joining the network requires two steps: a) One or more routers in the network need to accept new devices. On opening the network, the coordinator sends a broadcast request to all routers to accept new devices; b) The device needs to be searching for a network to join, i.e. for a router that accepts new connection. Typically, the device needs to be factory reset for this. How this is done, depends on the device: holding a reset button for several seconds, cycling the power in a certain pattern, or using touchlink. Touchlink typically only works for a limited time after the device is powered. Some devices need to be power-cycled after they are reset.

After the device has joined the network, the corresponding node becomes visible in the deCONZ GUI. Sometimes the new node is hidden behind an existing node - you might want to check the Node List panel.

If no new node becomes visible in the deCONZ GUI, power down all you ZigBee routers (except the coordinator), and move the device close to coordinator. If you cannot move the device, use a USB extension cable to move the ConBee, or use a USB power bank to move the Raspberry Pi with the RaspBee. If you're pairing an end-device, make sure the RaspBee or ConBee still has room for new end devices. As of firmware 0x26240500, they support up to 32 end devices; older firmware versions only support up to 10. EDIT It's fewer again with later firmware versions (due to memory restrictions), but I don't know how many.

You can still run the deCONZ GUI over VNC or SSH, when you run deCONZ on a headless server, see Access deCONZ GUI in headless setup.

2. Reading the device config

After the device has joined the network, the deCONZ core programme reads the device capabilities: its MAC (or IEEE) address and its network (or NWK) address, the Node Descriptor (with the list of endpoints), the Simple Descriptor per endpoint. You can force-read these in the deCONZ GUI, from the dropdown under the left of the two grey circles in the node. The IEEE and NWK address appear in the deCONZ GUI in the node; the Node Descriptor can be viewed in the Node Info panel. When the Simple Descriptors are read, a list of endpoints and device type and clusters per endpoint appear in the deCONZ GUI under the dropdown of the right of the two grey circles. Server clusters are blue; client clusters are grey.

If the endpoints and clusters don't appear, seem incomplete or incorrect, make sure you run a recent version of deCONZ. A lot of improvements for reading the config were introduced in version 2.05.42. EDIT There seem to be issues in 2.05.70 and 2.05.71 where deCONZ won't read the Simple Descriptors while the network is open. Close the network (or end the search for new devices), then select the dropdown, then re-open the network after the seconds dropdown appears.

For battery-powered end devices, there's a critical time window in which the config needs to be read. Typically they have an extended polling window after joining a new network, where their radio remains on. After that, they become unresponsive. You might need to trigger the device (pressing a button on switch, moving the magnet of a door sensor, waiving in front of a motion sensor, briefly pressing the reset button) to make sure it remains awake.

3. Creating the REST API resources

The REST API plugin exposes the device as one or more /lights and/or /sensors resources. Historically, all mains-powered routers were lights and all battery-powered end devices were sensors, but, today, this distinction no longer applies. You can best see /sensors as input-only resources (where the state attributes are read-only), and /lights as output (and input) resources (where the state attributes can be written). The uniqueid attribute provides the link between the device and the resource: it contains the MAC address, followed by the endpoint, followed by, for /sensors resources, the cluster.

The REST API plugin only creates resources when:

It knows that the network is open, so you need to open it through the REST API (or through an app using the REST API, e.g. by searching for new devices from Phoscon), instead of from the deCONZ GUI;
It has received the data needed to decide to create a resource.

The REST API plugin creates a /lights resource per endpoint based on:

Receiver On When Idle is set in the _Node Descriptor;
The Device Type from the Simple Descriptor;
The presence of a server OnOff cluster and, optionally server Level Control and Color Control clusters. Effectively, the REST API plugin will create the appropriate /lights resources for standard ZHA, ZLL, and ZigBee 3.0 lights, plug-in units, and wired in-wall switches and dimmers. Other devices (windows covering, vents, door locks, ...), and devices with non-standard device types or descriptors will have to be whitelisted explicitly, requiring a code change to the REST API plugin to support new devices.

The REST API plugin creates a /sensors resource per endpoint/cluster based on:

The MAC address prefix (which indicates the manufacturer);
The Manufacturer ID from the Node Descriptor;
The Model Indentifier from the Basic cluster;
The presence of a relevant client or server cluster. This means that each device needs to be whitelisted explicitly, requiring a code change to the REST API plugin to support new devices. Especially Xiaomi have a tendency to change the Model Identifier, causing newer versions of already supported devices not to be supported.

To support new devices, we need the information mentioned above. This is easiest provided through screenshots of the deCONZ GUI, see Request Device Support.

As with step 2, Reading the device config, there's a critical time for battery-powered devices to receive the information mentioned above. Especially for /sensors resources, the reading of the Model Identifier in the Basic cluster proves to be challenging. You can force-read this from the deCONZ GUI, by selecting Read Attributes in the Cluster Info panel.

4. Configuring the device

There's three ways for deCONZ to get input from devices:

The REST API plugin polls the device: it sends a request to the device to read its attributes and the device responds with the attribute values. For ZLL lights, this is the only way (the ZLL standard explicitly excludes attribute reporting for lights). For end-devices that poll very infrequently, this is very challenging, as the device is unreachable most of the time;
The device sends a report with the attribute values, at regular intervals, or when something has changed. Some devices (e.g. Xiaomi) send reports to the coordinator (NWK address 0x0000) out-of-the-box; other devices need to be told where to send the reports to and under what conditions to send a report;
The device sends a command to associated (groups of) lights to turn on, off, dim, or recall a scene. Some devices by default send a command to all lights (using a broadcast), other devices need to be told where to send the command. For most devices the commands are fixed (maybe influenced by a setting on the device), pro devices (e.g. ubisys) allow the commands to be configured on the device. The REST API plugin eavesdrops on these commands, and reverse-engineers the event that caused the command.

Note that linking a switch to a group in Phoscon (or the old web app) only works for devices that send commands. Most wireless switches do, except the Xiaomi switches, which only send attribute reports. Devices that send attribute reports require rules on the gateway to control lights. These rules are created through the REST API.

Telling a device where to send attribute reports or commands is called a binding in ZigBee speak. Normally the REST API plugin should create these bindings, but you can create them manually in the deCONZ GUI, using the Bind Dropbox panel. A binding is created from a cluster (on an endpoint) of the sending device to either the endpoint of the receiving device or to a group. For attribute reports, the binding needs to be from a server cluster to endpoint 0x01 of the coordinator (RaspBee or ConBee); commands typically use a binding from a client cluster to a group. As group commands are broadcast, the REST API plugin can eavesdrop on them. However, some devices (e.g. the IKEA Trådfri wireless dimmer) don't support group bindings. In this case a binding to coordinator needs to be created, in addition to a binding to (the endpoint containing a server cluster of) each associated light. See also here.

Telling a device how often and under what conditions to send an attribute report is done per attribute (per cluster per endpoint). Normally the REST APU plugin configures the reporting, when it receives confirmation that the binding was made, but you can create these manually in the deCONZ GUI, using the Cluster Info panel. Double click on the attribute for a pop-up windows with the reporting configuration.

When attribute reporting has been setup correctly, the deCONZ GUI updates the attribute value in the Cluster Info panel automatically. When attribute reporting or the commands have been setup correctly, the REST API plugin issues the corresponding web socket event(s). You can force reading the device's attributes from the deCONZ GUI (in the Cluster Info panel); there's no way to force reading the device from the REST API.

Again, for battery-powered devices, it might be challenging to setup the bindings or reporting configuration. The REST API plugin tries to do this after a new device is paired, when existing devices are found after restarting deCONZ, and when it finds it does not (or no longer) receive reports.