In the ever-growing Internet of Things, IoT, reality that we live in today, having efficient and reliable communication protocols is fundamental. MQTT, Message Queueing Telemetry Transport, is one of the most established and relied upon protocols within this space. MQTT operates as a lightweight and versatile messaging protocol, specifically designed for resource-constrained systems. This design also allows for the protocol to be easily scaled. These traits pushed MQTT and IoT to new heights together, and it’s currently seen as essential in the embedded systems field among many others.

To see the reason that MQTT is designed as it was, look towards the environment of origin. In the late 1990s, there was high demand for remote monitoring of oil pipelines. Dr. Andy Stanford-Clark and Arlen Nipper wanted to create a reliable communication protocol for this setting. Oil pipelines were often in or run through remote locations and these locations often came with limited infrastructure, extreme geographical and environmental climate conditions, and limited power sources. Because of this, the messaging protocol needed to be designed for devices in a network characterized by low bandwidth, high latency, and unreliable connectivity. Traditional protocols like HTTP, TCP/IP, etc. were not applicable, as they demanded things like continuous connections or substantial data overhead.

The key breakthrough in their solution is MQTT's publish/subscribe model, which allows devices to communicate without the need for a persistent connection. This approach was a response designed to conserve power and bandwidth due to the traits of oil pipelines as previously mentioned. This design also demonstrated simplicity, efficiency, and adaptability, making it an ideal solution for applications with sporadic connectivity leading to its widespread adoption even past oil lines.

The protocol was adopted by the OASIS organization, which works to develop standards for technologies in fields like IoT, which further refined it and standardized it. With this refined version of MQTT, its versatility shined and it helped solve many challenges in many IoT ecosystems. Some examples of fields reliant upon MQTT are: smart homes devices, due to the ease of handling and managing many sensors and then responses to sensors, industrial manufacturing, once again managing many sensors and process control, and anywhere you see a network of embedded systems that must communicate with each other, such as RFID tracking.

Now, to look more specifically at the design, MQTT operates with a publisher-subscriber architecture. In this, we see devices, known as publishers, publish messages to different topics or channels while recipients, known as subscribers, choose topics to receive messages from. These publishers and subscribers can be anything from microcontrollers, laptops, phones, etc. This structure imposes a middleman in the publisher and subscriber relationship, which allows for a sweeping sending of messages without direct connections or the publisher/subscriber devices even knowing of each other's existence.

This middleman is known as a broker. The broker will handle receiving the messages and sending them to the appropriate subscribers based on a list of predefined topics. A publisher will send information for a specific topic, and the broker will receive this message, and then appropriately store and forward it to the subscribers of this topic. Speaking of subscribers, due to the design of MQTT, the publisher doesn’t know of the subscribers existence much less their status. This can lead us to a question of reliability, how do we confirm that the subscribers are not missing messages if that is not an acceptable system behavior? The broker handles this as well as the message pathing.This is referred to as quality of service, QoS, which is individual for each topic.

Figure 1. Here is an example of the interaction between MQTT publishers and subscribers, where the broker is symbolized by the cloud in the center.

QoS levels determine the reliability of message delivery, which can be chosen to match the system’s desired behavior. MQTT supports three levels of QoS: 0 which is at most once, 1 which is at least once, and 2 which is exactly once. At most once means that the message will be sent once, and neither the sender nor receiver will acknowledge a successful delivery. At least once means the sender will continue sending messages until it receives an acknowledgment from the receiver. Exactly once means the sender and receiver will confirm connection before sending the message, so the message is only received a single time. As one would expect, these are in order of increasing overhead, so the QoS level is a trade off and the choice is made based on available resources, performance, and reliability requirements.

The broker will by default only retain/store one message at a time per topic. This allows a subscriber to still receive the most recent publish, even if they were not online or connected to the broker when the publish occurred. A broker will only retain a published message if the publisher marks the message specifically to be retained, meaning that you can have a less recent message be retained in the broker if you want to prioritize this message to offline subscribers. However, if a publisher sends a new message marked to be retained, it will replace the previously retained message. Alternatively, a broker can offer some configurations that will retain a history of published messages, but the owner must specifically design this complex configuration.

We know that when designing MQTT Nipper and Stanford-Clark were specifically trying to solve the issue of intermittent connections characteristic of oil pipelines. From this, another key feature of MQTT evolved and is the last will and testament (LWT). LWT solves the handling of unexpected disconnection events. LWT is a feature that can be used by either the publisher or the subscriber. When a client connects to the MQTT broker, one of the things they can specify is an LWT, and in the case of a sudden disconnection, the client will publish its LWT message to the topic whether it was a subscriber or publisher. This behavior allows clients to communicate their status to other clients, allowing for any emergency steps to be taken if unexpected client behavior is conveyed to the other clients in the system.

With growing concerns about IoT security, MQTT implementations have introduced robust security measures. MQTT supports transport layer security (TLS) and secure sockets layer (SSL) encryption for secure communication, safeguarding data transmission between clients and brokers. TLS/SSL protect brokers and clients with their cryptographic protocols that require an exchange of encryption keys for validation before a secure connection is made. Additionally, authentication mechanisms, such as username/password or client certificates, can be used instead so that only authorized clients can connect to the MQTT broker, preventing unauthorized access and potential data breaches. These security features are crucial in applications where sensitive data, such as personal or financial information, is exchanged between devices.

Many of the benefits of MQTT have already been highlighted when describing the functionality, but it is good to go through and summarize these. Efficient data exchange, reliability, scalability, and real time monitoring are all important benefits of MQTT. It demonstrates an efficient use of bandwidth and device resources and can ensure reliable communication in the event of bad connection thanks to the QoS and configuration. MQTT also allows for data communication to a large and variable number of devices or applications without any change to the system. Finally, MQTT operates in real-time allowing for monitoring and controlling a network of devices.

Another benefit to MQTT is its open source community. The community has developed many open source brokers and clients, which allows for a deep repository of examples and resources to improve the experience of using MQTT. The most notable example of this is Eclipse Mosquitto, an extremely popular open source MQTT broker. You can use this test broker to test whether your code is working as intended, without needing to set up your own broker or download anything for theirs. Additionally, many programming languages or software environments offer additional MQTT libraries, further simplifying use and implementation of MQTT and contributing to its widespread use.

So then the next question is what uses allow us to take advantage of the above benefits. The first thoughts are systems where we have limited network bandwidth, limited battery capacity of devices, a requirement of reliable real-time data exchange allowing for instantaneous changes or responses, and large-scale networks of numerous devices. Of course, the use case doesn’t need to check all these boxes, but these are some of the main appealing use cases. Some real examples are IoT devices, many companies that use embedded systems will need to communicate large amounts of data with constrained memory on systems. Smart home devices also are IoT systems, and these require instant responses to changes in the environment, such as turning on a thermostat or alarms in security systems. Mobile applications also often require easy data transfer between devices and servers, especially in applications that use real-time updates. Of course, there are plenty more examples and beneficial times to take advantage of MQTT, but this was just to give an idea.

Sadly no messaging protocol is one size fits all, so there are also some weaknesses of MQTT and cases where it would be better to pursue other options. Message ordering is not guaranteed across different topics with MQTT, so if you have a system where you need specific message sequences MQTT does not have that feature. In the most extreme of cases, other protocols have even more lightweight and less bandwidth-consuming implementations, so if you need the least bandwidth consumption and you'll only be communicating between a small number of devices, MQTT might be suboptimal.

Along the same lines as the message ordering issue, if you have a system where the messages must be confirmed and processed in order then something that can do those quicker like a message queue could be more appropriate. While not the biggest issue, it should be mentioned that the MQTT implementation can be complicated for someone unfamiliar with the protocol, and more depending on the system you’re implementing from, so someone new to MQTT can probably find an easier protocol to use if they don’t require MQTT.

There are other situations where a different protocol can be used over MQTT, but this was a general idea of some of those situations. Although some situations may have a hyper-specific protocol that suits them, it’s worth mentioning that due to the widespread familiarity with MQTT, many people will often use MQTT anyway. MQTT is extremely variable and can be configured for a wide range of uses, which often causes the slight optimality of a different messaging system to not be worthwhile for people to learn something besides MQTT.

In conclusion, MQTT is one of the strongest figures in the world of messaging protocols. Even with its creation early on in the world of IoT, MQTT has stayed relevant and continued to evolve alongside IoT. IoT and MQTT have mutually pushed each other to new heights and this is only expected to continue. MQTT will continue to offer and enable consistent and reliable flow of information in webs of devices and applications, and should always be heavily considered when choosing a messaging protocol.

It rose through the ranks right from creation due to its low system overhead, efficiency, scalability, and reliability in conventionally undesirable environments. It is arguably the best choice for applications requiring a similar list of specifications.

Although we have described many of the benefits and appealing characteristics of MQTT, it might not always match the needs of an application due to its complexity to set up, and lack of certain customizations. By weighing the advantages and disadvantages covered, one could make a more educated decision on whether MQTT is the best fit for a project.

[1] https://mqtt.org/

[2] https://psiborg.in/advantages-of-using-mqtt-for-iot-devices/

[3] https://mosquitto.org/