With the advent of the 5G era, the great vision of the IoT is becoming a reality. The number of connected IoT devices has reached 7 billion1 in 2018. In the next two years, smart water and electricity meters alone will exceed 1 billion2.
Massive device access and device management have brought great challenges to network bandwidth, communication protocols, and platform service architecture. For the IoT protocol , several key issues of IoT device communication must be specifically addressed, including that its network environment is complex and unreliable, its memory and flash memory capacity is small, and its processing capacity is limited.
MQTT protocol is an IoT communication protocol based on the Publish/Subscribe model. With its characteristics of simple and easy implementation, support for QoS, and small size of packet, it occupies half market of the Internet of Things protocol:
According to Arlen Nipper's self introduction on IBM Podcast , the original name of MQTT was
MQ TT, note the space between
TT, and its full name is MQ Telemetry Transport, which is a real-time data transmission protocol developed by him during a crude oil pipeline data acquisition and monitoring system (pipeline SCADA system) in Conoco Phillips. Its purpose is to allow the sensor to communicate with IBM's MQ Integrator via a limited bandwidth VSAT, Since Nipper is a professional in remote sensing and data acquisition and monitoring, he gave the name
MQ TT according to industry practice.
According to Nipper, MQTT must be simple and easy to implement, and support QoS (the equipment network environment is complex). It must be lightweight and bandwidth-saving (because bandwidth is expensive at that time), and must be data-independent (don't care about the payload data format). It must be continuous session awareness (knowing if the device is online at all times). Several core features of MQTT (version 3.1.1) will be introduced below, which respectively correspond to the implementation of these design principles.
The publish-subscribe model is a decoupling solution of the traditional Client/Server mode. The publisher communicates with the consumer through the Broker. The role of the Broker is to correctly send the received message to the consumer through some sort of
filtering rule. The benefits of the publish/subscribe model over the client/server model are:
In the MQTT protocol, the
filtering rules mentioned above are
Topic. For example: All messages published to the topic
news will be forwarded by broker to subscribers who have subscribed to
In the figure above, the subscribers subscribe to
news, and then the publisher publishes a message "some msg" to Broker and publishes it to the
news topic. The broker decides to forward this message to the subscriber by matching the topic.
MQTT topics have a hierarchical structure and support wildcards
+is a wildcard that matches a single level. For example,
#is a wildcard of one to multiple levels. For example,
MQTT topics are not created in advance. When a publisher sends a message to a topic, or when a subscriber subscribes to a topic, the broker automatically creates the topic.
The MQTT protocol minimizes the additional consumption of the protocol itself, and the message header requires only a minimum of 2 bytes.
The message format of MQTT is divided into three parts:
|Fixed-length header, 2 bytes, available in all types of message|
|Variable-length header, only in certain types of message|
|Payload, only in certain types of message|
The main message types of MQTT are:
Among them, the PINGREQ/PINGRESP and DISCONNECT packets do not require variable headers and there is no Payload, which means that their packet size consumes only 2 bytes.
In the variable-length header of the CONNECT packet, there is a Protocol Version field. To save space, there is only one byte. So the version number is not stored as the string "3.1.1", but the number 4 is used to represent the 3.1.1 version.
In order to adapt to different network environments of the equipment, MQTT has designed 3 QoS levels, 0, 1, 2:
QoS 0 is a "fire and forget" message sending mode: After a sender (possibly Publisher or Broker) sends a message, it no longer cares whether it is sent to the other party or sets up any resending mechanism.
QoS 1 includes a simple resending mechanism. After the Sender sends a message, it waits for the receiver's ACK. If it does not receive the ACK, it resends the message. This mode can guarantee that the message can arrive at least once, but it cannot guarantee that the message is repeated.
QoS 2 designed a slightly complicated resending and repeating message discovery mechanism to ensure the message will arrive only once.
MQTT does not assume that the device or Broker uses the TCP keepalive mechanism4, but designed a keepalive mechanism at the protocol layer: the Keepalive field can be set in the CONNECT packet to set the sending time interval of keepalive heartbeat packet PINGREQ/PINGRESP. When the PINGREQ of the device cannot be received for a long time, the Broker will think that the device is offline.
In general, keepalive has two functions:
For those devices that want to re-receive the messages that they missed during offline after re-online, MQTT has designed a persistent connection: In the CONNECT packet, you can set the CleanSession field to False, then Broker will store the following for the terminal:
MQTT has designed Last Will message to let the broker help the device publish a will message to the specified topic if it finds that the device is offline abnormally.
In fact, in some implementations of MQTT broker (such as EMQX), when the device goes online or offline, Broker publishes device status updates through certain system topics, which is more in line with the actual application scenario.
There are several popular MQTT Brokers so far:
MQTT broker implemented in C. The Eclipse organization also contains a number of MQTT client projects: https://www.eclipse.org/paho/#
MQTT broker developed in Erlang language that supports many other IoT protocols such as CoAP, LwM2M, etc.
MQTT broker developed with Node.JS that is easy to use.
MQTT broker also developed using Erlang
EMQ X MQTT IoT cloud service provides an online public MQTT 5.0 broker. You can quickly start learning, testing MQTT protocol or prototyping.
For the detailed accessing information for this MQTT broker, please visit the EMQ website: free public MQTT broker.
EMQ also provides the MQTT online client tool that can be accessed through a browser. This tool supports connecting to the MQTT broker through the normal or encrypted WebSocket ports, and cache connections for the next accessing.
The number of connected devices that are in use worldwide now exceeds 17 billion, with the number of IoT devices at 7 billion... https://iot-analytics.com/state-of-the-iot-update-q1-q2-2018-number-of-iot-devices-now-7b/ ↩
The estimated installed base of smart meters (electricity, gas and water) is expected to surpass the 1 billion mark within the next 2 years. https://iot-analytics.com/smart-meter-market-2019-global-penetration-reached-14-percent/ ↩
The message retention function of [EMQ X MQTT Broker](https://emqx.io) is implemented by the `emqx_retainer` plugin, which is enabled by default. By modifying the configuration of the` emqx_retainer` plugin, you can adjust the EMQ X Broker's retention message Location, restrict the number of retained messages and maximum payload length, and adjust the expiration time of retained messages.
This article will show you a「date」between MQTT client and CoAP client in the EMQ X world.
In this project, we will implement remote control LED lights via NodeMCU(ESP8266) and MQTT broker, and use the Arduino IDE to program NodeMCU ESP8266.