MQTT is an application layer protocol based on the TCP protocol and work in TCP/IP protocol cluster, and designed and developed by IBM in 1998. MQTT uses a lightweight publish/subscribe pattern for message transmission. It is designed to provide reliable network services for IoT devices in low-bandwidth and unstable network environments. Devices need to connect to the MQTT broker for messaging communication.
Among many IoT protocols, MQTT is accepted by more and more enterprises due to its openness, lightweight, energy-saving, universality, etc. It has become the fact standard of IoT communication protocols. However, due to the particularity of the industry or other reasons, there are a large number of communication protocols that are based on TCP/IP in the IoT industry: the scaled standard industry protocol(such as power, road traffic, industrial control), and the private protocol developed by companies, projects, and even individuals. Both new and old projects may encounter the dilemma of not being able to use the MQTT protocol.
TCP protocol communication involves network programming. Both the client and the server software needs to deal with complex network environments, business logic, and component architecture issues. In actual development, each component needs to balance performance and stability, and also needs to take security and usability into account, which has brought great difficulty to development.
Besides, the private TCP protocol has fewer users due to its relative niche or strong industry specificity, it often lacks an excellent client and server software. With the development of the business, the number of devices increases, the communication traffic exceeds expectations and the functional requirements increase and change, the potential problems in the entire business will be amplified:
As an excellent open source MQTT broker, based on supporting high-performance, large-scale standard MQTT protocol access to millions of devices, and supporting cluster level expansion, EMQ X expanded private TCP protocol access and integrated a large number of out-of-the-box functions: multiple ways of authentication, publish/subscribe ACL control, rule engine message processing, message storage and bridge, complete monitoring operation and maintenance system and external REST API control call, etc.
On the basis of high performance and high scalability, EMQ X opens up the upper-layer application to control the entire life cycle of device connection/communication and data interaction channels, allowing users to quickly complete the development of loT applications.
At present, EMQ X provides a complete private TCP access solution through server adaptation/device-side adaptation:
The connection of the EMQ X TCP private protocol eliminates the difference with the MQTT connection at the application layer. Taking the emqx-tcp plug as an example, some configuration items are as follows:
## Upsream topic ## ## Placeholder: ## - %c: ClientId of access client ## - %u: Username of access client tcp.proto.up_topic = tcp/%c/up ## Downstream topic. After the client access is successful, emqx-tcp will ## subscribe to this topic, to receive the message sent by the EMQ system ## to this type of client ## ## Placeholder:(Same as above) tcp.proto.dn_topic = tcp/%c/dn
From the configuration items, we can see the communication mode of private TCP protocol access:
tcp/%c/up, and other EMQ X clients including applications can subscribe to this topic to receive private TCP protocol device information;
tcp/%c/dnthrough the built-in REST API of EMQ X or any client.
Therefore, we have established two-way communication with the client that has established the private TCP protocol. At the same time, the client can use the built-in authentication components for authentication before accessing. When the message arrives, it can also use the publish/subscribe ACL components for accessing control. In the whole link, the rule engine, message bridge, and persistence components can also process this part of data after configuring relevant rules.
The service that EMQ provides customers with private payment development and customization, also includes private TCP protocol access adaptation projects.
The server adaptation aims to the existing private TCP protocol and adjusts the server function without changing the old network device. After the adaptation, the device can seamlessly migrate and connect to EMQ X. In the EMQ X system layer, the connection layer is responsible for processing the server Socket connection and MQTT protocol codec, and its functions are as follows:
In the current architecture design, to adapt to a specific private TCP protocol, only need to extend the corresponding protocol codec and message processing functions at the connection layer. After the user extends the EMQ X protocol adapter with plugins, the private protocol TCP device can directly access the EMQ X without going through the gateway/proxy server, allowing project development and use to fully enjoy the convenience brought by EMQ X.
For the new device, development and adjustment on the device side was done according to the EMQ private TCP protocol specification provided by EMQ and accessed SDK. After adaptation, the device can seamlessly migrate and connect to EMQ X.
Based on the experience of developing and customizing the private TCP protocol in several mature projects, EMQ has launched a universal access specification based on the TCP private protocol, EMQ private TCP protocol specification. It is more lightweight than the MQTT protocol and provides a complete solution for private TCP Access.
EMQ X TCP includes a private TCP protocol standard and corresponding access plug emqx-tcp. The design principles of the entire specification are as follows:
As an access module close to the end in the specification, emqx-tcp can divide the entire message exchange process into three parts: the client side, broker side and another side, according to its functional logic and the relationship of the entire system.
|<-- Terminal -->|<--------- Broker Side --------->|<--- Others --->| |<- Side ->| |<-- Side -->| +---+ PUB +-----------+ | D | INCOMING +----------+ PUB +---------+ -->| subscriber| | E |----------->| |----------->| |--/ +-----------+ | V | | emqx-tcp | | EMQ X | | I |<-----------| |<-----------| |<-- +-----------+ | C | OUTGOING +----------+ PUB +---------+ \--| publisher | | E | PUB +-----------+ +---+
Currently EMQ X TCP protocol specification and emqx-tcp plugins are distributed with EMQ X Enterprise. The users of EMQ X Enterprise can freely use this part of the function to driver device-side and develop communication, according to EMQ private TCP protocol specification, and private TCP protocol access can be adapted from the device side.
However, there are many kinds of the device side protocols in actual IoT projects. When the old projects or industry-related projects use other private TCP protocol specifications, the emqx-tcp plug is not directly applicable. At this time, it needs to adapt from the server.
The more value of EMQ X TCP protocol specification and emqx-tcp plugs is built on the experience of many projects, providing a mature solution for enterprise private TCP protocol customization.
Only EMQ X Enterprise Edition supports the private TCP protocol access adaptation. After the adaptation, EMQ X Enterprise and corresponding function plugins are delivered. Open source users with Erlang development experience can also develop and implement it by themselves.
Welcome to download and free try EMQ X Enterprise. If you have any requirements on private TCP protocol access customization and other privatization customization, you can contact EMQ: firstname.lastname@example.org.