# Redis
EMQX supports integration with Redis so you can save client messages and events to Redis. With Redis data bridge, you can use Redis for message caching and statistics of published/subscribed/discarded messages.
TIP
EMQX Enterprise Edition features. EMQX Enterprise Edition provides comprehensive coverage of key business scenarios, rich data integration, product-level reliability, and 24/7 global technical support. Experience the benefits of this enterprise-ready MQTT messaging platform (opens new window) today.
Prerequisites
- Knowledge about EMQX data integration rules
- Knowledge about data bridge
# Feature List
- [Connection pool](./data-bridges.md#Connection pool)
- [Async mode](./data-bridges.md#Async mode)
- [Batch mode](./data-bridges.md#Batch mode)
- Buffer queue
# Quick Start
In this section, we will illustrate how to leverage Redis to:
- Cache the last message of every client;
- Collect the message discard statistics.
# Install Redis
Install and run Redis via Docker:
# Start a Redis container and set the password to public
docker run --name redis -p 6379:6379 -d redis --requirepass "public"
# Access the container
docker exec -it redis bash
# Access the Redis server, use the AUTH command for authentication
redis-cli
127.0.0.1:6379> AUTH public
OK
# Verify the installation
127.0.0.1:6379> set emqx "Hello World"
OK
127.0.0.1:6379> get emqx
"Hello World"
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Now you have successfully installed Redis and verified the installation with the SET
and GET
commands. For more Redis commands, see Redis Commands (opens new window).
# Connect to Redis
We will create two Redis data bridges to illustrate the messaging caching and statistics features. The connection configurations for both data bridges are the same.
- Go to EMQX Dashboard, click Data Integration -> Data Bridge.
- Click Create on the top right corner of the page.
- In the Create Data Bridge page, click to select Redis, and then click Next.
- Input a name for the data bridge. Note: It should be a combination of upper/lower case letters and numbers.
- Set Redis Mode as the business needs, for example, single.
- Input the connection information. Input 127.0.0.1:6379 as the Server Host, public as the Password, and 0 for Database ID.
Now we have configured the connection information, for the Redis Command template, the setting differs a little depending on the feature to use.
# Message Caching
This section will illustrate how to use Redis to cache the last message from every client.
- Finish the Redis connection configuration.
- Configure Redis Command Template: Use the Redis HSET (opens new window) command and hash data structure to store messages, the data format uses
clientid
as the key, and stores fields such asusername
,payload
, andtimestamp
. To distinguish it from other keys in Redis, we add anemqx_messages
prefix to the message and separate it with:
# HSET key filed value [field value...]
HSET emqx_messages:${clientid} username ${username} payload ${payload} timestamp ${timestamp}
2
- Advanced settings (optional): Choose whether to use sync or async query mode as needed. For details, see Configuration parameters.
- Then click Create to finish the creation of the data bridge. A confirmation dialog will appear and ask if you like to create a rule using this data bridge, you can click Create Rule to continue creating rules to specify the data to be saved into Redis. You can also access the Rule page by clicking Data Integration -> **Rules **on EMQX dashboard.
# Create Rules
- Click Create on the top right corner of the page.
- Input
cache_to_redis
as the rule ID, and set the rules in the SQL Editor. Here we want to save the MQTT messages under topict/#
to PostgreSQL, we can use the SQL syntax below. Note: If you are testing with your SQL, please ensure you have included all required fields in theSELECT
part.
SELECT
*
FROM
"t/#"
2
3
4
- Then click the Add Action button, select Forwarding with Data Bridge from the dropdown list and then select the data bridge we just created under Data bridge. Then, click the Add button.
- Then click the Create button to finish the setup.
Now we have successfully finished the configuration for message caching.
# Message Discard Statistics
This section will illustrate how to use Redis for message discard statistics.
Note: Except for the Redis Command template and rules, the other settings are the same as the Message caching section.
Redis Command template
Use HINCRBY (opens new window) command below to collect the discarded messages under every topic.
# HINCRBY key field increment
HINCRBY emqx_message_dropped_count ${topic} 1
2
Each time the command is executed, the corresponding counter is incremented by 1.
Rule SQL
EMQX rules define 2 message discarding events, through which the rules can be triggered and recorded in Redis:
Event | Topic | Parameter |
---|---|---|
Messages are discarded during forwarding | $events/message_dropped | $events/message_dropped |
Messages are discarded during delivery | $events/delivery_dropped | $events/delivery_dropped |
The corresponding SQL is as follows:
SELECT
*
FROM
"$events/message_dropped", "$events/delivery_dropped"
2
3
4
# Test
Use MQTT X to send a message to topic t/1
to trigger a message caching event. If topic t/1
does not have any subscribers, the message will be discarded and trigger the message discard rule.
mqttx pub -i emqx_c -u emqx_u -t t/1 -m '{ "msg": "hello Redis" }'
Check the running status of the two data bridges, there should be one new Matched and one Sent Successfully message.
Check whether the message is cached.
127.0.0.1:6379> HGETALL emqx_messages:emqx_c
1) "username"
2) "emqx_u"
3) "payload"
4) "{ \"msg\": \"hello Redis\" }"
5) "timestamp"
6) "1675263885119"
2
3
4
5
6
7
Rerun the test, the timestamp
field should be updated.
Check whether the discarded messages are collected:
127.0.0.1:6379> HGETALL emqx_message_dropped_count
1) "t/1"
2) "1"
2
3
Repeat the test, the number on the counter corresponding to t/1
should also increase.