I am trying to transfer data from a sensor to our backend over MQTT, and it should not lose data. Setup:
- Self-hosted EMQX cluster, 3 nodes. Max queue size is the default 1000 messages.
- One producer, let’s say it is just a counter publishing messages 1, 2, 3 etc, with an interval of 10 per second. The topic has QoS 1.
- One consumer, that subscribes to the same topic. It’s a persistent session, with a long expiry interval (let’s say 3600 seconds).
- The producer is started, it creates the messages with increasing integers.
- The consumer is started for the first time, it picks up from the most recent integer (
- The consumer (or its connection) is interrupted, it does not consume data after
- The producer keeps on publishing integers.
- The consumer is restarted much later when
2000is published. It receives integers
2000immediately, but lacks
1000because they were dropped out of the queue.
One exception are messages that were ‘in flight’: already sent but not yet acknowledged. These are the last few, so depending on the timing, the consumer may also receive
16 and a few more.
Is there a way that the producer can be made aware that the broker is losing messages?