Newbie MQTT structure question

Hello

I am just getting started with MQTT. I have an embedded system which I am going to add an MQTT Client to, and then I will start working with the MQTT Broker.
I would like to describe the project I am working on, to hopefully get some feedback on the direction of the MQTT set up.
I apologise in advance. I am in the state of not knowing what I don’t know, so I may well be talking MQTT nonsense in the following.

Project Description

We would have a few thousand kiosks in the field
A user can buy a voucher online / in a store, enter the voucher number at the kiosk, and use a service that one time. The number of use-once vouchers would be in the hundreds of thousands. Each voucher would have a unique(say) 16-digit number.
When the voucher number is entered, the kiosk needs to verify (with a remote server) that the voucher number is valid, then provide a service to the user, then notify the remote server of the final voucher status (e.g. “used”).
When receiving the voucher verification request, and the voucher final status, the remote server would also want to know the kiosk ID, location, time/date and some other data.

MQTT

I am trying to figure out how the topics and messaging would be structured.
One way might be for each kiosk to create and publish a topic about itself (id, location etc.), and the remote server to subscribe to all kiosk topics. When a user enters a voucher, the kiosk could create and publish another topic about the voucher – e.g. my_project/kiosk_num/voucher_num?
But I was wondering if each voucher number could be a topic. Each kiosk could create a topic about itself, so that the remote server knows the status of the kiosks. But a kiosk could also create and publish a new voucher topic (e.g. my_project/voucher_num), when a new voucher number is entered. The remote server would subscribe to all (possible) voucher topics.

Thanks for reading this. Any feedback / pointers would be greatly appreciated.
Garrett

Hi.

EMQX can support hundreds of thousands of topics.

However, in your scenario, I do not recommend that you use the ID of each voucher as a topic, because they may only be valid for a short period of time, and too many such topics may cause some trouble for your precise permission control later.

And we also have another way to achieve your purpose, which is to use a common topic, and then use the voucher ID as the message content.

I think you’ve considered this approach, but you don’t seem inclined to choose it. Can you tell me why?

1 Like

Hi Maverick

thank you for our reply.

As I mentioned, I am new to MQTT, so I probably do not know how to best implement MQTT in our project.

The set up I was hoping to achieve was that all the voucher information would be stored in the broker, and that the back-end application would only have to connect to the broker infrequently (once an hour, or once a day).

This is my understanding of the implications of using a common topic, e.g. my-project/voucher-info, with the voucher ID in the message content:

First, the back-end application couldn’t store all the vouchers in the broker, as it could only publish one message to that topic – or maybe I am wrong – maybe it can publish 100,000 messages to that topic.

Second, when a device connects to the broker, because a user has entered a voucher ID, it couldn’t subscribe to the individual voucher topic anymore, and it couldn’t subscribe to the common topic, as it would then et all the 100,000 messages.

So, to use a common topic, it seems to me that the back-end application might have to be permanently connected to the broker, and subscribe to the common voucher-info topic, so that it could respond quickly to a device communicating with the broker.


For further information on what I am trying to achieve, I had snet an email earlier to someone else about it - so I will copy and paste it here.
Thanks,
Garrett


Project Overview

Our customer will have about 1,000 devices in the field – each device has a user interface (screen / keypad etc.).

To use the device, customers first purchase one or more “use once” vouchers.

Each voucher has a unique ID/code – let’s say 12 digits.

The customer enters the voucher number at the device keypad.

The device contacts a TCP server, grabs the voucher information, and determines if it user can use the device or not.

Later, the device will then send more information back to the server.

Currently, there are about 100 devices in the field, and these device communicate with a dedicated TCP server. We are looking to upgrade the systems and use an MQTT broker instead.

Existing System – description

  • The TCP server connects to a database which stores information all vouchers and all kiosks.
  • To use a service, a user purchases a voucher, and then enters the voucher number at a device keypad.
  • The device connects to the server and requests the voucher information.
  • Based on this information, the device either grants or denies the user access to a service.
  • Once the user has finished, the device again communicates with the server to update the status of the voucher (e.g. state change from “ready to use” to “used”).

Proposed System – description

  • A back-end application, acting as an MQTT Client, pre-populates (creates/publishes) one topic for each voucher in the MQTT Broker.
    • Let’s say there will be 100,000 vouchers used in a year – so these could all be created/published at the start, or in batches every month or so.
    • e.g. my-project/voucher-info/123456789012/payload
      • where payload might include the following:
        • voucher-state
        • voucher-type
        • publish-date-time
        • used-date-time
        • expire-date-time
        • device-used
  • When a device first powers up in the field, it would publish its status
    • e.g. my-project/ device-info/5454865/payload
      • where payload might include the following:
        • device-type
        • GPS-location
  • When a user enters a voucher number at a device, the device establishes communication with the MQTT Broker, and subscribes to the voucher topic, according to its unique ID
    • e.g. my-project/voucher-info/123456789012/payload
  • If this voucher/topic exists, the device will receive a message from the broker, with the latest message/payload, which could include:
    • voucher-state = “ready to use”
  • The device then grants the user access to a service.
  • When the user is finished, the device modifies the data received in the voucher payload, including the voucher-state (which is now “Used”) and the device-used fields, and then publishes back to this same voucher-info topic.
    • This would be a persistent message, so that the broker always holds the last received message per topic.
  • If a user was to enter this same voucher code at the same, or another device, the device would again subscribe to this voucher topic, the broker would send the topic message and payload to the device, and the device would see that the vouch-state is already “Used”, and therefore deny the user access to the service.
  • At some stage, e.g. once a day, the back end application can connect with the broker. The back application can subscribe to all voucher and device events
    • e.g.
    • my-project/voucher-info/#
    • my-project/device-info/#
  • In this way, the back-end application would receive one message for every device and voucher whose state has changed
  • Also, at some stage, the back-end application would probably want to remove the “used” vouchers from the broker – perhaps it has to publish an empty payload to the associated voucher topics

This is indeed a solution that I hadn’t thought of before, and it works.

The only thing you might need to watch out for is security issues, such as a malicious client that uses a voucher but doesn’t update the voucher’s state.

Thank you Maverick