Rules + Memory Usage

EMQX
1

I’m testing some scenarios to evaluate the pub/sub performance when there is a large set of rules to run through for each message. I’m running into a couple of issues and am looking for some input. Note that I’m running EMQX 5.0.23 locally via docker.

  1. After creating roughly 4k rules, I connect with a single client and publish a single message. This results in the client auto-disconnecting and the log indicates that this is due to “max heap size reached”:
    2023-04-24 10:46:05 2023-04-24T14:46:05.683136+00:00 [error] Process: <0.16617.0> on node 'emqx@172.17.0.2', Context: maximum heap size reached, Max Heap Size: 6291456, Total Heap Size: 6553052, Kill: true, Error Logger: true, Message Queue Len: 0, GC Info: [{old_heap_block_size,2487399},{heap_block_size,4145666},{mbuf_size,0},{recent_size,1043533},{stack_size,50},{old_heap_size,0},{heap_size,1992820},{bin_vheap_size,131116},{bin_vheap_block_size,121527},{bin_old_vheap_size,0},{bin_old_vheap_block_size,121527}]
    What are the memory units printed above? Is there any way to increase the size? I tried allocating more memory to the container but it did not appear to make much of a difference. And what is the relationship between max and total heap size?

  2. I am using the rule REST API to manage the rules. I notice the throughput for creating rules slows to a crawl after creating roughly 1k rules. Is there some API throttling behind the scenes?

Thanks in advance.

2

Hello,

What are the memory units printed above?

These are in machine words (8 bytes for 64-bit system)

Is there any way to increase the size?

Yes, also you can disable this protection completely, although it’s not recommended:

I notice the throughput for creating rules slows to a crawl after creating roughly 1k rules.

I don’t think there is throttling, it’s just adding rules getting slower, as we usually don’t expect to handle this many rules…
The problem with having so many rules is that each rule runs for each published message. That’s going to be 4000 invokations of the rule engine for each publish. Could you describe your problem, so we could, perhaps, come up with an optimization?

3

Thanks for the reply.

Could you describe your problem, so we could, perhaps, come up with an optimization?

We’d like to see if it is possible to take the payload from 1 or more topics, enhance it with some data, then republish it to another topic. This could result in many unique rules since the enhanced data is per customer.