# Alarm

EMQX has offered a built-in monitoring and alarm functionality for monitoring the CPU occupancy, system and process memory occupancy, number of processes, rule engine resource status, cluster partition and healing, and will raise an alarm in case of system malfunctions. The activation and deactivation of alarms will generate an alarm log and EMQX will publish an MQTT message with the topic of $SYS/brokers/<Node>/alarms/activate or $SYS/brokers/<Node>/alarms/deactivate. Users can subscribe to the topic to receive alarm notifications.

The payload in the alarm notification message is in JSON format and contains the following fields:

FieldTypeDescription
namestringAlarm name
detailsobjectAlarm details
messagestringHuman-readable alarm instructions
activate_atintegerA UNIX timestamp in microseconds representing the activation time of the alarm
deactivate_atinteger / stringA UNIX timestamp in microseconds representing the deactivation time of the alarm. The value of this field for the activated alarm is infinity.
activatedbooleanWhether the alarm is activated

Taking the alarm of high system memory usage as an example, you will receive an alarm message like below:

alarm massage

One system multifunction will be repeatedly reported. That is, if one alarm on high CPU usage alarm is activated, the system will not generate another alarm of the same type. The generated alarm will be automatically deactivated when the monitored metric returns to normal, or you can manually deactivate the alarm.

Users can view the current alarms and historical alarms on the Dashboard, or use EMQX HTTP API to query and manage alarms.

Users can use the configuration items to customize the alarm feature to meet their business needs. The following configuration items are currently available:

Configuration itemTypeDefault ValueDescription
sysmon.os.cpu_check_intervalduration60sCheck interval for CPU usage;
sysmon.os.cpu_high_watermarkpercent80%The high watermark of the CPU usage, the threshold to activate the alarm;
sysmon.os.cpu_low_watermarkpercent60%The low watermark of the CPU usage, the threshold to deactivate the alarm;
sysmon.os.mem_check_intervalduration60sCheck interval for memory usage;
sysmon.os.sysmem_high_watermarkpercent70%The high watermark of the system memory usage. The alarm will be activated when the total memory occupied reaches this value.
sysmon.os.procmem_high_watermarkpercent5%The high watermark of the process memory usage. The alarm will be activated when the memory occupied by a single process reaches this value.
sysmonn.vm.process_check_intervalduration30sCheck interval for the number of processes;
sysmon.vm.process_high_watermarkpercent80%The high watermark of the process occupancy rate; The alarm will be activated when this threshold is reached; Measured as a ratio of the number of created processes/maximum number limit.
sysmon.vm.process_low_watermarkpercent60%The low water mark of the process occupancy rate; The alarm will be deactivated when it goes below this threshold; Measured as a ratio of the number of created processes/maximum number limit.
sysmonn.vm.long_gcdisabled/durationdisabledWhether to enable Long GC monitoring
sysmon.vm.long_scheduledisabled/durationdisabledWhether to enable Long Schedule monitoring
sysmon.vm.large_heapdisabled/bytesizedisabledWhether to enable Large Heap monitoring
sysmon.vm.busy_portbooleantrueWhether to enable Busy Distribution Port monitoring
sysmonn.top.num_itemsinteger10Number of top processes per monitoring group
sysmon.top.sample_interlvalduration2sCheck interval for top processes
sysmon.top.max_procsinteger1000000Stop collecting data when the number of processes in the VM exceeds this value
sysmonn.top.db_hostnamestring""Hostname of the PostgreSQL database
sysmonn.top.db_portinteger5432Port of the PostgreSQL database
sysmon.top.db_usernamestring"system_monitor"Username of the PostgreSQL database
sysmon.top.db_passwordstring"system_monitor_password"Password in the PostgreSQL database
sysmon.top.db_namestring"postgres"Name of the PostgreSQL database