# Configure Load Balancer

Load Balancer (LB) balances the load among multiple network components and optimizes resource usage to avoid system malfunctions caused by overload. LB is not a mandatory component in EMQX, but it can bring some obvious system benefits, for example:

Balance the load of EMQX to avoid single node overload;
Simplify client configuration, the client only needs to connect to the LB and need not worry about the scaling within the cluster;
Reduce the load of EMQX clusters by TLS/SSL termination;
Improve cluster security, with LB configured at the front end of the cluster, unwanted traffic can be blocked to protect the EMQX cluster from malicious attacks.

This chapter introduces how to configure LB in EMQX.

# Architecture

For an EMQX cluster configured with LB, the LB handles the incoming TCP traffic and then distributes the received MQTT connection requests and messages to different EMQX nodes. The typical deployment architecture is as follows:

If SSL/TLS is enabled, it is recommended to terminate the SSL/TLS connection at LB, that is, to use SSL/TLS to secure the connection between clients and LB and then use TCP connection between LB and EMQX nodes, maximizing the performance of the EMQX cluster. The architecture is as follows:

TIP

You can also use DNS polling for load balancing for test or development purposes.

# Select an LB Product

Many load-balancing products are currently available, including open-source and commercial editions, and public cloud providers also have their load-balancing services.

LB products for public cloud:

Cloud provider	SSL Termination	LB Product
AWS (opens new window)	Yes	https://aws.amazon.com/elasticloadbalancing/?nc1=h_ls (opens new window)
Azure (opens new window)	Unknown	https://azure.microsoft.com/en-us/products/load-balancer/ (opens new window)
Google Cloud (opens new window)	Yes	https://cloud.google.com/load-balancing (opens new window)

LB products for private cloud:

Open-Source LB	SSL Termination	DOC/URL
HAProxy (opens new window)	Yes	https://www.haproxy.com/solutions/load-balancing.html (opens new window)
NGINX (opens new window)	Yes	https://www.nginx.com/solutions/load-balancing/ (opens new window)

The following section takes the HAProxy or NGINX as an example to illustrate how to configure an LB in EMQX cluster.

# Configure HAProxy in EMQX

Suppose you have a cluster with 2 EMQX nodes, with emqx1 on 192.168.0.2 and emqx2 on 192.168.0.3, you can follow the instructions below to add HAProxy as the LB.

# Prerequisites

HAProxy must be installed. For detailed introduction and installation of HAProxy, see HAProxy website (opens new window).
The proxy protocol must be enabled.

# Enable Proxy Protocol

To configure the HAProxy on port 1883, you first need to enable the configuration item proxy_protocol in emqx.conf by setting it to true, also specify the true source IP and the port number of the client:

Code Example:

listeners.tcp.default {
  bind = "0.0.0.0:1883"
  proxy_protocol = true
}

1
2
3
4

Where:

listeners.tcp.default is the name or identifier of the TCP listener configuration.
bind is the network interface and port the LB is bound, default: 1883.
max_connections is the maximum number of concurrent connections allowed by the listener, default: infinity.
proxy_protocol is to enable/disable the proxy protocol, it is a boolean, default: false.

TIP

For proxy protocols and that used in Haproxy, see https://www.haproxy.com/blog/haproxy/proxy-protocol

# Configuration Example

To configure HAProxy as the LB for EMQX and terminate the SSL connection, you can modify /etc/haproxy/haproxy.cfg following the code example below.

Note: The file path may differ based on your installation mode.

listen mqtt-ssl
  bind *:8883 ssl crt /etc/ssl/emqx/emq.pem no-sslv3
  mode tcp
  maxconn 50000
  timeout client 600s
  default_backend emqx_cluster

backend emqx_cluster
  mode tcp
  balance source
  timeout server 50s
  timeout check 5000
  server emqx1 192.168.0.2:1883 check-send-proxy send-proxy-v2 inter 10000 fall 2 rise 5 weight 1
  server emqx2 192.168.0.3:1883 check-send-proxy send-proxy-v2 inter 10000 fall 2 rise 5 weight 1

1
2
3
4
5
6
7
8
9
10
11
12
13
14

In this example, you have a cluster that handles a maximum of 50,000 concurrent connections (maxconn). You want to configure the HAProxy to monitor all incoming traffic encrypted in SSL (with SSL certificate located at /etc/ssl/emqx/emq.pem) on port 8883 and also to terminate the SSL connectoin, using the source load balancing algorithm.

# Configure NGINX in EMQX

Suppose you have a cluster with 2 EMQX nodes, with emqx1 on 192.168.0.2 and emqx2 on 192.168.0.3, you can follow the instructions below to add NGINX as the LB.

# Prerequisites

NGINX must be installed. For a detailed introduction and installation of NGINX, see NGINX website (opens new window).
The proxy protocol must be enabled.

# Enable Proxy Protocol

To configure the NGINX on port 1883, you first need to enable the configuration item proxy_protocol in emqx.conf by setting it to true, also specify the true source IP and the port number of the client:

Code Example:

listeners.tcp.default {
  bind = "0.0.0.0:1883"
  proxy_protocol = true
}

1
2
3
4

Where:

listeners.tcp.default is the name or identifier of the TCP listener configuration.
bind is the network interface and port the LB is bound, default: 1883
max_connections is the maximum number of concurrent connections allowed by the listener, default: infinity
proxy_protocol is to enable/disable the proxy protocol, it is a boolean, default: false.

TIP

For proxy protocols that are used in NGINX, see https://docs.nginx.com/nginx/admin-guide/load-balancer/using-proxy-protocol/ (opens new window).

# Configuration Example

To configure NGINX as the LB for EMQX and terminate the SSL connection, you can modify /etc/nginx/nginx.conf following the code example below.

Note: The file path may differ based on your installation mode.

Code Example:

listeners.tcp.default {
  bind = "0.0.0.0:1883"
  proxy_protocol = true
}

1
2
3
4

Where:

listeners.tcp.default is the name or identifier of the TCP listener configuration.
bind is the network interface and port the LB is bound, default: 1883
max_connections is the maximum number of concurrent connections allowed by the listener, default: infinity
proxy_protocol is to enable/disable the proxy protocol, it is a boolean, default: false.

TIP

For proxy protocols that are used in NGINX, see https://docs.nginx.com/nginx/admin-guide/load-balancer/using-proxy-protocol/.

# Configuration Example

To configure NGINX as the LB for EMQX and terminate the SSL connection, you can modify /etc/nginx/nginx.conf following the code example below.

Note: The file path may differ based on your installation mode.

stream {
  upstream stream_backend {
      zone tcp_servers 64k;
      hash $remote_addr;
      server 192.168.0.2:1883 max_fails=2 fail_timeout=30s;
      server 192.168.0.3:1883 max_fails=2 fail_timeout=30s;
  }

  server {
      listen 8883 ssl;
      status_zone tcp_server;
      proxy_pass stream_backend;
      proxy_buffer_size 4k;
      proxy_protocol on;
      ssl_handshake_timeout 15s;
      ssl_certificate     /etc/emqx/certs/cert.pem;
      ssl_certificate_key /etc/emqx/certs/key.pem;
  }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

In this example, you want to configure the NGINX LB that listens for incoming connections on port 8883 encrypted in SSL, terminates the SSL connection, and then forwards those connections to one of two upstream servers using the stream module.

← Cluster Security Performance Tuning (Linux) →