Load balancing methods

Learn about the available methods for balancing traffic between origin servers, how they work, which method is enabled by default, and how to change it. You will also learn about passive and active health checks.

Available load balancing methods

Load balancing is the distribution of traffic between origin servers. It is needed when you have more than one origin, so that the load is shared evenly and servers are not overloaded.

Load balancing between origin servers

We can configure round robin, sticky session, or IP hash. Round robin is enabled by default.

Round robin Sticky session IP hash

Balancing characteristic

Requests are distributed across servers independently of one another.

All requests within one session are sent to the same server.

All requests from one IP address are sent to the same server.

Mechanism

Requests are distributed in turn in proportion to server weights: the higher the weight, the more requests the server receives.

First, round robin selects the server for the client’s first request. Then, all other client requests in the current session are sent to the same server.

The load balancer takes the client IP address, processes it with a hash function, and calculates a result. The resulting number determines the server that receives requests from this IP address.

Benefits

The load is distributed between servers exactly as you configure it.

Better user experience: session state is preserved. For example, a shopping cart in an online store is not lost because the user stays connected to the same server that stores the items added to the cart.

Fewer switching events: the client consistently reaches the same server while their IP address does not change.

Limitations

The method does not retain client information: requests in one session can be sent to different servers. If the application does not store data centrally, user experience may suffer: shopping carts can be lost, users can be signed out, and so on.

If too many clients are assigned to one server, it can become overloaded.

If the IP address changes, for example, when a user enables a VPN, the client can reach another server. The session ends, and the user may lose unsaved data or be signed out.

Passive health check

✅

✅

✅

Can an active health check be enabled?

✅

❌

❌

How to enable and configure

Enabled by default.

You can configure origin servers and their weights in the management console.

Enabled on request through support.

After it is enabled, origin servers are configured through support rather than in the management console.

Enabled on request through support.

You can configure origin servers and their weights in the management console.

Round robin

How it works

Requests are distributed across servers in turn according to their weights. A weight is a number that you specify when adding an origin. The higher the weight, the more requests the server receives. If all weights are equal, requests are divided equally.

Weight does not equal the number of requests. If two servers have weights of 5 and 10, this does not mean that the second server waits until the first receives five requests. The load balancer distributes traffic evenly according to the weights. Since 5 is half of 10, requests are sent at a 1:2 ratio: one request to the first server, two to the second, then one to the first again, two to the second, and so on.

Each server operates in one of two modes: Primary or Backup. By default, traffic is sent only to Primary servers. Backup servers are used if all primary servers are unavailable.

Example

You have three servers: A, B, and C. The table below shows how the first eight requests are distributed with different settings:

Settings Request distribution

A: Primary, weight 1

B: Primary, weight 1

C: Primary, weight 1

A→B→C→A→B→C→A→B

A: Primary, weight 1

B: Primary, weight 3

C: Primary, weight 4

A→B→C→B→C→B→C→C

A: Primary, weight 1

B: Primary, weight 2

C: Backup, weight 1

When A and B are available:

A→B→B→A→B→B→A→B

When A is unavailable:

B→B→B→B→B→B→B→B

When A and B are unavailable:

C→C→C→C→C→C→C→C

How the system determines whether a server is available

A passive health check monitors responses from your server and counts errors with status codes 500, 502, 503, and 504. If there are 50 consecutive errors with these codes, the system considers the server unavailable and removes it from load balancing for three seconds. After three seconds, the server starts receiving requests again.

You can change the 5xx error threshold, which is 50 by default, and the removal duration, which is three seconds by default, in the origin settings.

For round robin, we can enable an active health check. Contact support, and we will configure it.

How to enable and configure

You do not need to enable round robin manually because it is enabled by default. You can change origin server weights in the management console.

IP hash

How it works

In short: the system selects the server that receives a request based on the digits in the client IP address. Requests from the same IP address are sent to the same server. If the server becomes unavailable, requests are temporarily sent to others.

In detail: when a request is sent, the system takes the client IP address and processes it with a hash function. This is a complex formula that includes calculations and value rearrangements, and returns a number. This number is divided by the number of available servers, taking their weights into account. The remainder identifies the server that receives the request.

Example

  1. You have three origin servers:

    • A, weight 1;

    • B, weight 2;

    • C, weight 3.

      Each server receives its own index number, starting from zero. If the weights differ, servers with a higher weight are assigned several index numbers so that they receive more requests. In this example, the index numbers are:

    • 0—A;

    • 1—B;

    • 2—B;

    • 3—C;

    • 4—C;

    • 5—C.

  2. Two clients send requests:

    • The first has IP address 192.0.2.112;

    • The second has IP address 203.0.113.202.

  3. The system processes the IP address with a hash function. For this example, imagine that the function adds all numbers in the IP address. In reality, hash functions are much more complex:

    • For the first client: 192+0+2+112=306;

    • For the second client: 203+0+113+202=518.

  4. The system finds the remainder after dividing the resulting numbers by the number of index numbers. In this example, the numbers range from 0 to 5, so there are six index numbers:

    • For the first client: 306/6=51. The numbers divide with no remainder, so the remainder is 0;

    • For the second client: 518/6=86.3. The remainder is 518-86*6=2.

  5. The system maps the remainders to server index numbers:

    • The first client request, with remainder 0, is sent to server A, which has index number 0;

    • The second client request, with remainder 2, is sent to server B, which has index number 2.

  6. Requests from the first client are always sent to server A, and requests from the second client are always sent to server B.

  7. If server A is unavailable, the first client requests are sent to another available server, B or C. When A becomes available again, the first client interacts only with it again.

A client with one IP address always reaches the same server. If the IP address changes, for example, when the user enables a VPN, the formula can produce a different number and the new request is sent to another server.

How the system determines whether a server is available

A passive health check monitors responses from your server and counts errors with status codes 500, 502, 503, and 504. If there are 50 consecutive errors with these codes, the system considers the server unavailable and removes it from load balancing for three seconds. After three seconds, the server starts receiving requests again.

You can change the 5xx error threshold, which is 50 by default, and the removal duration, which is three seconds by default, in the origin settings.

How to enable and configure

Contact support to configure IP hash load balancing for your resource.

IP hash does not support dividing origins into Primary and Backup servers. By default, all servers participate in load balancing as Primary servers.

If any servers must not receive traffic, contact support and we will exclude them from load balancing.

Sticky session

How it works

The first client requests are distributed using round robin. The system then assigns the client to a server for the duration of the session. The server that receives the first request receives all subsequent requests until the session ends. Even if the client IP address changes, requests are still sent to the same server.

In other words, sticky session is the distribution of client sessions across servers using round robin.

Example

  1. You have three origin servers: A, B, and C.

  2. A client opens a session, and the first request reaches server B. It is selected by round robin based on weights. All subsequent requests from this client in the session are sent only to server B.

  3. The client closes the session.

  4. The client opens a new session. Round robin sends the first request to server A. The remaining requests in the session are also sent to server A.

  5. If the client has a session with server A and it unexpectedly becomes unavailable, the client is moved to an available server selected by round robin, for example, server C. From then on, all requests are sent to C until the session is closed.

How the system determines whether a server is available

A passive health check monitors responses from your server and counts errors with status codes 500, 502, 503, and 504. If there are 50 consecutive errors with these codes, the system considers the server unavailable and removes it from load balancing for three seconds. After three seconds, the server starts receiving requests again.

You can change the 5xx error threshold, which is 50 by default, and the removal duration, which is three seconds by default, in the origin settings.

How to enable and configure

Contact support to configure sticky session load balancing for your resource.

Sticky session is available only for protection configurations in which SSL/TLS traffic is decrypted by Strictera.

Health checks

What is a health check?

A health check automatically determines which of your servers are currently working. If some servers become unavailable, the health check detects this and we send user requests only to working servers.

Passive and active health checks: the difference

A passive health check is enabled by default. It relies on actual responses to users. If a server returns 50 consecutive 5xx errors, it is removed from load balancing for three seconds. You can change the error threshold and recovery timeout in origin settings. The drawback is that some requests may still be sent to an unavailable server, causing delays.

An active health check is enabled on request. It sends test requests to servers. If there is no response, it immediately removes the server from load balancing. Its benefit is that users do not experience delays because traffic is distributed only to available servers.

How an active health check works

We support three check types:

  • HTTP—the system sends HTTP requests to a server and checks whether the response is valid;

  • TCP—the system checks whether a connection can be established without sending data;

  • SSL Hello—the system checks whether the server can complete the SSL handshake.

If a server does not respond or returns an error, it is temporarily removed from load balancing. When it passes the checks again, it returns to the pool.

We configure the following parameters based on your requirements:

  • The number of failed checks allowed before a server is removed;

  • The number of successful checks needed to return a server to load balancing;

  • The response timeout.

Example

You have three origin servers: A, B, and C.

Configuration: HTTP requests; remove a server after two failed checks; return it after three successful checks. The check interval is 10 seconds.

  1. 12:00:00—the system sends requests. Server B does not respond.

  2. 12:00:10—the check is repeated. Server B does not respond again and is removed from load balancing. Traffic is distributed only between A and C.

  3. 12:00:10–12:29:50—server B does not respond to any check. Traffic is distributed between A and C.

  4. 12:30:00—server B responds again. The system records the first successful response.

  5. 12:30:10—the second successful response.

  6. 12:30:20—the third successful response. Server B returns to the pool, and traffic is distributed between A, B, and C.

How to configure

Active health checks are available only for round robin load balancing. To enable them, contact support. We will configure everything.