Server Load Balancer: A Solution for Server Reliability Problems

During heavy traffic scenario, slow server response creates a huddle while surfing popular websites is a very common issue. To overcome this problem, the server load balancer comes in picture. A Server Load Balancer is a hardware or virtual software appliance that distributes the application workload across an array of servers, ensuring application availability, elastic scale-out of server resources and supports the health management of backend servers and application systems. This blog will provide information about how and by which algorithms we can solve this server reliability issues during peak hours.

Server Load Balancer…Makes operations Smooth

One of the popular company Amazon might be deploying lots of servers in the backend for the UI path. Different servers are deployed in the Amazon infrastructure which is running the same code on the Amazon UI. Now, how the Requests get into each of the servers? For example, a user using the Amazon website. How does Amazon know to re-direct a user to an instance? And how the new requests need to go to the new instance?
That is when SLB’s are useful. SLB helps to clear the traffic on servers and requests every single user to reach to the destination. SLB does the routing, for example, the user’s request 1 goes to server 1. Request 2 goes to the different server and so on.

Fig: Flow of Requests to different servers

All these requests can be from different users or from same user. So, this is server load balancing.

SLB Components:

SLB consists of the following components:

· SLB Instances

SLB instance is a running load balancing service that distributes the incoming traffic to backend servers. To use the SLB service, one has to create the SLB instance and then configure the instance with at least one listener and two backend servers.

· SLB Listeners

SLB listener checks the client request and proceed the request to backend servers according to the configured rules. SLB listeners also performs the health checks (checks whether the two backend servers work properly or not) on backend server.

· Backend Servers

Backend servers are the ECS instances added to the SLB instance to process the distributed requests. One can add the ECS instances to the default server group, a VServer group or an active server group for better management.

Fig: Overview of SLB Components

Algorithms:

Three different commonly used algorithms which companies prefer for setting up a SLB.

1. Round Robin: Sequential request distribution.

The request of every client goes in a sequence manner where the requests are re-directed to the different services in the Round Robin fashion. So, let’s say there are 4 servers in the network as shown in below diagram. The first request goes to the first server and second goes to the second and so forth. The same fashion is followed for all request in feature cycle. So, this is a Round Robin fashion where every server gets a new request every time.

Connection between description & figure is missing, edit contents.

Fig: Flow of Round Robin

2. Least Connections: Request sent to the least used server in the network.

To do this, the SLB needs to know which process or which server is having the least number of resources. It might take a while for the SLB’s to identify which are the servers, which are having least connections for the least resources. So, it needs to compute, or it needs to get a meta-data information from all the servers. So, this might be a little costlier compare to Round Robin fashion.

Fig: Flow of Least Connection

3. IP Hash: Request sent to the server based on Client IP.

IP hashing is useful when a client sent a request and the client’s request needs to be dedicatedly going to a set of servers i.e., when IP hashing is used. Re-direction can be done based on the client’s IP address and only those servers which are specific to that Client’s IP will be redirected to that network. This can be another strategy where some specific servers need to be given preference over other.

A typical example could be: if you see the IRCTC website from the client’s or general people will be re-directed to a bunch of server farms however the request which are coming from the internal IRCTC receptions that will be going to a different server, that way they will have more performance.

Fig: Flow of IP Hashing

Sticky Sessions (Session Information Persistence):

To understand the sticky sessions in SLB, Amzon.com is one of the best examples where a user using Amazon cart.

User added lots of information into the cart and this cart information is stored in the session which is basically stored in the client site browser. The browser usually caches all the session information and keeps it and every time new request is made this session information are shared.

The browser keeps the session ID which the server has generated the previous time when a user logged in. Same session id is given every time the request is sent.

If the requests are redirected to different servers, then the session id is tending to be lost. Because the first session which was generated by the server 1 might not be visible for the server 2. So, the session information is lost, and users’ cart will be disappeared.

This is the problem where a user has sessions which are persisted in their browsers. This is the general use case where a user will have to handle at their application end.

Fig: Load Balancer with and without Sticky Sessions

How to handle Session Information Persistence problem?

To overcome the problem of session persistence a user can use common cache for example, Redis Cache to store their information or need to allow the SLB to go to the same instance every time.

So, in general, SLB’s have the concept of Sticky Sessions and it do support sticky sessions because it knows that this session id comes for the first time and user can re-direct it to server 1. For the second time if a user gets the same request he/she needs to still re-direct it to server 1. Though a user will not consider the traffic or the other factors into considerations. This is called Session Persistence.

Most SLB’s do handle session Persistence because they know that need to re-direct to the same instance if the request is going to the same from the same client.

In general, let’s say if a user wants to handle it can go for caching methodism where they can use the session id to store in a different cache like Amazon does.

They store the cart information in the separate database and then route the request goes to a new instance and they are retrieving from a common data store.

The only difference between these two approaches are, the read operation might be more because users cart needs to be load on every time. However, if a user doesn’t want to sacrifice the performance then can re-direct the request to a single instance.

This is up to users to decide whether he/she wants to support sticky sessions or whether a user wants to eliminate sticky sessions in their network.[AB1]

Usage of SLB:

1. Distribute incoming traffic to the network by efficiently distributing request across multiple servers.

If there are servers which are unavailable or offline then those servers will be removed from the server load balancing setup and the request will not be re-directed to those servers.

So internally the SLB’s maintains the health check of all the servers which are available, and the requests are re-directed only to the servers which are available.

2. Reliability and high availability are maintained by redirecting requests only to the servers which are available in the network.

3. Ease of use in adding and removing servers in the network as per demand.

For example, let’s say Amazon want to scale up for a great Indian sale or universal sale, which is happening for some special occasion. They know the number of requests which are going to come for the UI are going to be more, so they can scale up and scale down based on the demand. So SLB helps in easily configuring by adding more servers with just a simple configuration change.

Conclusion!

When millions of users are accessing same website during specific time intervals, they face the problems of slow server loading which eventually leads to data loss again and again. In that tricky situations SLB address this issue, distributes all the incoming traffics wisely by creating replicas of servers & overcomes the above said issues. The features of SLB like reliability, time management, data security, data flow, cost-effectiveness makes SLB more popular to solve server reliability problems.

Click2Cloud Inc. is a privately held Cloud Technology Company that delivers innovative solutions and services in cloud infrastructure automation and migration t