JMC Research - Juan M. Casillas Web Site

Load Balancer Internals

Load Balancer internal diagram

Published at 18/09/2003 21:36:43
By Juan M. Casillas

In this section I will talk about the internal details of Load Balancer. Load Balancer has been written using C++ and some of the libraries I have developed (mainly libxml (to manage the configuration file), libnetcomm (to manage network connections ), libtool (some helpful code to manage strings, logs, etc), libthreadedserver (to manage the threaded servers)) so you need to have them installed and configured to make Load Balancer in your system. For detailed information, see the libraries? documentation. Load Balancer is built using a thread model, so it can handle thousand of connections per second.

Load Balancer uses heavily the plugin concept. A plugin is a piece of code (in Load Balancer?s case, is a full C++ class) that can be loaded in run time, and does some specific task. So you can write code to extend Load Balancer, without need to recompile Load Balancer, neither touching the Load Balancers? core code. In the following diagram you can see the working model of Load Balancer:

The first thing that Load Balancer does is accept connections from the clients (step 1). All the connections are done using TCP (Note: Load Balancer only supports TCP, because I don?t know if a UDP Load Balancer has meaning. If you think different from me, drop me some lines, I will insert UDP code in Load Balancer!). This first module get all the connections and store them into a FIFO queue (first in first out). This queue is protected using a semaphore structure from race conditions. In this queue only the client information & connection description is stored.

Next, when there are any free thread (a free thread is a thread that isn?t working in a client request) it get the connection request from the queue (step 2) and become to process it. The first thing that the thread does is checking the client is allowed to work with the Load Balancer (there are a filter that allows only connection from the configured addresses -- you can use regular expressions here). (step 3) (This security feature can be activated or deactivated from the configuration file).

Now, a call is done to the proxy plugin code. The proxy plugin also get another call to the chooser plugin, that selects what cluster is going to be used (if we choose a plugin that support per-request cluster switching: rule plugin) and choose a server for the cluster (step 4). Now, the proxy plugin connects the server and the client, and do their task (step 5). When closing the connection, some stats are printed in the Load Balances? log.

Also, Load Balancer has a polling interface, so all the servers are checked using some tests (also in the form of plugins), and some stats are collected and filed in the cluster & server database. Also this polling interface has the ability of take a server unavailable (e.g. when a server is unrecheable) so all the request go to another server. Also can bring back a server when the polling interface detects that this server is available (e.g. the server has been rebooted and it works again).