Dead Simple Deployment

Posted by ezmobius Wed, 26 Apr 2006 05:20:00 GMT

I have been experimenting with mongrel clusters for the past week or so. I tested pound, balance and finaly settled on pen http://siag.nu/pen/ for a load balancer in front of the mongrel cluster. Pen was the fastest out of those three load balancers in my tests, and is dead simple to set up.

I created a pool of 6 mongrels on my macbook pro and ran pen in front of them. I was able to get 800-900req/sec for static files and 70-150 req/sec for dynamic full blown rails pages. It was also rock solid. I abused the hell out of it with httperf and ab and was very impressed with the performance and stability.

I want to say a huge thank you to Zed Shaw (author of mongrel) and the folks supporting his effort finacially. Mongrel is an awesome addition to the ruby world.

Not once did I get any 500 errors or zombied mongrel processes.

Pen can do either round-robin balancing or ‘sticky session’ adaptive routing. ‘sticky sessions’ means that you can get session affinity between requests. But in my tests round-robin was faster by a little bit. And if sessions are in the db or memcached or drb then sticky sessions don’t matter.

Here is a simple way to test this setup out for yourself. First compile pen: 
$ curl -O http://siag.nu/pub/pen/pen-0.17.0.tar.gz
$ tar xzvf pen-0.17.0.tar.gz
$ cd pen-0.17.0
$ ./configure
$ make
$ sudo make install
Lets install the mongrel & mongrel_cluster gems 
$ sudo gem install mongrel mongrel_cluster --include-dependencies
mongrel_cluster is a gem plugin for mongrel that makes it easy to manage a cluster of mongrels. Thanks to Bradley Taylor for making this available. You can read his blog about it here: http://fluxura.com/articles/2006/04/24/easy-mongrel-clustering-with-mongrel_cluster. Lets configure our rails app to use it. cd into your rails application’s root directory and run: 
$ mongrel_rails cluster::configure -p 5000 -e production -a localhost
This will create the file: 
$ cat config/mongrel_cluster.yml
--- 
num-procs: 1024
docroot: public
timeout: 120
cwd: /Users/ez/ez
log_file: log/mongrel.log
port: 5000
config_script: 
debug: false
environment: production
servers: 6
pid_file: log/mongrel.pid
address: localhost
mime_map:
This is the config file that the cluster::configure just made. The only thing I changed was the number of servers: which i set to 6. Now lets start the mongrel cluster: 
$ mongrel_rails cluster::start
You can now stop or restart the cluster with these commands: 
$ mongrel_rails cluster::stop
or
$ mongrel_rails cluster::restart
Then once the cluster is running we can fire up pen. This command runs pen in debug mode in the foreground so you can see the requests being handled by different mongrels. This is also using round robin. If you turn off -d(debug) and -f(foreground) it will be a little bit faster still. 
$ sudo pen -r -a -f -d  localhost:80 localhost:5000 localhost:5001 \
localhost:5002 localhost:5003 localhost:5004 localhost:5005 localhost:5005

With pen running in the foreground and debug mode like this you can see pen balancing requests between our six mongrels. You will want to turn off the -d and -f switches when you want to run in production.

Then you can go use ab or preferably httperf to test the setup. I think you will agree this little pack of wolves…err.. mongrels is hungry! ;-)

I am very impressed with this setup and it consumes very little resources. Pen is the fastest, lightest HTTP load balancer that can be a front end for https as well, that I could find for free. Its about 5-10% faster then balance and pound in my tests. And I am getting better performance with this setup then I do with lighttpd->standalone fcgi listeners on my macbook.

Very cool stuff.

EDIT: With this type of setup you will need one IP address for each app you want to run. Or you can put lighty or apache in front of pen to use vhosting capabilities. I will post more instructions on how to do this if people need to see them.

Tags , , ,  | 69 comments