This (king-kong long) post is a continuation of a mini-guide that has already covered putting your Rails app under SVN version control, and deploying it with Capistrano. Now to explain how to serve it all up on a layered and scalable web server stack using Lighttpd, Pound and Mongrel. But first some obvious questions;
- What is Lighttpd ?
(aka. Lighty) – is an open-source all purpose webserver. With a small memory footprint, effective management of cpu-load, and an advanced feature set (FastCGI, CGI, Auth, Output-Compression, URL-Rewriting and more) LightTPD is the perfect solution for every server that is suffering load problems. Oh – and it can serve up PHP faster than a shiny red truck. - What is Pound ?
Pound is a reverse proxy, load balancer and HTTPS front-end for Web servers. Pound was developed to enable distributing the load among several Web-servers and to allow for a convenient SSL wrapper for those Web servers that do not offer it natively. It comes with different ‘balancing’ settings to easily configure load distribution. - What is Mongrel ?
It is a hybrid Ruby/C HTTP server designed to be small, fast and very secure. Written by Zed Shaw it is intended for hosting Ruby web applications using plain old HTTP rather than FastCGI or SCGI. A ‘Mongrel Cluster’ simply means a collection of Mongrel server proccess (one or more). Mongrel works well with the load-balancing offered by Pound (above), meaning as traffic increases to your Ruby based website, you can easily distribute the load across a number of Mongrel processes in your backend servers. Capistrano also has built in function to support management of your Mongrel Cluster – all good. - What is the point ?
The main reason I switched to this stack was because Dreamhost’s shared hosting environment serving Rails with Apache2/FCGI just wasn’t cutting it. Especially with a memory hungry Typo build and a bunch of sidebars. FCGI processes would crash, create CORE dumps, need to be reaped every hour and generally cause a lot of trouble. Thats not to say that you cant get a Rails site up and running under Apache2/FCGI – many people do it very well. However – when I finally got a new VPS server at Rimuhost (with root access) – I took the opportunity to try Lighttpd and Mongrel out.
So, Lighttpd is small, versatile and can serve up static pages (and PHP) super fast and Mongrel (see above) is better with the Ruby stuff. What we want to do ideally, is to proxy requests to our Rails app to Mongrel → from Lighttpd. We could use Lighttpd’s mod_proxy for this – BUT – as it stands right now, its a little bit, ‘not working right’. So, to get around that (or until it is fixed in Lighttpd 1.5) – we can use the ‘Pound’ proxy to sit between Lighttpd and Mongrel. If I could be bothered drawing a diagram it would probably look something like this – (gv) – clear as mud.
Starting Point
- You have got your Rails app under version control with SVN as described in the first tutorial
- You have got Capistrano working and deploying to your server as described in the second tutorial
- You have root access on your server for installing/configuring Lighttpd, Pound and Mongrel
Install Mongrel on your Server
- Get it’s dependencies, daemons and the gem_plugin. Also install mongrel_cluster, which contains a few tools which make managing a cluster of Mongrel servers easier.
sudo gem install daemons gem_plugin mongrel mongrel_cluster sendfile --include-dependencies - Note: this also includes support for ‘sendfile’ – an important part of the this layered stack – With ‘sendfile’, requests can be sent via a proxy (Pound) to your Mongrel servers directly via the local filesystem – Which is much faster than over a local HTTP pipe. Install sendfile on your server (debian) with;
sudo apt-get install sendfile - I’ll leave configuring Mongrel for now – First just test that mongrel is installed ok, try a ‘whereis’ on it, and then try running it to check its there. If not you may need to add its install directory to your $PATH variable;
whereis mongrel_rails mongrel_rails
Install Pound
- Follow the instructions here, make sure zLib and openSSL are installed first, then install Pound. If (during build/compiling) you experience errors (as I did) relating to libssl and/or libcrypto libraries not being present – then you will also need to install libssl-dev. Then try making and installing zLib/openSSL again. On Debian this would be;
apt-get install libssl-dev - If problems still persist after install (as they did for me!), you can try ‘faking’ the older library file verison with the new like so.
sudo ln -s libssl.so.0.9.7 /usr/lib/libssl.so.0.9.8 - Now configure the Pound config file (in /usr/local/etc/pound.cfg, or maybe in /etc/pound.cfg) – Set up Pound up on your server listening on port 6000, and directing requests onto lets say, 3 Mongrel Servers running on ports 6001, 6002 and 6003 (which I’ll setup next).
ListenHTTP Address 0.0.0.0 Port 6000 Service BackEnd Address 127.0.0.1 Port 6001 End BackEnd Address 127.0.0.1 Port 6002 End BackEnd Address 127.0.0.1 Port 6003 End Session Type BASIC TTL 300 End End End - You can run a check on your Pound configuration with;
sudo pound -v -c - You should now start the Pound proxy with;
sudo pound -v - Note: while Pound is running, it wont output errors or even messages to its own log files, instead you should monitor the system log file to see whats going on;
tail -f /var/log/sys.log (for debian, or system.log in OSX)
Configure Lighttpd
Lighttpd will be the main port of call for all HTTP requests to port 80. It is our main front-facing web server to the general browsing public, and it will pass off requests (for our Rails app) to the Pound proxy on port 6000; Your config/lighttpd.conf (normally in your main Rails directory) should look a little like this;
server.bind = "0.0.0.0"
server.port = 80
server.modules = ( "mod_rewrite", "mod_accesslog", "mod_fastcgi", "mod_compress", "mod_expire", "mod_proxy" )
server.error-handler-404 = "/dispatch.fcgi"
server.document-root = CWD + "/public/"
server.errorlog = CWD + "/log/lighttpd.error.log"
accesslog.filename = CWD + "/log/lighttpd.access.log"
#url.rewrite = ( "^/$" => "index.html", "^([^.]+)$" => "$1.html" )
compress.filetype = ( "text/plain", "text/html", "text/css", "text/javascript" )
compress.cache-dir = CWD + "/tmp/cache"
expire.url = ( "/favicon.ico" => "access 3 days",
"/images/" => "access 3 days",
"/stylesheets/" => "access 3 days",
"/javascripts/" => "access 3 days" )
# lighttpd configured with mod_proxy to only one backend server (pound)
proxy.balance = "fair"
proxy.server = ( "/" => ( ( "host" => "127.0.0.1", "port" => 6000 ) ) )
.
.
... more config stuff ... etc ...
.
- Note: Whatever your lighttpd.conf looks like, the important things to change here are to add mod_proxy to the modules and to add the proxy.balance and proxy.server variables. (You might have noticed I commented out the url.rewrite used in the Typo 4.0 build, for some reason this was causing errors and I havent figured out why yet.)
Configure Mongrel
- Finally we need to setup those 3 Mongrels on ports 6001, 6002 and 6003 – to do so, in your main Rails folder create a /config/mongrel_cluster.yml file like so;
cwd: /u/apps/yourdomainname.com/current port: "6001" environment: production address: 127.0.0.1 pid_file: log/mongrel.pid servers: 3
- Where cwd, is the full path to your Rails ‘current’ working directory – Remember, since we are using Capistrano, this will be the current/ symlink that points to the latest ‘release’ folder. Note the servers: 3, which creates 3 Mongrel processes, just change this number to specify how many Mongrel processes you want running.
- Note: It might be a good time to checkin all these changes to SVN at this point.
Fire it up!
- With Pound already running, we just need to start Lighttpd and our 3 Mongrels. I dont believe it matters in what order you do this, but i’ll start with Lighttpd, from your main Rails directory;
sudo ruby script/server -d
- Now start the 3 Mongrels barking, from within the main Rails directory;
sudo mongrel_rails cluster::start
- Thats it! open up your browser and check your site. You can tail -f to monitor for errors/messages in Mongrel, Pound and Lighttpd like so;
tail -f /var/log/sys.log (will show pound errors and other sys info) tail -f rails_dir/log/mongrel.log (mongrel info) tail -f rails_dir/log/lighttpd.access.log (lighttpd access log) tail -f rails_dir/log/lighttpd.error.log (lighttpd error log)
Help! its not working!
- Use the following command to list running processes that are using open ports on your system;
sudo lsof -i -P - Use the following command to port scan your server;
sudo nmap -sT -O localhost - To start debugging problems, check your logs and try those port mapping commands (above) first !
- Feel free to drop me an email (matt [at] hiddenloop [dot] com)about any part of this tutorial, or post a comment and i’ll try to get back to you.
To Do
You’ll probably want to setup /etc/init.d/ scripts for Lighttpd, Mongrel and Pound to start automatically when your server reboots. Maybe i’ll post about this later. Also some tidying up could be done to extract the Lighttpd and Mongrel configuration files out from inside the Rails app directory. AND ideally, its better practice to create a specific user (rather than root) for running the processes for this stack.
Coming soon
In the next (and last) part of this guide, I’ll go through what it takes to adding a copy of the Mint webstats package into your (now nicely setup) Rails app. Showing you how to;
- make sure it gets ignored by version control
- is skipped from any Capistrano deployments
- is served up directly by Lighttpd and PHP
The procedure for doing this with Mint, applies to any non-versioned/non-rails folder you want to place in your Rails app directory structure. And also explains how to mingle serving PHP and Rails from within this stack.
References
