Matthew Hutchinson

Part3. Lighttpd, Pound and Mongrel with Rails

no comments yet, post one now

This (king-kong long) post is a continuation of a mini-guide that has already covered putting your Rails app under SVN version control, and deploying it with Capistrano. Now to explain how to serve it all up on a layered and scalable web server stack using Lighttpd, Pound and Mongrel. But first some obvious questions;

  • What is Lighttpd ?
    (aka. Lighty) – is an open-source all purpose webserver. With a small memory footprint, effective management of cpu-load, and an advanced feature set (FastCGI, CGI, Auth, Output-Compression, URL-Rewriting and more) LightTPD is the perfect solution for every server that is suffering load problems. Oh – and it can serve up PHP faster than a shiny red truck.
  • What is Pound ?
    Pound is a reverse proxy, load balancer and HTTPS front-end for Web servers. Pound was developed to enable distributing the load among several Web-servers and to allow for a convenient SSL wrapper for those Web servers that do not offer it natively. It comes with different ‘balancing’ settings to easily configure load distribution.
  • What is Mongrel ?
    It is a hybrid Ruby/C HTTP server designed to be small, fast and very secure. Written by Zed Shaw it is intended for hosting Ruby web applications using plain old HTTP rather than FastCGI or SCGI. A ‘Mongrel Cluster’ simply means a collection of Mongrel server proccess (one or more). Mongrel works well with the load-balancing offered by Pound (above), meaning as traffic increases to your Ruby based website, you can easily distribute the load across a number of Mongrel processes in your backend servers. Capistrano also has built in function to support management of your Mongrel Cluster – all good.
  • What is the point ?
    The main reason I switched to this stack was because Dreamhost’s shared hosting environment serving Rails with Apache2/FCGI just wasn’t cutting it. Especially with a memory hungry Typo build and a bunch of sidebars. FCGI processes would crash, create CORE dumps, need to be reaped every hour and generally cause a lot of trouble. Thats not to say that you cant get a Rails site up and running under Apache2/FCGI – many people do it very well. However – when I finally got a new VPS server at Rimuhost (with root access) – I took the opportunity to try Lighttpd and Mongrel out.

So, Lighttpd is small, versatile and can serve up static pages (and PHP) super fast and Mongrel (see above) is better with the Ruby stuff. What we want to do ideally, is to proxy requests to our Rails app to Mongrel → from Lighttpd. We could use Lighttpd’s mod_proxy for this – BUT – as it stands right now, its a little bit, ‘not working right’. So, to get around that (or until it is fixed in Lighttpd 1.5) – we can use the ‘Pound’ proxy to sit between Lighttpd and Mongrel. If I could be bothered drawing a diagram it would probably look something like this – (gv) – clear as mud.

Starting Point

  • You have got your Rails app under version control with SVN as described in the first tutorial
  • You have got Capistrano working and deploying to your server as described in the second tutorial
  • You have root access on your server for installing/configuring Lighttpd, Pound and Mongrel

Install Mongrel on your Server

  • Get it’s dependencies, daemons and the gem_plugin. Also install mongrel_cluster, which contains a few tools which make managing a cluster of Mongrel servers easier.
    sudo gem install daemons gem_plugin mongrel mongrel_cluster sendfile --include-dependencies
  • Note: this also includes support for ‘sendfile’ – an important part of the this layered stack – With ‘sendfile’, requests can be sent via a proxy (Pound) to your Mongrel servers directly via the local filesystem – Which is much faster than over a local HTTP pipe. Install sendfile on your server (debian) with;
    sudo apt-get install sendfile
  • I’ll leave configuring Mongrel for now – First just test that mongrel is installed ok, try a ‘whereis’ on it, and then try running it to check its there. If not you may need to add its install directory to your $PATH variable;
    whereis mongrel_rails
    mongrel_rails

Install Pound

  • Follow the instructions here, make sure zLib and openSSL are installed first, then install Pound. If (during build/compiling) you experience errors (as I did) relating to libssl and/or libcrypto libraries not being present – then you will also need to install libssl-dev. Then try making and installing zLib/openSSL again. On Debian this would be;
    apt-get install libssl-dev
  • If problems still persist after install (as they did for me!), you can try ‘faking’ the older library file verison with the new like so.
    sudo ln -s libssl.so.0.9.7 /usr/lib/libssl.so.0.9.8
  • Now configure the Pound config file (in /usr/local/etc/pound.cfg, or maybe in /etc/pound.cfg) – Set up Pound up on your server listening on port 6000, and directing requests onto lets say, 3 Mongrel Servers running on ports 6001, 6002 and 6003 (which I’ll setup next).
    ListenHTTP
      Address 0.0.0.0
      Port    6000
      Service
        BackEnd
          Address 127.0.0.1
          Port    6001
        End
        BackEnd
          Address 127.0.0.1
          Port    6002
        End
        BackEnd
          Address 127.0.0.1
          Port    6003
        End
        Session
          Type BASIC
          TTL  300
        End
      End
    End
  • You can run a check on your Pound configuration with;
    sudo pound -v -c
  • You should now start the Pound proxy with;
    sudo pound -v
  • Note: while Pound is running, it wont output errors or even messages to its own log files, instead you should monitor the system log file to see whats going on;
    tail -f /var/log/sys.log  (for debian, or system.log in OSX)

Configure Lighttpd

Lighttpd will be the main port of call for all HTTP requests to port 80. It is our main front-facing web server to the general browsing public, and it will pass off requests (for our Rails app) to the Pound proxy on port 6000; Your config/lighttpd.conf (normally in your main Rails directory) should look a little like this;

server.bind = "0.0.0.0"
server.port = 80

server.modules           = ( "mod_rewrite", "mod_accesslog", "mod_fastcgi", "mod_compress", "mod_expire", "mod_proxy" )

server.error-handler-404 = "/dispatch.fcgi"
server.document-root     = CWD + "/public/"
server.errorlog          = CWD + "/log/lighttpd.error.log"
accesslog.filename       = CWD + "/log/lighttpd.access.log"

#url.rewrite              = ( "^/$" => "index.html", "^([^.]+)$" => "$1.html" )

compress.filetype        = ( "text/plain", "text/html", "text/css", "text/javascript" )
compress.cache-dir       = CWD + "/tmp/cache"

expire.url               = ( "/favicon.ico"  => "access 3 days",
                             "/images/"      => "access 3 days",
                             "/stylesheets/" => "access 3 days",
                             "/javascripts/" => "access 3 days" )

# lighttpd configured with mod_proxy to only one backend server (pound)
proxy.balance = "fair"
proxy.server  = ( "/" => ( ( "host" => "127.0.0.1", "port" => 6000 ) ) )

.
.
... more config stuff ... etc ...
.
  • Note: Whatever your lighttpd.conf looks like, the important things to change here are to add mod_proxy to the modules and to add the proxy.balance and proxy.server variables. (You might have noticed I commented out the url.rewrite used in the Typo 4.0 build, for some reason this was causing errors and I havent figured out why yet.)

Configure Mongrel

  • Finally we need to setup those 3 Mongrels on ports 6001, 6002 and 6003 – to do so, in your main Rails folder create a /config/mongrel_cluster.yml file like so;
    cwd: /u/apps/yourdomainname.com/current
    port: "6001"
    environment: production
    address: 127.0.0.1
    pid_file: log/mongrel.pid
    servers: 3
  • Where cwd, is the full path to your Rails ‘current’ working directory – Remember, since we are using Capistrano, this will be the current/ symlink that points to the latest ‘release’ folder. Note the servers: 3, which creates 3 Mongrel processes, just change this number to specify how many Mongrel processes you want running.
  • Note: It might be a good time to checkin all these changes to SVN at this point.

Fire it up!

  • With Pound already running, we just need to start Lighttpd and our 3 Mongrels. I dont believe it matters in what order you do this, but i’ll start with Lighttpd, from your main Rails directory;
    sudo ruby script/server -d
  • Now start the 3 Mongrels barking, from within the main Rails directory;
    sudo mongrel_rails cluster::start
  • Thats it! open up your browser and check your site. You can tail -f to monitor for errors/messages in Mongrel, Pound and Lighttpd like so;
    tail -f /var/log/sys.log (will show pound errors and other sys info)
    tail -f rails_dir/log/mongrel.log (mongrel info)
    tail -f rails_dir/log/lighttpd.access.log (lighttpd access log)
    tail -f rails_dir/log/lighttpd.error.log (lighttpd error log)

Help! its not working!

  • Use the following command to list running processes that are using open ports on your system;
    sudo lsof -i -P
  • Use the following command to port scan your server;
    sudo nmap -sT -O localhost
  • To start debugging problems, check your logs and try those port mapping commands (above) first !
  • Feel free to drop me an email (matt [at] hiddenloop [dot] com)about any part of this tutorial, or post a comment and i’ll try to get back to you.

To Do

You’ll probably want to setup /etc/init.d/ scripts for Lighttpd, Mongrel and Pound to start automatically when your server reboots. Maybe i’ll post about this later. Also some tidying up could be done to extract the Lighttpd and Mongrel configuration files out from inside the Rails app directory. AND ideally, its better practice to create a specific user (rather than root) for running the processes for this stack.

Coming soon

In the next (and last) part of this guide, I’ll go through what it takes to adding a copy of the Mint webstats package into your (now nicely setup) Rails app. Showing you how to;

  • make sure it gets ignored by version control
  • is skipped from any Capistrano deployments
  • is served up directly by Lighttpd and PHP

The procedure for doing this with Mint, applies to any non-versioned/non-rails folder you want to place in your Rails app directory structure. And also explains how to mingle serving PHP and Rails from within this stack.

References

Ghibli

1 comment

ghibli.jpg

As I mentioned before, Ive joined the Amazon rental service, and have taken to getting out movies I wouldnt normally think of buying. Its been a good opportunity to see both Spirted Away and Howling Wolf’s moving Castle – two excellent shows from Studio Ghibli, that prove that great characters and good storytelling, will always beat overstated visual effects and no substance.

Over the next few weeks, Film Four will be showing some of these movies for free (if you have Freeview/Satellite) – including Laputa, Castle in the Sky – so you’ve no excuse.

August 07, 2006 10:59 by

Part 2. Setup Capistrano on the Rails app and run a test deploy to your server

1 comment

Starting Point

  • You have got your Rails app under version control with SVN as described in the first tutorial
  • First make sure you have Capistrano installed, you can get it as a free Ruby gem with;
    gem install capistrano
  • ‘Apply’ capistrano to your Rails app on the dev box, this creates some config files locally for Capistrano deployment to work.
    cap --apply-to /u/apps/sitename.com sitename.com
  • Edit the Capistrano’s deploy configuration in /u/apps/sitename.com/config/deploy.rb to set the correct roles for your server;
    set :repository, "svn+ssh://matt@dagobah/svn/#{application}"
    role :web, "server"
    role :app, "server"
    role :db,  "server"
    set :deploy_to, "/u/apps/#{application}"
    
  • From within the Rails directory (/u/apps/sitename.com) run the rake ‘setup’ task to remotely setup Capistrano’s directories on your server;
    rake remote:exec ACTION=setup
  • Try a test deploy to the server (may ask for password during sudo) this will;
    • Checkout the latest revision of your application from your remote repository to the /u/apps/sitename.com/releases directory on your server
    • Update (or create) the /u/apps/sitename.com/current symlink so it points to this new revision
      rake deploy
  • If all goes well, your Rails app should be checked out under /u/apps/sitename.com/current on your remote server. Run ‘rake deploy’ again if you like and see what happens. A new ‘release’ will be checked out to /u/apps/sitename.com/releases/timestamp/ and the symlink at /u/apps/sitename.com/current will be automatically setup to point to it.

Capistrano Tips

  • Obviously with lots of deploys, you’ll start getting loads of release directories on your server, all containing your entire Rails app. You can run the ‘cleanup’ command to remove older releases. In config/deploy.rb – set the following to say how many releases you’d like to keep after a cleanup
    set :keep_releases, 3
  • Then in the Rails directory run;
    cap cleanup

So there you have it, you can now deploy using Capistrano to your server from your dev box and your Rails app is safely under SVN version control. There is loads more you can do with Capistrano (see the links below)

In the next part of this guide, I’ll go through building the hosting stack (lighttpd/pound/mongrel) on your server. And making use of Capistrano to help with deploying your versioned Rails app around an unversioned folder containing the mints/ stats package (which we’ll also get running under PHP with lighttpd)

References

Part 1. Setting up SVN on a working Typo build (or any Rails app)

no comments yet, post one now

So in this post its all about getting your existing Typo (or rails app) under version control with SVN.

Starting Point

  • Have a rails site working and using lighttpd on your local dev box under /u/apps/sitename.com
  • Your remote server has SVN installed (try svn -v) and Rails/Lighttpd installed
  • Make sure you have an ssh key setup for accessing your server from your dev box
  • If you will have multiple users checking in and out and using Capistrano/SVN – set them all up a with ssh keys and create a new ‘developer’ group on the server (that has permissions on svn repository folders and /u/apps/)
  • Note: if you make a mistake, you can use this command to clean away all svn folders from the CWD downwards
    find . -type d -name '.svn' -print0 | xargs -0 rm -rdf 

Create SVN repository on svn server, checkin all rails code, then check out a working copy again

  • Login to the server and create a top level directory svn to hold your repositories;
    mkdir /svn
  • Create a new repository in it with;
    svnadmin create /svn/sitename.com
  • On your dev box, import the entire rails app to this new repository on ‘server’ with a simple comment;
    svn import -m "initial import" /u/apps/sitename.com svn+ssh://matt@server/svn/sitename.com
  • On dev box, rename /u/apps/sitename.com to sitename.com.stepaside – you could delete it before checking out your working copy, but its safer to do this in case anything goes wrong.
    mv /u/apps/sitename.com /u/apps/sitename.com.stepaside
  • Now checkout from the new svn repository to your dev box with;
    svn co svn+ssh://matt@server/svn/sitename.com /u/apps/sitename.com/
  • If all is ok, you can now delete sitename.com.stepaside
    rm -r /u/apps/sitename.com.stepaside

SVN Tips

  • You can run svn update, to update your current working copy with the latest from the server repository;
    svn update /u/apps/sitename.com
  • You can run svn commit -m “comment” to commit all changes from your working copy to the repository on the server
    svn commit -m "your comment" /u/apps/sitename.com
  • You can run svn status to see what files have changed or are not under version control;
    svn status /u/apps/sitename.com
  • You can add files/folders to svn’s ignore list for a particular file under version control using propedit; e.g. to set whats to be ignored in the config/ folder;
    svn propedit svn:ignore config/
  • This will popup a text editor (usually vim) – allowing you to specify files/folders inside config/ to ignore – e.g. .rb or deploy etc. In vim, press I to start editing (inserting), edit the file and press Esc and ‘ZZ’ to save and exit. (easy!) – Of course for these ignores to take place you’ll have to do an svn commit.
  • IMPORTANT for these ignores to work on files/folders that are already under SVN version control, you’ll first have to un-version and delete them; So back up the file/folder first then SVN remove it with;
    svn remove /path/to/file
  • Then place the backed up file/folder into your working copy again (if its a folder, delete any .svn folders in it) (Its a real pain, but ‘svn status’ is your friend during this process)
  • You can run svn status —no-ignore to see what files you’ve added to svn’s ignore list (these don’t get version controlled
    svn status --no-ignore /u/apps/sitename.com

References

Intro. All kinds of 'newness'

1 comment

So here we are, your looking at a fresh new install of Typo 4.0, with my custom theme, migrated blog content and mint stats running on a new VPS host, all served up on a platter with a new hosting stack (Lighttpd/Pound/Mongrel) – So basically, all kinds of ‘newness’.

As I mentioned – in the next couple of posts, ill try to walk through what it took to set this up. There are lots of guides already out there that take you through parts of this process, but I am attempting to gather my entire experience together and apply it to an existing Rails application (this Typo blog) – Working with a real Rails app like this, you come across ‘gotchas’ that you wouldnt normally see with a fresh vanilla Rails.

I have taken to using a blank page on my wiki scratchpad to log what I do, when I do it. What you get below is basically a cut & paste from there. Im working with a Mac/OSX as my local development machine and a Debian Rimuhost VPS as my server. Im assuming you have root access on both boxes to do this. And while you could do all this with Windows – I choose not to (I’ve been down that road before)

Terms Used

  • server (production/svn server) – the remote box hosting the svn repository and live website, and its alias on your local dev box
  • dev box – local client machine contains working copy of site
  • sitename.com – domain name of site
  • CWD – abbreviation for current working directory
  • Rails directory – the top level Rails dir, i.e. the one containing app/ config/ public/ etc.
  • start reading PART 1

Primer

no comments yet, post one now

primer.jpg

I highly recommend watching Primer, A sci-fi puzzle thriller, it explores the effects of an accidental invention on its two creators. Winner of two Sundance film festival awards it was shot with only $7000. Since joining the Amazon DVD rental service, Ive taken to renting things I wouldnt normally consider buying – but this is a great show. Thanks to petemc for the tip.

Coming soon...

no comments yet, post one now

After deciding to take a break from freelance work for the next couple of months, Ive had some time to sort a few things out here on this blog (and in doing so, learnt a thing or to).

Very soon I’ll be moving this site to my shiny new VPS server at Rimuhost, now armed with root access – all manner of crazy things can happen. Ive spent some free time over the last couple of weeks doing the following;

  • Tidied up all the CSS here
  • Moved all my Rails development from Windows to OSX
  • Setup a new VPS Debian server with Rimuhosting
  • Upgraded to Typo 4.0 and migrated this site’s theme into it
  • Switched my photos back to a Flickr Pro account from 23HQ
  • Moved this site to using SVN / Capistrano for deployment
  • Changed from using Apache FCGI, to a full Lighttpd, Pound and Mongrel stack.
  • Survived the London heat

So be prepared for all sorts of nerd like ‘how-to’ posts – as I attempt to share the knowledge and gotchas I experienced during all this (including the London heat) None of the changes to this site have been rolled out yet, its still sitting on Dreamhost running under Apache/FCGI.

Why bother? – The whole process has been a bit of learning exercise for myself and it means I can now quickly develop, deploy and host rails applications, faster and more reliably – which is good -you see.

Superman Returns (without Zod)

1 comment

I can still taste the excess popcorn after last nights visit to the largest screen in Britain, the London IMAX Cinema – What better place to see the most expensive film ever made on the biggest screen ive ever seen, with the Super-est Man in comic book history !

Depsite neck injuries from sitting one row from the front – the film was very enjoyable. And even without General Zod making an appearance, it was much better than expected.

Share your RSS OPML ?

no comments yet, post one now

I’d be interested (maybe just plain nosey) to flick through other people’s subscribed feeds, and see what you’re reading. Export your subscribed feeds as OPML, upload and post a comment with a link.

Here is my RSS OPML file, – I check most all of these feeds on a daily basis. You can import this into any feedreader for reading at your leisure.

OPML ? is an XML format for outlines. Originally developed as a native file format for an outliner application, it has since been adopted for other uses, the most common being to exchange lists of RSS feeds between RSS aggregators.

Its probably already been done, but perhaps there’s scope here for an OPML delicious type – ‘share my feeds’ web app. Quick – to the bat mobile !

Edit: Its Been Done, already – Note to self, always Google before posting …

July 11, 2006 04:57 by

Job Descriptions

1 comment

Many of you may know I work as a Technical Project Manager. Today a fellow TPM came up with this response when asked for a short job description;

Fellow TPM

I am a synergistic facilitator empowered to provide technical solutions across convergent and parallel business workstreams, as well as mediating end-point QA utilising prescriptive enforcement and non-contractual guidance to achieve timely admissable delivery.

What’s your job description ?

June 22, 2006 08:38 by
← (k) prev | next (j) →