articles tagged with lighttpd

Part4. Having a Mint

8 comments

Following on from part 3 you should now have your Rails application nicely hosted and deployed using Capistrano from your SVN repository. All good – but what happens when you want to serve another public web folder into the mix e.g. in public/myfolder, but you DON’T want it under version control ? Why you ask ?

Well lets take this example further, what if this folder contained a copy of the Mint stats package in public/mint – i.e. PHP code, that you wanted to execute and run. Since new versions of Mint are released fequently, and i’m often adding and removing ‘Peppers’ (read Mint Plugins), I see no need to put Mint under version control. Purists might argue differently, but I dont collect stats on my local dev box either so it makes no sense to have it there after a checkout.

Assuming your convinced I’ll start off getting Mint up and running on the server, with PHP through Lighttpd and stats being recorded. Then i’ll add some extra functions to the Capistrano deployment, allowing us to deploy ‘around’ this folder on the public server.

Configuring Lighttpd with PHP and FastCGI

First ensure your server already has PHP installed and is configured with the FastCGI module;

$> php -v
PHP 5.1.4 (cgi-fcgi) (built: Aug  2 2006 23:53:20)
Copyright (c) 1997-2006 The PHP Group
Zend Engine v2.1.0, Copyright (c) 1998-2006 Zend Technologies

If you dont see the (cgi-fcgi) part, then you’ll need to re-configure PHP with the FastCGI module. There are instructions for installing PHP and configuring modules at PHP.net – and elsewhere on the web – Google is your friend.

Now configure your config/lighttpd.conf file (do this locally on your dev machine, checkin to SVN and redeploy – or just edit on the server to get things working then apply the same changes locally, checkin & deploy), in config/lighttpd.conf;

server.modules = ( "mod_rewrite", "mod_accesslog", "mod_fastcgi", "mod_compress", "mod_expire", "mod_proxy" )

fastcgi.server = (".php" => ("127.0.0.1" =>
  ("socket" => CWD + "/tmp/sockets/php.socket",
    "bin-path" => "/usr/local/bin/php",
    "bin-environment" => ("PHP_FCGI_CHILDREN" => "1", "PHP_FCGI_MAX_REQUESTS" => "5000")
  ))
)

# lighttpd proxy server (pound) - no proxy for mint please
$HTTP["url"] !~ "^/mint/" {
  proxy.balance = "fair"
  proxy.server  = ( "/" => ( ( "host" => "127.0.0.1", "port" => 6000 ) ) )
}

Things to note here are;

  • Add mod_fastcgi to your server.modules if its not there already.
  • fastcgi.server is configured with your server’s PHP bin path, and a tmp socket (which can be anywhere).
  • We DON’T want to proxy requests for Mint through Pound to Mongrel – since this is a PHP app, we just want Lighttpd to deal with it using FastCGI – hence the need for the if statement on the pound proxy configuration.

Setting up Mint (e.g. in Typo)

First grab your licensed copy of Mint for your domain – and drop the mint/ folder into current/public/mint to the main Rails directory on your server. Next add the following url.rewrite to your lighttpd configuration;

  # configure for mint access
  url.rewrite = ("^/mint/$" => "/mint/index.php", "^/mint/\?(.*)" => "/mint/index.php?$1")

This url.rewrite is nessecary to ensure that Lighttpd doesnt treat files or folders under /mint/ as Rails specific. Again this should also be added in your local copy, and checked in to SVN. DON’T deploy with Capistrano just yet – because doing so will check out a new release and archive the existing current/ folder (and hence remove our /current/public/mint) – You’ll also want to add the following javascript in the header of any page where stats should be reported;

<script src="/mint/?js" type="text/javascript"></script>

Setting up Capistrano

In order to keep using Capistrano to deploy to your server with mint in the current/public/mint folder – we need to deploy around it. There are probably better ways to do this – in the Capistrano deployment recipe file (config/deploy.rb) – I added two action functions; one occurs before deployment starts, the other after. The functions basically move the mint/ folder out of the way (to the top level shared folder) while Capistrano does its stuff. So in config/deploy.rb;

  # executed before deployment
  task :before_deploy, :roles => [:web, :app] do
    # copy the mint/ and files/ folders to holding area in shared/
    puts "before deploy ---> copy mint and files to shared from current"
    run "sudo mv #{deploy_to}/#{current_dir}/public/mint  #{shared_dir}/mint"
    run "sudo mv #{deploy_to}/#{current_dir}/public/files  #{shared_dir}/files"
  end

  # executed after deployment
  task :after_deploy, :roles => [:web, :app] do
    # copy the mint/ and files/ folders back from holding area in shared/
    puts "after deploy ---> copy mint and files from shared to current"
    run "sudo mv #{shared_dir}/mint #{deploy_to}/#{current_dir}/public/mint"
    run "sudo mv #{shared_dir}/files #{deploy_to}/#{current_dir}/public/files"
  end

You’ll also see i’m doing the same thing for a current/public/files folder – This folder is used by Typo for uploaded files for blog entries. Without these actions in place, each Capistrano deploy would clear out the files/ folder on your server.

Trying it out

Check the changes in, make sure the mint folder is on your server (and correctly configured) and run a new Capistrano deploy. During this you should see the before and after tasks running (you may will be asked for a password to sudo). You should then see your copy of mint up and running, like so; /mint/

Thus ends the mini-guide; Any suggestions, comments or questions are appreciated. Normal useless entries will resume here as of today.

References

Part3. Lighttpd, Pound and Mongrel with Rails

no comments yet, post one now

This (king-kong long) post is a continuation of a mini-guide that has already covered putting your Rails app under SVN version control, and deploying it with Capistrano. Now to explain how to serve it all up on a layered and scalable web server stack using Lighttpd, Pound and Mongrel. But first some obvious questions;

  • What is Lighttpd ?
    (aka. Lighty) – is an open-source all purpose webserver. With a small memory footprint, effective management of cpu-load, and an advanced feature set (FastCGI, CGI, Auth, Output-Compression, URL-Rewriting and more) LightTPD is the perfect solution for every server that is suffering load problems. Oh – and it can serve up PHP faster than a shiny red truck.
  • What is Pound ?
    Pound is a reverse proxy, load balancer and HTTPS front-end for Web servers. Pound was developed to enable distributing the load among several Web-servers and to allow for a convenient SSL wrapper for those Web servers that do not offer it natively. It comes with different ‘balancing’ settings to easily configure load distribution.
  • What is Mongrel ?
    It is a hybrid Ruby/C HTTP server designed to be small, fast and very secure. Written by Zed Shaw it is intended for hosting Ruby web applications using plain old HTTP rather than FastCGI or SCGI. A ‘Mongrel Cluster’ simply means a collection of Mongrel server proccess (one or more). Mongrel works well with the load-balancing offered by Pound (above), meaning as traffic increases to your Ruby based website, you can easily distribute the load across a number of Mongrel processes in your backend servers. Capistrano also has built in function to support management of your Mongrel Cluster – all good.
  • What is the point ?
    The main reason I switched to this stack was because Dreamhost’s shared hosting environment serving Rails with Apache2/FCGI just wasn’t cutting it. Especially with a memory hungry Typo build and a bunch of sidebars. FCGI processes would crash, create CORE dumps, need to be reaped every hour and generally cause a lot of trouble. Thats not to say that you cant get a Rails site up and running under Apache2/FCGI – many people do it very well. However – when I finally got a new VPS server at Rimuhost (with root access) – I took the opportunity to try Lighttpd and Mongrel out.

So, Lighttpd is small, versatile and can serve up static pages (and PHP) super fast and Mongrel (see above) is better with the Ruby stuff. What we want to do ideally, is to proxy requests to our Rails app to Mongrel → from Lighttpd. We could use Lighttpd’s mod_proxy for this – BUT – as it stands right now, its a little bit, ‘not working right’. So, to get around that (or until it is fixed in Lighttpd 1.5) – we can use the ‘Pound’ proxy to sit between Lighttpd and Mongrel. If I could be bothered drawing a diagram it would probably look something like this – (gv) – clear as mud.

Starting Point

  • You have got your Rails app under version control with SVN as described in the first tutorial
  • You have got Capistrano working and deploying to your server as described in the second tutorial
  • You have root access on your server for installing/configuring Lighttpd, Pound and Mongrel

Install Mongrel on your Server

  • Get it’s dependencies, daemons and the gem_plugin. Also install mongrel_cluster, which contains a few tools which make managing a cluster of Mongrel servers easier.
    sudo gem install daemons gem_plugin mongrel mongrel_cluster sendfile --include-dependencies
  • Note: this also includes support for ‘sendfile’ – an important part of the this layered stack – With ‘sendfile’, requests can be sent via a proxy (Pound) to your Mongrel servers directly via the local filesystem – Which is much faster than over a local HTTP pipe. Install sendfile on your server (debian) with;
    sudo apt-get install sendfile
  • I’ll leave configuring Mongrel for now – First just test that mongrel is installed ok, try a ‘whereis’ on it, and then try running it to check its there. If not you may need to add its install directory to your $PATH variable;
    whereis mongrel_rails
    mongrel_rails

Install Pound

  • Follow the instructions here, make sure zLib and openSSL are installed first, then install Pound. If (during build/compiling) you experience errors (as I did) relating to libssl and/or libcrypto libraries not being present – then you will also need to install libssl-dev. Then try making and installing zLib/openSSL again. On Debian this would be;
    apt-get install libssl-dev
  • If problems still persist after install (as they did for me!), you can try ‘faking’ the older library file verison with the new like so.
    sudo ln -s libssl.so.0.9.7 /usr/lib/libssl.so.0.9.8
  • Now configure the Pound config file (in /usr/local/etc/pound.cfg, or maybe in /etc/pound.cfg) – Set up Pound up on your server listening on port 6000, and directing requests onto lets say, 3 Mongrel Servers running on ports 6001, 6002 and 6003 (which I’ll setup next).
    ListenHTTP
      Address 0.0.0.0
      Port    6000
      Service
        BackEnd
          Address 127.0.0.1
          Port    6001
        End
        BackEnd
          Address 127.0.0.1
          Port    6002
        End
        BackEnd
          Address 127.0.0.1
          Port    6003
        End
        Session
          Type BASIC
          TTL  300
        End
      End
    End
  • You can run a check on your Pound configuration with;
    sudo pound -v -c
  • You should now start the Pound proxy with;
    sudo pound -v
  • Note: while Pound is running, it wont output errors or even messages to its own log files, instead you should monitor the system log file to see whats going on;
    tail -f /var/log/sys.log  (for debian, or system.log in OSX)

Configure Lighttpd

Lighttpd will be the main port of call for all HTTP requests to port 80. It is our main front-facing web server to the general browsing public, and it will pass off requests (for our Rails app) to the Pound proxy on port 6000; Your config/lighttpd.conf (normally in your main Rails directory) should look a little like this;

server.bind = "0.0.0.0"
server.port = 80

server.modules           = ( "mod_rewrite", "mod_accesslog", "mod_fastcgi", "mod_compress", "mod_expire", "mod_proxy" )

server.error-handler-404 = "/dispatch.fcgi"
server.document-root     = CWD + "/public/"
server.errorlog          = CWD + "/log/lighttpd.error.log"
accesslog.filename       = CWD + "/log/lighttpd.access.log"

#url.rewrite              = ( "^/$" => "index.html", "^([^.]+)$" => "$1.html" )

compress.filetype        = ( "text/plain", "text/html", "text/css", "text/javascript" )
compress.cache-dir       = CWD + "/tmp/cache"

expire.url               = ( "/favicon.ico"  => "access 3 days",
                             "/images/"      => "access 3 days",
                             "/stylesheets/" => "access 3 days",
                             "/javascripts/" => "access 3 days" )

# lighttpd configured with mod_proxy to only one backend server (pound)
proxy.balance = "fair"
proxy.server  = ( "/" => ( ( "host" => "127.0.0.1", "port" => 6000 ) ) )

.
.
... more config stuff ... etc ...
.
  • Note: Whatever your lighttpd.conf looks like, the important things to change here are to add mod_proxy to the modules and to add the proxy.balance and proxy.server variables. (You might have noticed I commented out the url.rewrite used in the Typo 4.0 build, for some reason this was causing errors and I havent figured out why yet.)

Configure Mongrel

  • Finally we need to setup those 3 Mongrels on ports 6001, 6002 and 6003 – to do so, in your main Rails folder create a /config/mongrel_cluster.yml file like so;
    cwd: /u/apps/yourdomainname.com/current
    port: "6001"
    environment: production
    address: 127.0.0.1
    pid_file: log/mongrel.pid
    servers: 3
  • Where cwd, is the full path to your Rails ‘current’ working directory – Remember, since we are using Capistrano, this will be the current/ symlink that points to the latest ‘release’ folder. Note the servers: 3, which creates 3 Mongrel processes, just change this number to specify how many Mongrel processes you want running.
  • Note: It might be a good time to checkin all these changes to SVN at this point.

Fire it up!

  • With Pound already running, we just need to start Lighttpd and our 3 Mongrels. I dont believe it matters in what order you do this, but i’ll start with Lighttpd, from your main Rails directory;
    sudo ruby script/server -d
  • Now start the 3 Mongrels barking, from within the main Rails directory;
    sudo mongrel_rails cluster::start
  • Thats it! open up your browser and check your site. You can tail -f to monitor for errors/messages in Mongrel, Pound and Lighttpd like so;
    tail -f /var/log/sys.log (will show pound errors and other sys info)
    tail -f rails_dir/log/mongrel.log (mongrel info)
    tail -f rails_dir/log/lighttpd.access.log (lighttpd access log)
    tail -f rails_dir/log/lighttpd.error.log (lighttpd error log)

Help! its not working!

  • Use the following command to list running processes that are using open ports on your system;
    sudo lsof -i -P
  • Use the following command to port scan your server;
    sudo nmap -sT -O localhost
  • To start debugging problems, check your logs and try those port mapping commands (above) first !
  • Feel free to drop me an email (matt [at] hiddenloop [dot] com)about any part of this tutorial, or post a comment and i’ll try to get back to you.

To Do

You’ll probably want to setup /etc/init.d/ scripts for Lighttpd, Mongrel and Pound to start automatically when your server reboots. Maybe i’ll post about this later. Also some tidying up could be done to extract the Lighttpd and Mongrel configuration files out from inside the Rails app directory. AND ideally, its better practice to create a specific user (rather than root) for running the processes for this stack.

Coming soon

In the next (and last) part of this guide, I’ll go through what it takes to adding a copy of the Mint webstats package into your (now nicely setup) Rails app. Showing you how to;

  • make sure it gets ignored by version control
  • is skipped from any Capistrano deployments
  • is served up directly by Lighttpd and PHP

The procedure for doing this with Mint, applies to any non-versioned/non-rails folder you want to place in your Rails app directory structure. And also explains how to mingle serving PHP and Rails from within this stack.

References

← (k) prev | next (j) →