articles tagged with script

My primary backup is Time Machine set to auto-backup my entire machine to a single 1TB drive (including all my iTunes media and applications). This way I can do a full restore if I ever need to. BUT, what happens when that drive fails? I needed a remote, off-site backup for my essential files. The stuff I just couldn’t afford to loose if Time Machine died.

I wanted a solution that was simple, fast, efficient (incremental and deletes redundant files), secure and wouldn’t cost the earth. After researching different options (MobileMe, Dropbox, S3, iDrive, mozy) I found that I could simply use rSync over SSH with my Dreamhost. Here’s how …

I already use Dreamhost to host some legacy websites, and they offer unlimited storage space on even their cheapest hosting plan. Per/Gb its way more cost effective than Amazon S3. I setup the following bash script to run these rsync commands on a daily cron interval (where mydhserver.dreamhost.com is my remote dreamhost server)

#!/bin/bash

# send via rsync using SSH
rsync -azP --delete --delete-excluded --exclude-from=/Users/matt/.rsync_exclude.txt /Users/matt/Documents mydhusername@mydhserver.dreamhost.com:~/rsync_backup
rsync -azP --delete --delete-excluded --exclude-from=/Users/matt/.rsync_exclude.txt /Users/matt/Sites mydhusername@mydhserver.dreamhost.com:~/rsync_backup
rsync -azP --delete --delete-excluded --exclude-from=/Users/matt/.rsync_exclude.txt /Users/matt/work mydhusername@mydhserver.dreamhost.com:~/rsync_backup
rsync -azP --delete --delete-excluded --exclude-from=/Users/matt/.rsync_exclude.txt /Users/matt/resources mydhusername@mydhserver.dreamhost.com:~/rsync_backup
rsync -azP --delete --delete-excluded --exclude-from=/Users/matt/.rsync_exclude.txt /Users/matt/workbench mydhusername@mydhserver.dreamhost.com:~/rsync_backup

I have a separate rsync_backup user setup on my Dreamhost account using an SSH key pair without passphrase. Make sure sure that ~/rsync_backup exists on your remote server before running this. The .rsync_exclude.txt file simply contains a list of file patterns to exlude from backups, mine looks like this;

Steam Content
.svn
.DS_Store

rSync will use SSH by default when sending your data (so its encrypted over the wire). However, you should remember that your backed up data (on the remote machine) is NOT encrypted, it is only as secure as the server it resides on.

So rSync is cheap, simple, fast (you can tweak SSH options to make it faster/slower), incremental and secure. Here’s a few helpful links on configuring your rSync backups further.

• Slicehost article on rSync (dated snapshot approach)
• General tips/tweaks for the rSync command
• Rsnapshot (a perl based snapshot tool, based on rSync)

Working on the rails day in and out now, I’ve found the following aliases to come in handy;

# General Commands
alias ls='ls -al'

# TextMate, mate all of current dir and crucial rails folders only
alias et='mate . &'
alias ett='mate app config lib db public test vendor/plugins &'

# RAILS,  (run these from your rails folder)

# rails scripts
alias ss='./script/server'
alias sc='./script/console'
alias sg='./script/generate'
alias sp='./script/plugin'
alias mr='mongrel_rails start'

# rails testing 
alias att='autotest'
alias tu='rake test:units'
alias tf='rake test:functionals'    

# tail logs
alias tl='tail -f ./log/development.log'
alias tt='tail -f ./log/test.log'      

# clean the logs
alias ctl='cp /dev/null ./log/test.log'
alias cdl='cp /dev/null ./log/development.log'

I should credit Peep Code for the idea. To use, (e.g. in OSX) place the above in a ~/.bash_aliases file and in ~/.bash_profile, load it in with this command;

if [ -f ~/.bash_aliases ]; then . ~/.bash_aliases ; fi

Also, (and before I forget it myself) – here’s a quick session cleaner command to put in your cron, (for day old, mysql session clearage action); rather than build a rake task, or extra controller to clean them out.

#!/bin/bash

cd /u/apps/matthewhutchinson.net/current
echo "<== CRON TASK ==> clear day old sessions data on matthewhutchinson.net"
ruby script/runner -e production "ActiveRecord::Base.connection.delete(\"DELETE FROM sessions WHERE updated_at < NOW() - INTERVAL 1 DAY\")"