articles by matt

As expected the yearly, ‘threaten to leave your mobile phone operator’ pitch worked a treat – giving me a great new contract deal and a newly released handset all for free :) – I choose to ignore the slightly increased monthly rate, since I’ll be saving on calls, at least thats what I tell myself – in fact I usually say; I’d be wasting money if I hadn’t gone for this deal.

In part of our new re-think in developing Skribe we have decided to evaluate a bunch of blogging applications and services on the web. For anyone who might be interested here is my (positive) review of Typo – which i’ve been happily using here for a few months now. Its not a complete review by any means, and i’ve really only highlighted points that are relevant to our development and planning work. But read on if you like;

What is Typo ?

‘Typo is a lean engine that makes blogging easy’. Its an open source web application for a personal blog site, that can be installed (i.e. just unzipped and configured) on your web server. Its free and can be downloaded from here – the source files are also available in a public subversion repository. The latest version (as of Jan 06) is 2.6.0 (released Nov 05)

The first revision of Typo was written to power too biased, Tobias Luetke’s web blog. While he was waiting for a client at Starbucks he wrote down the wrong time for the appointment so naming it Typo seemed quite fitting. It was also one of the first open source Rails Blogging applications on the web – and perhaps still is.


What technology/software does it use ?

Typo is proudly powered by Ruby On Rails, and can be configured to link to either a MySQL or Postgres database. The server must support Rails, Ruby, and CGI. FastCGI is recommended for better performance. The blog can run on any HTTP server supporting Rails, including, WebBrick (Ruby web server), Apache (& Apache2) or Lighttpd


How is the database structured ?

After unpacking the release, you use a .sql file included in the release to generate a blank set of tables in the database. Installing Typo (although simple enough) is NOT for non-techies. Your expected to have a little knowledge of mySQL and FTP’ing files.

The db includes the following key tables; (there are 15 tables in all, but I’m only summarizing these important ones)

• articles – created by the blog author(s), this holds the main content, title, author and id of each article. Body of the article is stored in 2 fields, one with the html encoded text, the other with the plain text with the markdown syntax. This means that the html parsing engine (e.g. markdown) is only used when articles are added/edited and not on the fly when rendering pages. The table also includes a permalink field – for direct url linking to the article page. And allow comments/trackbacks for individual articles.
• comments – linked to an article_id, stores the actual comment (and the comment html in a separate field) – includes IP address, date/time, author, email, and url. Only author and comment are required. Note that IP logging is only done on comments, since you obviously need to be logged in to post articles.
• categories – Categorisation on articles is achieved using this table and the following fields, id, name, position, permalink. A separate articles_categories mapping table simply maps articles to these categories. Any article can belong in none or more categories.
• page_caches – contains a unique id and url to all pages (XML and html) cached by typo – I assume ruby caches pages frequently, you are given no control of this process in the admin, except clearing the cache. I haven’t looked into how typo caches pages – but in general use I have found it to be a little biased in caching, and sometimes I find myself clearing the cache in the admin to see immediate changes in my published site.
• sessions – instead of using a temp session file, sessions (for the admin) are stored in this db table. Using a db table for session data is proven to be more secure than a file – especially on a shared hosting environment.
• settings – general blog settings are stored here in a name/value arrangement – this is good for 2 reasons; it means settings aren’t stored in a hard to reach file – AND using name value pairs, more settings can easily be added to the table – should newer versions of typo require them.
• trackbacks – Trackbacks to your blog articles are stored here. Trackback is a mechanism used in a blog to show, around an entry, a list of other blogs that refer to it. I haven’t used it much, but it seems straightforward enough. Warning most of the trackbacks on my blog are from spam sites. So I’ve disabled them for now. I don’t think typo has very good (or any) spam protection on trackbacks.
• sidebars – used by typo to keep track of what sidebar components are installed on the blog, their order and when they were last updated. Typo allows you to easily drag and add sidebar components like, Flickr images feed, delicious link feed, your blog article archive, plain html box, standard RSS feeds etc. Its a powerful feature that’s easy to use and is one thing typo does very well.

What templating system does it use ? – if any

Typo uses the Rails, MVC approach, with templates clearly separated in the view from the rest of the application. Typo allows ‘themes’ to be easily applied using the admin. Download and unzip the theme (which contains graphics, rhtml, javascript files etc.) – then use the admin to simply activate it.

Typo takes care of the rest – copying the necessary files from the theme to the public/ and views folders. I’m in the process of re-skinning typo right now, so I’ll know more about this whole process soon.

How does it perform ?

Lots of blogs including my own use Typo with no problems. The main issues with performance arise with the hosting environment and how they have configured their web server running Rails and FastCGI. I found that on Dreamhost, I had to write a cronscript to kill wandering FastCGI processes when Typo was used by more than a couple of users. It turns out this is not Typo’s fault, but Dreamhost’s. Generally page loading speed is very good and the application runs smoothly.

What does it do well ?

To summarise – Typo is very good at the following;

• Importing RSS feed data from other web apps
• Spam protection on comments
• Applying Themes
• Using html parsing engines (like markdown and textile)
• Caching html and XML
• Easy to use admin – very simple
• Keeping things simple – db structure is straight forward
• Easy to extend features and develop for the blog
• Compatible with desktop blog clients like "MarsEdit"http://ranchero.com/marsedit/ etc.

What can be improved ?

• Spam protection on trackbacks
• No image/file uploading features
• Admin is completely separate from the published website, tighter integration could help when making quick changes
• Hard to install/configure for non tech users
• More control over caching or better choice of caching periods
• HTML parsing engines (markdown/textile) may be complicated for non web savvy users to learn
• No formal migration process – to pull entries/comments from another blog, like MoveableType etc.
• No backup or content extraction process – i.e. theres no way to download a copy of your Typo content.

What, if anything, may be of use in Skribe ?

Its hard to compare Typo directly to Skribe, given that Typo is a standalone blogging application that must be installed – opposed to Skribe, a web application offering a blogging service to a large number of interconnected users. To summarise then – the following points and features should be considered in planning Skribe development.

• Uploading Images, Files, Resources for use in Blog entries
• Importing and displaying of external feed data – e.g. standard RSS and additional extra support for Flickr and other services
• Good Page caching for speed, more control over your own page cache options
• Spam protection via a bad IP list and online spam listing (and spam protection on trackbacks)
• Commenting and Trackbacks on/off on individual entries
• Easier UI for content editing
• Use of Textile/Markdown for advanced editing
• Storing both parsed xHTML and non-parsed body text (again for speed)
• Tighter integration of admin features with published content, e.g. in-place editing
• Themes – not sure we want to go with this in Skribe, as an alternative, I suggest we create a simple API that will allow developers to hook into Skribe and get Skribe content out.
• Skribe must be compatible with desktop blogging clients in one form or another

Having never owned a laptop (or a Mac), I am looking forward to getting a MacBook Pro sometime in February. Glad I waited, almost bought a Powerbook a few months ago.

Well, im sure a lot of you have heard already, but in just over 4 days last week I have;

• Left Belfast, and moved to London
• Finished setting up my own company Hiddenloop Ltd.
• Started a new full-time contract as Technical Project Manager with the BBC, working 2 days
• Drove back to Belfast, (via Stranraer) leaving my car back in N.Ireland
• Flown back to London on the 8th
• Got very little sleep

So im here now, and in probably one of the busiest weeks of my life, I seemed to have landed on my feet, moving into a great shared house within 15mins walking distance to White City and 5 mins from East Acton Underground.

As a rule, I wont be blogging about my job at the BBC, all I can say is that I have joined as a Technical Project Manager for initially 6 months, on a high(ish) profile project that is due to launch soon. Having been here just 4 days now, things are going well so far.

Forming a company was something I’ve had in mind for a while, taking on more and more contract work. Hiddenloop (and this website) are going through a complete redesign over the next couple of months. So more information will be available soon on what Hiddenloop will do as a company etc.

Who knows what will happen, its entirely possible I will be back in full-time employment by the end of this year – the point is that I am giving this a go. Listening to the Venture Voice Podcasts has been great encouragement, and prove that with some hard work, a bit of talent and a good idea – small people can make it big – (but not too big)

RyanC from 37Signals has also got a great series of articles on the challenges of setting up a new small business.

London seems great so far, its a big change from living in Belfast for the last seven years, there are some great web agencies over here (no dis-respect). And while rent is expensive, being able to walk to work really helps. I have a brother and sister over here as well, which is dead handy for figuring out where things are etc.

No doubt I’ll be posting here more frequently now, so stay tuned, who knows what will happen …

If your developing Ruby on Rails web apps on Windows (or any platform really) why not try RadRails

The IDE itself is running on top of a very light version of Eclipse, it allows database queries to be fired off right in the console and has integrated support for working with subversion repositories. Its promising all you could ask for under one roof.

I came across this recently and first impressions are good. There is even a blog and podcast you can follow on its development.

Ive just had my first blog comment spamming experience. Despite Typo’s best efforts – over the last week or so about 20 comments have been posted across all my blog articles selling all kinds of nasty stuff.

And much to your disappointment, I have removed these comments, so you will have to use another site to get your prescription drugs – or check your spam inbox.

For now you can only comment on articles less than 7 days old, until I jump into typo and sort this out a little better.

Ruby on Rails 1.0 has been released! There’s even a new website for the framework, designed by the guys at 37Signals; quoted from the site;

> 15 months after the first public release, Rails has arrived at the big 1.0. What a journey! We’ve gone through thousands of revisions, tickets, and patches from hundreds of contributors to get here. I’m incredibly proud at the core committer team, the community, and the ecosystem we’ve raised around this framework.

I recently had the rather unpleasant task of writing some PHP to compare a CSV file (with some 22,000+ entries) with a mySQL database. With the CSV file holding the master copy of data, it would update/insert and delete from mySQL. The script needed to run as a daily Cron on my (shared) Dreamhost box.

This would normally be simple enough, using a status field on the CSV file to indicate fields that had been updated. Unfortunately there was no status field, and none could be added. In fact the CSV file could not be modified at all. The only way to check if a row had been modified was to do a field by field comparison on every row.

I started off with a single script that imported the CSV to an Array, and also extracted all rows from the db table. With some looping to search through all rows and all fields in each row, I got the script to work. Great! (I Thought)

But with 22,000 rows looping ~22,000 other rows, (22,000 × 22,000 = 484 million loops ) – in short the script took minutes to execute, and if left long enough it ate up 100% CPU usage (through php). Even using a exponential back-off search took too long.

On Dreamhost, if any script you run nears 100% CPU usage, its killed automatically. A major rethink was required. So I decided to split the script in two.

• script 1 – would create a temporary table in the database and simply import the CSV file into it – row by row.

• script 2 – running a few minutes later, would then compare the two tables using mysql queries (rather than a php loop search) – after performing all updates/inserts and deletes the temporary table would be destroyed.

The comparison script (2), works by looking for id matches between the two tables, and marking any rows found. If found – both rows are fetched and a field by field comparison is made to check if an UPDATE statement is needed.

Finally any rows not marked as found in the master CSV file were added, and any rows not marked as found in the DB were deleted.

Using 2 tables for the comparison rather than looping and searching in php, meant that the strain was now on mySQL (rather than PHP). Dreamhost seems to tolerate this, and the php script execution time is reduced from minutes to seconds.

And, why am I explaining all this ? – you ask,

1. So I can remember what on earth I did. and;
2. Im curious to know if anyone can think of a better way to do this. Bearing in mind the limiting factors,
the CSV file CANNOT be altered in any way. It has to execute in seconds and the CPU usage cannot approach 100%

After weeks of absence from the online gaming lark, I jumped back in last night for a quick bout on DoD:Source.

Day Of Defeat is the only game I have been playing since 2001, when its 3.0 release was a mod for Halflife (1).
DoD:Source arrived earlier this year and has taken the game to a whole new level.

Ive had other brief stints on HL2:Deathmatch, Counterstrike, Battefield 2, and being a fan of WW2 history and books may make me biased, but DoD is simply the best game i’ve ever played. To be sucessful it forces teamwork and tatics. And while tough for beginners and sticking to realism, its not completely insane – like Ghost Recon, where wind direction and drag infulence bullet trajectories.

For almost a year or so at college I was a member of the Iron&Steel division, a U.S. based clan that doesnt take themselves too seriously and focuses on skill and teamplay. Taking part in tournaments and leagues was good crack for a while, and Ive made a few friends from it all.

You’ll usually find me on the servers at Sunlit Games – they are also based in the States so the pings are little higher, but the quality of play and teamwork are second to none. Many IronAndSteel regulars hang out there and the servers can get busy after 9pm GMT. Its very tough on there, defintely more so that a normal pub server – but that will only make you a better player. (or so I tell myself). They also tend to try out and host some of the best custom maps available.

With the exception of Halflife 1&2, I only play multi-player PC games. Single player games just seem pointless when your beating a predictable cpu. (and consoles were fun for 10 or 20 mins, when I was 15) – saying that Goldeneye 64 has yet to be beaten for 4 player carnage around a TV – (it even forced me to buy an old N64 on ebay a few months ago)

Unfortunately there doesnt seem to be enough time in the day anymore to warrant a 4hr marathon on DOD till the small hours. But I get some practice in when I can.

Oh well, back to work …

BTW – if you’d like to shoot at me (online), my steam account is m.hutchinson@ijug.org or playername G.Costanza – (dont ask)

How often do you post on your blog (comments welcome) – is it completely random, weekly, daily, every hour (?)

Ive failed in trying to stick to almost daily posts, it was going well for a while and its not that the novelty has worn off, its just hard to find the time.

On a completely seperate note, Im thinking of buying a new 15inch G4 powerbook for some much needed mobile computing. Having never owned a Mac before – Some advice would be good on any problems I might come across, where to buy, networking with other machines etc.

You are now allowed to comment on this post – I permit you..