css (1) jQuery (7) linux (38) mac (70) php (29) python (1) svn/git (9) trac (4) ubuntu (1) virtualbox (6) vista (2) windows (14)

Sunday, 19 February 2012

High Availability or Backups?

Clients often ask me to make their sites and services "Highly available". By this they mean they want some form of redundancy and their server to have as much uptime as possible.

This in itself is not a bad thing and it's not massively complicated to set up (the complexity depends on the software involved).

However one thing they never ask me is "Can you set up a backup solution". Which seems odd to me, in my experience a decent backup solution is far more valuable than a HA setup.

What might you need HA for?
1. Server hardware failure
2. Network failure
3. Data centre goes boom!
4. Scheduled maintenance requires a server to go offline

Out of all of those 4 is probably the most common and that would normally be for reboot due to software updates which takes around a minute, a minute of down time every few months for most sites is no problem. (Over 1 year if you rebooted once per month at 1 minute downtime per reboot that would be 99.9999% uptime)

What might you need backups for?
1. Server hardware failure
2. Data centre goes boom!
3. User error corrupts files/databases
4. Server is compromised

So 1 and 2 are in both HA and Backups, granted HA will respond quicker and with fresher data than backups, if your data is critical (and I mean really critical i.e. financial stuff) then having a database replica is a good idea.

User error, this happens more often than you'd think, a typo in an SQL update clause for example can kill a database, this is where backups are a must, HA can't help you here since all changes will be replicated to all servers.

Server is compromised, heven forbid this ever happens to you but you should always be prepared. If it does the safest solution is to load a backup from a date before the intrusion, fix the hole (keeping the server offline while fixing it) then you're good to go again. Backups in this case also provide a history of file changes which can help pinpoint when you were compromised.

Now consider that HA requires duplicate servers to deal with problems that rarely happen (in the past year all my pingdom graphs have 99.99%+ uptime), which makes it expensive, backups are a lot cheaper storage solutions such as Amazon S3 cost peanuts in comparison to a second server.

So think to yourself. Do I really need HA? Do I have a backup plan in place first?

I'm sure if you think about it you'll agree backups are more important than HA.