Multi data center redundancy – sysadmin considerations
Last week, I considered the implications for multi data center redundancy on your applications. This post will look at considerations for the sysadmin – network and server level failover, plus [...]
Multi data center redundancy – application considerations
A few months ago we finished a long running multi data center redundancy project to allow our server monitoring service, Server Density, to survive the complete failure of our primary [...]
High availability with Pound load balancer
Following our migration to SoftLayer and their release of global IPs, we started to implement our multi-datacenter plan by replacing dedicated (static IP) load balancers with Pound and a global IP. [...]
Logging for fun and profit
Once you have more than a few servers and applications running, handling all those logs is going to become a problem. Not in the sense of managing rotation – syslog [...]
Internal dev talks with Google Hangout
Every Friday we have a short full company meeting where we run through what we’ve been working on and what was deployed, then discuss customer feedback and the roadmap for [...]
You can’t just have a sysadmin on call
Reading through the Azure cloud outage root cause analysis from the other week made me think about the problem they encountered, what was needed to fix it and the timeline [...]
Cloud storage failures – the perfect storm
When the Qantas A380 suffered engine failure after taking off from Singapore in November last year, many things went wrong at the same time: The failure of the giant Rolls-Royce [...]
Replacing RabbitMQ with MongoDB
Back in April we ditched RabbitMQ in favour of building our own simple queuing system using MongoDB. This has been in production since then and has been working very well. [...]
The self aware system
When we first started with our server monitoring tool, Server Density, we had a single slice at Slicehost. Those were simple times – if it or any service running on [...]
On shortened field names in MongoDB
Last night we got almost 10k unique visitors in just a few hours because of a post disputing a minor point of a post I wrote in July 2009. The [...]