Saving $500k per month buying your own hardware – cloud vs colocation
Written by David Mytton — Subscribe now.
This post was originally published on GigaOm on 7th Dec.
Last week I compared cloud instances against dedicated servers showing that for long running uses such as databases, it’s significantly cheaper if you do not use the cloud, but that’s not the end of it. Since you are still paying on a monthly basis then if you project the costs out 1 or 3 years, you end up paying much more than it would have cost to purchase the hardware outright. This is where buying your own hardware and colocating it becomes a better option.
Continuing the comparison with the same specs for a long running database instance, If we price a basic Dell R415 with x2 processors each with 8 cores, 32GB RAM, a 500GB SATA system drive and a 400GB SSD, then the one time list price is around $4000 – more than half the price of the SoftLayer server at $9,468/year in the previous article.
Of course, the price you pay SoftLayer includes power and bandwidth and these are fees which depend on where you locate your server. Power usage is difficult to calculate because you need to actually stress test the server to figure out the maximum draw and then run real workloads to see what your normal usage is.
My company, Server Density, just started experimenting with running our own hardware in London. We tested our 1U Dell with very similar specs as discussed above was using 0.6A normally but stress tested with everything maxed out to 1.2A. Hosting this with the ISP who supplies our office works out at $161/month or $1932/year (it would work out cheaper to get a whole rack at a big data centre but this was just our first step).
This makes the total annual cost look as follows:
Remember, again, that this is a database server so whilst with Rackspace, Amazon and SoftLayer you pay that price every year, after the first year with colocation the annual cost drops to $1932 because you already own the hardware. Further, the hardware can also be considered an asset which has tax benefits.
Server Density is still experimenting at on small scale but I spoke to Mark Schliemann VP of Technical Operations at Moz.com because they run a hybrid environment. They recently moved the majority of their environment off AWS and into a colo facility with Nimbix but are still using AWS for processing batch jobs (the perfect use case for elastic cloud resources).
Moz worked on detailed cost comparisons to factor in the cost of the hardware leases (routers, switches, firewalls, load balancers, SAN/NAS storage & VPN), virtualization platforms, misc software, monitoring software/services, connectivity/bandwidth, vendor support, colo and even travel costs. Using this to calculate their per server costs means on AWS they would spend $3,200/m vs $668/m with their own hardware. Projecting out 1 year results in costs of $8,096 vs AWS at $38,400.
Moz’s goal for the end of Q1 2014 is to be paying $173,000/month for their own environment plus $100,000/month for elastic AWS cloud usage. If they remained entirely on AWS it would work out at $842,000/month.
Optimizing utilization is much more difficult on the cloud because of the fixed instance sizes. Moz found they were much more efficient running their own systems virtualized because they could create the exact instance sizes they needed. Cloud providers often increase CPU allocation alongside memory when in real world uses you tend to need one or the other. Running your own environment allows you to optimize this and was one of the big areas Moz have used to improve their utilization. This has helped them become much more efficient with spend.
Right now we are able to demonstrate that our colo is about 1/5th the cost of Amazon but with RAM upgrades to our servers to increase capacity we are confident we can drive this down to something closer to 1/7th the cost of Amazon.
Colocation has its benefits once you’re established
Colocation looks like a winner but there are some important caveats:
- First and foremost, you need in-house expertise because you need to build and rack your own equipment and design the network. Networking hardware can be expensive and if things go wrong, you need to have the knowledge about how to deal with the problem. This can involve support contracts with vendors and/or training your own staff. However, this does not usually require hiring new people because the same team that has to deal with cloud architecture, redundancy, failover, APIs, programming, etc, can work on the ops side of things running your own environment.
- The data centers chosen have to be easily accessible 24/7 because you may need to visit at unusual times. This means having people on-call and available to travel, or paying remote hands at the data center high hourly fees to fix things.
- You have to purchase the equipment upfront which means large capital outlay but this can be mitigated by leasing.
So what does this mean for the cloud? On a pure cost basis, buying your own hardware and colocating it is significantly cheaper. Many will say that the real cost is hidden with staffing requirements but that’s not the case because you still need a technical team to build your cloud infrastructure.
At a basic level, compute and storage are commodities. The way the cloud providers differentiate is with supporting services. Amazon has been able to iterate very quickly on innovative features, offering a range of supporting products like DNS, mail, queuing, databases, auto scaling and the like. Rackspace has been slower to do this but is now starting to offer similar features.
Flexibility of cloud needs to be highlighted again too. Once you buy hardware you’re stuck with it for the long term but the point of the example above was that you had a known workload.
Considering the hybrid model
Perhaps a hybrid model makes sense, then? This is where I believe a good middle ground is and we can see Moz making good use of such a model. You can service your known workloads with dedicated servers and then connect to the public cloud when you need extra flexibility. Data centers like Equinix offer Direct Connect services into the big cloud providers for this very reason, and SoftLayer offers its own public cloud to go alongside dedicated instances. Rackspace is placing bets in all camps with public cloud, traditional managed hosting, a hybrid of the two and support services for OpenStack.
And when should you consider switching? Dell(s dell) cloud exec Nnamdi Orakwue said companies often start looking at alternatives when their monthly AWS bill hits $50,000 but is even this too high?