Written by David Mytton

It’s been my experience that all cloud providers share at least one property – variable performance. In this context, “cloud” means “virtualised instances”, of which there could be many running on a single host. In contrast to a private cloud where you have complete control of the host, public clouds give you no visibility of the underlying hardware. This means someone else could affect the performance of your VM e.g. suddenly bursting CPU usage, or performing high i/o operations on local storage (this isn’t to say it’s not possible in a private cloud, just you have more control in those situations).

As a result, you can usually see variable performance depending on:

• Time of day: variable demand for the whole infrastructure all the way down to the host you’re on. This is the case for consumer sites e.g. accessing Facebook more often at lunchtime as well as B2B sites e.g. accessing a document management system during working hours.
• Time of year: Black Friday/Cyber Monday sales vs summer holiday lulls.
• Real world events: major sporting events or certain news incidents.
• Internet events: an Amazon data centre going down causing load to be transferred to another region.
• Anything else you can think of!

This is fine for use cases where you can easily deal with the changing performance but some applications need guaranteed performance. Let’s look at some specific cases.

• Load balancer. Should I run in the cloud? No. If all your traffic is running through a single or set of load balancers you want to be sure that traffic won’t be slowed down by contention. How much you are affected will depend on how the load balancer is implemented. For example, we use pound which is entirely in memory and is less likely to be hit by performance issues due to CPU or i/o contention. In our own testing we occasionally saw CPU performance drop significantly (high load) on our VMs due to host activity without adversely affecting the load balancer. However, as a critical portion of the infrastructure we elected to run these on dedicated nodes.
• Web server. Should I run in the cloud? Yes. Web servers are ideal to run in the cloud so long as you have an intelligent load balancer which can detect how loaded each node is. This is often done through response time checks so with many nodes on different hosts, you can mitigate individual host performance problems. Web servers also more likely to be candidates for quick scaling to handle more traffic, which virtualisation makes easier.
• Database server. Should I run in the cloud? Maybe. Databases are the classic example where you need maximum i/o performance and so should always run on dedicated instances with dedicated storage. However, with the availability of new i/o optimised instance types and EBS from the likes of Amazon, this decision is no longer clear cut. In our case for Server Density we run our primary data store on dedicated hardware but have several lower usage/priority clusters on virtualised instances. What performance guarantee options your cloud provider offers will come into play here.
• Mail servers. Should I run in the cloud? Yes. But does anyone really run their own mail servers these days?! Mail is one of the few applications where a few seconds (or more) delay isn’t that important so are a good candidate for running on the cloud.

Of course in reality, these choices are rarely sweepingly generalised yes/no answers for “should I run in the cloud?”. The above is more of a general guideline with some ideas of what to think about when you’re really asking the question “How much will I be affected if performance suddenly drops?”. The answer to that will tell you whether you should be running that application on virtualised instances or not, and will often change over time as usage grows (or shrinks).

Enjoy this post? You may also like How to configure nginx as a load balancer