Building the Server Density World Map

Every 6 weeks we run a “Random week”. During this time we stop working on our current projects and instead create something different that we think will be interesting/useful/educational and, most importantly, fun! Being a design/frontend guy, I’m a big fan of the visual side of things and straight away decided that my next RW project would be some sort of data visualisation based on the server data we have stored as part of Server Density, our server monitoring tool.

[Take a look at the map and then continue reading!]

One of the most common events within Server Density is a customer’s server checking in with our service to provide us with its latest metrics. I decided that displaying these check-in events on a large world map would be an interesting way of observing the overall Server Density usage around the world. With my idea sorted, I moved on to deciding *how* I should build it.

One of the benefits of Random week is that it provides me with an opportunity to experiment with new tools/languages outside of my normal environment. One such language that has been gaining a lot of momentum lately is CoffeeScript, a language that compiles into JavaScript and is:

an attempt to expose the good parts of JavaScript in a simple way.

I’ve heard an incredible amount of positive talk regarding CoffeeScript in the last few months and figured this would be the ideal time to give it a whirl. Similarly, complimentary mentions of tools to extend CSS, namely LESS and SASS, have saturated my Twitter stream lately and promises of “making CSS fun again” had me intrigued.

With an idea and my tools ready to go, I had a realisation – I know absolutely nothing about cartography, or the process of converting location data into usable coordinates on a map. This was obviously a problem, so of course, I did what most people do to find out about a subject: I headed straight on over to the Cartography page on wikipedia, which in turn led me to the Map projection article. After reading the “Which map is best?” section, which suggests that although the “Mercator” projection is the most popular, it is rectangular while the Earth is spherical and therefore does not represent the continents accurately and should be avoided, I decided that I would rather use a map that fills my rectangular monitor and sacrifice some accuracy (Sorry Earth!).

After a bit of photoshopping fun, I had my world map. Next I needed some location data to plot. Not being much of a Python guru, I asked one of our engineers, Ryan, to whip up a script that I could run on our servers collection to convert each server’s IP address into longitude/latitude data. For this we used the excellent SimpleGeo service, specifically the SG Context API, and made a single pass over all of the data resulting in a usable set of server locations. With some further wikipedia-ing on the projection formulas for Mercator I was able to determine x and y coordinates of a point on the map and managed to hook up the server locations to the visual.

Now came the problems. It turns out that animating thousands of check-in events every second was perhaps a tad ambitious. The good news was that the check-in markers were being plotted in the correct locations on the map. The bad news: Lag. Lots of lag.

After a much needed facepalm, I limited the data size to something more manageable and began to see improvements. However, it was still pretty obvious that the browser was struggling. Some quick debugging suggested that jQuery’s animate method was the culprit, so I did a small experiment by replacing the animate call with CSS3 transform inside a CSS3 animation, a feature that allows keyframe animation right in the CSS. The beauty of -webkit-transform, specifically the scale3d() function in my case, is that it is able to make use of hardware acceleration, moving all of the work to the GPU and improving performance dramatically.

Of course the downside to using -webkit prefixed CSS properties is that they will only work in webkit based browsers such as Chrome or Safari. However, with this being a Random week project my usual requirements for cross-browser support went out the window and I figured if it works in my primary browser (Chrome) I’d be happy.

By limiting the data to a subset of our servers database, as well as making use of hardware accelerated CSS3 properties, the world map was working nicely. In fact, it’s fairly mesmerising watching all the markers pulse as a server checks in with us.

Initially I planned for this to be an internal project, however, we decided it’d be pretty cool if everybody could see it, so head on over to http://www.serverdensity.com/map/ to check it out for yourself. Remember you’ll need a Webkit-based browser (it works best in Chrome!)

Lessons learned

• Cartography is a tricky business. There are a lot of different approaches, and a lot of disagreement over which approach is best. No doubt I could’ve spent more time researching this and made a more educated decision. However, I’m fairly pleased with how the world map turned out so I shall consider it successful.
• CoffeeScript enables me to produce much cleaner/readable/manageable code compared to my previous JavaScript. The CS documentation and “The Little Book on CoffeeScript” provided an excellent introduction to the basics of CoffeeScript as well as an overview of idioms and practices that might not be immediately obvious to the traditional JavaScript developer like myself. I’ll definitely be using CoffeeScript in all of my future projects.
• SASS is a great improvement to CSS, enabling me to write better reusable code. I will be using it wherever possible in future projects.

Our Random week projects are always a great reminder that it’s good to take some time from your day-to-day work every once in a while and try something new.