How to create a Monitoring Dashboard

By Jonathan Sundqvist, Engineer at Server Density.
Published on the 30th June, 2016.

Over the years, Server Density has gone from processing 40GB per month, to more than 250TB. That is a lot of data.

We manipulate that data in various ways producing graphs, monitoring dashboards, and various other tools used by sysadmins to troubleshoot and spot trends in their infrastructure.

But the question has risen recently: can we do more with this stockpile of data? Rather than letting it linger in standby, why not look for ways to tap into this vast resource and make it accessible to everyone out there, sysadmins and non-sysadmins alike.

monitoring dashboard

Making data human

We are not breaking any new ground here. Information is beautiful, flowingdata, infosthetics, datavisualization and numerous others have embarked on a mission to make data accessible to the world. Not just industry insiders but everyone else as well.

Which begs the question: what is the monitoring community doing with all those billions of data points they collect and accumulate each second? How can we unlock its hidden value, not just for our benefit but for everyone else’s too?

The community has worked hard to make monitoring dashboards easy to understand, but what we are missing is a way to communicate infrastructure insights in a way that is more compelling and human-friendly. I was inspired by Stack Exchange and thought: This is something we could give to everyone!

Mind you, even if you don’t have a massive architecture and tons of data, it still makes sense to visualise it for your customers. It’s a great way to demonstrate your investment, illustrate the breadth of your customer base and maybe even advertise the performance and reliability of your infrastructure. So, yes, visualisations are a great marketing tool too.

With all that in mind I set out to create a monitoring visualisation tool. I wanted to avoid the hassle of coding if at all possible. I also wanted the visuals to refresh automatically as data grows, almost like a dynamic infographic of sorts.

So, here is version 1.0 of our dashboard.

Generating the monitoring dashboard

The dashboard is generated every time it gets published. It pulls data from the Server Density API and populates the widgets on the dashboard (Want to use it with another monitoring solution? Just raise a PR!).

We use Travis CI to trigger a build every 24 hours using nightli.es. The .travis.yml contains the settings so that Travis CI can generate the graph. Note that there are a few extra settings you need to configure so Travis CI can work for you.

If you want to publish it manually all you need to do is to use the command make publish and it will be hosted on GitHub pages. Another option is to set up a simple cron job on a server you own. This triggers the make publish command on specific intervals of time. The way to configure this is through a yaml file. Each metric has its own subsection in the configuration file:

monitoring dashboard

Here you can decide how you would like to transform the data and then display it. What’s more, with just a few CSS tweaks you can easily come up with your own ways of displaying the data.

Summary

What use is collecting gobs of monitoring data if we don’t put it to work? Extracting, visualizing, and sharing infrastructure insights with your teams, partners, customers and everyone else, is a fantastic and fun way to highlight all the great work your sysadmins are doing behind the scenes.

Head over to the repo, read the instructions and start visualising your data in no time.

Free eBook: 4 Steps to Successful DevOps

This eBook will show you how we i) hacked our on-call rotation to increase code resilience, ii) broke our infrastructure, on purpose, to debug quicker and increase uptime, and iii) borrowed practices from the healthcare and aviation industry, to reduce complexity, stress and fatigue. And speaking of stress and fatigue, we’ve devoted an entire chapter on how we placed humans at the centre of Ops, in order to increase their productivity and boost the uptime of the systems they manage. What are you waiting for, download your free copy now.

Help us speak your language. What is your primary tech stack?

What infrastructure do you currently work with?

Articles you care about. Delivered.

Help us speak your language. What is your primary tech stack?

Maybe another time