Premium Hosted Website & Server Monitoring Tool.

(Sysadmin / Devops blog)

visit our website

Blog   >   Metrics   >   Visualizing server metrics with sparklines in the terminal

Visualizing server metrics with sparklines in the terminal

As part of our company random week where we work on our own project ideas, I set out to provide a new tool for every ops toolbelt – a Server Density monitoring CLI util.

The goal was a well behaved command line frontend to our server monitoring API that would let you query and get results in human machine readable formats. This would not only make it easier to utilise our API in existing systems, getting around having to write calls in native languages/platforms but also providing you with a quick and simple query tool from within your terminal, the place you’re most likely to be interested in server/service monitoring data.

With that in mind allow me to introduce densli.

densli

What makes a well behaved CLI app?

  • Clear and well defined -h/--help output – this is fulfilled by the excellent Python standard library module optparse
  • Takes data via STDIN as well as via options/arguments – fulfilled by optparse and Kenneth Reitz’ clint
  • Outputs information and output data to STDOUT and all errors to STDERR (and tells you exactly what went wrong, not just throwing an exception)
  • Allows you to silence extra info output to STDOUT (in densli’s case with a -q/--quiet option) so that you can pipe to another process and not deal with extra noise in the output
  • Allow options to be stored in a config file and have that config file in a sane OS-dependent location – hat tip to the clint library making this easy
  • Returns non-zero exit codes in the result of an error

As well as these requirements there are, in my opinion, other concerns when designing the user interface for the command line; just because something is text based doesn’t mean it isn’t also a visual interface. densli uses terminal colours to indicate information and error output, to quickly identify the type of information being output. Of course in case of vision impairment all information and error messages are clearly defined by their text too.

Output format is another consideration. I had planned to provide a more “human” readable output (via an option) using column based display, however the large variety of output from our API made this more difficult and I discovered that JSON reformatted with indentation (thanks to the standard library json or installable simplejson modules) was just as easy to read, while at the same time remaining as fully machine parsable JSON. I would like to add other options in the future however, such as a CSV output.

Sparklines

One special output format that densli does provide is a sparklines graph output for calls to the metrics API. Provide a metrics range, use the -s or --spark flag and you’ll get pretty mini-graphs output right in the terminal like so:

This accomplished using another random week creation, the py-sparkblocks Python module, which takes a list of numbers like [0.1, 3, 5.1, 2.2, 0.02] and returns a unicode string of block characters (unicode characters 9601-9608) representing the range of numbers by relative block height. py-sparkblocks is based on the great CLI app spark by Zach Holman, and it in turn by the original Twitter sparklines generating excel spreadsheet by Alex Kerin.

These “sparkblock” graphs, as I termed them, posed another problem: how best to display a graph of lots of data in a width limited terminal, especially when even a small range of metrics could result in 5 lines of graph output. To mitigate this issue I needed a way of condensing my list of metric numbers down to a shorter yet proportionate list of numbers. After much brain-wracking, I settled on using a naive reduction based on a moving average.

Effectively, I create a list of lists pivoted on a division, e.g. divided by 20 (to ultimately display a maximum of 20 characters, but this can be changed with the max_graph_width config option), so for every 1/20 units I have a list of 1/20 units, and for each of these sub-lists I take the mean average of the units, ultimately giving me a averaged list of 20 units representing the overall proportion of the larger list.

The maths for this isn’t perfect – the proportion won’t always be completely accurate down to the minute detail (especially for large spikes or other points that fall way out of the bell curve over a longer range), but for an overall view of a range of metrics it does the job quickly and in relatively easy to understand few lines of Python.

As it was pointed out to me that one big use case for densli would be quickly looking at a small range, e.g. the past 30 minutes, I also added a -t/--timeago option that takes a flexible relative time format, and works well with the --spark option to quickly get an idea about the recent metrics for a device.

Some example use cases for using densli in your day to day workflow:

densli is of course open source and available via Github for forking and hacking. Any and all pull requests considered, especially if you feel the urge to add linear interpolation to the sparkblock graphs, or just fancy adding to the docs! Keep an eye out in the future for native OS packages to install densli but for the time being you can install it as a Python package:

pip install densli