Time Series Charts with React, Redux and D3

By Richard Powell, Frontend Engineer at Server Density.
Published on the 29th March, 2017.

For 2 years Server Density used Backbone and Rickshaw to render server and website monitoring data into time-series charts. Rickshaw is a great library but we had to write a lot of code to bend it to our requirements. Time was not kind to that code. The codebase migrated towards React from Backbone and then from Backbone to Redux, but this code remained static and became an area we were hesitant to touch. Every codebase has these areas, unfortunately for us ours was the time series code.

We have just finished a complete rewrite of our time series charts using React, D3 and Redux. Now this has been released to customers, I’m going share how we developed this code and its high level architecture. If you like React, Redux, Data Visualisation, architectural challenges or if you want insight on approaching a large re-write then read on!

Gathering Requirements

Server Density’s time series charts have pretty well-defined requirements:

  • Line and stacked area render types.
  • Spark-line, single axis or multiple axis charts.
  • Complete control over scales.
  • Tooltips with lots of data that work in tandem with every chart on the page.
  • Chart design is important to us, so efficient use of pixels and a polished design.
  • Handle large amounts of data whilst still remaining performant.
  • The ability to click through to the snapshot view on specific data points.

These were requirements that absolutely had to be met but I had some soft requirements in mind too. Mainly, I wanted to make sure that our developers would not have to worry about complexities like layout and scales. Instead I wanted our developers only concerns to be the helper functions being used, the props specified and any components composed.

Early Development

So that we could be sure what we developed worked in production, I grabbed a sample of the time series data that our charts render. This way we could plug the new charts into the existing data layer and limit the refactor to the view layer.

To develop the charts inside our application without worrying about keeping pages up to date with master, I added a test page. Gradually we would add multiple examples, and requirements were fulfilled.

(Easter Egg: This page still exists and can be viewed by manually browsing to /react-charts-test-page when you log in to your account).

At Server Density all our engineers get a random week every 2-3 months. This is a week where we can work on anything we like, so long as it has some link to the product or company. I chose to work on time series charts, which is how this project to build a core part of our product started out. Each week I considered the requirements and fleshed out the architecture. This way, we soon had control over the y axis, 2 render types and support for multiple axis:

Early graphing prototype

By the end of my second random week we had stacked area charts, tooltips, y axis with dynamic widths, colour control, axis that were entirely optional and the presentation was a lot more polished:

Design added

This progress was encouraging but it was deceptive. 80% of functionality existed, but that only took 20% of the development time. The remaining 80% would be spent on additional tooltip functionality, separating component concerns, unit tests and architecture refactors. Out of all of this it was the architecture refactors that took the most time, but I think it was worth it.

Architecture

The architecture of the new charts loosely follows 3 statements: React for composability, Redux for performance, helpers wrapping D3 for data visualisation heavy lifting. I’m going to discuss how React & Redux fit into the architecture now.

Configuration Through Component Composition

Very quickly we saw that multiple composable, declarative components made it easy to specify a chart’s configuration. But what does this mean?

Consider the following spark-line example:

<Chart
    width={600}
    height={100}
    data={chartData}
    scales={chartScales}
>
    <LineChart/>
</Chart>

In this example the chart is 600 pixels wide and 100 pixels high and it renders a line chart. Now let’s look at an example for a chart with multiple axis:

<Chart
    width={600}
    height={100}
    data={chartData}
    scales={chartScales}
>
    <YAxis position="left"/>
    <LineChart/>
    <YAxis position=“right”/>
    <XAxis />
</Chart>

This chart will also be 600 pixels wide and 100 pixels tall. It will render a line chart, has an x axis and multiple y axis. If that explanation was not needed then I believe the component API is declarative.

This architecture has advantages beyond simple configuration:

  • The parent chart component can position child components so that the axis uses only the space they need and the chart uses the rest. Because this intelligence is abstracted, the developer needs only specify one dimension and it will just work.
  • Properties that are needed by multiple child components can be calculated once by the parent chart component and passed via props to children transparently. Components can be specialised. Our most specialised component for example is a mouse events component that doesn’t render anything the user can see. All it does is translate mouse positions to points in time before deciding if it should inform the wider world of the event.

Performance Through Redux

By default React has quite a simple mental model. Data down, actions up, if something changes re-render everything below it. Whilst this works very well 90% of the time, there are situations when it won’t be fast enough. We had one of those situations, which is demonstrated in this GIF:

Performance Through Redux

In this GIF each chart responds to a mouse event on every other chart by updating the tooltip. Unfortunately, because every mouse event would re-render every chart component, large data sets incurred a noticeable lag. That’s because we were using an architecture like so:

Old architecture

Fortunately, React-Redux provides a convenient way to connect state to specific components bypassing the component hierarchy entirely: containers. Instead of using setState above the chart components, we could use Redux to update the store state. React-Redux connect could ensure that only the components that had to update would receive new props. The new architecture would look something like this:

New architecture

The important point is that only one component renders when the event or action occurs.

This pattern is certainly not unique to our time series charts, in-fact it’s pretty common to React and Redux applications. What is interesting though is the properties our chart tooltips have that make this pattern so useful, specifically:

    • Events can happen frequently, in this case every time the mouse moves.
    • State needs to be shared amongst multiple components, in this case every chart tooltip.
    • It is difficult to make the render methods of those components fast, in this case it is because we need to calculate so much for each render. Arguably we could have cached the calculation results in component state, but that would have added a great deal of complexity.
    • shouldComponentUpdate can only yield limited returns, in our case because of the complexity of our props and the fact that a child may need to update, whilst a parent does not.

The next time you have a performance issue with React, perhaps you might consider the above 4 points and arrive at the conclusion that Container components are the way forward.

Is The Architecture Successful?

From a developer’s perspective React, Redux and D3 is a match made in heaven. React gives you easy composability, Redux gives you easy and performant state management and D3 excels at the data visualisation heavy lifting. These 3 tools have allowed us not only to fulfil the requirements, but to establish an architecture that has made previously complex functionality quite trivial. For example highlighting when data is missing:

Missing data example

Or improving accessibility for color blind users whilst also making it easier to track a series on a busy chart:

Highlighting

But have users noticed the difference? Yes, in many ways. My fondest memory in this respect is whilst we were testing the new charts internally, our team only wanted to look at the new charts and not the old ones, commenting on the many improvements. We also know of customers who appreciate the extra visual polish and the extra robustness around interactions. We know which customers reported issues the new charts have now fixed.

The Future

The big “coming soon!” headline has to be drag to zoom (which benefits greatly from the architecture described above):

Zoom

We know customers would like to see new render types and more control over the y axis. Personally I’d also like to overlay alert data onto charts. But I’m most interested in what you would like to see, so please email, tweet, or comment below.

I am working to open source this code, but it will take some time. It’s tricky extracting code like this from a large codebase and it would be no use without good documentation and examples. If you’d like to see this code then please email, tweet or comment below.

  • receptor7

    Richard, I understand you have written the chart components yourself from scratch. This is some great work :) Have you looked at opensource libraries like react-d3? They might have already solved some of your requirements.

    • Richard Powell

      Hey, thanks for the kind words and thanks for taking the time to comment.

      I looked at quite a few libraries but unfortunately none fitted our requirements well.

      Often libraries will offer the headline features but they won’t work exactly as you need. Sometimes this will be complete deal breakers like incompatible data granularity or not handling missing data. Other times it will be a long list of small things like how axis are drawn, ticks are configured, tooltips are customised, events that are missing etc… These issues add up and eventually compromise either user or developer experience. Time series charts are core to our product offering, so I did not want to compromise.

      That being said I did learn a great deal from other libraries. For example I spent the most time looking at http://software.es.net/react-timeseries-charts/ which influenced a lot of the early, and since refactored, architecture.

  • moshensky

    Hi Richard, thank you for this post. I’d like to explore your source code.

    • Richard Powell

      Hi Nikita, sure thing.

      Please could you email hello@serverdensity.com and I will get back to you.

      Richard

  • juanda95

    I’d like to explore the source code too.
    Congratulations on the project it looks really amazing.

  • Ian Wright

    Interesting article, if you’re willing I’d be up for taking a look at your code too. I’m building custom visualizations in D3 for much of the same reasons that you highlighted below – and while we have heavy use of react/redux in other parts of the product I’ve not touched it for our charts as I’ve yet to seen a strong benefit.

    I guess I’d ask – what was the main drive for use Redux/React to do this and what advantages has it brought about? As I believe our current architectures are actually quite similar – we use a layering system and an event/notification system to communicate between these layers not too dissimilar to your React-Redux image example. I’m wondering if there are any advantages over the two.

    • Richard Powell

      Hi Ian, Thanks for taking the time to comment and sorry for my slow response. I’ve been on vacation. If you’d like to email hello@serverdensity.com we’ll happily send you some source code.

      An early version of this actually used a mediator for events, which might be similar to the events system you describe. It wasn’t very idiomatic though and it lacked some of the advantages of a global store. There are too main advantage from what I see:

      1. In my experience event systems are a bit of a black box and they get very hard to see what effects what.
      2. As more functionality is added you do not need to make more components aware of the mediator, it’s all just the store.

      On this second point a concrete example: Recently I’ve been implementing drag to zoom. One behaviour of zooming is that if the user changes the date via the date-picker we must remove the zoom. By using a global store the two components (zoom & date picker, which are un-related) can remain dumb and de-coupled. Yes, a mediator could be shared between them, but in my view that won’t scale as well as the global store will.

  • Ryan Lee

    Great Article!, Thanks Richard
    I am developing simmlar things to use in my office so I’d like to explore the source code too.

    • Richard Powell

      Hi Ryan, If you haven’t already please could you email hello@serverdensity.com and we’ll give you a link with the source code.

  • wiesson

    I encountered performance issues while displaying large amount of data with SVG based graph libraries (such as D3). How did you solve this?

    And how did you manage D3 and the virtual-dom?

    • Richard Powell

      Thanks for taking the time to comment.

      With rendering performance SVG can only be so fast, eventually large data sets will hit a performance bottleneck. We’ve observed this on dashboards with 6 months of data and lots of series but we haven’t fixed it yet as it is a very uncommon use case. Our most likely course will be to decrease the granularity of the data. This can be a valid approach if the granularity exceeds that which the user can distinguish (100 points in a 50pixel space, for example) and time series databases offer this kind of functionality. Another, but more involved option, is to render to canvas. D3 supports this. This will be faster, but there will still be a bottleneck eventually.

      Mostly we don’t use D3 for rendering (we calculate our own axis for example). As such we don’t need to worry about connecting D3 to the virtual DOM. One hack we do have is that we use D3 to generate the d attribute of paths, which we then copy over to JSX attributes. This is a bit nasty but easy to refactor later.

      Currently our time series data comes from old Backbone collections but we are in the process of moving this to Redux. We only refresh data every minute so I don’t expect performance problems, in-fact I expect to achieve performance optimisations like preventing duplicate requests, caching data and making smaller requests.

Articles you care about. Delivered.

Help us speak your language. What is your primary tech stack?

Maybe another time