What’s in your Backpack? Modular vs. Monolithic DevelopmentLeave a Comment
While building version 2.0 of our Server Monitoring agent, we reached a point where we had to make a choice.
We could either ship the new agent together with every supported plugin, in one single file. Or we could deploy just the core logic of the agent and let users install any further integrations, as they need them.
This turned out to be a pivotal decision for us. And it was much more than technical considerations that advised it.
Let’s start with some numbers.
How Much Does Your File Weigh?
Simple is better than complex.
The Zen of Python
The latest version of our agent allows Server Density to integrate with many applications and plugins. We made substantial improvements in the core logic and laid the groundwork for regular plugin iterations, new releases and updates.
All that extra oomph comes with a relatively small price in terms of file size. Version 2.0 has a 10MB on-disk footprint.
If we were to take the next step and push every compatible plugin into a single package, our agent would become ten times “heavier”. And it would only keep growing every time we support something new.
Moving is Living
Question: But agent footprint is not a real showstopper, is it? Why worry about file sizes when I can get everything I need in one go?
There is something to be said about the convenience of the monolithic approach. You get everything you need in one serving.
And yet, it is the nature of this “component multiplicity” that makes iterations of monolithic applications slower.
For example, when a particular item (say, the Python interpreter or Postgres library) is updated by the vendor, our users would have to wait for us to update our agent before they get those patches. Troubleshooting and responding to new threats would therefore—by definition—take longer. This delay creates potential attack vectors and vulnerabilities.
Even if we were on-the-button with every possible plugin update (an increasingly impossible feat as we continue to broaden our plugin portfolio), the majority of our users would then be exposed to more updates than they actually need.
Either way, it’s a lousy user experience.
To support all those new integrations—without introducing security risks or needless headaches for our users—was not easy. It took a significant amount of development time to come up with an elegant, modular solution that is simple—and yet functional—for our customers.
The result is a file that includes the bare minimum: agent code plus some specific Python modules.
Flexibility and Ease of Use
To take advantage of all the new supported integrations, users may choose to install additional plugins as needed.
Question: Doesn’t that present challenges in larger / diverse server environments?
Sysadmins continue to embrace Puppet manifests, Chef configuration deployment and Ansible automation—tools designed to keep track of server roles and requirements. It’s easier than ever to stay on top of what plugin goes to what server. Automation and configuration utilities can remove much of that headache. Since we tie into standard OS package managers (deb or RPM packages), we simply work with the existing tools everyone is already used to.
By packaging the plugins separately we get to focus on what we control: the logic inside our agent. Users only ever download what they need, and enjoy greater control of what’s sitting on their servers. The end-result is a flexible monitoring solution that adapts to our users (rather than the other way around).
The 1.x to 2.0 agent upgrade is not automatic. Existing installations will need to opt-in. We’ve made it easy to upgrade with a simple bash script. Fresh installs will default to version 2.0. The 1.x agent will still be available (but deprecated). All version 1.x custom plugins will continue to work with the new agent too.
Truth is ever to be found in simplicity, and not in the multiplicity and confusion of things.
The modular vs. monolithic debate has been going on for decades. We don’t have easy answers and it’s not our intention to dismiss the monolithic approach. There are plenty of examples of closed monolithic systems that work really well for well-defined target users.
Knowing our own target users (professional sysadmins), we know we can serve them better by following a modular approach. We think it pays to keep things small and simple for them, even if it takes significantly more development effort.
As we continue improving our back-end, our server monitoring agent will support more and more integrations. Employing a modular development, means prompt updates with fewer security risks. That’s what our customers expect, and that’s what drives our decisions.
What about you? What approach do you follow?