Analytics as Code

(This post starts out as a newsletter issue. Subscribe to receive our newsletter in the future)

Some of you might know that I was a data engineer in a previous life. I wrote pipelines, poked Redshift whenever it fell over, and schlepped data from system to system.

(Now I run a business intelligence software company, so maybe I haven't changed jobs that much. Oh well).

I want to tell you a story about that old life.

Most companies at the time, circa 2010-2014, used Jenkins as a continuous integration tool. My company was no different. We would write code, push it to our main repository, and then the code would be pulled down to our Jenkins server where Jenkins would run a build, followed by our test suite.

If our tests failed, everyone got an alert.

Over time, however, the Jenkins server became this special snowflake that nobody really dared to touch. And no wonder: Jenkins was configured using a hodgepodge of scripts and a graphical user interface. Someone set up the server, and then someone else modified Jenkins, and then someone else fixed a broken Jenkins configuration, and so on, until nobody was really sure how to set up our continuous integration server the same way ever again.

So nobody did.

I tell this story because business intelligence software reminds me of this. (Yes, yes, Holistics included). If you want to set up dashboards in any mainstream BI tool, it's likely that you'll have to do so through a combination of SQL and GUI-poking.

But it's worth asking: does it have to be this way? And also: are there benefits of adopting code as a source of truth?

We certainly think so. In the years since Jenkins, much of the CI/CD world adopted programmatic configuration. Heck, most of the developer infrastructure world shifted to infrastructure-as-code. The idea was compelling, once discovered: configure your systems once, express it as code, and then chuck it into a version control system.

At Holistics, we've been playing with this vision of the future for a few months now. Some of you probably know about it, if you've poked around our home page. We're getting close to testing analytics-as-code with early customers, and we think we might be on to something.

You can check out the landing page here; I'm probably going to muse about some of these ideas in the blog/newsletter in the coming weeks.