We move relentlessly from an analog world to a digital one. With so many startup developments and technology process innovations emerging like moss in moist air, many mature companies have begun to move towards a "startup style" working environment. This in turn means that large companies must engage in culture-change and process-transformation efforts within an industrial context.
These efforts are often called 'digital transformation efforts', or simply 'transformation efforts'.
What do we mean when we say digital transformation? This is a little tricky — because digital transformation looks different for each company, it is somewhat challenging to come up with a definition of digital transformation that applies to all companies. However, broadly speaking, we define digital transformation as the integration of digital technology into all business areas that results in fundamental changes in how businesses operate and how they deliver meaningful value to customers.
But are companies really doing digital transformation, or is it simply a manual mindset that is being forcibly digitized, like an organization blindly moving spreadsheets from Microsoft Office to Google Suite?
According to a Forbes article (citing research from global consulting firm McKinsey), about 70% of companies fail to successfully complete large-scale digital transformation programs — a set that includes prestigious companies like GE, Ford, and Procter & Gamble! Digital transformation is simply really difficult.
An interesting nuance to this trend is that business intelligence is subject to the same digital transformation headwinds. On the face of it, this is an odd thing to say. Business intelligence remains an important function within the corporation because it is a requirement for speedy business decision-making. As a result, the field of business intelligence has been around for more than 60 years ... and BI functions were in fact one of the first to be digitized within the modern corporation!
However, legacy BI platforms are beginning to show their age. Such systems enforce specialization amongst members in the data team (dividing data teams into analyst, engineer, and business user), create dependencies within a collection of tools, and demand that organizations employ entire departments with specialized expertise, in order to use such large-scale enterprise solutions.
Here's a concrete example, for reference. In a large multi-national bank, one of the data analysts we spoke to told us that every new report requested by a business unit must go through a complicated internal request-and-cost-accounting process. For every new report, an analyst must make a request to a data engineer for an ETL pipeline in order to produce a dataset with the required set of attributes. It takes about a month for this request to pass through and for the engineer to deliver the dataset for consumption. If a miscommunication occurs between engineer and analyst, another month must pass before the error is corrected.
This is all well and good if the bank faces no competition. But we live in a time of globalization and digital markets. Nimble-footed startups and market entrants from global competitors threaten the bank's competitiveness.
Take Amazon, for example: its business units are able to plug into data systems within days, not weeks. Or Airbnb: its operation teams are able to access internal data reporting systems without ever interacting with a member of the data team. Neither of these companies rely on legacy business intelligence systems.
The truth is that legacy business intelligence tools are limited by context in which they first arose. In the early years of digital transformation, computing power was limited and very expensive. As a result, BI tools did one thing, and one thing well, and were designed to work in concert with other tools. You had to purchase a data warehouse, a large computing cluster, an ETL tool, and a visualization tool, all of them separately. You then had to set up an entire department to keep the whole shebang humming.
To put this simply, early digital transformation efforts consisted of 1) the procurement of a large set of software and hardware tools, and then 2) the setting up of teams to manage the procured infrastructure. This was an incredibly complicated process. As such, we shouldn't be very surprised by the McKinsey statistic of a 70% failure rate for large-scale digital transformation programs.
But there is good news: a new generation of BI tools have emerged in the past 10 years to kill the inefficiencies of these legacy systems. They have taken advantage of technological advancements in the decades since business intelligence first emerged, and are beneficiaries of the industry-wide move to the cloud.
These tools are tools like Looker, Sisense, Holistics, dbt, and Chartio. They run on massively scalable cloud-based data warehouses like BigQuery, Redshift, and Snowflake. They allow companies to run lean data teams, and to decentralize access to business data throughout the company.
More importantly, however, these tools are designed to reduce 'silos' within the company. No longer must an analyst request a new pipeline in order to deliver important and timely intelligence to a business unit. With these new tools, operators are able to grab the data they need whenever they need it, with less dependence on data analysts or data teams in general. Data engineering becomes less complicated. Data teams take on the role of enablers, instead of becoming bottlenecks.
This emerging trend should be of interest to all us, and not just to those of us who are data geeks. At Holistics, we are beginning to see a second wave of digital transformation within older industries, some of whom are becoming our customers. Newer startups and SMEs may the ground running using this newer generation of tools. But older companies are slowly engaging in digital transformation using such tools as well. They do so because it is cheaper, faster, and less complex to implement business intelligence this way. They do so because it reduces the risk of transformation failure.
This transformation is how we think companies will work with data in the future, and we're excited to be a part of it.
Sign up for our BI newsletter
Insights from practitioners around the globe.
In your inbox. Every week.
No spam, ever. We respect your email privacy. Unsubscribe anytime.
From SQL Queries To Beautiful Charts
Connect to your database and build beautiful charts with Holistics BILearn More
"Holistics is the solution to the increasingly many and complex data requests from the operational teams"
Tang Yee Jie
Senior Data Analyst, Grab