fb

High-level: What a typical analytics setup looks like

First, it is important for you to get a high-level picture of an analytics setup. In this chapter, we talk about the most common setup for an analytics system. In certain places, you will see others doing things differently, but if you look closely, all analytics systems boil down to the same approach.

Let’s get started.

Most basic analytics setups will be broken into three steps:

  1. Collect, consolidate and store data in a central data warehouse.
  2. Process data: transform, clean up, aggregate and model the data that has been pushed to a central data warehouse.
  3. Present data: visualize, extract, or push data to different services or users that need them.

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/da11bbd9-3ce1-4d36-a700-a8a4dfa35776/Untitled.png

Step 1: Collect, Consolidate and Store Data

Before you can analyze your organization’s data, data from multiple sources must be pulled into a central analytics database. This database is usually called a data warehouse, and the process by which this transfer happens is commonly called ETL (Extract Transform Load).

Chapter 2 of the book will go into more detail about this step.

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/9c3eef27-1b04-4d26-a9a8-eb54cc012558/Untitled.png

Since we’re talking about a big picture view in this chapter, there are only two key components you need to understand.

  1. The ETL Process

    As mentioned earlier, ETL stands for ‘Extract, Transform, and Load’. This refers to the fact that you extract data from sources, transform that data into a queryable form for analysis, and then load that data into your data warehouse.

    In Chapter 2, we will explore:

- How do you setup an ETL process? What ETL technology should you choose?
- Why and how the industry is moving from ETL to a process called ELT (Extract, Load, Transform).
- How is ELT different from ETL and how do they both compare?

As ELT is now a more prominent approach than ETL, we will speak more about the ELT approach in this book.
  1. The central analytics database, or “data warehouse”

    This is the place where most of your analytics activities will take place. In this book we’ll talk about:

- Why do you need a data warehouse?
- How do you build one?
- What data warehouse technologies should you choose?

So what do you get at the end of this step?

  • You will have a data warehouse powerful enough to handle your analytics workload
  • You have a process in place that syncs all raw data from multiple sources (CRM, app, marketing, etc) into the central data warehouse.

Once you have these two pieces set up, the next step is to turn raw data into meaningful data for analytics.

Step 2: Processing Data (Transform & Model Data)

In this guide, we assume that you extract data from your data sources and load that data to your warehouse as a first step. The second step is about transforming that data. The process of transforming the data into a form that is useful for business analysis is commonly called ‘data modeling’.

This step is necessary because raw data is often not ready to be used for reporting. Raw data will often contain extraneous information — e.g. duplicated records, test records, or metadata that is only meaningful to the production system — which is bad for analytics.

Therefore, you usually need to apply a “processing step” to such data. You’ll have to clean, transform and shape the data to match the logic necessary for your business’s reporting.

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/3b8fca3c-e2c7-4fac-b8f0-75e7d713fc52/Untitled.png

This step usually involves two kinds of operations:

  • Modeling data: apply business logic and formulae onto the data
  • Transforming data: clean up, summarize, and pre-calculate data

Chapter 3 goes into more detail about these two operations, and compares a modern approach (which we prefer) to a more traditional approach that was developed in the 90s.

Beginner readers take note: usually, this is where you’ll find most of the fun — and complexity! — of doing data analytics.

At the end of this step, you’ll have a clean set of data that’s ready for analysis and reporting to end users.

Step 3: Presenting & Using Data

Now that your data is properly transformed for analytics, it’s time to make use of the data to help grow your business. This is where people hook up a “reporting/visualization tool” to your data warehouse, and begin making those sweet, sweet charts.

Chapter 4 will focus on this aspect of data analytics.

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/30ad6fcf-682a-4d48-8e58-51a7bf7863c3/Untitled.png

Most people think of this step as just being about dashboarding and visualization, but it involves quite a bit more than that. In this book we’ll touch on a few applications of using data:

  • Data reporting, which we’ve already covered.
  • Data exploration: how letting end users freely explore your data lightens the load on the data department
  • Data delivery: how pushing analytics directly to business users helps with spreading insights throughout your organization
  • Data alerts: why it’s a good idea to get notified when certain trigger conditions occur in your data
  • Data actions: how to make an automated workflow to push data from one system to another

Since this step involves the use of a BI/visualization tool, we will also discuss:

  • The different types and mechanism of BI tools
  • A set of criteria with which to choose BI tools