What’s happening in the modern data stack: Drivers, trends, and opportunities

  • Second, there are more users of this data within every organization. Businesses no longer have one data analytics team that generates one report focusing on one type of data or a handful of static metrics. Teams across different business units need more consistent access to data to measure KPIs and answer function-specific questions very quickly.
  • Third, there are more use cases for data. Data’s value becomes imperative even at the highest levels of an organization, with CEOs/CFOs/CMOs demanding quick innovation/iteration to drive efficiency and value for their businesses. At the executive level, the use cases for the data we have today are limitless.
Source: Bessemer State of the Cloud 2021, Internal Analysis
  1. Integration tools (ETL/ELT), which extract data from sources and process it through data pipelines into one set for insights
  2. The storage/management layer, which includes data warehouses/lakes — the central location where processed data is stored. The warehousing layer also includes transformation tools, which turn raw data into clean data for specific use cases
  3. Business analytics (BI) and AI/ML tools sit at the top of the stack and often plug directly into the data warehouse, allowing members of organizations to drive functional insights
Source: Capital G
  1. ETL/ELT: Extract, Transform, Load (the key process that occured within the integration layer) became Extract, Load, Transform, as data no longer needed to be trimmed down through to save storage costs before entering the warehouse. And reverse ETL tools emerged on the warehouse back-end with specific use cases for BI teams
  2. Transformation: In conjunction with ELT, data manipulation moved into the warehousing layer
Source: Capital G
  1. Self-serve business analytics tools
  2. Next-gen go-to-market platforms
  3. ML workflow tools (ML-ops tools)
Source: Company discussions

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store