Sacra Logo

How has the world of data engineering changed since the introduction of dbt?

Julia Schottenstein

Product Manager at dbt Labs

Imagine that before you could do any of your work, someone gave you a tool that was really not that great for your job—for instance, something legacy like Talend or Informatica where it's always broken. Or maybe you had to go and ask a data engineer who held the keys to production to create the data assets you needed in your warehouse for you. 

That was frustrating, because when you needed to work, you had this big dependency on a different team to help get your work done. There was a lot of gatekeeping around letting analysts contribute to prod. 

And pre-dbt that was the norm for data transformation. You need to have proper software engineering best practices—like version control, documentation, reviews on your pull requests, CI checks, and tests—and without that, you shouldn't be allowed to push code to prod.

What dbt did was create a path forward such that people who were close to the business logic could also be the ones to deploy their pipelines and create their data tables. dbt created an easy way to collaborate and productionize data transformation work that was accessible to more people.

Find this answer in Julia Schottenstein, Product Manager at dbt Labs, on the business model of open source
lightningbolt_icon Unlocked Report