dbt — Data Build Tool for SQL Transformations

What it is

dbt (data build tool) turns SQL into a software-engineering discipline for the analytics layer of your data warehouse. Analytics engineers write modular SELECT statements, and dbt compiles them into views, tables, and incremental models with dependency graphs, tests, documentation, and CI.

dbt supports Snowflake, BigQuery, Redshift, Databricks, PostgreSQL, Spark, and many more data platforms through adapter plugins. It is used by thousands of data teams to manage their transformation layer.

How it saves time or tokens

dbt eliminates the manual management of SQL dependencies and execution order. Instead of maintaining a list of which queries must run before others, dbt infers dependencies from ref() calls and builds a DAG automatically. Tests (not_null, unique, accepted_values, custom) run against every table after each build, catching data quality issues before they reach dashboards. The docs site is auto-generated from the project, eliminating separate documentation maintenance.

How to use

1. Install dbt with an adapter:

pip install dbt-postgres

1. Scaffold a project and run:

dbt init analytics && cd analytics
# Configure ~/.dbt/profiles.yml with warehouse credentials
dbt deps          # Install packages
dbt seed          # Load reference CSVs
dbt run           # Materialize models
dbt test          # Run schema + data tests
dbt docs generate && dbt docs serve  # Browse lineage

1. Write a model as a SELECT statement:

-- models/marts/dim_customers.sql
WITH customers AS (
    SELECT * FROM {{ ref('stg_customers') }}
),
orders AS (
    SELECT * FROM {{ ref('stg_orders') }}
)
SELECT
    c.customer_id,
    c.name,
    c.email,
    COUNT(o.order_id) AS total_orders,
    SUM(o.amount) AS lifetime_value
FROM customers c
LEFT JOIN orders o ON c.customer_id = o.customer_id
GROUP BY 1, 2, 3

Example

Testing and documenting a dbt model:

# models/marts/schema.yml
version: 2
models:
  - name: dim_customers
    description: Customer dimension with lifetime value
    columns:
      - name: customer_id
        tests:
          - unique
          - not_null
      - name: email
        tests:
          - unique
          - not_null
      - name: lifetime_value
        tests:
          - not_null
          - dbt_utils.accepted_range:
              min_value: 0

Related on TokRepo

• Database tools — More data tools on TokRepo.
• Automation tools — Browse data pipeline automation.

Common pitfalls

• Not using ref() for model dependencies breaks the DAG and causes stale data. Always reference other models with {{ ref('model_name') }}.
• Running dbt run without dbt test means data quality issues go undetected. Always run tests after builds in your CI pipeline.
• Writing all models as tables wastes warehouse storage. Use views for lightweight transformations and tables only for heavily queried models.