DBT E-COMMERCE DATA TRANSFORMATION PIPELINE

The E-Commerce Data Transformation Pipeline is a complete analytics engineering project that builds a dimensional data warehouse for e-commerce operations. It demonstrates the modern data stack approach by combining dbt for transformation logic and DuckDB for lightweight, file-based analytical storage.

The project ingests raw CSV data for users, products, orders, and promotions, then transforms it through a structured dbt pipeline. Staging models clean and prepare the source data, intermediate models handle complex joins and enrichments, and mart models publish clean dimension tables and a fact table optimized for analytics.

All transformed data is served through an interactive Streamlit dashboard that enables business stakeholders to explore KPIs, product performance, and customer insights without writing SQL. The pipeline is designed to be reproducible, testable, and easily extensible for additional data sources or metrics.

TECH STACK

KEY FEATURES

• Modular dbt transformation pipeline with staging, intermediate, and mart layers
• Automated data quality tests and validation using dbt expectations
• Interactive Streamlit dashboard for exploring transformed analytics
• CSV data generation for reproducible development and testing
• Lightweight analytics warehouse using DuckDB with zero infrastructure overhead
• Dimensional modeling with conformed dimensions and a fact table