From dbt Cloud to Core: How a Tech Company Saved $100K+ and Got the Flexibility dbt Cloud Couldn't Offer
Zero-downtime migration with better tooling on the other side
Tony Zeljkovic
2025-04-15
- Industry: Technology
- Duration: ~2 months
- Team: 1–2 Narona Data consultants
- Stack: dbt Core, VS Code devcontainers, custom CLI wrapper, YAML linter, pytest, CI/CD
- Key Results: >$100K annual savings | 30% faster development | Zero downtime migration
Executive Summary
A high-growth tech company had outgrown dbt Cloud. The managed platform couldn't meet their development environment requirements, and every workaround made the team slower. Narona Data migrated them to dbt Core in two months with zero downtime — and delivered custom tooling that made the team 30% more productive than they were on dbt Cloud. The migration saved over $100K annually in licensing costs.
Situation
A fast-growing technology company relied on dbt Cloud for data transformations. The analytics engineering team was growing, and dbt was central to their data workflow.
dbt Cloud had been a good starting point — it simplified setup and provided a managed environment. But as the team's requirements grew more heterogeneous, the managed platform's constraints started to bite.
Complication
The core issue was capability, not just cost:
dbt Cloud couldn't do what the team needed. The team required a development environment that could connect through the company VPN, integrate with lineage tools, support custom database querying, and work seamlessly on local machines. dbt Cloud couldn't deliver this — and workarounds were eating into development time.
Best practices couldn't be enforced. The team needed to prevent expensive patterns like building all upstream models during development. dbt Cloud had no mechanism for this. Without guardrails, costly mistakes were a regular occurrence.
YAML configuration was out of control. Files were inconsistent across environments, missing configurations were only caught in production, and existing tools like dbt-checkpoint lacked the extensibility the team needed.
The $100K+ annual licensing cost made the decision easier — but the team would have migrated for capability alone. The savings were a bonus, not the driver.
Resolution
Narona Data delivered three interventions in an 8-week engagement with a two-person team. The migration was sequenced so the team was never without a working environment.
1. VS Code Devcontainer Environment
The foundation was a custom development environment that matched or exceeded dbt Cloud's capabilities while running locally.
Narona Data deployed a VS Code devcontainer with:
- Warehouse connection through the company VPN
- Built-in lineage tracking and database query tools
- Compatibility with both local machines and remote environments (GitHub Codespaces)
Each team member received the environment, documentation, and a scheduled onboarding session to get set up.
Why local-first: After conversations across the organization, local development was the clear preference. The team wanted their existing machine setups, VPN access, and tool integrations — not another managed cloud environment.
2. Custom dbt CLI Wrapper
Once the team was working in the new environment, Narona Data addressed the best-practices gap that dbt Cloud couldn't fill.
A custom, extensible CLI wrapper around the dbt CLI automatically configured builds to use defer commands and blocked specific costly patterns — like building all upstream models for expensive model chains.
Why a wrapper: Changing team habits through documentation doesn't work. The wrapper made best practices the default and bad practices impossible, without adding friction to the development flow.
3. YAML Linter and Governance Testing
The final piece tackled YAML configuration quality:
Real-time YAML linter — Integrated into the development environment, highlighting missing or incorrect configurations as engineers wrote them. Problems caught in the editor, not in production.
pytest governance framework — A custom testing suite that parsed dbt manifests and validated configurations against organizational standards in CI/CD. Extensible for any governance rule the team needed.
Why custom over dbt-checkpoint: The client had tried dbt-checkpoint and found it insufficiently extensible. A pytest-based approach gave them full control over governance rules and integrated cleanly into their existing CI/CD process.
Results
| Metric | Before | After | Business Impact |
|---|---|---|---|
| Annual platform cost | dbt Cloud licensing | >$100K saved | Budget redirected to data team growth |
| Development speed | Slowed by workarounds | 30% faster | Analytics engineers ship more, iterate faster |
| Migration downtime | — | Zero | No disruption to production workflows |
| Best practice enforcement | None (manual/hopeful) | Automated via CLI | Expensive development mistakes prevented |
| YAML quality | Inconsistent, caught in prod | Real-time linting + CI/CD | Configuration errors caught before merge |
Cost savings are measured (licensing elimination). Development speed improvement is directional based on team feedback.
Ready to Talk?
Spending six figures on dbt Cloud and wondering if you've outgrown it? Narona Data offers a free consultation to assess whether dbt Core with custom tooling would be a better fit for your team.