From dbt Cloud to Core: How a Tech Company Saved $100K+ and Got the Flexibility dbt Cloud Couldn't Offer

Audit & Team Assessment

The engagement started with an audit of the existing dbt Cloud setup, team interviews, and security policy review.

Key Findings

• dbt Cloud couldn't meet the team's development environment requirements — VPN access, lineage integration, custom querying
• The team preferred local development over another managed cloud environment
• No mechanism existed to enforce best practices (defer commands, blocking expensive upstream builds)
• YAML configuration quality was inconsistent — errors caught in production, not during development
• Existing tools (dbt-checkpoint) lacked the extensibility the team needed for custom governance

Migration Strategy

A phased approach was chosen: devcontainer first (foundation), then CLI wrapper (best practices), then YAML tooling (governance). Each phase was sequenced so the team always had a working environment — zero downtime was a hard requirement.

VS Code Devcontainer

The foundation of the migration: a custom development environment that matched or exceeded dbt Cloud's capabilities.

What Was Built

A VS Code devcontainer with warehouse connection through VPN, lineage tracking, and database query tools. Compatible with both local machines and GitHub Codespaces for flexibility.

Why devcontainers over alternatives: The team needed a reproducible environment that worked identically across machines, supported VPN, and didn't require each engineer to maintain their own dbt setup. Devcontainers encapsulated everything — one setup, every machine.

What This Enabled

The team could develop dbt models locally with full warehouse access, lineage visibility, and query capabilities — things that had required workarounds or were impossible in dbt Cloud.

Custom dbt CLI Wrapper

With the team working in the new environment, the next gap to close was best-practice enforcement.

What Was Built

An extensible CLI wrapper around the dbt CLI that:

• Auto-configured builds to use defer commands (avoiding full upstream rebuilds)
• Blocked specific expensive patterns like building all models in long dependency chains
• Was easily extensible for future rules as the team's patterns evolved

Why a wrapper instead of documentation: Telling engineers "don't build all upstream models" doesn't prevent it. The wrapper made the right thing automatic and the wrong thing impossible — without adding steps to the development flow.

Iterative Refinement

The wrapper went through several iterations with the team. The initial rules were too aggressive in some cases and not aggressive enough in others. Through experimentation, the team landed on patterns that improved efficiency without creating friction.

YAML Linter & Governance Testing

The final build phase tackled configuration quality — the last major gap from the dbt Cloud migration.

Real-Time YAML Linter

A custom linter integrated into the development environment that highlighted missing or incorrect YAML configurations as engineers wrote them. Errors surfaced in the editor immediately, not in production.

pytest Governance Framework

A testing suite that parsed dbt manifests and validated configurations against organizational standards. Integrated into CI/CD, it caught governance violations before merge.

Why custom over dbt-checkpoint: The client had tried dbt-checkpoint and found it couldn't be extended for their specific governance rules. A pytest-based approach gave full control — any rule that could be expressed as a manifest query could become a test.

What This Enabled

YAML quality became enforceable and measurable. The team went from "we hope configurations are consistent" to "CI blocks inconsistent configurations."

Team Onboarding

With the devcontainer ready, each team member needed to be set up and comfortable before the old environment was retired.

Onboarding Approach

Documentation was written for the new environment, and scheduled 1:1 sessions were held with each team member to get them set up. This wasn't a group demo — each engineer's machine and workflow had specific considerations.

Why individual sessions: Engineers had different local setups, VPN configurations, and workflow habits. Group training would have papered over individual issues. One-on-one sessions ensured everyone was actually productive, not just "onboarded."

This phase ran in parallel with the CLI wrapper work — onboarding didn't block the next build phase.

Migration Complete

The migration was complete with zero downtime — the team transitioned seamlessly from dbt Cloud to dbt Core with custom tooling.

What Made It Work

Three factors combined:

1. Phased migration with zero downtime — The team always had a working environment. No "dark period" where productivity dropped.
2. Custom tooling that exceeded dbt Cloud — The devcontainer, CLI wrapper, and YAML linter weren't just replacements — they were upgrades. The team was more productive after the migration, not just cheaper.
3. Iterative refinement — The CLI wrapper rules and governance tests were tuned through experimentation with the team, not imposed top-down. Adoption was natural because the tools fit the team's actual workflow.