Pipelines as YAML: the problem
CI/CD pipelines are almost always declarative YAML (GitHub Actions, GitLab CI, Jenkins Declarative, CircleCI, etc.). Problems: not locally testable, vendor lock-in, limited reuse, no rich abstractions. When a pipeline fails in CI after 20 minutes, there’s only one way to debug: re-run.
The Dagger vision
Pipelines as code in real languages: Python, TypeScript, Go, Java, Elixir, PHP. Each step is a typed input/output function. Execution on BuildKit (Docker build engine) for portability, cache and parallelism.
The release
Dagger is published in March 2022 by Solomon Hykes (Docker cofounder), Sam Alba, Andrea Luzzardi. Apache 2.0 licence. Written in Go. First version uses the CUE language; from version 0.3 it natively supports Go, TypeScript, Python via Dagger SDK.
Architecture
- Dagger Engine — container (BuildKit wrapper) that executes pipelines
- Dagger SDK — library per language (Go, TS, Python, PHP, Elixir, Java, Rust)
- GraphQL API — engine exposes typed schema
- Content-addressed cache — layers and steps cached globally
- Portable — runs in CI (GitHub Actions, GitLab CI, Jenkins, local machines)
import dagger, anyio
async def main():
async with dagger.Connection() as client:
python = (
client.container()
.from_("python:3.12-slim")
.with_directory("/app", client.host().directory("."))
.with_workdir("/app")
.with_exec(["pip", "install", "-r", "requirements.txt"])
.with_exec(["pytest"])
)
await python.stdout()
anyio.run(main)The same code runs on dev macbook, GitHub Actions, Jenkins, GitLab: zero difference.
Dagger Cloud
Dagger Inc commercial product: visualisation, shared cache, telemetry, team collaboration. Free tier for Open Source.
Early versions
- 0.1 (March 2022) — CUE only
- 0.3 (June 2022) — Go SDK
The roadmap announces TypeScript and Python SDKs in the coming months.
Competitors / complements
- GitHub Actions, GitLab CI — YAML pipelines, not portable
- earthly — similar build framework, Earthfile DSL
- Pants, Bazel — monorepo build systems
- Drone CI — YAML
- Jenkins Pipelines — Groovy
Dagger complements existing CI (GitHub Actions invokes dagger call) rather than replacing them.
In the Italian context
Dagger is emerging in Italy, with early experiments in:
- Python/TS teams with complex monorepos
- Multi-cloud companies wanting portable pipelines
- AI/ML pipelines — a future Python SDK will be able to orchestrate training workflows
- Platform engineering — Dagger as backend of custom platforms
References: Dagger (March 2022). Solomon Hykes (Docker cofounder), Sam Alba, Andrea Luzzardi. Apache 2.0 licence. Written in Go. BuildKit engine. Initial SDK in Go. Dagger Cloud (commercial).