Pipelines as YAML: the problem
CI/CD pipelines are almost always declarative YAML (GitHub Actions, GitLab CI, Jenkins Declarative, CircleCI, etc.). Problems: not locally testable, vendor lock-in, limited reuse, no rich abstractions. When a pipeline fails in CI after 20 minutes, there’s only one way to debug: re-run.
The Dagger vision
Pipelines as code in real languages: Python, TypeScript, Go, Java, Elixir, PHP. Each step is a typed input/output function. Execution on BuildKit (Docker build engine) for portability, cache and parallelism.
The release
Dagger is published in March 2022 by Solomon Hykes (Docker cofounder), Sam Alba, Andrea Luzzardi. Apache 2.0 licence. Written in Go. First version uses the CUE language; from version 0.3 it natively supports Go, TypeScript, Python via Dagger SDK.
Architecture
- Dagger Engine — container (BuildKit wrapper) that executes pipelines
- Dagger SDK — library per language (Go, TS, Python, PHP, Elixir, Java, Rust)
- GraphQL API — engine exposes typed schema
- Content-addressed cache — layers and steps cached globally
- Portable — runs in CI (GitHub Actions, GitLab CI, Jenkins, local machines)
import dagger, anyio
async def main():
async with dagger.Connection() as client:
python = (
client.container()
.from_("python:3.12-slim")
.with_directory("/app", client.host().directory("."))
.with_workdir("/app")
.with_exec(["pip", "install", "-r", "requirements.txt"])
.with_exec(["pytest"])
)
await python.stdout()
anyio.run(main)The same code runs on dev macbook, GitHub Actions, Jenkins, GitLab: zero difference.
Dagger Cloud
Dagger Inc commercial product: visualisation, shared cache, telemetry, team collaboration. Free tier for open source.
Versions
- 0.1 (March 2022) — CUE only
- 0.3 (June 2022) — Go SDK
- 0.6 (May 2023) — TS, Python SDK
- 0.9 (November 2023) — engine rewrite, Module system
- 0.12 (2024) — Function service, cross-language modules
Competitors / complements
- GitHub Actions, GitLab CI — YAML pipelines, not portable
- earthly — similar build framework, Earthfile DSL
- Pants, Bazel — monorepo build systems
- Drone CI — YAML
- Jenkins Pipelines — Groovy
Dagger complements existing CI (GitHub Actions invokes dagger call) rather than replacing them.
In the Italian context
Dagger is emerging in Italy but used in:
- Python/TS teams with complex monorepos
- Multi-cloud companies wanting portable pipelines
- AI/ML pipelines — Python Dagger for training workflow
- Platform engineering — Dagger as backend of custom platforms
Adoption growing since 2024 especially after module system (0.9+).
References: Dagger (March 2022). Solomon Hykes (Docker cofounder), Sam Alba, Andrea Luzzardi. Apache 2.0 licence. Written in Go. BuildKit engine. SDK: Go, TS, Python, PHP, Elixir, Java, Rust. Dagger Cloud (commercial).