Salt: parallel remote execution and configuration management over ZeroMQ

Linux Services & Systems

Domains, hosting, certified email, email infrastructure, network services and Linux systems. Open Source infrastructure support and management.

Discover →

An answer to existing CM slowness

In 2011 the Infrastructure as Code landscape is dominated by Puppet and Chef, both based on Ruby models and periodic polling architectures. Thomas S. Hatch, a system administrator, needed a tool with a specific property: speed in executing parallel commands on hundreds of servers simultaneously.

From this need comes Salt (originally Salt Stack), a Python open source project whose first public commit is in February 2011. In the following months releases 0.8-0.9 (summer-autumn 2011) consolidate core functionality. Apache 2.0 licence.

The architecture

Salt adopts a master/minion model similar to Puppet, but with substantial differences:

• ZeroMQ transport — high-performance pub/sub pattern, instead of the Puppet model’s HTTP/REST. Asynchronous commands, streaming-aggregated responses
• Minion persistent connection — each minion maintains a TCP connection with the master, eliminating handshake overhead
• Broadcast — the master can emit a command to all minions simultaneously with sub-second latency
• Grains (analogous to Puppet’s Facter) — minion introspection system, used for command targeting
• Pillars — separate data store for secrets and per-host configurations

Practical result: running salt '*' cmd.run 'uptime' across 1000 nodes takes seconds, not minutes.

Two paradigms: execution modules and states

Salt exposes two complementary modes:

Execution modules — imperative

Ad-hoc commands run on target minions via CLI salt:

salt 'web*' pkg.install nginx
salt 'web*' service.restart nginx
salt '*' cmd.run 'uptime'

Hundreds of modules (pkg, service, file, cmd, user, network, mysql, postgres, apache, selinux, grains, …) cover common operational tasks.

States — declarative

Desired configuration described in YAML/Jinja2 files (.sls — SaLt State):

nginx:
    pkg.installed: []
    service.running:
        - enable: True
        - require:
            - pkg: nginx

/etc/nginx/nginx.conf:
    file.managed:
        - source: salt://nginx/nginx.conf
        - require:
            - pkg: nginx
        - watch_in:
            - service: nginx

States are analogous to Puppet manifests or Chef cookbooks, but with YAML syntax instead of custom DSL. The advantage: familiarity for many sysadmins already used to YAML files.

Orchestration

Salt introduces the concept of orchestration: states coordinating actions across multiple minions in sequence, useful for complex deployments (e.g. rolling updates with load balancer, database migration with failover). Orchestration is written as states on the master, not on minions, and uses the salt.state module to invoke remote executions.

Comparison with Puppet and Chef

Salt vs. Puppet:

• Puppet: Ruby, periodic polling, custom DSL
• Salt: Python, push/pub-sub, YAML

Salt vs. Chef:

• Chef: Ruby, pull model, client/server
• Salt: Python, push/pub-sub

Salt vs. Ansible (Ansible releases in February 2012):

• Ansible: Python, agentless over SSH
• Salt: dedicated agent, faster at scale but more intrusive

Each tool has trade-offs: Salt wins in scalability and immediacy of ad-hoc operations; loses to Ansible on setup simplicity (no target installation).

The SaltStack company

SaltStack Inc. is formally founded in 2012 by Thomas Hatch to sustain development. Commercial model: open source core + SaltStack Enterprise with additional features (GUI, reporting, compliance). In 2020 VMware will acquire SaltStack, integrating it in its offering (vRealize Automation SaltStack Config).

Adoption

Salt rapidly attracts adoption in contexts with strong scale needs:

• LinkedIn — internal infrastructure management
• Apple — some operational infrastructures
• Cisco — network automation
• VMware itself (before acquisition) — some divisions
• European hosting providers — managing massive VM fleets

In the Italian context

As of 2011 Salt has just been born; Italian adoption is experimental. In the following three years (2012-2014) it will find space in:

• Italian hosting providers — large fleet management
• Telco — equipment configuration automation
• Universities and HPC research centres — cluster management

Puppet and Chef remain more widespread overall; Salt carves niches where execution speed matters. After 2013-2014 Ansible will become the most formidable competitor, thanks to agentless simplicity.

References: Salt (SaltStack), Thomas S. Hatch, first commit February 2011. Apache 2.0 licence. ZeroMQ transport. Grain/Pillar system. SaltStack Inc. (2012), VMware acquisition (2020).