Loki: log aggregation "like Prometheus, but for logs"

Linux Services & Systems

Domains, hosting, PEC, email infrastructure, network services and Linux systems. Open Source infrastructure support and management.

Discover →

Low-cost logs

Elasticsearch is powerful but expensive: indexes every word of every log → fast disks, high RAM, complex licensing from 2021 (SSPL). Many teams just want to search by labels (namespace, pod, app) and filter text with grep, not full-text search.

The release

Loki is released on 14 December 2018 by Grafana Labs at KubeCon Seattle 2018. Lead engineers: Tom Wilkie (ex-Weaveworks, Prometheus maintainer) and David Kaltschmidt. Written in Go, AGPLv3 licence (since 2021, Apache 2.0 before).

Philosophy

“Like Prometheus, but for logs”:

• Index only on labels (not content)
• Compressed log content archived on object storage
• Economic storage — S3, GCS, Azure Blob, MinIO
• LogQL query language similar to PromQL

{app="webapp", env="prod"} |= "error" | json | status_code >= 500

Architecture

• Distributor — receives logs, distributes to writers
• Ingester — in-memory batches, flushes to storage
• Querier — executes LogQL, aggregates from ingester + storage
• Compactor — index compaction
• Storage — object storage (S3-compatible) + boltdb-shipper/TSDB index
• Query frontend — query parallelisation

Deploy patterns: Single binary (monolith), Simple Scalable, Microservices (1000+ pods).

Promtail

Companion log collector:

• Promtail — Grafana agent for collecting logs (path, journald, K8s)
• Fluent Bit, Fluentd — support Loki output as alternatives

Integration

• Grafana — built-in UI, Explore tab for LogQL
• Alerting — Grafana alerts based on log counts/patterns
• Recording rules — metrics derived from logs
• Tempo + Loki — trace ↔ log correlation

Competitors

• Elasticsearch (via Filebeat/Logstash) — powerful full-text index but expensive
• Splunk — commercial, enterprise
• Datadog Logs, New Relic Logs — SaaS

In the Italian context

Loki is very widespread for cost-contained log aggregation:

• Italian MSPs and hosting providers
• Banks for infrastructure logs (not full-text transactional)
• Digital PA — PSN projects, multi-tenant clusters
• Italian B2B SaaS
• Development teams for application logs
• Home-lab and local dev environments

The Grafana + Prometheus + Loki stack is starting to take shape as an affordable alternative to Datadog/Splunk for Italian cloud-native teams.

References: Loki (14 December 2018, KubeCon Seattle). Grafana Labs. Tom Wilkie, David Kaltschmidt. AGPLv3 licence (since 2021). Written in Go. LogQL query language. S3/GCS/Azure Blob storage + object-storage index.