Unified logging layer
Log ingestion in distributed environments is fragmented: app stdout, syslog, journald, rotational files, cloud services. Each source speaks a different format. A unified layer is needed to collect, parse, normalise, enrich, route to heterogeneous backends (Elasticsearch, S3, Kafka, Loki, Datadog).
Fluentd
Fluentd is released in October 2011 by Treasure Data (Sadayuki Furuhashi, also author of MessagePack). Written in Ruby + C, Apache 2.0 licence. Donated to CNCF in November 2016, graduated in April 2019.
<source>
@type tail
path /var/log/app.log
tag app.web
</source>
<match app.**>
@type elasticsearch
host es.example.com
</match>source → filter → match pipeline, plugin ecosystem of 1000+ modules (input, parser, filter, output).
Fluent Bit
Fluent Bit is released in 2015 by Eduardo Silva within the Treasure Data / Fluent ecosystem, written in pure C, ~450 KB RAM footprint. Designed for edge, containers, IoT. Shares CNCF Graduated status with Fluentd under the Fluent organisation.
Typical usage: Fluent Bit as collector on every node/pod, Fluentd as central aggregator with complex transformations.
Kubernetes logging
Standard pattern:
- Fluent Bit DaemonSet on every node → reads
/var/log/containers/*.log - Fluentd aggregator (optional) → advanced parsing
- Output: Elasticsearch/OpenSearch, Loki, S3, Kafka, Azure Log Analytics, Google Cloud Logging
Competitors
- Logstash (Elastic, 2009) — JVM-heavy, part of ELK, SSPL licensing since 2021
- Vector (Datadog, 2019) — Rust, modern, unified logs+metrics+traces
- Filebeat (Elastic) — lightweight, part of Elastic stack
- Promtail (Grafana) — collector specific for Loki
- rsyslog, syslog-ng — legacy UNIX
In the Italian context
Fluent Bit is omnipresent in Italian Kubernetes clusters:
- Banks with ELK/OpenSearch stack
- Telco — TIM, Vodafone for infrastructure log aggregation
- Digital PA — national cloud projects (PSN)
- B2B SaaS with multi-tenant logging
- Hosting providers and MSPs
Many Italian integrators propose EFK (Elasticsearch + Fluent Bit + Kibana) or Loki + Fluent Bit + Grafana stacks.
References: Fluentd (October 2011). Treasure Data, Sadayuki Furuhashi. Fluent Bit (2015, Eduardo Silva). Apache 2.0 licence. CNCF Fluentd graduated April 2019; Fluent Bit in the same Fluent organisation. 1000+ plugin ecosystem.