Unified logging layer
Log ingestion in distributed environments is fragmented: app stdout, syslog, journald, rotational files, cloud services. Each source speaks a different format. A unified layer is needed to collect, parse, normalise, enrich, route to heterogeneous backends (Elasticsearch, S3, Kafka, Loki, Datadog).
Fluentd
Fluentd is released in October 2011 by Treasure Data (Sadayuki Furuhashi, also author of MessagePack). Written in Ruby + C, Apache 2.0 licence. Donated to CNCF in November 2016.
<source>
@type tail
path /var/log/app.log
tag app.web
</source>
<match app.**>
@type elasticsearch
host es.example.com
</match>source → filter → match pipeline, plugin ecosystem of 1000+ modules (input, parser, filter, output).
Fluent Bit
Fluent Bit is released in 2015 by Eduardo Silva within the Treasure Data / Fluent ecosystem, written in pure C, ~450 KB RAM footprint. Designed for edge, containers, IoT.
Typical usage: Fluent Bit as collector on every node/pod, Fluentd as central aggregator with complex transformations.
Kubernetes logging
Standard pattern:
- Fluent Bit DaemonSet on every node → reads
/var/log/containers/*.log - Fluentd aggregator (optional) → advanced parsing
- Output: Elasticsearch/OpenSearch, Loki, S3, Kafka, Azure Log Analytics, Google Cloud Logging
Competitors
- Logstash (Elastic, 2009) — JVM-heavy, part of ELK
- Filebeat (Elastic) — lightweight, part of Elastic stack
- rsyslog, syslog-ng — legacy UNIX
In the Italian context
Fluent Bit is omnipresent in Italian Kubernetes clusters:
- Banks with ELK/OpenSearch stack
- Telco — TIM, Vodafone for infrastructure log aggregation
- Digital PA — national cloud projects (PSN)
- B2B SaaS with multi-tenant logging
- Hosting providers and MSPs
Many Italian integrators propose EFK (Elasticsearch + Fluent Bit + Kibana) stacks.
References: Fluentd (October 2011). Treasure Data, Sadayuki Furuhashi. Fluent Bit (2015, Eduardo Silva). Apache 2.0 licence. CNCF donation November 2016. 1000+ plugin ecosystem.