Fluentd + Elasticsearch + Kibana, your on-premise logging platform

JM Robles
5 min readSep 11, 2022

TL;DR: How to set up an open source logging platform on Kubernetes

No, it’s not an Egyptian hieroglyph!

Motivation

Logs are mandatory.

We need a way to record and monitor our application's status in real production-grade applications.

In another post, I explored how you can use an OpenTelemetry-based stack to have traces of the requests which pass through our application.

But, what happens if we need to analyze errors or warnings generated from our system?

In the past, the old school way was to connect with the remote server via SSH (or even telnet) and check for the access.log or error.log generated by Apache, Nginx, …

Nowadays we have a lot of solutions for logging capabilities. In the commercial/cloud stack we found AWS CloudWatch or Azure Monitor.

We have open source alternatives too: Sentry, ELK, …

Sentry is awesome… but it’s so heavy to run it for your “small” cluster.

--

--

JM Robles

AIOps @ Montevive.ai / Backend / Platform engineer #llm #rag #llamaindex #langchain #go #python #kubernetes #terraform #devops