TL;DR: How to do modern python logging with a fluent logging collector
Motivation
Python's out-of-box logging facilities are excellent.
It’s a pluggable system that allows us to customize the logging format, add as many handlers as we need, configure it using config files and so on.
On the other hand, fluentd is a great open-source logging collector solution.
In the last post, we saw how to set up our own logging platform with fluent, Elasticsearch and Kibana.
This post, in a certain way, is an organic continuation of that.
On the one hand, we’ll see how simple is to do logging with Python to fluent. On the other, we’ll see different logging approaches under “modern Python logging” label.
Fluent HTTP library for Python
Until this very moment, under PyPI repository, we can only find a fluent library for the native transport format, not over HTTP.
So, I decided to create a new one for fluent with HTTP transport.
Here it is.
Its development only took a couple of hours. It was very easy to create it.
The repository: github.com/jmrobles/fluent-http
Hello world with Fluent Http
It’s very pretty straightforward to start working with the library.
We start running a simple fluentd container where our samples will write the logs.
First, it’s necessary to create a fluent.conf file with our input handler set to HTTP.
