TL;DR: How to add spans to your Istio traces in Python
Motivation
We discovered how to enable Istio in your Kubernetes cluster in the last post.
Thanks to its auto instrumentation feature out of the box, Istio creates a new trace for each new incoming request. If the request “travels” among some services (pods) in our cluster, each Istio sidecar appends a new span under the trace.
With the Jaeger UI, we can view these traces and their spans.
But what happens if we have to instrument our app to know the SQL to execute in the database, the latency in our handler, or more details about a call to our ML model in production?
Today we discovered how to do it in Python with the Opentelemetry SDK.
Opentelemetry SDK
Opentelemetry is an open-source collection of APIs, formats, libraries, and SDK with the aim to simplify and create a universal observability platform.
With Opentelemetry is very easy to change the tracing or metrics collector without major changes in our codebase.
It’s important to highlight that the project is in the incubation phase under Cloud Native Computing Foundation (CNCF).
It has SDK and libraries for the most used languages: Go, JavaScript, Rust, PHP… and, obviously, Python.
For Python, there are some package-level auto-instrumentation libraries for the most used web frameworks like Flask (Flask Instrumentation).
These libraries use “black magic” to perform the instrumentation (black magic = patching objects on the fly).
But to be honest, I don’t pretty much like black magic. I prefer to have control of what is under the hood. On the other hand, in many cases, you can’t do it with an auto-instrumentation.
