Adding contextual data to Python logging

Photo by Joshua Aragon on Unsplash

Recently I faced an exciting challenge. I needed to create a robust logging system integrated with an external API. One of the requirements was to have specific contextual data logged whenever it was available. The main issue was that a lot of it was spread across the whole application, nested within different functions.

The most trivial approach of passing down necessary objects through the functions would bloat the code unnecessarily. I started to think about how to leverage context managers to collect the data. After several iterations, together with the team, I came up with an elegant solution, which gave us extra fluff we could use in the app. Let me show you the code and explain the reasoning behind it. Ready? Let’s move on!

Setting up logging context handler

First, we will create a data store for our context manager. It needs to have three capabilities:

• we need to be able to add more contextual data to it
• it should return a specific attribute value by its name
• it should be able to remove contextual data when no longer needed.

To achieve this, we will create a stack with dictionaries as their elements. The dictionaries will contain a set of contextual data to be used in logging. Let’s see how it looks:

Let’s take a deeper look at the add method because that’s where the magic happens.

Whenever we add new data to the context, add clones the dictionary from the top of the stack and updates it with newly added content. This way, we won’t need to iterate over the whole stack to get an attribute hidden deep within it.

I mentioned a context manager a couple of times, but where is it? Let me fix that right away!

Creating context manager

There’s nothing fancy, just a simple function. Whatever you use as the argument will be pushed to the handler defined above. After the successful…