Behind the Scenes: Python Decorators
Let's complete our previous visit to Python decorators with a deep dive into the insights of this amazing feature.
Hello, Hypers!
In a previous post, we talked about Python decorators. In case you missed it, here it is. If you want an introduction to Python decorators and how we can use them, please read this first (also, you’ll find many references to this article in this post):
This week in Under the Hype, we will take a deeper look into Python decorators and learn how they work behind the scenes and why they are a great tool for programmers.
Let’s get started!
What is a Python Decorator?
In the previous post about Python decorators, we said that decorators are not an exclusive Python feature. A decorator is a higher-order function (a function that takes another function as an argument) that returns a modified version of the input function.
We also saw that one of the cool things about Python decorators was how we use them, removing the friction of making extra function calls to get the expected result. However, it is not all about notation and syntax sugar. Python decorators are quite powerful and we can apply them in many different situations.
In our previous visit to the Python decorators world, we implemented many useful examples and explored some use cases. But some of those examples could seem like magic. In this post, we are going to explain the magic trick!
Function Decorators
Let’s start with an example!
In this example, the add
function is wrapped by the wrapper
function inside log_execution
. When add
is called, wrapper
prints the function name and arguments before calling the original function and prints the return value after the function executes.
Behind the scenes, the Python interpreter is reassigning the add
function. The previous code is translated into something like this:
This is the effect of writing @log_execution
before a function definition, just a reassignment of the original function. The new function is the result of evaluating the decorator in the original function.
You can get an idea of how decorators with parameters work. It is exactly the same! For example, suppose that our log_execution
decorator receives an initial_log
parameter.
Now the translation would be:
Both code snippets are functionally equivalent. But, of course, using the @ to decorate is way more readable and safe. These examples about function decorators helped us to demystify Python decorators a little bit and they will help us to understand the other decorator’s usages and implementations.
Class Decorators
Class decorators operate similarly to function decorators but they modify classes. They can add new methods, override existing ones, or alter class attributes. This can be especially useful for adding common functionality to multiple classes.
Here's an example:
In this example, add_repr
adds a __repr__
method to any class it decorates, which provides a string representation of the instance by iterating over its __dict__
attributes. This is more generic and works with any class.
Just to give you more context and to add value to this post, the __repr__
is another magic method that returns a string representation of the Python object. This representation should be a valid Python expression. When a class has a __repr__
method and we print an instance of the class, we get the result of the __repr__
method.
Clarification! If the class has a
__str__
method, then the__str__
is called instead of the__repr__
. The__str__
is supposed to return a “nice” representation, while the__repr__
is supposed to return a more technical representation. That’s why we usef’{k}={v!r}’
in the previous decorator. The!r
is used to get the result of__repr__
instead of__str__
.
I feel this could be a whole new post :)
Besides all the technical specs about magic methods, nothing changes behind the scenes. In the previous example, the Python Interpreter will reassign the class definitions. That’s how we get instances with a __repr__
method already implemented.
Conclusion (or Why Should I Know All This Stuff?)
At this point, you should be able to figure out what happens behind the scenes when we implement a decorator with a class. You only need to remember that the __call__
magic method allows us to use a class instance as a function. If you don’t remember how to implement a decorator with a class, please go to the precious article about decorators referenced at the beginning of this post.
But, what’s the point of all this? Are decorators so important? Well… yes and no. You can code whatever you want without them. On the other hand, you can significantly increase the quality of your code if you include them in your toolbox.
What makes a codebase a good codebase? There are a lot of engineering standards to look after when evaluating a codebase. But one of the most important is the Open/Closed Principle.
This principle says that classes (and functions) should be open to be extended and adapted but closed to be modified. In simpler words, we should avoid directly modifying a class or function implementation, instead, we should use other “indirect mechanisms” to adapt them to our new necessities.
A function is implemented in a given context. Modifying the function requires us to know that context and predict the possible consequences. This tends to be too complex and costly in the long run. Features like inheritance, polymorphism, and decorators allow us to adapt functions and classes without modifying the original implementation.
This is just a fundamental benefit of using decorators and I think most cases can be reduced to this Open/Closed Principle. But if you think a little bit more about it I’m sure you will come up with additional benefits of using decorators.
There are still other technical details in the air. For example, how can our decorated functions access the variables and parameters defined in the decorator implementation no matter how many times we call them? This is possible because of closures. But this is the topic for another post. Let me know if you would like me to dive into this in the future!
And that’s it! I hope you have enjoyed the post. It was a bit technical but I did my best to make it easy to digest. If you like this kind of article let me know. I’ll be happy to share more content about Engineering and Programming Languages.
See you next Tuesday!