Python Decorators — Tutorial

Python Decorators — Tutorial


Estimated Reading Time: 7 minutes


Introduction

It’s been absolutely ages since I’ve had the time or inclination to write an informational blog post, but development of a new room has finally given me an excuse!

This post will have a look into Python decorators — a really cool construct that allows you to pre-process functions; very useful when working with Flask apps, or Discord bots, where they are used for things like routes and authentication, or to specify commands. Decorators are a tricky concept to understand to begin with, but they are incredibly powerful once you know how to write them.

The key use for decorators is to code a single function which can then be applied to many other functions: for example, an authentication function which can be used to protect many different routes in a Flask application.

This will not be a particularly long post, so let’s dive right in.


Overview

If you’ve worked with either Flask or discord.py then you’ll definitely have already encountered function decorators. In Flask they may appear as routes:

@app.route("/")

In discord.py they might designate a command:

@bot.command()

Two very different uses, but both doing fundamentally the same thing — executing before the function and doing something with it.

Decorators are usually referenced in their shorthand format: an @ symbol above the function being pre-processed, however, there are other ways to call them (which make more sense, but are much less neat) — we will see those later on.

The concept of a decorator is heavily reliant on two key aspects of Python:

  • Scoping
  • Treating basically everything as an object

Decorators effectively work by defining a function which takes another function as an argument. This function contains an inner (nested) function which does all of the heavy lifting. The outer function then returns the entire inner function as an object.

The best way to understand decorators is by building one, so let’s have a look at an example.


Simple Example

Here is the code for a very simple decorator example:

#!/usr/bin/env python3

def decorate(function):
    def handler():
        return function().upper()
    return handler

@decorate
def returnMsg():
    return "Hello World"

print(returnMsg())

This code is in three sections. First of all, the decorator itself:

def decorate(function):
    def handler():
        return function().upper()
    return handler

Next, the function that we are decorating:

@decorate
def returnMsg():
    return "Hello World"

Finally, calling the function:

print(returnMsg())

This will print out the upper-case version of the message:

Demonstration that the example works and that the decorated function prints out the upper case version of the message
Demonstration of the working example

So, how does it work?

Let’s take a closer look at that decorate() function.

We have an outer function (decorate(function)) and an inner function (handler()). decorate() returns the entire handler() function as an object.

Inside the handler() function is where the pre-processing occurs. When decorate() is called, it takes a function as an argument. Python scopes mean that the inner function is able to access all of the objects available to the outer function, meaning that the handler() function can also access the function that is being passed to decorate(). In this example, handler() executes the function that is being passed, then converts whatever it returns into upper case. The decorate() function then returns the entire inner function as an object, effectively returning a procedure — a set of instructions for how to handle a function that’s passed in. This is unlikely to make much sense just now, so don’t worry if you can’t follow it just yet!

As mentioned previously, there is another (less clean, but easier to follow) way to call a decorator, which may make things a little clearer:

#!/usr/bin/env python3

def decorate(function):
    def handler():
        return function().upper()
    return handler

def returnMsg():
    return "Hello World"

response = decorate(returnMsg)
print(type(response))
print(response)
print(response())

This is the same program, but instead of using the @ syntax, we are calling the decorate() function manually (providing the function we want to process as an argument) and storing the result in a variable called response. When this is executed, it prints three things:

<class 'function'>
<function decorate.<locals>.handler at 0x7f028fed6670>
HELLO WORLD

This tells us that the type of response is a function — this is to be expected: we know that the decorate() function returns a function object. The second line just tells us the function address — not too useful, but helps to illustrate that this is just a function. Finally we call the function stored in response, resulting in the expected “HELLO WORLD”.

So, to sum all of that up, decorators take a function as an argument and return a function in response. The function being returned pre-processes the function being passed as an argument — we will look at a better example of this next.

The @ shorthand notation can be thought of as basically saying to Python: “Take the function below this line and pass it to the decorator I am specifying here, then execute the function that the decorator returns to you”.


Authenticator Example

Let’s have a look at a slightly more complex example: a decorator which handles authentication. This might come in handy for, say, a Flask app. This code wouldn’t be much use itself, but we will look at a more advanced version in the next example, which would be a big improvement.

Here is the code:

#!/usr/bin/env python3
import getpass

def authenticator(func):
    def processor():
        username = input("What is your username? ")
        password = getpass.getpass("What is your password? ")
        if username == "ag" and password == "decoratorsAreCool":
            func()
        else:
            print("Access Denied")
    return processor

@authenticator
def main():
    print("Access was granted and the main function is now running!\nWelcome to the secret club.")

main()

When run, the program attempts to execute main(), but the decorator gets called first. It asks us for a username and password. If the username is “ag” and the password is “decoratorsAreCool” then it executes the main() function — otherwise it prints out “Access Denied” and exits without executing the main() function.

Here it is in action:

Demonstration that the authenticator program works as described
Authenticator Demonstration

This shows really clearly that the decorator is executed before the function, with the function only being called if and when the decorator wants it to be called.

In reality the decorator would likely not be asking for credentials (it might be using Flask session storage to determine whether the user should have access to the function, for example), but this really nicely demonstrates how decorators can control whether the function is executed or not.


Functions with Arguments

Hopefully by now you have a pretty good idea about how decorators work (if not, please let me know in the comments below and I’ll try to explain it better!). There is still one thing that is unclear though: how the heck are we meant to pass arguments into the decorator without calling the decorated function pre-emptively?

For example, let’s say we have a function (main) which takes a string argument for the user’s name:

def main(name):
    print(f"Hello, {name}!")

Nice and simple.

What happens if we put this into our authenticator from before though?

#!/usr/bin/env python3
import getpass

def authenticator(func):
    def processor():
        username = input("What is your username? ")
        password = getpass.getpass("What is your password? ")
        if username == "ag" and password == "decoratorsAreCool":
            func()
        else:
            print("Access Denied")
    return processor

@authenticator
def main(name):
    print(f"Hello, {name}!")    

main("Muiri")
---
Traceback (most recent call last):
  File "/tmp/decorator-demo/./args.py", line 18, in <module>
    main("Muiri")
TypeError: processor() takes 0 positional arguments but 1 was given

We get a big, fat error: that’s what.

Effectively, the argument we gave to main() is being passed into the inner function of the decorator, which doesn’t expect any parameters.

We could rewrite the inner function so that it also takes a single string argument, but then we lose the one big advantage of decorators: the fact that they can be applied to any function.

Fortunately, there is a way to pass all arguments into the function: *args and **kwargs.

These special parameters allow us to capture every argument and keyword argument passed into a function, which means we can pass them straight through to the decorated function:

#!/usr/bin/env python3
import getpass

def authenticator(func):
    def processor(*args, **kwargs):
        username = input("What is your username? ")
        password = getpass.getpass("What is your password? ")
        if username == "ag" and password == "decoratorsAreCool":
            func(*args, **kwargs)
        else:
            print("Access Denied")
    return processor

@authenticator
def main(name):
    print(f"Hello, {name}!")    

main("Muiri")
Demonstration of the above code in action.
*args, **kwargs demonstration

Functools: Wraps

What we’ve covered so far is great. It works, and in most instances will work perfectly. Behind the scenes there is still a little problem though — namely that the decorator overwrites the function attributes with its own.

For example, let’s look at a slight variation of our very first example:

#!/usr/bin/env python3

def decorate(function):
    def handler():
        return function().upper()
    return handler

@decorate
def returnMsg():
    return "Hello World"

print(returnMsg.__name__)

We would expect the name of the returnMsg() function to be returnMsg. What is it actually?

Demonstration that the decorator overwrites the correct name for returnMsg() (which is returnMsg) with the name of the inner function (handler)
Demonstration of the error

handler. The inner function of the decorator (handler()) overwrote the name of the decorated function, along with the rest of its inbuilt attributes. In most cases this won’t really be an issue, but it’s still a good idea to avoid it!

The solution is a standard module called functools — specifically a function (an inbuilt decorator!) called wraps(). Functools provides function that are designed to help when designing functions which modify other functions (e.g. decorators). wraps() is designed to solve the problem that we just encountered: it ensures that the original attributes remain applied to the decorated function. With wraps() in use, our final program looks like this:

#!/usr/bin/env python3
from functools import wraps

def decorate(function):
    @wraps(function)
    def handler():
        return function().upper()
    return handler

@decorate
def returnMsg():
    return "Hello World"

print(returnMsg.__name__)

Notice the @wraps decorator above the inner function of our custom decorator.

When executed, this program now returns the correct name for the returnMsg() function:

Demonstration that after invoking the wraps() function decorator on the inner function of our custom decorator, the returnMsg function now retains the correct attributes
Program now returns the correct name for the returnMsg function

Conclusion

That concludes our whistle-stop tour of Python decorators. Hopefully they now make sense, if they didn’t before.

If anything is still unclear, please feel free to post it as a comment below. Otherwise I can highly recommend playing around with these for yourself. This is a relatively complex topic, but a very valuable one.

Leave a Reply

Your email address will not be published. Required fields are marked *

Enter Captcha Here : *

Reload Image

Scroll Up