slinkp blog
Thread locals are an interesting idea: a way to give each thread its own storage, useful for global state that you don’t want to share between threads.
The obvious caveat is that threadlocals are still effectively global (for the current thread), and like all global state, should be treated with great suspicion. If you can do without one, you’re probably better off.
Let’s look at the Python official docs about threading.local. Here is all of it:
class threading.local
A class that represents thread-local data. Thread-local data are data whose values are thread specific. To manage thread-local data, just create an instance of local (or a subclass) and store attributes on it:
mydata = threading.local()
mydata.x = 1
The instance’s values will be different for separate threads.
For more details and extensive examples, see the documentation string of the _threading_local module.
Okay, so let’s play with that example and see what happens.
>>> import threading >>> >>> mydata = threading.local() >>> mydata.x = 'hello' >>> class Worker(threading.Thread): . def run(self): . mydata.x = self.name . print mydata.x . >>> w1, w2 = Worker(), Worker() >>> w1.start(); w2.start(); w1.join(); w1.join() Thread-1 Thread-2 >>> print mydata.x hello Cool! Each thread does indeed set and read its own data in the x attribute.
Mutable threadlocals part 1: oops
Let’s try the same exact example again with a different value for x . Something mutable. Say, a dict.
>>> import threading >>> mydata = threading.local() >>> mydata.x = <> >>> >>> class Worker(threading.Thread): . def run(self): . mydata.x['message'] = self.name . print mydata.x['message'] . >>> w1, w2 = Worker(), Worker() >>> w1.start(); w2.start(); w1.join(); w2.join() Exception in thread Thread-2: Traceback (most recent call last): File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner self.run() File "", line 3, in run AttributeError: 'thread._local' object has no attribute 'x' Exception in thread Thread-1: Traceback (most recent call last): File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner self.run() File "", line 3, in run AttributeError: 'thread._local' object has no attribute 'x' Um. Wat. So mydata.x exists if it’s assigned an immutable value, and does not exist if it’s assigned a mutable value?
I don’t fully understand this.
It’s a bit irksome that this is glossed over in the published python docs and lies here waiting to bite unsuspecting users. Remind me to submit a docs patch.
Mutable threadlocals part 2: Subclass threading.local
But here’s a better way, based on a close read of the docstring that was mentioned in the docs. It works if you subclass threading.local and create your mutable object inside __init__ .
You can see what happens if you do that by reading the _threading_local source: both __new__ and __getattribute__ are defined such that your __init__() gets run for each thread as needed, with locking around it. The Worker class here is the same as above:
>>> import threading >>> class MyData(threading.local): . def __init__(self): . self.x = <> . >>> >>> mydata = MyData() >>> >>> class Worker(threading.Thread): . def run(self): . mydata.x['message'] = self.name . print mydata.x['message'] . >>> >>> w1, w2 = Worker(), Worker() >>> w1.start(); w2.start(); w1.join(); w2.join() Thread-1 Thread-2 >>> print mydata.x <> What if you wanted to create a mutable value explicitly per thread, instead of inside __init__() ?
That’s actually easier, just do that inside code that’s run per thread instead of globally, and assign it. No local subclass needed. Eg. in this case inside the run method:
>>> mydata = threading.local() >>> >>> class Worker(threading.Thread): . def run(self): . mydata.x = 'message': self.name> . print mydata.x['message'] . >>> w1, w2 = Worker(), Worker() >>> w1.start(); w2.start(); w1.join(); w2.join() Thread-17 Thread-18 Thread-Local Data in Python
You can use thread-local data by calling threading.local() and sharing the instance between threads.
In this tutorial you will discover how to use thread-local data storage in Python.
Need Data Unique to Each Thread
A thread is a thread of execution in a computer program.
Every Python program has at least one thread of execution called the main thread. Both processes and threads are created and managed by the underlying operating system.
Sometimes we may need to create additional threads in our program in order to execute code concurrently.
Python provides the ability to create and manage new threads via the threading module and the threading.Thread class.
Sometimes we may need to store data that is unique for each thread.
Python provides a thread-local object that can be used to store unique data for each thread. The thread-local storage provides an alternative to having to extend the threading.Thread class and define instance variables for thread-specific data.
What is thread-local data exactly?
Run your loops using all CPUs, download my FREE book to learn how.
What is Thread-Local Data?
Thread-local data storage is a mechanism in multi-threaded programming that allows data to be stored and accessed in a way that is private to each thread.
It may be called “thread-local storage“, “thread-private“, or simply “thread-local“.
Typically this involves creating an instance of a thread-local object that is shared between objects on which data is set and retrieved.
Data stored and retrieved on the thread-local object may have the same variable names across threads, meaning the same code can be used across threads, a common approach when using worker threads.
Importantly, the reads and writes to the thread-local object are unique and private at the thread-level. This means one thread may write a variable with the name “address” and another thread may read or write a variable with an identical name but it will not interact with the variable stored by the first thread.
If executing the same code and using the same thread-local instance, then each thread has its own private version of a named variable and its assigned value in the thread-local storage.
This can be useful when multiple threads need to store local data such as a partial solution or a temporary variable and need to use the same code while executing, such as the same instance of an object.
Now that we know what thread-local data is, how do we use it?
Confused by the threading module API?
Download my FREE PDF cheat sheet
How to Use Thread Local-Data
Threads can store local data via an instance of the threading.local class.
First, an instance of the local class must be created by calling the threading.local() function.