Killing Multithreaded Python Programs with Ctrl-C

If you have ever done multithreaded programming in Python you have probably found it frustrating that you can't simply hit Ctrl-C in the terminal and have it exit like a normal Python process. Instead you have to put the process in the background (Ctrl-D) and then either "kill %%" or kill the PID. The good news is that it doesn't have to be this way. After experimenting a bit I finally figured out why it doesn't work normally and what you have to do to make it work.

Normally when I write a threaded program in Python it looks something like this...

The problem is with this program is that if you hit Ctrl-C it doesn't do anything. The reason is that join() is a blocking operation. As a result the process will only receive the signal for Ctrl-C when join() becomes unblocked, which in this case will never happen.

In order to handle Ctrl-C with multiple threads you can use the following code:

This code does a few things differently in order to make it handle Ctrl-C as we would like. First, instead of using join() we use join(timeout) which tries to join a thread but will timeout if it does not occur after the timeout elapses. This allows the main thread of execution to continue doing other things, in particular waiting for a KeyboardInterrupt to be thrown which is what Ctrl-C raises. Since join will return upon timeout, we need to keep any threads which aren't None and respond to isAlive().

The next thing is that if child threads never return or take a really long time you need a way to notify the child that it should die. This is accomplished by the kill_recieved flag in the Worker class. When that flag is set by the parent process the child knows that it should finish up what it is doing and return.

The last thing is something that caught me off guard a bit. Initially, in the main() while loop I was trying to catch all exceptions that came up by using "try...except Exception:". As it turns out Exception does not include KeyboardInterrupt, meaning that Ctrl-C's that are raised in that block will not be caught. If you instead use "try...except KeyboardInterrupt:" or just "try..except:" it will work as you expect it to.

So there you have to exit multithreaded Python programs using Ctrl-C.


Unknown said...

Very nice. I discovered this when working on CRAWL-E. I did some event triggering (I think) but it's fairly similar.


何一鸣 said...

good,same to my solution,here is my detail summary in chinese:

Unknown said...

You can always set your threads to "daemon" threads like:
t.daemon = True

And whenever the main thread dies all threads will die with him ^^

Anonymous said...

Hmm... pastie no longer has the code snippets. Can you post 'em within the blog entry?

Umar Said said...

Running your code longer than 10 seconds failed to respond to ctrl+c

marcojovi said...

This solved a problem I had where the function inside the thread (do_something in your case) was blocking. I used to stop the thread by changing a global variable (used the same way you used kill_received), but that didn't work if the function was already being executed (that's understandable actually). Using a local "class" variable works, although I don't fully understand why... any hints?

devplayer said...

"Killing Multithreaded Python Programs with Ctrl-C" again lost the the code snippets that the blog refers too. I hope you could refresh it.

Post a Comment