Break the function after certain time
I want to continue the for loop if function A takes more than five seconds by skipping it so I won’t get stuck or waste time. By doing some search, I realized a subprocess or thread may help, but I have no idea how to implement it here.
I think having a check in the function you call to avoid getting stuck would be a smarter move, unless you are doing some heavy calculations or something has gone very wrong, you should not be stuck for 5 seconds, if you have a lot of data then maybe multiprocessing would be the way to go.
@Padraic Cunningham Thanks for replay. It won’t work in my case since the function may experience network or CPU idling.
@user2372074: Do you only need this to work on Mac OS X (or at least only on Mac OS X and other reasonably modern Unix and Unix-like systems)? If so, the answer is definitely simpler. (Signals are easy in Python; Windows APCs or similar mechanisms are not…)
5 Answers 5
I think creating a new process may be overkill. If you’re on Mac or a Unix-based system, you should be able to use signal.SIGALRM to forcibly time out functions that take too long. This will work on functions that are idling for network or other issues that you absolutely can’t handle by modifying your function. I have an example of using it in this answer:
Editing my answer in here, though I’m not sure I’m supposed to do that:
import signal class TimeoutException(Exception): # Custom exception class pass def timeout_handler(signum, frame): # Custom signal handler raise TimeoutException # Change the behavior of SIGALRM signal.signal(signal.SIGALRM, timeout_handler) for i in range(3): # Start the timer. Once 5 seconds are over, a SIGALRM signal is sent. signal.alarm(5) # This try/except loop ensures that # you'll catch TimeoutException when it's sent. try: A(i) # Whatever your function that might hang except TimeoutException: continue # continue the for loop if function A takes more than 5 second else: # Reset the alarm signal.alarm(0) This basically sets a timer for 5 seconds, then tries to execute your code. If it fails to complete before time runs out, a SIGALRM is sent, which we catch and turn into a TimeoutException. That forces you to the except block, where your program can continue.
I’m pretty sure you’re supposed to do that… unless it’s effectively the same question and deserves exactly the same answer, in which case it’s probably better to close as a dup than to copy your code over.
@TheSoundDefense: No, that’s not right, the OP found the right solution. His change (using class ) means he’s defining a subclass of Exception , which is exactly what he wants. Your original code (using def ) just defines a function whose parameter happens to be named Exception , and then tries to use that function as an exception class.
@TheSoundDefense: Yeah, I figured. Who hasn’t made that kind of silly mistake? And who’s ever noticed it in his own code without at least 6 hours of bashing his head against the monitor?
Maybe someone find this decorator useful, based on TheSoundDefense answer:
import time import signal class TimeoutException(Exception): # Custom exception class pass def break_after(seconds=2): def timeout_handler(signum, frame): # Custom signal handler raise TimeoutException def function(function): def wrapper(*args, **kwargs): signal.signal(signal.SIGALRM, timeout_handler) signal.alarm(seconds) try: res = function(*args, **kwargs) signal.alarm(0) # Clear alarm return res except TimeoutException: print u'Oops, timeout: %s sec reached.' % seconds, function.__name__, args, kwargs return return wrapper return function @break_after(3) def test(a, b, c): return time.sleep(10) >>> test(1,2,3) Oops, timeout: 3 sec reached. test (1, 2, 3) <> If you can break your work up and check every so often, that’s almost always the best solution. But sometimes that’s not possible—e.g., maybe you’re reading a file off an slow file share that every once in a while just hangs for 30 seconds. To deal with that internally, you’d have to restructure your whole program around an async I/O loop.
If you don’t need to be cross-platform, you can use signals on *nix (including Mac and Linux), APCs on Windows, etc. But if you need to be cross-platform, that doesn’t work.
So, if you really need to do it concurrently, you can, and sometimes you have to. In that case, you probably want to use a process for this, not a thread. You can’t really kill a thread safely, but you can kill a process, and it can be as safe as you want it to be. Also, if the thread is taking 5+ seconds because it’s CPU-bound, you don’t want to fight with it over the GIL.
There are two basic options here.
First, you can put the code in another script and run it with subprocess :
subprocess.check_call([sys.executable, 'other_script.py', arg, other_arg], timeout=5) Since this is going through normal child-process channels, the only communication you can use is some argv strings, a success/failure return value (actually a small integer, but that’s not much better), and optionally a hunk of text going in and a chunk of text coming out.
Alternatively, you can use multiprocessing to spawn a thread-like child process:
p = multiprocessing.Process(func, args) p.start() p.join(5) if p.is_alive(): p.terminate() As you can see, this is a little more complicated, but it’s better in a few ways:
- You can pass arbitrary Python objects (at least anything that can be pickled) rather than just strings.
- Instead of having to put the target code in a completely independent script, you can leave it as a function in the same script.
- It’s more flexible—e.g., if you later need to, say, pass progress updates, it’s very easy to add a queue in either or both directions.
The big problem with any kind of parallelism is sharing mutable data—e.g., having a background task update a global dictionary as part of its work (which your comments say you’re trying to do). With threads, you can sort of get away with it, but race conditions can lead to corrupted data, so you have to be very careful with locking. With child processes, you can’t get away with it at all. (Yes, you can use shared memory, as Sharing state between processes explains, but this is limited to simple types like numbers, fixed arrays, and types you know how to define as C structures, and it just gets you back to the same problems as threads.)
Ideally, you arrange things so you don’t need to share any data while the process is running—you pass in a dict as a parameter and get a dict back as a result. This is usually pretty easy to arrange when you have a previously-synchronous function that you want to put in the background.
But what if, say, a partial result is better than no result? In that case, the simplest solution is to pass the results over a queue. You can do this with an explicit queue, as explained in Exchanging objects between processes, but there’s an easier way.
If you can break the monolithic process into separate tasks, one for each value (or group of values) you wanted to stick in the dictionary, you can schedule them on a Pool —or, even better, a concurrent.futures.Executor . (If you’re on Python 2.x or 3.1, see the backport futures on PyPI.)
Let’s say your slow function looked like this:
def spam(): global d for meat in get_all_meats(): count = get_meat_count(meat) d.setdefault(meat, 0) += count def spam_one(meat): count = get_meat_count(meat) return meat, count with concurrent.futures.ProcessPoolExecutor(max_workers=1) as executor: results = executor.map(spam_one, get_canned_meats(), timeout=5) for (meat, count) in results: d.setdefault(meat, 0) += count As many results as you get within 5 seconds get added to the dict; if that isn’t all of them, the rest are abandoned, and a TimeoutError is raised (which you can handle however you want—log it, do some quick fallback code, whatever).
And if the tasks really are independent (as they are in my stupid little example, but of course they may not be in your real code, at least not without a major redesign), you can parallelize the work for free just by removing that max_workers=1 . Then, if you run it on an 8-core machine, it’ll kick off 8 workers and given them each 1/8th of the work to do, and things will get done faster. (Usually not 8x as fast, but often 3-6x as fast, which is still pretty nice.)
Как остановить выполнение функции в Python?
Для остановки выполнения функции в Python можно использовать ключевой оператор return . Когда функция достигает этого оператора, она прекращает выполнение и возвращает указанное значение.
def func(): print('Часть функции, где код сработает') x = 11 return x # Функция возвращает значение переменной x и завершает свою работу. print('Эта часть функции - нет') y = 22 return y a = func() print(a) # => Часть функции, где код сработает # => 11 Exit a Function in Python
- Implicit Return Type in Python
- Explicit Return Type in Python
Every program has some flow of execution. A flow is nothing but how the program is executed. The return statement is used to exit Python’s function, which can be used in many different cases inside the program. But the two most common ways where we use this statement are below.
- When we want to return a value from a function after it has exited or executed. And we will use the value later in the program.
def add(a, b): return a+b value = add(1,2) print(value) Here, it returns the value computed by a+b and then stores that value which is 3 , inside the value variable.
def add(a, b): if(a == 0): return elif(b == 0): return else: sum = a + b return sum value = add(0,2) print(value) Here, if the values of either a or b are 0 , it will directly return without calculating the numbers’ sum. If they are not 0 then only it will calculate and return the sum .
Now, if you implement this statement in your program, then depending upon where you have added this statement in your program, the program execution will change. Let’s see how it works.
Implicit Return Type in Python
Suppose we have a function inside which we have written using an if statement, then let’s see how the program behaves.
def solution(): name = "john" if(name == "john"): print('My name ',name) solution() The solution() function takes no arguments. Inside it, we have a variable called name and then check its value matches the string john using the if statement. If it matches, we print the value of the name variable and then exit the function; otherwise, if the string doesn’t match, we will simply exit it without doing anything.
Here, you might think that since there is no return statement written in the code, there is no return statement present. Note that the return statement is not compulsory to write. Whenever you exit any Python function, it calls return with the value of None only if you have not specified the return statement. The value None means that the function has completed its execution and is returning nothing. If you have specified the return statement without any parameter, it is also the same as return None . If you don’t specify any return type inside a function, then that function will call a return statement. It is called an implicit return type in Python.
Explicit Return Type in Python
Whenever you add a return statement explicitly by yourself inside the code, the return type is called an explicit return type. There are many advantages of having an explicit return type, like you can pass a value computed by a function and store it inside a variable for later use or stop the execution of the function based on some conditions with the help of a return statement and so on. Let’s see an example of the explicit type in Python.
def Fibonacci(n): if n 0: print("Fibo of negative num does not exist") elif n == 0: return 0 elif n == 1 or n == 2: return 1 else: return Fibonacci(n-1) + Fibonacci(n-2) print(Fibonacci(0)) This is a program for finding Fibonacci numbers. Notice how the code is return with the help of an explicit return statement. Here, the main thing to note is that we will directly return some value if the number passed to this function is 2 or lesser than 2 and exit the function ignoring the code written below that. We will only execute our main code (present inside the else block) only when the value passed to this function is greater than 2 .
Sahil is a full-stack developer who loves to build software. He likes to share his knowledge by writing technical articles and helping clients by working with them as freelance software engineer and technical writer on Upwork.