Skip to content

set_timeout() fails to terminate execution when IO Latency is high #11971

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mikhainin opened this issue Aug 14, 2023 · 3 comments
Open

set_timeout() fails to terminate execution when IO Latency is high #11971

mikhainin opened this issue Aug 14, 2023 · 3 comments

Comments

@mikhainin
Copy link
Contributor

Description

I'm not sure if this is a bug or not and I haven't found the proper channel to ask (shall I raise a discussion in Internals?).

We faced a problem in our production cluster. Our setup is following:

SAPI: fpm
max_execution_time: 28
hard_timeout: 2

When Opcache is reloading, we're having excessive IO pressure, which is expected - PHP starts reading everything from disk, etc... However, scripts started executing for 5+ minutes - that wasn't expected.

Further investigation showed that zend_set_timeout_ex uses setitimer(ITIMER_PROF) by default, which means the time the process spends in IOWait doesn't seem to count.

So, basically, even though our gateway cannot wait for such long - it expects that application will finish earlier, PHP still performs the request. One more backside of such behaviour - the restart of Opcache takes longer - it waits until all the workers finish execution (or become killed by opcache when opcache.force_restart_timeout is reached).

This problem doesn't seem to appear when we use ITIMER_REAL. I can provide how we tested this.

I can see a couple of solutions here:

  • Switch to ITIMER_REAL (it looks like there are some concerns about the Apache module)
  • Choose one over another for some SAPIs when we are sure that there shouldn't be a conflict (e.g. FPM, CLI)
  • Make this behaviour configurable (either at runtime or compile time)
  • Something else?
@KapitanOczywisty
Copy link

There is RFC to add wall time limit https://wiki.php.net/rfc/max_execution_wall_time

@mikhainin
Copy link
Contributor Author

Oh, that's interesting @KapitanOczywisty! As far as I can see it didn't go forward. Any reason for that?
Would it be possible to re-open that activity?

@KapitanOczywisty
Copy link

I wasn't following progress of that, but you can read about that: #6504 https://externals.io/message/112492

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants