| Author |
|
nono303
Joined: 20 Dec 2016
Posts: 215
Location: Lille, FR, EU
|
 Posted: Wed 20 May '20 12:22 Post subject: mod_fcgid latency on high concurrency |
 |
|
Hi @all!
I open this topic to (re?)start a discussion around the mod_fcgid latency in case of high concurrency.
This subject has already been discussed (cf. https://bz.apache.org/bugzilla/show_bug.cgi?id=53693) with a revert in mod_fcgid 2.3.9
Being faced with the problem (thumb gallery whose images are generated in php via mod_fcgid involving concurrency from several tens of processes without lazy-load) I have randomly latencies of (exactly) one second on 1 to several images.
This is explained in the mod_fcgid code if all the processes are busy: | Code: | | apr_sleep (apr_time_from_sec (1)); |
https://github.com/pagespeed/mod_fcgid/blob/master/modules/fcgid/fcgid_bridge.c#L456
In your point of view, what would be the best way to go around this bottleneck?1) Reducing the waiting time apr_sleep (apr_time_from_msec (XX)); maybe with XX externalized as the time to spawn a new process depends on many parameters
2) Directly force a new process spawn and try to handle request to it more than 2 times (see patch https://www.mail-archive.com/dev@httpd.apache.org/msg55268.html which I am currently testing)
3) Other ideas? On configuration side? Many thanks for your advices!
++NoNo |
|
| Back to top |
|
nono303
Joined: 20 Dec 2016
Posts: 215
Location: Lille, FR, EU
|
| Posted: Wed 20 May '20 13:47 Post subject: |
|
|
...and here is my config:
| Code: | FcgidInitialEnv PHPRC "C:/serveur/php7"
<Files ~ (\.php)>
FcgidWrapper "C:/serveur/php7/php-cgi.exe -c C:/serveur/php7/php.ini" .php
</Files>
FcgidInitialEnv TEMP "R:/php-httpd_temp"
FcgidInitialEnv TMP "R:/php-httpd_temp"
FcgidInitialEnv windir "C:\WINDOWS"
FcgidInitialEnv SystemRoot "C:\Windows"
FcgidInitialEnv SystemDrive "C:"
FcgidInitialEnv PROCESSOR_ARCHITECTURE "AMD64"
FcgidInitialEnv PROCESSOR_IDENTIFIER "Intel64 Family 6 Model 42 Stepping 7, GenuineIntel"
FcgidInitialEnv PROCESSOR_LEVEL "6"
FcgidInitialEnv PROCESSOR_REVISION "2a07"
FcgidInitialEnv NUMBER_OF_PROCESSORS 4
FcgidInitialEnv PHP_FCGI_MAX_REQUESTS 0
FcgidInitialEnv PHP_FCGI_CHILDREN 0
FcgidFixPathinfo 1
FcgidWin32PreventOrphans On
FcgidMaxRequestsPerProcess 0
FcgidMaxProcesses 40
FcgidMaxProcessesPerClass 40
FcgidMinProcessesPerClass 16
FcgidIdleScanInterval 1
FcgidErrorScanInterval 1
FcgidZombieScanInterval 1
FcgidIdleTimeout 60
FcgidProcessLifeTime 60
FcgidConnectTimeout 1
FcgidIOTimeout 600
FcgidBusyTimeout 600
FcgidBusyScanInterval 600
FcgidMaxRequestLen 134217728
FcgidOutputBufferSize 0
FcgidPassHeader Authorization |
|
|
| Back to top |
|
James Blond
Moderator
Joined: 19 Jan 2006
Posts: 7442
Location: EU, Germany, Next to Hamburg
|
| Posted: Wed 20 May '20 16:33 Post subject: |
|
|
The re-spawn takes time. I noticed that, too.
You set FcgidProcessLifeTime to one minute. I set it to 0 (endless). Every minute mod_fcgid kills the spawned php-cgi process.
Why do you let the module kill the php process so early? That also causes a log flood[1]. It has only been fixed in trunk. But nobody dares a release of 2.3.10. I run the .10 for years without any issues.
--- edit ---
My config[2]
--- /edit ---
[1] https://bz.apache.org/bugzilla/show_bug.cgi?id=54597
[2] https://www.apachelounge.com/viewtopic.php?t=2394 |
|
| Back to top |
|
nono303
Joined: 20 Dec 2016
Posts: 215
Location: Lille, FR, EU
|
| Posted: Sun 24 May '20 14:53 Post subject: |
|
|
Thanks @James Blond for your feedback!
Yes, FcgidProcessLifeTime set at 1 minute is really too short but setting it to endless it’s not possible for me with php in a multiple vhost context (different kind of php usage that may have memory issue like big persistent footprint or leaks)
I tried it and all php-cgi process were zombified after some hours… like with fcgi-proxy)
I also run mod_fcgid built from the trunk (2.3.10 as I understand…) and never had issue, just this frequent "1 second latency" on high concurrency.
I’ll try to make a FcgidMinSpareProcesses patch… for fun ^^ |
|
| Back to top |
|
James Blond
Moderator
Joined: 19 Jan 2006
Posts: 7442
Location: EU, Germany, Next to Hamburg
|
| Posted: Sun 24 May '20 20:41 Post subject: |
|
|
| Well I use the Thread Safe version (TS) and don't have such issues at all. |
|
| Back to top |
|
nono303
Joined: 20 Dec 2016
Posts: 215
Location: Lille, FR, EU
|
| Posted: Fri 05 Jun '20 9:44 Post subject: |
|
|
After 1 week testing TS instead of NTS (but process spawned by fcgid doesn't share context except opcache, so...) nothing changes for me with long FcgidProcessLifeTime value : some child are not reachable after some long time.
I know that some of my php script are quite dirty.
Fyi, after testing the 3 patches (see https://bz.apache.org/bugzilla/show_bug.cgi?id=53693 & https://www.mail-archive.com/dev@httpd.apache.org/msg55262.html) the one from Merijn van den Kroonenberg seems to be the best compromise without any 503 and lower latencies |
|
| Back to top |
|
James Blond
Moderator
Joined: 19 Jan 2006
Posts: 7442
Location: EU, Germany, Next to Hamburg
|
| Posted: Fri 05 Jun '20 15:38 Post subject: |
|
|
I'm a bit out of context after such a long time looking that the code of mod_fcgid
basicly what that patches removes is
| Code: |
if (i > 0 || j > 0 || count_busy_processes(r, &fcgi_request)) {
- apr_sleep(apr_time_from_sec(1));
|
removing i (connected ipc handle) seems okay.
removing j (FCGID_APPLY_TRY_COUNT) also okay
The condition also calls count_busy_processes which sets a lock. That might cause a bottle neck.
So maybe only removing that function call and lowering the sleep time to re reasonable might be it. This is only my thinking with you trying  |
|
| Back to top |
|
federicoemartinez
Joined: 13 May 2023
Posts: 3
Location: Argentina
|
| Posted: Sat 13 May '23 14:03 Post subject: |
|
|
Would it be possible to you to publish the patched version with the patch by Merijn van den Kroonenberg so I can try it? I think I have hit the same issue but hasn't been able to configure a windows machine to build mod_fcgid yet. |
|
| Back to top |
|
nono303
Joined: 20 Dec 2016
Posts: 215
Location: Lille, FR, EU
|
|
| Back to top |
|
bagu
Joined: 06 Jan 2011
Posts: 193
Location: France
|
| Posted: Sun 14 May '23 21:02 Post subject: |
|
|
Apache don't load fcgid module if i replace mod_fcgid.so 2.3.10 vs17 by yours.
Is there something else to do than just replace ? |
|
| Back to top |
|
federicoemartinez
Joined: 13 May 2023
Posts: 3
Location: Argentina
|
| Posted: Mon 15 May '23 1:24 Post subject: |
|
|
Thanks a lot! |
|
| Back to top |
|
nono303
Joined: 20 Dec 2016
Posts: 215
Location: Lille, FR, EU
|
|
| Back to top |
|
bagu
Joined: 06 Jan 2011
Posts: 193
Location: France
|
| Posted: Mon 15 May '23 19:50 Post subject: |
|
|
| Ok, it was a download problem. Thanks |
|
| Back to top |
|
