I am using Yii 1 with PHP 7.1.33 running on an Apache/2.4.41 server macOS Catalina 10.15.3
In my application some of the requests are dying unexpectedly and randomly like this
In this image two calls are being made to the same api but one is succeeding and other one is returning no response.
I know the parameters are different but even when I am doing a die dump at the start of controller method, I'm getting the same result, first one succeeding, second one failing.
The same error is occurring with other apis which are in no way related to this api. And when this error is occurring, I am getting
[Tue Mar 17 16:34:39.572039 2020] [core:notice] [pid 697] AH00052: child pid 1233 exit signal Segmentation fault (11)
in my apache logs.
Related
question is pretty simple even if not so obvious.
I managing a website that's showing issues with connection timeout, making the web service unaccessible and not working properly when it happens.
The actual environment is WooCommerce on a self-managed secured VPS.
php7, php-fpm, wordpress 5.+, apache2, mysql, cPanel, centOs
For a couple of time the site became unreachable showing Connection Timeout error, even if all services was up and running.
How to track or log requests originating the connection timeout error?
For example, with an application ran on the server that:
log every requests that occurs with more than 30 seconds execution time
the moment of the request
the request URL.
Any software, wordpress, cPanel extension or service in order to detect the issue?
I get the solution in matter of detecting the situation when it happens.
The key information can be found in PHP-FPM log (deducted that by the fact that restarting PHP-FPM and not Apache2 was solving the problem, due to a probable PHP-FPM thread appended).
i.e., in a cPanel enviroment you would find the log at the path:
/opt/cpanel/ea-php$$/root/usr/var/log/php-fpm/error.log
In the file the following lines are noteful:
[Tue Jul 16 09:55:30.677693 2019] [proxy_fcgi:error] [pid 17283] (70007)The timeout specified has expired: [client 162.158.91.192:16946] AH01075: Error dispatching request to : (polling), referer: https://www.example.com/my-resource.php
[Tue Jul 16 10:00:30.738947 2019] [proxy_fcgi:error] [pid 17283] (70007)The timeout specified has expired: [client 162.158.91.192:16946] AH01075: Error dispatching request to : (polling), referer: https://www.example.com/my-resource.php
So I've been able to inspect the shown resource and replicate and solve the issue.
I am trying to install Wordpress 5 on PHP 7.3 today on Apache/2.4.37 (Ubuntu) and it is constantly throwing errors like this:
[Sun Dec 09 23:36:47.885206 2018] [core:notice] [pid 17094] AH00052:
child pid 17162 exit signal Segmentation fault (11)
It's just redirecting to this page: https://localhost/wordpress/wp-admin/setup-config.php without any response and above error in apache's error log.
What exactly is causing this and how can I fix this?
disable xdebug if you have it enabled, while it comes out of beta for php7.3
Apache (2.4.25 FPM/FastCGI) stop responding and crashes after a while on my wordpress multisite website.
For some reason it used to run fine several days until I changed the wordpress(mu) domain using this script github.com/interconnectit/Search-Replace-DB (probably not the reason).
When restarted it works for about an hour then choked again.
I have added a mysql slow query log but its empty.
Apache error_log has multiple errors like this one:
[Wed Sep 06 08:50:27.941819 2017] [proxy_fcgi:error] [pid 25444:tid 140610719610624] (70007)The timeout specified has expired: [client x.x.x.x:53398] AH01075: Error dispatching request to : (polling)
and occasionally this error appears too:
[Wed Sep 06 09:13:33.296777 2017] [core:notice] [pid 10710:tid 140611211331392] AH00051: child pid 25582 exit signal Segmentation fault (11), possible coredump in /opt/bitnami/apache2
another occasional error (expected due to the multiple AH01075 timeout errors):
[Mon Sep 04 20:18:58.758718 2017] [mpm_event:error] [pid 19928:tid 140675798046528] AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
Stopping mysql server kills my site but seems to keep apache working fine, at least for an hour or two that i`ve been testing.
Is 457452 value is normal for mysqld.bin in the following process list? (ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS)
2128 /sbin/init
2400 php-fpm: master process (/opt/bitnami/php/etc/php-fpm.conf)
3840 -bash
4244 sshd: ubuntu [priv]
6928 /usr/bin/gonit
11036 /opt/bitnami/apache2/bin/httpd.bin -f /opt/bitnami/apache2/conf/httpd.conf -DDISABLE_BANNER
11320 /opt/bitnami/apache2/bin/httpd.bin -f /opt/bitnami/apache2/conf/httpd.conf -DDISABLE_BANNER
11364 /opt/bitnami/apache2/bin/httpd.bin -f /opt/bitnami/apache2/conf/httpd.conf -DDISABLE_BANNER
11592 /opt/bitnami/apache2/bin/httpd.bin -f /opt/bitnami/apache2/conf/httpd.conf -DDISABLE_BANNER
20592 /opt/bitnami/apache2/bin/httpd.bin -f /opt/bitnami/apache2/conf/httpd.conf -DDISABLE_BANNER
57668 php-fpm: pool wordpress
57716 php-fpm: pool wordpress
65252 php-fpm: pool wordpress
67608 php-fpm: pool wordpress
68776 php-fpm: pool wordpress
457452 /opt/bitnami/mysql/bin/mysqld.bin --defaults-file=/opt/bitnami/mysql/my.cnf --basedir=/opt/bi
PHP Version 5.6.30
mysql.bin Ver 14.14 Distrib 5.6.36
Ubuntu 14.04.5 (bitnami image on AWS)
Memory usage while apache not responding: used:883mb of 992mb
php.ini script memory_limit = 128M
Was a bit long to put it in a comment, so replying as an answer.
Are you logging any php errors? From what you presented so far, my theory would be that:
You used that script to do a search & replace of db but it executed poorly causing malformed or corrupted data.
That caused your php scripts that rely on mysql to go haywire due to malformed or corrupted data.
Because your php on fcgi isn't responding, your apache waits for it and holds onto the resources.
All the requests are hanging and now you run out of resources and start seg-faulting.
And then stupid apache, seeing that you're out of child count asks you to raise it not realizing you're already exhausted on physical resource beyond reason. Typical apache...
Because apache is now out of free children, it can't take anymore requests and it appears as if all of apache is broken even though it's merely clogged up by bad fcgi processes.
If my theory is correct, you should.
Check that none of the mysql data is corrupted
Check that the data that was replaced in mysql was as what you expected
Check for PHP error logs as they should be logging something if they can.
You should isolate the PHP processes that are actually the cause of clogging up and remove it if cleaning up data alone wasn't enough. Ideally, no matter how bad the data, the php program should've died cleanly.
Good luck hunting.
I'm running a PHP script that converts a few hundred thousand records from a remote MySQL server to the local one. A few times during the transfer, the error_logs shows ...
[Wed Jul 03 17:02:36 2013] [notice] Graceful restart requested, doing restart
[Wed Jul 03 17:02:37 2013] [notice] SSL FIPS mode disabled
[Wed Jul 03 17:02:37 2013] [notice] Apache/2.2.22 (Unix) mod_ssl/2.2.22 OpenSSL/1.0.0-fips mod_bwlimited/1.4 PHP/5.3.16 configured -- resuming normal operations
It's rare, and I can't find a reason for it, but the script seems to continue as though nothing happened. Testing if data was lost would be very cumbersome. So I'm wondering, what happens if a MySQL query is interrupted? And why does PHP seems to ignore the restart entirely, and just continue running as though nothing happened? Any thoughts on how/what to test?
That's an Apache log, not a PHP log. Your script is probably running in CLI (command line) so an Apache restart won't affect it. Otherwise it may have been restarted with -k graceful which will allow child processes to complete.
So I'm wondering, what happens if a MySQL query is interrupted
If MySQL is restarted, the script will fail (unless the programmer incorporated some extra robust error handling to deal with that situation).
I have an application which interacts with a database. Suddenly and occasionally, pages are showing me Server connection Reset error in my Web browser. More surpisingly, accessing on a localhost page is triggering an alert on avast.
If I refresh pages using Ctrl+R, it happens occasionally. PHP is not showing any error messages, and it seems like the server is taking more time to respond than usual.
I am using wamp with apache 2.4, PHP 5.4.3. I am clueless as to where to start debugging or where to the problem is.
[Sun May 13 13:01:14 2012] [warn] Init: Session Cache is not configured [hint: SSLSessionCache]
[Sun May 13 13:01:14 2012] [notice] Apache/2.2.22 (Win32) mod_ssl/2.2.22 OpenSSL/0.9.8x configured -- resuming normal operations
[Sun May 13 13:01:14 2012] [notice] Server built: May 13 2012 12:51:11
[Sun May 13 13:01:14 2012] [notice] Parent: Created child process 3660
Apache server interrupted...
arn] Init: Session Cache is not configured [hint: SSLSessionCache]
[Sun May 13 13:01:15 2012] [notice] Child 3660: Child process is running
[Sun May 13 13:01:15 2012] [notice] Child 3660: Acquired the start mutex.
[Sun May 13 13:01:15 2012] [notice] Child 3660: Starting 64 worker threads.
[Sun May 13 13:01:15 2012] [notice] Child 3660: Starting thread to listen on port 80.
[Sun May 13 13:01:15 2012] [notice] Child 3660: Starting thread to listen on port 80.
[Sun May 13 13:01:28 2012] [notice] Parent: Received shutdown signal -- Shutting down the server.
[Sun May 13 13:01:28 2012] [notice] Child 3660: Exit event signaled. Child process is ending.
[Sun May 13 13:01:29 2012] [notice] Child 3660: Released the start mutex
[Sun May 13 13:01:30 2012] [notice] Child 3660: All worker threads have exited.
[Sun May 13 13:01:30 2012] [notice] Child 3660: Child process is exiting
[Sun May 13 13:01:30 2012] [notice] Parent: Child process exited successfully.
UPDATE:
When 'connection request' occurs if is use cachegrind it shows partial list of callstack of methods. means it does not run all the code. it is showing some require_once calls and that it. next time if i retry to get the page, page executes and shows the whole callstack.
When 'connection request occurs' it shows
18 different functions called in milliseconds (1 runs, 18 shown)
after retrying
220 different functions called in 329 milliseconds (2 runs, 220 shown)
I dont know why it is showing 2 runs. also it is taking more time to execute page. before it was doing it less than 100 ms.
Restart you computer and close/disable all running applications including anti-virus keeping only a minimal set of running applications. Close everything even those applications that you are sure can't interfere - you never know..
Make sure PHP shows all errors/warnings:
error_reporting(E_ALL);
ini_set('display_errors', '1');
Make sure you review every warning you receive from PHP. It may give you a clue.
Try to isolate a problem. Comment a chunk of code you suspect to cause the problem. Keep commenting until stop receive the error. Then start uncommenting until you find a problematic place. This way you can isolate a problematic code and once you see it, you may understand the problem.
Add lots of statements that will write to log file (or just echo). Then you can analyze the log file and understand at which point error happens helping you to isolate the problem...
Eventually you will find the problematic code block and will be able to track the problem. Hopefully :)
to restart Apache via PHP use the follow code in PHP
exec('/etc/init.d/httpd graceful');
and also look in the manual... and change the path to your httpd
Change apache listen port from 80 to 8080
file httpd.conf , change listen 80 to listen 8080 and restart apache
Re-try disabling temporarly the antivirus
Maybe it is not a right solution, but why not just try other WAMP version or maybe setup your system clock correct?. And have you tried to run Apache in minimal configuration? You may disable all extensions and modules (inc php itself) and if web server will work as expected you may turn one by one you modules. I don't know about your environment but you may also try to change working threads number and other values to minimal. I bet it doesn't help but at least you'll try.
If you have mod_security enabled try disabling and see if the same happens.. sometimes you can have over vigilant mod_security that will just go to http fail (sometimes can be as simple as inserting data with an ampersand into the database) and won't write any error to the server logs making it hard to troubleshoot, sounds similar to what you're experiencing, I would disable mod_security re-start server and see if the behavior has stopped!
It looks like some other people have encountered the same issue using Wamp:
Have you gone through each of the points enumerated in this post belonging to the official forum:
http://forum.wampserver.com/read.php?2,67660
Are you sure you don't have an include/require loop ?
That kind of loop make PHP eat up too much memory and apache kill itself to avoid it.
Or maybe some kind of 404 loop error ...
That kind of loop can happen if you have 404 errors sometimes. Here is an example :
You have a layout with an image which is missing, so it triggers a 404 errors. To display the page, the 404 errors add the layout around the error page which contains the missing image, which triggers another 404 errors.
Hope that helps.