Related
This question already has answers here:
net::ERR_CONNECTION_RESET when large file takes longer than a minute
(7 answers)
Closed 3 years ago.
I have a site that used to be able to upload large files (large being > 10 or 20mb) but no longer can. I've been debugging this for hours at this point.
All php values are set ludicrously high:
post_max_size = 512M
upload_max_filesize = 512M
memory_limit = 1024M
max_execution_time = 600
max_input_time = 600
I've also set TimeOut 600 in httpd.conf.
Essentially, if I add a large file to an upload field, it never uploads. I can witness the "Uploading (1%)..." in the lower left in chrome showing the file start uploading. It will count up, sometimes even reaching 100%, then start over again at 0 and start counting up again, eventually failing with an ERR_CONNECTION_RESET message.
The eventual failure seems to happen after a random amount of time, sometimes 24 seconds, simetimes 3 minutes.
I tried a 170mb file and it will always get to 16 or 17% before it restarts. That always takes something like 22 seconds. Then, it will restart at 0 and count up to 16 or 17% again, then restart again. It ultimately fails with the ERR_CONNECTION_RESET message sometimes after restarting once, sometimes after restarting 4 or 5 times.
I also tried a 30mb file. This one will always reach right around 100% before restarting.
df -h shows plenty of file space remaining, and I was able to upload files fine via SFTP confirming that there is indeed sufficient hard disk space.
Files also upload fine using the exact same application on my development server, so I can rule out any application issues.
Smaller files also upload fine on the production server, i've tried files as large as 3 or 5mb with no issue.
I'm able to execute code like:
echo "start";
sleep(60);
echo "stop";
without any hiccup on production, so it isn't timing out all requests, only the uploads.
I've tried multiple browsers, and this is happening from multiple client locations.
There is never an error in any log I can find in /var/log/httpd.
I'm not running mod security. Nowhere in my application are any of the php settings overwritten. It's a pretty standard installation of apache and php.
The production server is Amazon Linux running Apache/2.4.39 and I've tried it on php 7.1 and php 7.2 and got the same result, both using mod_php.
I am well into the "banging head against wall" stage of this issue. Does anyone have any ideas what I can do to debug this?
Finally got this to work. Thanks to net::ERR_CONNECTION_RESET when large file takes longer than a minute
I had to add RequestReadTimeout header=0 body=0 to my httpd.conf file
It couldn't be within a vhost definition, at least I tried that hours ago and nothing. But, circled back and tried it again in httpd.conf and it worked.
TG.
I have a multipart file upload in a form with a php backend. I've set max_execution_time and max_input_time in php.ini to 180 and confirmed on the file upload that these values are set and set TimeOut 180 in Apache. I've also set
RewriteRule .* - [E=noabort:1]
RewriteRule .* - [E=noconntimeout:1]
When I upload a 250MB file on a fast connection it works fine. When I'm on a slower connection or a network link conditioner to artificially slow it down, the same file times out and on Chrome gives me net::ERR_CONNECTION_RESET after 1 minute (and 5 seconds) reliably. I've also tried other browsers with the same outcome, just different error messages.
There is no indication to an error in any log and I've tried both on http and https.
What would cause the upload connection to be reset after 1 minute?
EDIT
I've now also tried to have a simple upload form that bypasses any framework I'm using, still timeouts at 1 minute.
I've also just made a sleep script that timeouts after 2 and a half minutes, and that works, page takes around 2.5 minutes to load so I can't see how it's browser or header related.
I've also used a server with more RAM to ensure it's not related to that. I've tested on 3 different servers with different specs but all from the same CentOS 7 base.
I've now also upgraded to PHP 7.2 and updated the relevant fields again with no change in the problem.
EDIT 2
The tech stack for this isolated instance is
Apache 2.4.6
PHP 5.6 / 7.2 (tried both), has OPCache
Redis 3.2.6 for session information and key / value storage (ElastiCache)
PostgreSQL 10.2 (RDS)
Everything else in my tech stack has been removed from this test area to try and isolate the problem. EFS is on the system but in my most isolated test it's just using EBS.
EDIT 3
Here some logs from the chrome network debugger:
{"params":{"net_error":-101,"os_error":32},"phase":0,"source": {"id":274043,"type":8},"time":"3332701830","type":69},
{"params": {"error_lib":33,"error_reason":101,"file":"../../net/socket/socket_bio_adapter.cc","line":216,"net_error":-101,"ssl_error":1},"phase":0,"source": {"id":274043,"type":8},"time":"3332701830","type":56},
{"phase":2,"source":{"id":274038,"type":1},"time":"3332701830","type":159},
{"phase":1,"source": {"id":274038,"type":1},"time":"3332701830","type":164},
{"phase":1,"source": {"id":274038,"type":1},"time":"3332701830","type":287},
{"params": {"error_lib":33,"error_reason":101,"file":"../../net/socket/socket_bio_adapter.cc","line":113,"net_error":-101,"ssl_error":1},"phase":0,"source": {"id":274043,"type":8},"time":"3332701830","type":55},
{"params":{"net_error":-101},"phase":2,"source": {"id":274038,"type":1},"time":"3332701830","type":287},
{"params":{"net_error":-101},"phase":2,"source":{"id":274038,"type":1},"time":"3332701830","type":164},
{"params":{"net_error":-101},"phase":2,"source":{"id":274038,"type":1},"time":"3332701830","type":97},
{"phase":1,"source":{"id":274038,"type":1},"time":"3332701830","type":105},
{"phase":2,"source":{"id":274038,"type":1},"time":"3332701830","type":105},
{"phase":2,"source":{"id":274043,"type":8},"time":"3332701830","type":38},
{"phase":2,"source":{"id":274043,"type":8},"time":"3332701830","type":38},
{"phase":2,"source":{"id":274043,"type":8},"time":"3332701830","type":34},
{"params":{"net_error":-101},"phase":2,"source":{"id":274038,"type":1},"time":"3332701830","type":2},
I went through a similar problem, in my case it was related to mod_reqtimeout by adding:
RequestReadTimeout header=20-40, MinRate=500 body=20, MinRate=500
to httpd.conf did the trick!
You can check the documentation here.
Hope it helps!
Original source here
ERR_CONNECTION_RESET usually means that the connection to the server has ceased without sending any response to the client. This means that the entire PHP process has died without being able to shut down properly.
This is usually not caused by something like an exceeded memory_limit. It could be some sort of Segmentation Fault or something like that. If you have access to error logs, check them. Otherwise, you might get support from your hosting company.
I would recommend you to try some of these things:
Try cleaning the browser's cache. If you have already visited the page, it is possible for the cache to contain information that doesn’t match the current version of the website and so blocks the connection setup, making the ERR_CONNECTION_RESET message appear.
Add the following to your settings:
memory_limit = 1024M
max_input_vars = 2000
upload_max_filesize = 300M
post_max_size = 300M
max_execution_time = 990
Try setting the following input in your form:
In your processing script, increase the session timeout:
set_time_limit(200);
You might need to tune up the SSL buffer size in your apache config file.
SSLRenegBufferSize 10486000
The name and location of the conf file is different depending on distributions.
In Debian you find the conf file in /etc/apache2/sites-available/default-ssl.conf
A few times it is mod_security module which prevents post of large data approximately 171 KB. Try adding/modifying the following in mod_security.conf
SecRequestBodyNoFilesLimit 10486000
SecRequestBodyInMemoryLimit 10486000
I hope something might work out!
Incase anybody else runs into this - there is also a problem with this relating to PHP-FPM. If you dont set "ProxyTimeout" in your httpd.conf - PHP-FPM uses a default timeout of one minute. It took me several hours to figure out the problem as I initially was thinking of all the normal settings like everyone else.
I had the same problem. I used the resumable file upload method where if the internet is disconnected and reconnects back then the upload resumes from the same progress.
Check out the library https://packagist.org/packages/pion/laravel-chunk-upload
Installation
composer require pion/laravel-chunk-upload
Add service provider
\Pion\Laravel\ChunkUpload\Providers\ChunkUploadServiceProvider::class
Publish the config
php artisan vendor:publish --provider="Pion\Laravel\ChunkUpload\Providers\ChunkUploadServiceProvider"
In my opinion it maybe relative to one of them:
About apache config (/etc/httpd2/conf ou /etc/apache2/conf):
Timeout 300
max_execution_time = 300
About php config ('php.ini'):
upload_max_filesize = 2000M
post_max_size = 2000M
max_input_time = 300
memory_limit = 3092M
max_execution_time = 300
About PostgreSQL config (execute this request):
SET statement_timeout TO 0;
About proxy, (or apache mod_proxy), it maybe also be due to proxy timeout configuration
in case anyone has the same issue, the problem I encountered is that the http request has to go through proxy sever and waf, small files upload is ok, but with large files the tcp connection automatically closed, how to validate:
simply change your hosts setting point the domain to the web server ip address (or you may use firefox with no-proxy if there is no waf), if your problem gone then it's the caused by the proxy or the waf in between your web server and the browser
Connection-Reset occurs when php process dies without proper error message.
Changing oracle client version from 19 to 12c and then appropriately configuring in php.ini solved the connection reset issue for our team.
I'm fairly new to PHP, but I'm having a recurring issue via multiple different scripts and servers when uploading images via ShareX to my server with a custom script, specifically this one.
I've migrated servers (I was on a shared host, now I'm on a VPS), and have since changed to using this script, but I'm still having the issue and I don't know what exactly the problem is.
The issue (does not occur 100% of the time, but it does most of the time; sometimes it works after retrying) is that uploading images over a certain size, about 250-500KB times out or fails most of the time. After 60 seconds, I get a 502 error (Bad Gateway) on ShareX.
I've looked up common solutions to similar problems ("large" files timing out in PHP), and have checked the following variables in my PHP.ini file.
max_execution_time = 60
max_input_time = 60
memory_limit = 128M
post_max_size = 8M
When uploads are successful, it takes a few seconds in total to upload and get the link of the uploaded image returned, but when it fails, it's always 60 seconds and then error. There is no middle ground, it's either it succeeds instantly or times out after 60 seconds.
I don't know exactly how to go about finding what exactly the error (if any) is. When it happens, ShareX reports a (502) Bad Gateway error, the 'Response:' is just the source code of the page (the script is set up to redirect you to this page if it detects you aren't uploading anything or it fails), and the 'Stack Trace' is the following:
StackTrace:
at System.Net.HttpWebRequest.GetResponse()
at ShareX.UploadersLib.Uploader.UploadData(Stream dataStream, String url, String fileName, String fileFormName, Dictionary`2 arguments, NameValueCollection headers, CookieCollection cookies, ResponseType responseType, HttpMethod method, String requestContentType, String metadata)
Edit: My server is behind cloudflare, and I read that cloudflare might cause problems. However, I've checked the settings and the maximum upload size is set at 100MB on cloudflare, and pausing it doesn't seem to help.
Edit: I removed the limit on post_max_size which was 8M and it seems to have partly fixed the issue. I can now upload things up to about 3MB but after that it always fails with a custom error message from the script.
When increasing file POST limits, you may need to change at least 2 settings:
upload_max_filesize = 30M
post_max_size = 32M
Dont think it has anything to do with CloudFlare. See if you can check the error log for Apache if the above settings dont work.
I am using jquery ajax to send the url of a file (csv file) located on the server to my php script so as to process it.
The csv file contains telephone calls. If i have a file with even 10.000 calls everything is ok. But if i try a big file with like for example 20000 calls then i get an Ajax Error 0 . I check for server responce with firebug but i get none.
This behaviour occurs after like 40mins of w8ing for the php script to end. So why do i get this error on big files only? Does it have to do with apache, mysql or the server itself? Anyone able to help will be my personal hero cause this is driving me nuts.
I need a way to figure out whats happening exactly but firebug wont return a server responce. Any other way i can find out whats happening?
I checked the php error log and it reports nothing on the matter
Thanks in advance.
The script may have timed out:
See your php.ini file
max_execution_time
max_input_time ;# for the max time an input can be processed
Were your PHP.ini is depends on your enviroment, more information: http://php.net/manual/en/ini.php
Check:
max_input_time
This sets the maximum time in seconds a script is allowed to parse input data, like POST and GET. It is measured from the moment of receiving all data on the server to the start of script execution.
max_execution_time
This sets the maximum time in seconds a script is allowed to run before it is terminated by the parser. This helps prevent poorly written scripts from tying up the server. The default setting is 30. When running PHP from the command line the default setting is 0.
Also
Your web server can have other timeout configurations that may also interrupt PHP execution. Apache has a Timeout directive and IIS has a CGI timeout function. Both default to 300 seconds. See your web server documentation for specific details.
First enable php error by placing below code at top of the php file.
error_reporting(E_ALL);
Then as Shamil explained in this answer, checkout your max_execution_time settings of your php.
To check max_execution time, open your php.ini file and search for that, and then change it to a maximum value of say one hour (3600).
I hope this will fix your issue.
Thank you
The Issue
When uploading files of around 8MB or over, I recieve a 500 Internal Server Error.
All PHP settings in php.ini are correct
maxAllowedContentLength has been set in the web.config
Server Info
As one can probably tell from the maxAllowedContentLength, I am running IIS 7.5, with FastCGI and PHP 5.3.17
Additional Info
I have tried so many different things to get this working but simply cannot find the issue.
However, I have found the following bits of info that may help figure out the root of this problem:
When uploading files (larger ones) using the Media Wiki that I have on the server, I receive the same error, this goes to show that it is not an error in my code.
Most importantly - I managed to upload an 18MB file in the Plesk File Manager, this obviously means that Plesk was able to get around this config issue. I have tried to copy all of the Plesk Control Panel settings over to this domain in IIS but this does not seem to work.
The error is being returned before the script is executed, as I have tried writing exit; at the top to try to get a blank screen, but this is ignored and the 500 error is returned.
I think that the issue lies within the configure command part of the PHP configuration, because when I change the handler mapping of the .php files to use the Plesk php-cgi.exe instead of the usual one, I do not get the 500 Internal Error. Having said that, I cannot leave it on this PHP version as it is Plesk's own exe and there are other configuration issues.
The reason why I think it may be to do with the configure command, is simply because this differs hugely from one phpinfo() to the other.
If you have any ideas or suggestions, please post them. I have tried everything to my knowledge and cannot seem to fix this. If only it was Linux...
Thanks in advance
UPDATE 1
Forgot to add, there are no errors being returned in the PHP error log. As for IIS errors, I do not know where to look
UPDATE 2
This is what I have placed in my web.config file:
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="2147483647" />
</requestFiltering>
</security>
UPDATE 3
With your help, we have managed to get the error displayed by IIS. This is what I am receiving:
PHP Warning: POST Content-Length of 12221448 bytes exceeds the limit
of 8388608 bytes in Unknown on line 0
Is that to do with post_max_size?
UPDATE 4
PHP settings as follows (from phpinfo()):
post_max_size = 64M
memory_limit = 128M
max_file_uploads = 20
max_execution_time = 6000
upload_max_filesize = 64M
UPDATE 5
Lastly, just in case anybody can spot any potential issues, Plesk is able to upload large files absolutely fine, so I assumed that their php-cgi.exe was compiled differently. When I read a phpinfo() of their configuration the configure command information was very different:
My configuration:
cscript /nologo configure.js "--enable-snapshot-build"
"--disable-isapi" "--enable-debug-pack" "--without-mssql"
"--without-pdo-mssql" "--without-pi3web"
"--with-pdo-oci=C:\php-sdk\oracle\instantclient10\sdk,shared"
"--with-oci8=C:\php-sdk\oracle\instantclient10\sdk,shared"
"--with-oci8-11g=C:\php-sdk\oracle\instantclient11\sdk,shared"
"--enable-object-out-dir=../obj/" "--enable-com-dotnet=shared"
"--with-mcrypt=static" "--disable-static-analyze"
Plesk's Configuration:
cscript /nologo configure.js "--enable-debug-pack" "--enable-cli"
"--enable-cgi" "--enable-isapi" "--enable-one-shot" "--enable-pdo"
"--enable-intl" "--with-openssl=shared" "--with-pdo-odbc"
"--with-iconv" "--with-xml" "--with-xsl" "--with-mysql"
"--with-mysqlnd" "--with-mysqli" "--with-pdo-sqlite"
"--with-pdo-mysql" "--with-curl=shared" "--enable-mbstring"
"--enable-mbregex" "--with-imap=shared" "--enable-sockets"
"--enable-shmop" "--enable-soap"
UPDATE (ANSWER)
This is extremely weird as the phpinfo() info is saying one thing, but it is obviously being ignored, not sure why.
If I change the post_max_size in Plesk, for that particular domain/sub-domain, then nothing is changed (although it appears to have changed in the phpinfo()). However, if I actually change the post_max_value in the php.ini then this fixes the issue.
The reason why this is not a good way to fix this, is simply because when Plesk updates, the php.ini is overwritten as PHP is updated and resultantly the changes made to the php.ini are lost. Which means that everytime that Plesk updates I will need to make changes tot he php.ini. This is why Plesk offers the ability to change PHP settings without making changes to the php.ini.
Can anybody think of why PHP is ignoring the local value and reverting to the value in the php.ini, even though the php.ini states that the local value is different?
If you look at the source code of PHP, you can see on the file php-5.4.8-src\main\rfc1867.c line 706-709 this:
if (SG(post_max_size) > 0 && SG(request_info).content_length > SG(post_max_size)) {
sapi_module.sapi_error(E_WARNING, "POST Content-Length of %ld bytes exceeds the limit of %ld bytes", SG(request_info).content_length, SG(post_max_size));
return;
}
Same is there also in file php-5.4.8-src\main\SAPI.c.
So, the message PHP Warning: POST Content-Length of 12221448 bytes exceeds the limit of 8388608 bytes in Unknown on line 0 is about post_max_size setting. You have confirmed from using phpinfo() that you have this setting configured correctly, but it seems to be using the default value of 8M anyway.
As to why, see this thread:
As it turns out, on Windows, you can only set ini directives that are
marked PHP_INI_USER per directory. Unfortunately,
upload_max_filesize and post_max_size are both PHP_INI_PERDIR.
From the PHP docs at
http://php.net/manual/en/configuration.changes.php
The settings for the directory would be active for any script running from this directory or any subdirectory of it. The values
under the key should have the name of the PHP configuration directive
and the string value. PHP constants in the values are not parsed.
However, only configuration values changeable in PHP_INI_USER can be set this way, PHP_INI_PERDIR values can not.
So even though Plesk has an interface to change those directives, and
even though phpinfo() picks up on them, they do nothing to change
the actual max upload sizes. Plesk should not allow you to change
those on Windows, and phpinfo() should not report the change, but
what can you do.
So, it's post_max_size, and it needs to be set on php.ini. Plesk setting simply will not work, even though phpinfo says otherwise. I also opened a bug entry on phpinfo behaviour as there didn't seem to be an entry for it.
This is a fairly common error and is due to the fact that the size of data being uploaded does not match file size: even if you POST max size is not exceeded by the file size, it could be by the uploaded data size.
See this page in the PHP manual.
; Maximum size of POST data that PHP will accept.
post_max_size = 8M
Another source of troubles (for VERY large texts) is UTF8 encoding. You might find yourself with a "six megabytes" TEXTAREA that is actually 6 mega*characters*, and with international codepoints it might run to, say, 8.2 megabytes. Thus you get an apparently contradictory situation of "six megabytes data exceed the configured 8 megabytes limit".
Update
You report two apparently contradictory facts:
PHP settings as follows (from phpinfo()):
post_max_size = 64M
and
PHP Warning: POST Content-Length of 12221448 bytes exceeds the limit of 8388608 bytes
It is clear from the PHPINFO that the limit for POST is 64M. Yet the error says that the limit is 8M (the default). So it seems to me that your code is talking to two different PHP implementations (Two different virtual hosts? A CGI version and a non-CGI version in the same host? Two different machines?)
IIS will re-use the FastCGI processes. You will need to kill off any old processes to get php.ini to reload.
Edit the FastCGI module and edit 'monitor changes to file' and select the php.ini file. This will force the child processes to restart whenever you save an edit.
You could turn the limits to -1, that way, you won't ever have troubles about the size of the files.
It is probably no the best solution as you are basically saying "if I don't see it, it doesn't exist", but believe, it's really reliable and will always work.