I have a really odd problem. I am using an upload form to upload videos. Sometimes I have to try twice to upload a file so I know it works but these files take a long time to upload so I don't want the end-user getting mad if the process fails. Also, this works 100% of the time on my test machine so I am thinking there is a config problem.
The file is 330mb and I set upload_max_filesize and post_max_size to 500mb. The max_execution_time and max_input_time are set to 60000 for testing purposes. memory_limit is what I think may be the problem. It is set to 128mb. Does it need to be higher to have a consistent upload success rate? Anybody know of any other problems that could cause things to go wrong?
You're right in assuming memory_limit is your culprit.
Taken from php.net.
post_max_size (int)
Sets max size of post data allowed. This setting also affects file upload. To upload large files, this value must be larger than upload_max_filesize.
If memory limit is enabled by your configure script, memory_limit also affects file uploading. Generally speaking, memory_limit should be larger than post_max_size. When an integer is used, the value is measured in bytes. Shorthand notation, as described in this FAQ, may also be used. (...)
Related
Today i was making a file upload for avatars, everything works great, it resizes the images etc, but occasionally when selecting a large and invalid file it produces this error :
Warning: POST Content-Length of 52091839 bytes exceeds the limit of 8388608 bytes in Unknown on line 0
You did not select a file to upload.
This does not happen all the time, just occasionally. Normally it just gives the correct error message when a file is too big.
Does anyone have an idea where this error comes from, and why it shows?
Thanks!
Sounds like you need to increase post_max_size in php.ini. If you have not already also increased upload_max_filesize, you are likely to need to increase it as well.
# php.ini
# Allow huge files:
# Post usually needs to be bigger than file upload size!
post_max_size = 256M
upload_max_filesize 128M
Update your post_max_size in php.ini to a larger value
upload_max_filesize sets the max file size that a user can upload while post_max_size sets the maximum amount of data that can be sent via a POST in a form. That might be the reason why you get big file error sometimes when you try upload a single file of large size but when you try to upload multiple files the above error is raised.
This has to do with the way the file is coded when uploaded.
Basically it is "seen" as a very large POST.
So you want your php.ini post_max_size larger than your upload_max_filesize value.
Depending on the encoding you use, it should be from 60% to 100% larger.
Otherwise you might have a max file size of 5 M, but this gets encoded to 8.01 M; when that happens, the check for file size passes, but the one for post body size fails. Hence your error.
I have a question - I have a file uploader and each user is allowed to upload max_size:30M.
Now I'd like to know from your experiences what I should set up in php.ini.
Here are the options I've already changed, but I'm not sure if that is the best:
upload_max_filesize = 30M
post_max_size = 30M
max_execution_time = 300
max_input_time = 300
memory_limit = 32M
Here I think memory_limit is a bit low. And is there something more I have to include?
When I tried to upload more than 30M, Firefox crashed.
Thanks for your answers.
If you work heavily with files you should split the file and upload it in separate parts, it doesn't choke the server as much and you can check the progress, also you don't need that much memory and excecution time.
For some reason someone said it's not true... Well I don't think Youtube or upload sites like Megaupload (rip) use 2gb of memory and 2 hours of maximum excecution time to upload the file of one user, they split the file, in whatever way they can, if you need to upload 30mb files, and you use standard PHP your webserver will choke until the upload finishes, unless you don't care about that, it's not recommended.
The easiest solution is to upload it by parts, you can do it using Flash/Silverlight/JS/HTML5 or whatever way you prefer, client-side, and then joining them server-side.
I am uploading files and having trouble finding an answer to this question. Some of the time my file upload fails at move_uploaded_file() so I was told this could be a memory problem. Unfortunately there is no documentation available that I could find to answer this question.
The file I am uploading is 400mb and I set upload_max_filesize and post_max_size to 500mb. The max_execution_time and max_input_time are set to 60000 for testing purposes. My memory_limit is set to default 128mb. Should this be changed? How can you calculate the optimal balance of memory_limit vs upload_max_filesize and post_max_size for say a 400mb file?
If the upload fails, you shouldn't even be attempting to do a move_uploaded_file. The proper bare-bones upload handling method is
if ($_FILES['yourfile']['error'] !== UPLOAD_ERR_OK) {
... upload failed, bail out
die("Failed upload");
}
// got here, upload must've worked
move_uploaded_file(...);
The error codes are detailed here, and will tell you if the file was too large to handle due to PHP settings.
As for the PHP settings themselves. Your post_max_size should be the largest allowable file size, plus whatever other data you're submitting alongside the file upload. memory_limit must be larger than the max file size, plus extra to allow for script overhead. PHP's execution time limits do not come into play for uploads, as the script timers are not started until AFTER the upload is completed.
Bottom-line:
Do I need to be concerned about setting post_max_filesize >> memory_limit?
Details:
This answer suggests that uploaded files do not need to fit within php’s memory_limit. The php docs suggest that the entire post should fit within php’s memory limit.
I find the docs surprising and I’m hoping someone can elaborate. For example take the following php configs:
; config A
memory_limit = 50M
upload_max_filesize = 100M
post_max_filesize = 1000M
max_file_uploads = 10
and
; config B
memory_limit = 50M
upload_max_filesize = 10M
post_max_filesize = 1000M
max_file_uploads = 100
With these configurations I’d expect to be able to:
upload 10x100mb files to server A, and 100x10mb files to server B. I would also expect that: Working with any one of the 10 files uploaded to server A is a problem (100Ms of file in a 50M bag…).Working with any 1 of the 100 files uploaded to server B is okay (10 < 50). While experimenting with less round but equivalently related numbers, I’ve found these expectations hold true.
This experience would lead me to say that "generally the memory_limit should be larger than the upload_max_filesize"; instead, the php docs say:
generally speaking, memory_limit should be larger than post_max_size.
Why and what happens if it isn't?
When my php code is executed I see no evidence that all of the posted files are in memory. It seems to me that all I’ve got is a $_FILES array of paths to files found exclusively on disk. Is php holding the whole post in memory at some point prior to my ability to introspect the environment? Do I need to be concerned about setting post_max_filesize >> memory_limit?
Aside:
Violating the manual's rule does not result in a grossly broken server (w/ php5.3 apache2.2 debian 6).
PHP will accept uploaded files that are individually smaller than upload_max_filesize and together take less than post_max_size bytes. The PHP documentation is wrong with regard to memory_limit which does not need to hold the file contents posted.
The following configuration works with both Apache2 module and CGI, and accepts files smaller than 1G.
upload_max_filesize = 1G
post_max_size = 1G
memory_limit = 32M
Do I need to be concerned about
setting post_max_filesize >>
memory_limit?
Only if you plan on reading an entire file into memory and the file that you read in is larger than the space you have allocated to PHP (i.e. memory_limit), in which case you'll run out of memory.
My own personal experience is that you HAVE to have a memory_limit higher than post_max_size and upload_max_size.
The post_max_size refers to the entirety of the POSTed data. This includes any form fields that may have been included with the file itself. The upload_max_size is the largest allowable size a file can be within that upload.
For instance. with a post_max_size of 10mb and a upload_max_size of 1mb, you could upload 9 files, each 1mb in size, Why 9 files? because part of the POST data is the file metadata - filename, mimetype, file size, etc... This all takes up some space, so your 9 files will actually take up 9.01megabytes or so. The 0.99 leftover is too small for another file, so you can't upload that 10th, even though it fits within the upload_max_size limit.
As for memory_limit, not only do you have to have enough "room" for the files that were uploaded, you have to remember that this limit applies to the script as a whole. A memory_limit of 10mb would allow for only a 9megabyte file to be uploaded, because PHP itself and all the associated code and libraries will suck up (say) 1 megabyte already.
Even though the files aren't held in memory - they get dumped out to temp files as soon as possible, they are passed in to PHP from Apache via STDIN. PHP has to read the files from that stream and copy them out to the temporary files you use in the ['tmp_name'] section of the $_FILES array.
For whatever reason, PHP seems to be basically doing "file_get_contents()" and slurping the files up in bulk, rather than doing a streaming-type copy. Hence requiring a memory_limit that exceeds the largest allowed file size.
Hi im quite new to PHP, i have created a form for very large csv files to be uploaded to my server. Some one mentioned to me that the browser can time out due to the uploading file being to big, is this true? and if so, can it be prevented?
Thanks for your help!
You need a proper value for the following php.ini settings:
max_input_time (not max_execution_time!)
upload_max_filesize
post_max_size
and maybe
memory_limit
There are some configuration directives that can cause large uploads to fail if their values are too small:
PHP
max_input_time Maximum time in seconds a script is allowed to parse input data, like POST, GET and file uploads
upload_max_filesize Maximum size of an uploaded file.
post_max_size Maximum size of post data allowed.
Apache
TimeOut Amount of time the server will wait for certain events before failing a request
LimitRequestBody Restricts the total size of the HTTP request body sent from the client
There are probably some more than this.
A good way to work around the poor handling of large file uploads in php, is to use an uploader like JUpload which will split the file into chunks before sending them. This also has the benefit for your users that they get a proper progress feedback while uploading, and they can upload multiple files in one go.
I was able solve this problem using the following settings, you could use different values but you get the idea:
For my server, I put these lines in a ".user.ini" file inside the script directory, your server may look for a different file, if you do a phpinfo('user_ini.filename') on the server it will spit out the file you need to put your values in
max_execution_time = 1800
max_input_time = -1
post_max_size = 100M
upload_max_filesize = 100M
memory_limit = 256M
When uploading very large files, you have to change 4 configuration variables:
upload_max_filesize
post_max_size
memory_limit
time_limit
Time limit may be increased at runtime with set_time_limit().
A script is allowed to run, by default, for something like 30 seconds. You can use the set_time_limit() function to alter this. Also, if your user will need to upload large files, you'll need to change the post_max_size and/or the upload_max_filesize values in your php.ini file.
Also, if you want to just extend your timeout limit globally, you can change max-execution-time in php.ini.
Yes it is true. File upload is done through a POST request and requests in general are subject to timeout. You should be able to reconfigure your environment for a longer request timeout.
It's not just timeouts that can cause problems. There are some limits on the maximum size of file that can be uploaded. These limits can be changed in the php.ini file:
post_max_size
upload_max_filesize
memory_limit
Check out http://uk.php.net/ini.core for details.
My answer is not directly related to your original question, but if you have a reverse proxy load balancer in front of your PHP script, the load balancer can timeout or block large uploads. Always check your load balancer's configuration if you support file uploads. Just like PHP, most load balancers default settings for uploads are pretty small.
If changing any of the above parameters doesn't seem to make any difference, it can be that a html form somewhere contains the name MAX_FILE_SIZE as a hidden field.
<input type="hidden" name="MAX_FILE_SIZE" value="10000000">
In the example above, any file over 10MB will not be uploaded.