I have image hosting. All image requests redirect to a PHP script with mod_rewrite. PHP script uses function fread() and displays picture from another file. I want to know, does this use a lot of processor time or not?
It depends on how much you think "a lot of processor time" is, but from what you're describing, the processing time required by mod_rewrite and PHP is trivial compared to the I/O time to read the image from disk and send it over the network.
If you're concerned about speed, caching the images in memory will probably have the most benefit.
Yes, this is putting considerable strain on the web server, because the PHP interpreter has to be initialized for every small resource request, and passes through the data. The consensus is that this is not a good thing to do on a high-traffic website.
Why are you doing this, are you resizing images?
You will run out of memory before hitting CPU limit :-)
Reading/writing file is not CPU-intensive task, but each apache process created for that can eat up to 50 MB of RAM.
If you want to send images fast and secure, you should look into X-SendFile - this allows your php scripts to tell your Webserver to send files not directly accessible by url using something like header('X-SendFile: /path/to/the/file');
For apache there is mod_xsendfile (http://tn123.ath.cx/mod_xsendfile/) though labeled beta, it has proven to be very stable in production, and the its sourcecode is rather small and can be audited easily.
Related
I have a PHP script that serves alot of smaller files (>100,000) with sizes up to 10mb. It basically loads the requested file into memory and serves it to the client. Because of access control I cannot serve these files by apache directly and need a script wrapped around it.
If there is high traffic (>150mbit) my hdd is heavily used and represents a limit for scaling. I had the idea that I could use memcached to reduce the hdd load since I have 10gig of ram available but memcached has a max item size of 1MB. Then I thought I could use PHP-APC but its behaviour if the cache runs out of memory (complete reset) isn't acceptable.
What would you do to reduce the IO load?
Thanks
What would you do to reduce the IO load?
I have never worked with it myself, but the X-Sendfile method may be helpful for taking away a bit of the load. It passes the task of actually serving the file back to Apache.
I think you can't do this unless you have 2 HDD which would split these files.
I would use PHP-APC to load these files into the cache.
apc_add(), apc_fetch() and apc_delete() are what you want. You can ensure you don't overflow by using apc_cache_info() to determine free memory levels. You can also set apc.user_ttl INI setting to prevent total cache clearing on fill.
Set things up on a test server, subject it to high load (with ab or the like) and check your stats using apc.php. Tweak, tweak and tweak some more!
You could use a CDN that supports access control.
If you want to continue serving it yourself, though, there are various approaches you could take. You'll always want to avoid serving the file through PHP though, because that is your bottleneck. None of these are very elegant though.
Store the files outside of the HTTP root and generate a new symlink every X minutes. The symlinks are deleted after Y time. Your PHP authentication script would then simply redirect the user to the URL in which the (temporarily valid) symlink exists. Very short PHP execution time, files served by Apache.
Keep the files inside the HTTP root, but change the rewrite rules in a .htacess file instead to achieve the same thing.
Reduce IO load by storing the most frequently accessed files in a ramdisk, integrate them with the regular file system by some mounting magic or symlinks and then let Apache handle the rest.
You need either mod_xsendfile for Apache2 or nginx with X-Accel-Redirect. There is also similar a solution for lighttpd. Nginx can also serve from memcached.
If you're thinking about storing frequently used files in tmpfs, don't. That's not a real solution because even if you serve files right from the disk subsequent requests will hit the system cache and you'll get similar to tmpfs speed.
I have a problem here, maybe someone has already gone through this before.
A system controller is serving php downloads, it reads information from files and sends the client as a download. The system works perfectly. The problem is that speed is always low, always less than the 300kb/se times less than 100kb / s for the user.
The server has a 100mbps link 6mbps free and the customer has, then it should be downloaded at 600kb / s. Something is holding the output of php. I've tried searching on the buffers of apache but found nothing on this issue.
Does anyone have any idea what might be happening?
PHP really isn't built for processing large files. It has to read that entire file into memory and then output it. It sounds like you're sending a reasonable amount of traffic through PHP, if 100kb/s - 300kb/s per user is too slow, via something like readfile() which is a bad idea. Instead, I suggest taking a look at mod_xsendfile (if you're using Apache) or it's equivalent for your web server of choice (e.g. I prefer nginx, and would use XSendFile for this).
In PHP then, you can just do this: header('X-Sendfile: ' . $file);. The server intercepts the header, and sends that file. It allows you the benefits of what you're doing with PHP, and the speed of the web server directly reading the file.
I created an simple web interface to allow various users to upload files. I set the upload limit to 100mb but now it turns out that the client occasionally wants to upload files 500mb+.
I know what to alter the php configuration to change the upload limit but I was wondering if there are any serious disadvantages to uploading files of this size via php?
Obviously ftp would be preferable but if possible i'd rather not have two different methods of uploading files.
Thanks
Firstly FTP is never preferable. To anything.
I assume you mean that you transferring the files via HTTP. While not quite as bad as FTP, its not a good idea if you can find another of solving the problem. HTTP (and hence the component programs) are optimized around transferring relatively small files around the internet.
While the protocol supports server to client range requests, it does not allow for the reverse operation. Even if the software at either end were unaffected by the volume, the more data you are pushing across the greater the interval during which you could lose the connection. But the biggest problem is that caveat in the last sentence.
Regardless of the server technology you use (PHP or something else) it's never a good idea to push that big file in one sweep in synchronous mode.
There are lots of plugins for any technology/framework that will do asynchronous upload for you.
Besides the connection timing out, there is one more disadvantage in that file uploading consumes the web server memory. You don't normally want that.
PHP will handle as many and as large a file as you'll allow it. But consider that it's basically impossible to resume an aborted upload in PHP, as scripts are not fired up until AFTER the upload is completed. The larger the file gets, the larger the chance of a network glitch killing the upload and wasting a good chunk of time and bandwidth. As well, without extra work with APC, or using something like uploadify, there's no progress report and users are left staring at a browser showing no visible signs of actual work except the throbber chugging away.
You all know about restrictions that exist in shared environment, so with that in mind, please suggest me a php function or something with the help of which I could stream my videos and other files. I have a lot of videos on the server, unlimited bandwidth and disk space, but I am limited in ram and cpu.
Don't use php to stream the data. Use a header redirect to point to the URL of the actual file. This will offload the work onto the webserver which might run under a different user id and is better optimized for this task.
Hmm, there is XMoov that acts as a "streaming server" but does not much more than serve a file byte by byte, with a few additional options and settings. It promises random access (i.e. arbitrary skipping within a video) but I haven't used it myself yet.
As a server administrator, though, I would frown on anybody using PHP to serve huge files like that because of the strain it puts on the server. I would generally not regard this to be a good idea, and rent a streaming server instead if at all possible. Use at your own risk.
You can use a while loop to load bits of the file, and then sleep for some time, and then output more, and sleep... (that would be the only way to limit the CPU usage).
RAM shouldn't be a problem, as you will just dump parts of the file, so you don't need to load it into RAM.
I have a PHP page that gets its content by making an HTTP request to another site on the same server, using file_get_contents. Both sites run in Apache 2 which calls PHP using suPHP (which is FastCGI, right?)
How significant is the overhead of this call? Does Apache do a lot of processing before sending a request to PHP?
An alternative way to make the call would be for the first site to exec('php /the/other/script.php some parameters'). Would this be faster, or is the overhead of spawning a process bigger than that of going through Apache?
Apache's over head is going to depend on whats configured for that site host, for example https, htaccess checks, rewriting, etc.. Those things can stack up. Now i dont think it would be much strain overhead wise comparatively but you are going to have the time it taks to generate the response which depending on the nature of the external pages oyure calling could be signifigant in some situations.
With that said, i dont nessecarily see a problem with making the calls through apache. But i do think that as you suggest exposing the php directly would be better. I think maybe reading up on SOA in general might help you gain some insight on how best to implement.
Unfotunatly installing PHP as cgi, you will loose alot of performace, because eachtime you have to create a new process for it.
So best method is to install PHP as apache modul