Each page on my website is rendered using PHP.
Each PHP file uses around 10 includes. So for every page that is displayed, the server needs to fetch 10 files, in addition to the rest of its functions (MySQL, etc).
Should I combine them into a single include file? Will that make ANY difference to the real-world speed? It's not a trivial task as there would be a spaghetti of variable scope to sort out.
Include files are processed on the server, so they're not "fetched" by the browser. The performance difference of using includes vs. copy and pasting the code or consolidating files is so negligible (and I'm guessing we're talking about in the 10 ms to 100 ms range, at the absolute most), that it isn't at all worth it.
Feel free to include and require to your heart's content. Clean code is substantially more important than shaving less than 100 ms off a page load. If you're building something where timing is that critical, you shouldn't be using PHP anyway.
What takes time is figuring out where the files are actually located in the include path. If you got multiple locations in your include path, PHP will search each location until it either finds the file or fails (in which case it throws an error). That's why you should put the include path where most of the included files are to be found on top of the include path.
If you use absolute paths in your include path, PHP will cache the path in the realpath cache, but note that this gets stale very quickly. So yes, including ten files is potentially slower than including one large file, simply because PHP has to check the include path more often. However, unless your webserver is a really weak machine, ten files is not enough to make an impact. This gets only interesting when including hundreds of files or have many locations to search, in which case you should use an OpCode cache anyway.
Also note that when including files, it is not good practice to include each and every file right at the beginning, because you might be including files that are never called by your application for a specific request.
Reference
http://de2.php.net/manual/en/ini.core.php#ini.include-path
http://de2.php.net/manual/en/ini.core.php#ini.sect.performance
http://en.wikipedia.org/wiki/List_of_PHP_accelerators
Although disk I/O operations among the biggest performance-eaters, a regular site won't notice any sensible number of includes.
Before you hit any problems with includes, you probably already would have some opcode cache that eliminates this problem too.
include\ andrequires` only open file on the server side, but that might be time consumming depending on the hardware/filesystem, etc.
Anyway, if you can, use autoloader. Only needed files will be loaded that way.
Then if you think included files are a source of slowdown (and I think there is a lot of other points to look for improvement before), you can try to automatically merge the files. You still have one file per class when developping, but you can build a file that contains each class' definition to have only one include (something like cat <all your included file>.php > to_include.php).
Related
Even if there seem to exist a few duplicate questions, I think this one is unique. I'm not asking if there are any limits, it's only about performance drawbacks in context of Apache. Or unix file system in general.
Lets say if I request a file from an Apache server
http://example.com/media/example.jpg
does it matter how many files there are in the same directory "media"?
The reason I'm asking is that my PHP application generates images on the fly.
Once created, it places it at the same location the PHP script would trigger due to ModRewrite. If the file exists, Apache will skip the whole PHP execution and directly serve the static image instead. Some kind of gateway cache if you want to call it that way.
Apache has basically two things to do:
Check if the file exists
Serve the file or forward the request to PHP
Till now, I have about 25.000 files with about 8 GB in this single directory. I expect it to grow at least 10 times in the next years.
While I don't face any issues managing these files, I have the slight feeling that it keeps getting slower when requesting them via HTTP. So I wondered if this is really what happens or if it's just my subjective impression.
Most file systems based on the Berkeley FFS will degrade in performance with large numbers of files in one directory due to multiple levels of indirection.
I don't know about other file systems like HFS or NTFS, but my suspicion is that they may well suffer from the same issue.
I once had to deal with a similar issue and ended up using a map for the files.
I think it was something like md5 myfilename-00001 yielding (for example): e5948ba174d28e80886a48336dcdf4a4 which I then put into a file named e5/94/8ba174d28e80886a48336dcdf4a4. Then a map file mapped 'myfilename-00001' to 'e5/94/8ba174d28e80886a48336dcdf4a4'. This not-quite-elegant solution worked for my purposes and it only took a little bit of code.
When is a PHP include file parsed? At startup, or during execution?
My web forms call a single php script. Depending on the arguements passed in the URL, a switch/case condition determines what the script will do. Each "case" within the switch has its own include files.
If include files are parsed during initial load, then my php script will take up more memory/time to process which leads me to believe having individual php files called from my web form is better, than having one which includes what it needs.
If include files are parsed when needed (thus, when a branch of the code reaches a specific case statement, that it then performs the include) it tells me my code will be reasonably conservative on memory.
So.... my question... When is a PHP include file parsed? At initial load, or during execution?
(note... I failed to find the answer here, and I have read http://php.net/manual/en/function.include.php)
Files are included if and when the include statement is reached at runtime. To very succinctly summarise what that means, the following file is never going to be included:
if (false) {
include 'foo.php';
}
Since you're concerned about memory usage from too many includes, I feel that a bit more detail will be useful over and above a direct answer to your question.
Firstly, to directly answer you, PHP files are parsed as soon as they are loaded -- if a file contains a syntax error, you will be told of that immediately; it won't wait till it gets to that line of code. However subsequent files are only included if the specific line of code containing the include statement is executed.
You're concerned about memory usage, but having a lot of included files is generally not a major memory issue, nor a major performance issue. Indeed, most modern PHP applications of any size will use a framework library that load hundreds of PHP files for every page load. Memory and performance issues are far more likely to be caused by bugs within your code rather than simply loading too much code.
If you are concerned about memory and performance from this, you should consider using PHP's OpCache feature. With this feature enabled, PHP stores a cache in memory of the compiled state of all the files it has included within a system. When it runs the page again, therefore, it does not need to actually load or parse anything when it encounters an include statement; it simply fetches it from the cache.
Using OpCache you can write your code with a very large number of files to include, and without any performance penalty at all.
The good news is that OpCache is enabled by default in recent PHP versions, and is completely transparent to the developer -- you don't even need to know that it's there; the only difference you'll see between it being turned on and off is your site running faster.
So, firstly, make sure your PHP version is up-to-date (v5.5 or higher). Then make sure OpCache is enabled in your PHP.ini file. Then just sit back and stop worrying about these kinds of things.
File included with include statement are parsed during exection. When your php code hits a include statement it will start parsing the file to see what is in there.
From w3schools
The include (or require) statement takes all the text/code/markup that
exists in the specified file and copies it into the file that uses the
include statement.
There is other questions with a similar topic:
In PHP, how does include() exactly work?
I've got nearly a million images for my site and they are stored in one folder on my windows server.
Since opening this folder directly on desktop drive me and my CPU crazy I am wondering that whether fetching one of them using my PHP script for a HTTP request is also laborious. So, will separating them into different folders improve the performance?
No, the performance does not depend on the number of files that are in a directory. The reason why opening the folder in Windows explorer is slow is because it has to render icons and various other GUI related things for each file.
When the web server fetches a file, it doesn't need to do that. It just (more or less) directly goes to the location of the file on the disk.
EDIT: Millions is kind of pushing the limits of your file system (I assume NTFS in your case). It appears that anything over 10,000 files in a directory starts to degrade your performance. So not only from a performance standpoint, but from an organizational standpoint as well, you may want to consider separating them into subdirectories.
Often the best answer in a case like this is to benchmark it. It shouldn't be too hard to create a program that opens 1000 hard-coded file names and closes them. Run the test on your million-plus directory and another directory containing only those 1000 files being tested and see if there's a difference.
Not only does the underlying file system make a difference, but accessing over a network can affect the results too.
Separating your files into seperate directories will most likely help performance. But as mark suggests it's probably worth benchmarking
Just something I wonder about when including files:
Say I want to include a file, or link to it. Should I just for example:
include("../localfile.php");
or should I instead use
include("http://sameserver.com/but/adirect/linkto/localfile.php");
Is one better than the other? Or more secure? Or is it just personal preference?
Clearly it would be a necessity if you had a file that you would include into files in multiple directories, and THAT file includes a different file, or is there some other way of doing that?
Reading a file is much faster than making an HTTP request and getting the response. Never include(a_uri) if you can help it.
Use $_SERVER['DOCUMENT_ROOT'] if you want to calculate a complete file path for your include.
As said before, definitely include a local file and not do an HTTP request (which takes more time, is not cached and the contents are technically viewable to all the world, if he knows where to look for it).
One more small detail, if you use full paths to your included files, it will even be faster then relative paths, especially if you use some kind Byte Code Cache.
Definitely include the local file, because the php script doesn't really know or care that you're including a script on your local server, so the url path causes an http request, and network latency from http requests is pretty much the bottleneck for rendering any html page in general, the fewer of them you have, the better off you're going to be.
Personally, I try to avoid using include and require in general, in favor of require_once, because using require_once means that you are writing your code reusably instead of writing code that executes immediately when you include it. Pull in class definitions, pull in function libraries, but try to avoid code that executes immediately when you include it, because that will make it harder to reuse.
If your question is about keeping it so you don't have to change a billion paths when you move from staging to production, go with this little tidbit I learned:
define('BASE_DIR', '/path/to/root/');
Then use BASE_DIR in all of your path references. When it's time to move your site, just change that definition to the new path (which should just be / at that point).
In addition to what other people say, these invocations will have different results, since the remove invocation will execute php output, not the file contents. Unless you stop php from processing the file, in which case you're exposing your code to the world which is also not necessarily what you actually want to.
Always include locally, cause if you include remote someone can create a different file and do nasty things. And the other problem you can't really test this with remote includes. As far as i know you should use require_once instead...
The question might prompt some people to say a definitive YES or NO almost immediately, but please read on...
I have a simple website where there are 30 php pages (each has some php server side code + HTML/CSS etc...). No complicated hierarchy, nothing. Just 30 pages.
I also have a set of purely back-end php files - the ones that have code for saving stuff to database, doing authentication, sending emails, processing orders and the like. These will be reused by those 30 content-pages.
I have a master php file to which I send a parameter. This specifies which one of those 30 files is needed and it includes the appropriate content-page. But each one of those may require a variable number of back-end files to be included. For example one content page may require nothing from back-end, while another might need the database code, while something else might need the emailer, database and the authentication code etc...
I guess whatever back-end page is required, can be included in the appropriate content page, but one small change in the path and I have to edit tens of files. It will be too cumbersome to check which content page is requested (switch-case type of thing) and include the appropriate back-end files, in the master php file. Again, I have to make many changes if a single path changes.
Being lazy, I included ALL back-end files inthe master file so that no content page can request something that is not included.
First question - is this a good practice? if it is done by anyone at all.
Second, will there be a performance problem or any kind of problem due to me including all the back-end files regardless of whether they are needed?
EDIT
The website gets anywhere between 3000 - 4000 visits a day.
You should benchmark. Time the execution of the same page with different includes. But I guess it won't make much difference with 30 files.
But you can save yourself the time and just enable APC in the php.ini (it is a PECL extension, so you need to install it). It will cache the parsed content of your files, which will speed things up significantly.
BTW: There is nothing wrong with laziness, it's even a virtue ;)
If your site is object-oriented I'd recommend using auto-loading (http://php.net/manual/en/language.oop5.autoload.php).
This uses a magic method (__autoload) to look for a class when needed (it's lazy, just like you!), so if a particular page doesn't need all the classes, it doesn't have to get them!
Again, though, this depends on if it is object-oriented or not...
It will slow down your site, though probably not by a noticable amount. It doesn't seem like a healthy way to organize your application, though; I'd rethink it. Try to separate the application logic (eg. most of the server-side code) from the presentation layer (eg. the HTML/CSS).
it's not a bad practice if the files are small and contains just definition and settings.
if they actually run code, or extremely large, it will cause a performance issue.
now - if your site has 3 visitors an hour - who cares, if you have 30000... that's another issue, and you need to work harder to minimize that.
You can migitate some of the disadvantages of PHP code-compiling by using XCache. This PHP module will cache the PHP-opcode which reduces compile time and performance.
Considering the size of your website; if you haven't noticed a slowdown, why try to fix it?
When it comes to larger sites, the first thing you should do is install APC. Even though your current method of including files might not benefit as much from APC as it could, APC will still do an amazing job speeding stuff up.
If response-speed is still problematic, you should consider including all your files. APC will keep a cached version of your sourcefiles in memory, but can only do this well if there are no conditional includes.
Only when your PHP application is at a size where memory exhaustion is a big risk (note that for most large-scale websites Memory is not the bottleneck) you might want to conditionally include parts of your application.
Rasmus Lerdorf (the man behind PHP) agrees: http://pooteeweet.org/blog/538
As others have said, it shouldn't slow things down much, but it's not 'ideal'.
If the main issue is that you're too lazy to go changing the paths for all the included files (if the path ever needs to be updated in the future). Then you can use a constant to define the path in your main file, and use the constant any time you need to include/require a file.
define('PATH_TO_FILES', '/var/www/html/mysite/includes/go/in/here/');
require_once PATH_TO_FILES.'database.php';
require_once PATH_TO_FILES.'sessions.php';
require_once PATH_TO_FILES.'otherstuff.php';
That way if the path changes, you only need to modify one line of code.
It will indeed slow down your website. Most because of the relative slow loading and processing of PHP. The more code you'd like to include, the slower the application will get.
I live by "include as little as possible, as much as necessary" so i usually just include my config and session handling for everything and then each page includes just what they need using an include path defined in the config include, so for path changes you still just need to change one file.
If you include everything the slowdown won't be noticeable until you get a lot of page hits (several hits per second) so in your case just including everything might be ok.