I'm coding a site in PHP and getting "pretty urls" (also hiding my directories) by directing all requests to one index.php file (using .htaccess). The index file then parses the uri and includes the requested files. These files also have more than a couple of includes in them, and each may open up a MySQL connection. And then those files have includes too, which open sql connections. It goes down to about 3-4 levels.
Is this process CPU and memory intensive, both from the PHP includes and opening (and closing) MySQL connections in each included file?
Also, would pretty urls using purely htaccess use less resources?
PHP Overheads
The answer re the logical decomposition of your app into a source hierarchy depends on how your solution is being hosted.
If you use a dedicated host/VM then you will probably have mod_php+Xcache or equiv and the answer will be: no, it doesn't really hit the runtime since everything gets cached in-memory at the PHP Opcode level.
If you use a shared hosting service then it will impact performance since any PHP scripts will be loaded through PHP-cgi probably via suPHP and the entire source hierarchy that is included will need to be read in and compiled per request. Worse, on a shared solution, if this request is the first in say 1 min, then the servers file cache will have been flushed and marshalling this source will involve a lot of physical I/O = seconds time delay.
I administer a few phpBB forums and have found that by aggregating common include hierarchies for shared hosting implementations, I can half the user response time. Here are some are articles which describe this in more detail (Terry Ellison [phpBB]). And to quote one article:
Let me quantify my views with some ballpark figures. I need to emphasise that the figures below are indicative. I have included the benchmarks as attachments to this article, just in case you want to validate them on your own service.
20–40. The number of files that you can open and read per second, if the file system cache is not primed.
1,500–2,500. The number of files that you can open and read per second, if the file system cache is primed with their contents.
300,000–400,000. The number of lines per second that the PHP interpreter can compile.
20,000,000. The number of PHP instructions per second that the PHP interpreter can interpret.
500-1,000. The number of MySQL statements per second that the PHP interpreter can call, if the database cache is primed with your table contents.
For more see More on optimising PHP applications in a Webfusion shared service where you can copy the benchmarks to run yourself.
MySQL connection
The easiest thing to do here is to pool the connection. I use my own mysqli class extension which uses a standard single-object-per-class template. In my case any module can issue a:
$db = AppDB::get();
to return this object. This is cheap as it is an internal call involve half a dozen PHP opcodes.
An alternative but traditional method is to use a global to hold the object and just do a
global $db;
in any function that need to use it.
Footnote for Small Applications
You suggested combining all includes into a single include file. This is OK for stable production, but a pain during testing. Can I suggest a simple compromise? Keeps them separate for testing but allow loading of a single composite. You do this in two parts (i) I assume each include defines a function or class, so use a standard template for each include, e.g.
if( !function_exists( 'fred' ) ) {
require "include/module1.php";
}
Before any loads in the master script simple do:
#include "include/_all_modules.php";
This way, when you are test you delete _all_modules.php and the script falls back to loading individual modules. When you're happy you can recreate the _all_modules.php. You can event do this server side by a simple "release" script which does a
system( 'cp include/[a-z]*.php include/_all_modules.php' );
This way, you get the best of both worlds
It depends on the MySQL client code, I know for one that connections often get reused when opening a MySQL connection with the same parameters.
Personally I wouldd only initialize the database connection in the front controller (your index.php file), because everything should come through there anyway.
You could use the include_once() or require_once() methods to ensure that PHP only parses them once, thus saving processing time. This would be particularly valuable if you suspect that your code might attempt to include files more than once per script execute.
http://php.net/manual/en/function.include-once.php
I would imagine that using .htaccess to parse URLs would always use more resources than any other method, purely because those rules would be activated upon every single .php file request your server encountered.
Related
I am trying to reduce the number of requests on my site (to improve page speed). In one file, I have 10 separate php require statements calling 10 different php files.
My question is, are these 10 require statements considered as 10 separate requests? By replacing the require statements with actual contents from the called php file can I reduce the number of requests?
I would greatly appreciate if someone could please clarify this form. Please note that I am not an experience programmer or web designer. Thanks!
As the first comment mentions; requests are only served in response to clients; none of the language constructs of PHP should be described as requests.
As simple as I can make it ...
Execution is a three stage process:
Load file from disk
Compile into Zend's intermediate form
Execute intermediate form
A cache improves performance by usurping those parts of Zend normally responsible for loading files from disk and compiling it to Zend's intermediate representation; such that, if possible, the loading and compilation stage of execution can be bypassed, improving performance.
Without an Opcode Cache
When a request is made such as
GET /index.php HTTP/1.1
The web server servicing the request is invoked in the normal way, which in turn invokes the PHP interpreter. The interpreter then loads index.php from disk, compiles it into Zend's intermediate representation, and begins executing the code.
When a script contains a statement such as
include "my_awesome_code.php";
The (same) interpreter loads my_awesome_code.php from disk, compiles it into Zend's intermediate representation and executes it.
With an Opcode Cache
When a request is made such as
GET /index.php HTTP/1.1
The web server servicing the request is invoked in the normal way, which in turn invokes the PHP interpreter. Rather than loading index.php from the disk, the cache retrieves the code from memory where it is stored in something very close to the form of Zend's intermediate representation. Preparing the cached code (which means copying from shared memory and finalizing it's form) for execution is much faster than loading from disk and compiling to that representation first.
When a script contains a statement such as
include "my_awesome_code.php";
The codes are again retrieved from the cache, bypassing compilation once again.
Should the cache not be able to find index.php or my_awesome_code.php in shared memory, it invokes the parts of Zend it usurped; resulting in the normal loading from disk and compilation of those codes into Zend's intermediate representation. The cache intercepts the resulting intermediate code and stores it in shared memory before allowing it to be executed for the first time.
Like my comment, it's depending how PHP is installed.
If you using PHP as Mod (mod_php), your Webserver create only one PHP process. All requests will be handled here.
But if you have implement PHP as an CGI-Wrapper, each request starts an PHP instance.
To perform your loading time, use Opcode-Caches like eAccelerator, APC or other for PHP Scripts. Other solution is to handle requests with the right way: Cache statical files like Stylesheets, Javascripts, Images,..
Edit
An other solution is: optimize your PHP scripts. A good inspector is New Relic, here you can see, what scripts have very large execution time. I had create an Blog post of New Relic with their features, but it's only in german: http://hovida-design.de/php-die-macht-der-performance/
I am in the process of writing an API for my website, along with a pretty large class to process all of the API requests.
Most pages, if not all all the pages on the website will send at least one request to the api on load. The most important priority for this website is efficiency and resultantly, very quick server processing.
I am therefore seeking a little bit of advice when it comes to classes and certain PHP functions.
Firstly, it looks like the class I am writing will probably end up being around 3000 lines of code. If this is initialised on each page, ignoring the fact that only one or two of the functions within the class will be used per page, will this make the API much slower? Should I be looking at separate files with extensions to the class for each class method?
Secondly, I currently have all of my connections to the various databases in separate files within a directory. Within each connection is the function mysql_pconnect(). At the moment, I only require these files when needed. So if the method needs a connection to the x database, then I simply place require(connection...) into the method. Is it bad to include files within a class?
I am asking, because the only other solution is to require all of the connections at the top of the page so that the class can access them without requiring them for each method. This is why I would like to know whether having several connections at once, even if they are not used, is taxing on the server?
So three questions really:
Does initiating a large class at the beginning of each page slow down the script runtime, even if only one class method is being used? Is this why people use 'class extends class'?
Is it bad to 'require()' files within a class?
Do unused connections to a mysql database slow down the runtime of a script?
No, an unused MySQL connection won't consume much (if any) cpu time, though it will occupy a bit of memory to handle the various bits of 'state' which have to be maintained on a per-connection basis.
However, note that MySQL's connection protocol is actually fairly "light-weight". Maintaining a pool of persistent connections sounds attractive, but the cost of establishing a new connection is already very low anyways.
Persistent connections are a quick fix to solving connection overhead, but they do bring in issues. The worst one being abandoned connections can leave the connections in an indeterminate state (in-progress transactions, changed server variables/configurations, etc...) and you can quite easily create inadvertent deadlocks unless you're very careful.
If you are writing an api to serve data from your db to multiple 3rd parties you are better off not letting them query the db, not even through a webservice (unless you have second-to-second db mutations that these 3rd parties need immediately)
Better way would be to write xml file(s) to a protected location and use a webservice to auth a 3rd party and serve the xml file(s).
Then update the xml periodically.
I have a file which contains application specific properties (e.g. timeouts etc.) which I have placed in a my_app.ini properties file and which I read using parse_ini_file().
How/where should I make this code run exactly once for all my user requests (I do not want my application to read the .ini file on every request) ?
I though of putting the config file reading code in a separate PHP and including this with include_once/require_once in all pages that require access to the config variables. But:
Will this mean that the file will be
read once for every different page
(so different times since I have
multiple pages with
require_once('config_setup.php'))
?
Is require_once/include_once
executed only once across all user
requests ?
I am interested in possibly hearing other (better) ways how to do this.
PHP doesn't keep state between multiple requests. This means that standalone PHP cannot store variables in memory between requests. Sessions are saved to disk (by default), config files needs to be reparsed, etc.
You can however use extensions such as APC or Memcache/Memcached to store variables in memory. These extensions needs to be compiled/installed on the server so that their functions are available.
Parsing your INI will not be a bottleneck for your application. For simplicity's sake, it's easier just to parse it everytime. That 0.00001 ms you'll be saving otherwise isn't worth it.
include_once() and require_once() has nothing to do with persistance. It just tells PHP to only include the script if it wasn't already included during this request.
include/require_once mean that the file will only be loaded once during the processing of a single request; however, whenever a new request is received, it will be read again as the script is parsed.
In addition, for what you want to do here, you would have to read the configuration and place its value in the session; then, if the specific key you use in the session array is defined, you do not need to reparse the file:
if ( !array_key_exists( 'config' , $_SESSION ) ) {
$_SESSION['config'] = parse_ini_file ( 'whatever.ini' );
}
However, this approach has a major disadvantage - if the configuration file changes, users that are already on the site will keep on using the previous version of the file.
First of all, reading the *.ini is hardly going to be the bottleneck of any service in normal circumstances, so really think whether the advantages you seek (which are...?) outweigh the disadvantages. I have no problem scanning INI files, although I usually use native PHP to do some settings as long as no non-PHP programmer needs to reach them. Of course, people are now going to advocate key/value stores like APC & Memcached because of the persistence issue, but that will only be an advantage if your file is (relatively) huge for an ini file, otherwise it's just overhead if you don't use it already.
The only way I know how to get a (scalar) setting across requests without extensions in Apache easily is to do some SetEnv business in the host configuration (so scalar values are just read from the environment), but those are kludgy at best, and require a server reload or restart to pick up alterations.
Stay with the ini file, or possibly with a straight PHP file with settings, in which case APC could grant you a small advantage by caching the file until it changes.
I've heard of two caching techniques for the PHP code:
When a PHP script generates output it stores it into local files. When the script is called again it check whether the file with previous output exists and if true returns the content of this file. It's mostly done with playing around the "output buffer". Somthing like this is described in this article.
Using a kind of opcode caching plugin, where the compiled PHP code is stored in memory. The most popular of this one is APC, also eAccelerator.
Now the question is whether it make any sense to use both of the techniques or just use one of them. I think that the first method is a bit complicated and time consuming in the implementation, when the second one seem to be a simple one where you just need to install the module.
I use PHP 5.3 (PHP-FPM) on Ubuntu/Debian.
BTW, are there any other methods to cache PHP code or output, which I didn't mention here? Are they worth considering?
You should always have an opcode cache like APC. Its purpose is to speed up the parsing of your code, and will be bundled into PHP in a future version. For now, it's a simple install on any server and doesn't require you write or change any code.
However, caching opcodes doesn't do anything to speed up the actual execution of your code. Your bottlenecks are usually time spent talking to databases or reading to/from disk. Caching the output of your program avoids unnecessary resource usage and can speed up responses by orders of magnitude.
You can do output caching many different ways at many different places along your stack. The first place you can do it is in your own code, as you suggested, by buffering output, writing it to a file, and reading from that file on subsequent requests.
That still requires executing your PHP code on each request, though. You can cache output at the web server level to skip that as well. Crafting a set of mod_rewrite rules will allow Apache to serve the static files instead of the PHP code when they exist, but you'll have to regenerate the cached versions manually or with a scheduled task, since your PHP code won't be running on each request to do so.
You can also stick a proxy in front of your web server and use that to cache output. Varnish is a popular choice these days and can serve hundreds of times more request per second with caching than Apache running your PHP script on the same server. The cache is created and configured at the proxy level, so when it expires, the request passes through to your script which runs as it normally would to generate the new version of the page.
You know, for me, optcache , filecache .. etc only use for reduce database calls.
They can't speed up your code. However, they improve the page load by using cache to serve your visitors.
With me, APC is good enough for VPS or Dedicated Server when I need to cache widgets, $object to save my mySQL Server.
If I have more than 2 Servers, I like to used Memcache , they are good on using memory to cache. However it is up to you, not everyone like memcached, and not everyone like APC.
For caching whole web page, I ran a lot of wordpress, and I used APC, Memcache, Filecache on some Cache Plugins like W3Total Cache. And I see ( my own exp ): Filecache is good for caching whole website, memory cache is good for caching $object
Filecache will increase your CPU if your hard drive is slow, and Memory cache is terrible if you don't have enough memory on your VPS.
An SSD HDD will be super good speed to read / write file, but Memory is always faster. However, Human can't see what is difference between these speed. You only pick one method base on your project and your server ( RAM, HDD ) or are you on a shared web hosting?
If I am on a shared hosting, without root permission, without php.ini, I like to use phpFastCache, it a simple file cache method with set, get, stats, delete only.
In Addition, I like to use .htaccess to cache static files like images, js, css or by html headers. They will help visitors speed up your page, and save your server bandwidth.
And If you can use .htaccess to redirect to static .html cache if you cache whole page is a great thing.
In future, APC or some Optcache will be bundle into PHP version, but I am sure all the cache can't speed up your code, they use to:
Reduce Database / Query calls.
Improve the speed of page load by use cache to serve.
Save your API Transactions ( like Bing ) or cURL request...
etc...
A lot of times, when it comes to PHP web applications, the database is the bottleneck. As such, one of the best things you can do is to use memcached to cache results in memory. You can also use something like xhprof to profile your code, and really dial in on what's taking the most time.
Yes, those are two different cache-techniques, and you've understood them correctly.
but beware on 1):
1.) Caching script generated output to files or proxies may render problems
if content change rapidly.
2.) x-cache exists too and is easy to install on ubuntu.
regards,
/t
I don't know if this really would work, but I came across a performance problem with a PHP script that I had. I have a plain text file that stores data as a title and a URL tab separated with each record separated by a new line. My script grabs the file at each URL and saves it to its own folder.
Then I have another page that actually displays the local files (in this case, pictures) and I use a preg_replace() to change the output of each line from the remote url to a relative one so that it can be displayed by the server. My tab separated file is now over 1 MB and it takes a few SECONDS to do the preg_replace(), so I decided to look into output caching. I couldn't find anything definitive, so I figured I would try my own hand at it and here's what I came up with:
When I request the page to view stuff locally, I try to read it from a variable in a global scope. If this is empty, it might be that this application hasn't run yet and this global needs populated. If it was empty, read from an output file (plain html file that literally shows everything to output) and save the contents to the global variable and then display the output from the global.
Now, when the script runs to update the tab separated file, it updates the output file and the global variable. This way, the portion of the script that actually does the stuff that runs slowly only runs when the data is being updated.
Now I haven't tried this yet, but theoretically, this should improve my performance a lot, although it does actually still run the script, but the data would never be out of date and I should get a much better load time.
Hope this helps.
My question is whether or not using multiple PHP includes() is a bad idea. The only reason I'm asking is because I always hear having too many stylesheets or scripts on a site creates more HTTP requests and slows page loading. I was wondering the same about PHP.
The detailed answer:
Every CSS or JS file referenced in a web page is actually fetched over the network by the browser, which involves often 100s of milliseconds or more of network latency. Requests to the same server are (by convention, though not mandated) serialized one or two at a time, so these delays stack up.
PHP include files, on the other hand, are all processed on the server itself. Instead of 100s of milliseconds, the local disk access will be 10s of milliseconds or less, and if cached, will be direct memory accesses which is even faster.
If you use something like http://eaccelerator.net/ or http://php.net/manual/en/book.apc.php then your PHP code will all be precompiled on the server once, and it doesn't even matter if you're including files or dumping them all in one place.
The short answer:
Don't worry about it. 99 times out of 100 with this issue, the benefits of better code organization outweigh the performance increases.
The use of includes helps with code organization, and is no hindrance in itself. If you're loading up a bunch of things you don't need, that will slow things down -- but that's another problem. Clarification: As you include pages, be aware what you're adding to the load; don't carelessly include unneeded resources.
As already said, the use of multiple PHP includes helps to keep your code organized, so it is not a bad idea. It will become a problem when too many includes are used, because the web server will have to perform an I/O operetation for each include you have.
If you have a large web application, you can boost it using a PHP accelerator, which caches data and compiled code from the PHP bytecode compiler in shared memory. If you have lots of PHP includes in a specific file they will be performed just once. Further calls to that file will hit the cache, so no PHP require will be performed.
It really depends on what you want to do, I mean if you have a piece of code that is used all the time it is really convenient to include it instead of copying and pasting all the time and that will make your code more clear and not slower, but if you include all the functions or classes you have written in your files without using them of course thats not a good practice...I would suggest using a framework (like codeigniter or something else you find convinient) because it really helps clearing this things out... good luck!
The only reason I'm asking is because
I always hear having too many
stylesheets or scripts on a site
creates more HTTP requests and slows
page loading. I was wondering the same
about PHP.
Do notice that an HTTP request is several orders of magnitude slower that a PHP include.
Several HTTP requests -> Client has to request and accept over the wire several files
Several PHP includes -> Client has to request and accept only one file
The includes, obviously, will have a server penalty. But by your question... don't sweat it. Only on really large-scale PHP projects will you face such problems.