What can I do to increase the performance/speed of my PHP scripts without installing software on my servers?
Profile. Profile. Profile. I'm not sure if there is anything out there for PHP, but it should be simple to write a little tool to insert profiling information in your code. You will want to profile function times and SQL query times.
So where you have a function:
function foo($stuff) {
...
return ...;
}
I would change it to:
function foo($stuff) {
trace_push_fn('foo');
...
trace_pop_fn('foo');
return ...;
}
(This is one of those cases where multiple returns in a function become a hinderance.)
And SQL:
function bar($stuff) {
trace_push_fn('bar');
$query = ...;
trace_push_sql($query);
mysql_query($query);
trace_pop_sql($query);
trace_pop_fn('bar');
return ...;
}
In the end, you can generate a full trace of the program execution and use all sorts of techniques to identify your bottlenecks.
One reasonable technique that can easily be pulled off the shelf is caching. A vast amount of time tends to go into generating resources for clients that are common between requests (and even across clients); eliminating this runtime work can lead to dramatic speed increases. You can dump the generated resource (or resource fragment) into a file outside the web tree, and then read it back in when needed. Obviously, some profiling will be needed to ensure this is actually faster than regeneration - forcing the web server back to disk regularly can be detrimental, so the resource really does need to have heavy reuse.
You might also be surprised how much time is spent inside badly written database queries; time common generated queries and see if they can be rewritten. The amount of time spent executing actual PHP code is generally pretty limited, unless you're using some sub-optimal algorithms.
Neither of these are limited to PHP, though some of the PHP "magicy" approaches/functions can over-protect one from thinking about these concerns. For example, I recently updated a script that was using array_search to use a binary search over a sorted array, and gained the expected exponential speedup.
Really consider using XDebug profiler: it helps with checking how much a certain function is being executed against what you would have expected.
I try to decrease instructions while improving code readability by replacing logic with array-lookups when appropriate.
It's what Jeff Atwood wrote in [The Best Code is No Code At All][1].
Also, avoid loops inside another
loop, and nested if/else statements.
Short functions. Sometimes a lot of
code does not need to be executed
when the result-value is already
known.
Unnecessary testing:
if (count($array) === 0) return;
can also be written as:
if (! $array) return;
Another function-call eliminated!
[1]: http://www.codinghorror.com/blog/archives/000878.html"The Best Code is No Code At All"
You can optimized the code with two basic things:
Optimizing PHP associated library and server
Go through https://www.digitalocean.com/community/articles/how-to-optimize-apache-web-server-performance Or
You can use profiling tool like xhprof to view what part of your code can by optimized and here is the link to follow: http://michaelsanford.com/compiling-xhprof-for-php-5-4/
Optimizing your code using code profiler and code analyzer
You need to install Netbeans in order to use this plugin.
Here are the steps you need to follow:
1) Open NetBeans then select option from menu bar Tools > Plugin. Then search plug-in name "phpcsmd" in the available plug-in tab and install it from there.
2) Now open the terminal and be there as the super user by typing command "sudo su".
3) Install PEAR library (if it is not installed) into your system by running following commands into your terminal
a) wget http://pear.php.net/go-pear.phar
b) php go-pear.phar
As we need this for the installation of further addons.
4) Then run the command
"pear config-set auto_discover 1"
This will be used to set auto discover the path "true" for the required plug-ins. So they get install to the desired location automatically.
5) Then run below command to install PHP code sniffer.
"pear install --alldeps pear/PHP_CodeSniffer"
6) Now to install the PHP Mess Detector by running following command
"pear install --alldeps phpmd/PHP_PMD"
If you get the error like "invalid package name/package file "phpmd/PHP_PMD"" while installing this module. You need to use this "pear channel-discover pear.phpmd.org" command to get rid of this error. After this command you can run the above command again to install Mess detector.
7) Now to install the PHP Depend by running following command
"pear install --alldeps pdepend/PHP_Depend"
8) Now install the PHP Copy Paste Detector by running following command
"pear install --alldeps phpunit/phpcpd"
9) Then run the command
"pear config-set auto_discover 0"
This will be used to set auto discover path "false".
10) Then open net beans and follow the path Tools>Options>PHP>PHPCSMD
There is no magic solution, and attempting to provide generic solutions could well just be a waste of time.
Where are your bottlenecks? For example are your scripts processor/database/memory intensive?
Have you performed any profiling?
including files is slow, and requiring them is even slower. If you use __autoload for including every class then that will add up. for example.
I'm always a bit wary of trying to be too clever in terms of code optimisation, if it sacrifices code clairty. If you need to make code obscure to make it fast, would it not be cheaper to upgrade hardwear instead of wasting your time trying to tweak code? Processor cycles are cheaper than programmer cycles, after all.
The ones I can think of...
Loop invariants are always a
good one to watch.
Write E_STRICT and E_NOTICE
compliant code, particularly if you
are logging errors.
Avoid the # operator.
Absolute paths for requires and
includes.
Use strpos, str_replace etc. instead of regular expressions whenever possible.
Then there's a bunch of other methods that might work, but probably wont give you much benefit.
Whenever I look at performance problems, I think the best thing to do is time how long your pages take to run, and then look at the slowest ones. When you get these real metrics, you can often improve performance on the slowest ones by orders of magnitude, either by fixing a slow SQL query or perhaps tightening up the code a bit.
This of course requires no new hardware or special software, just a critical eye on the existing code.
That said, this will only work for so long... if you really are getting enough traffic to hit the limits of your hardware, and/or there is some code that is just inherently slow and really required, you will have to look at other possibilities.
I'm responsible for a large reporting system and we track the slowest reports kind of like that. I fire a unique key into the db when the report starts and then when it finishes I can determine how long it took. I'm using the database because that way I can detect when pages timeout (which happens a lot more than I'd like)
Follow some of the other advice first like profiling and making good resource allocation decisions, e.g. caching.
Also, take into account the performance of outside resources like your database. In MySQL you can check the slow query log for example. In addition make sure you didn't design your database an forget about it. Optimizing your queries (again for MySQL) against real data can pay of big.
Rasmus Lerdorf gave some good tips in his recent presentation "Simple is Hard" at FrOSCon '08. If you are using a bytecode cache (and you really should be using one), include path misses hurt a lot, so optimize your require/require_once.
You can use profiling tool like xhprof to view what part of your code can by optimized !
1) Use latest version of PHP
The core team is working hard on improving the performance of PHP in every version.
2) Use a bytecode cache
Since PHP 5.5 a bytecode cache has been added to PHP named OPcache. Using OPcache can make a huge difference especially since PHP 7. It receives improvements in every PHP version and might even get a JIT implementation in the future.
3) Profiling
While developing profiling gives you great insight what exactly is happening. This helps a lot finding bottlenecks in your code.
One of the most used tools is XHProf but is not officially supported anymore and has issues with PHP >= 7. An alternative when you want to profile PHP >= 7 is Tideways which is a fork of XHProf.
Related
I would like to test some variants of transaction concurrency in PostgreSQL and for that I need a script which would force two transaction to start at exactly the same time. Something that does not requires manual intervention ;)
Any ideas?
You can homebrew this by taking a LOCK on a table, setting up your transactions, then releasing the lock by rolling back the transaction that got the lock. See this prior answer and its links for details on this approach. While I demonstrated it using three psql sessions it's equally viable to do it with bash co-processes, a Python script using psycopg2 and the multiprocessing or threading modules, etc. Fairly simple to do. Update: In fact here's an example I just wrote in python3.
For more sophisticated tests, grab the PostgreSQL source code and use the "isolationtester" tool in src/test/isolation which lets you write recipes that do complex orderings of commands. It doesn't support being built with PGXS (though such support would probably be pretty trivial to add) so you have to compile the whole PostgreSQL source tree, but that's quick enough. It'll run against your existing PostgreSQL so there's no need to install the one you compiled.
See src/test/isolation/README for more about the isolationtester tool. The docs are a little thin on the ground since it's an internal testing tool, but the existing tests cases should help you get started. Feel free to improve it to meet your needs and submit patches :)
I have a small question about crawling a web page in PHP. I have to crawl about 90 000 products on one big eshop. I tried it in PHP, but one product takes about 2-3 sec and that's bad. Any tips, how to do it faster? Maybe a C++ multithread version? But what about time of a HTTP request? I mean, is it PHP's limitation or not? Thank you for the tips.
That's an extremely vague question. When you benchmarked the code you have, what was the slowest part? Was it network transfer times? Using a different language (or multiple threads) won't change that.
Was it time spent parsing the page? How are you doing that? If you're using an XML library to parse the entire DOM, could you get away with just looking for keywords (or even regular expressions)? That's less precise (and in some sense less correct) but perhaps it's faster.
What algorithms are you using for your analysis? Would other data structures provide better performance? As one simple example, if you spend a lot of time iterating over an array, perhaps a hash map is more appropriate.
PHP can be run in multiple processes. What happens if you kick off multiple instances of your script at once (on different pages)? Does the total time decrease?
Ultimately you've described a very general problem so I can't offer very specific solutions, but there is no inherent reason why PHP is inappropriate for this task. When you've identified what's slow (regardless of what language you're using) you should be able to more precisely address how to fix it.
I don't think it's PHPs problem but it could be depending on connection speed/computer speed. I've never had a speed problem with PHP/cURL though.
Just do multiple threads (ie. multiple connections at once), I suggest you use cURL but that's only because I'm familiar with it.
Here's a guide I've used for multiple threads for scraping with cURL:
http://semlabs.co.uk/journal/object-oriented-curl-class-with-multi-threading
Be VERY careful not to accidentally cause a denial of service situation with your scripts. But I'm sure you're already away of that possibility.
If your program is running slowly, my advice would be to run a profiler on it, and analyse why it's running slowly.
This advice applies to any language, but in the case of PHP, the profiler software you need is called xDebug.
This is a PHP extension, so you need to install it into your server. If you're running on an ISP's server, then you may not have permission to do this, but you can always install it with PHP on your local PC and run your tests there.
Once you've got xDebug installed, switch on the profiling features in PHP.ini (see the xDebug documentation for instruction on this), and run your program. It will then generate profiler files, which can be used to analyse what the program is doing.
Download KCacheGrind to perform the analysis. This will generate call tree information, showing exactly what happened as the program ran, and how long every function call took.
With this information, you can look for the function calls that are running slowly, and work out what's happening. Usually the reason for slow code is some kind of inefficiency in the way something is written; xDebug will help you find it.
Hope that helps.
You have 99% probability that PHP is NOT the problem. It is rather the eshop webserver or any other network latency.
I know this for sure because I have been doing this for months now, and even if your code has lots of regular expressions, data scraping is really fast in PHP.
The solution to speed this ? Pre cache all the website with a command line crawler since disk space is cheap. curl can do this, and httrack as well. It will be much faster and stable than PHP doing the crawling.
Then let PHP do the parsing alone, you will see hopefully PHP chomping dozens of pages per minute, hope this helps :)
I'm trying to track down issues with an application [modx] I have several of these sites [about 10] on my server & was wondering how I can see what php is doing.
Pages on these sites are extremely slow while the same sites in dev are fine as are other php applications on the server.
I tried using xdebug to get an idea of what php was doing while processing these pages & where the bottleneck was occurring, but it only appeared to want to do anything on an error [there are no errors being thrown]
Any suggestions on how to track this down?
[linux/Centos5/php5.2.?/apache2]
Xdebug and webgrind are a nice way to see where your bottel necks are...
Read XDEBUG_PROFILE and Webgrind
Set up the php.ini to have xdebug profile your code on every run or if a special param is passed, then setup webgrind to read from the same directory xdebug writes its profile dumps to.
Webgrind will show you what functions and set of functions require the most time, it breaks it down and makes it easy to find slow and/or inefficient code. (eg. your script is calling "PDOStatement->execute" 300 times on a fast query [Or calling it once and a massively slow one] taking up 90% of the execution time).
The most commonly used tool, for finding bottlenecks in PHP, would be Xdebug. But you should also manually examine the codebase.
There are three different areas where you will have to focus on:
frontend performance
SQL queries
php logic itself
.. and the impact on the perceived speed is in this order.
You should start by running ySlow, and make sure that your site follows the guidelines as closely as possible.
The next step would be tracking down what SQL queries are executed, and (assuming you are using mysql) try to run them with EXPLAIN. Also, check the queries themselves. There might be some extremely stupid code there, like ORDER BY RAND() or use of LIKE in huge tables.
And the last stage would fixing it all would a hard looks at the code itself. Both on PHP and JavaScript side of things.
Also , you should upgrade to PHP 5.3, because your version is extremely outdated.
Usually when you don't know what you're looking for, you cannot spot it with tools like xdebug or other plugins/debug bars etc built into CMS/Framework, new relic is the simplest solution - you'll be able to spot bottlenecks after few min.
while new relic is a paid app, you can test if for free for first 14 days - it's more than enough to find problem.
It's great because it integrates all other tool's and data sources you usually use:
xdebug, cpu & i/o monitoring, mysql slowlog, queries log.
It will also show you if your app is slow on php/DB/frontend/network.
You should try it out instead of wasting time for debugging with other tools.
here is a guide for centos installation: https://newrelic.com/docs/php/php-agent-installation-redhat-and-centos
When building some of my PHP apps, a lot of the functionality could be coded using PEAR/PECL modules, however, the fact that some people using it may not have the access to install things, It poses a puzzler for me.
Should I forsake some users to use PEAR/PECL for functionality, where these will allow me to have a system coded up quicker than if I wrote my own functionality, but eans that it will exclude certain people from using it.
It partly depends on how much time you have, and the purpose of the project. If you're just trying to make something that works, go with PEAR/PECL. If you're trying to learn to be a better programmer, and you have the time, then I'd recommend taking the effort to write your own versions. Once you understand the innards of whatever you're trying to replace, you may want to switch to the PEAR/PECL version so that you're not wasting time reimplementing what has already been implemented...
...but on the other hand, preexisting tools don't always do exactly what you need, and sometimes have overhead that doesn't do you any good. This is why Unix command-line tools are so small and narrow of purpose; nobody really needs a version of 'ls' that can do anything besides what 'ls' can currently do. Your version of whatever PEAR library will, by virtue of being written by you, do exactly what you need doing. It requires some careful thought...
...but on the gripping hand, don't spend too much time thinking about it. Spend five minutes, make a decision, and start coding. Even if you make the wrong decision, you'll at least have gotten more practice coding. :-)
Save on development time by developing with the pear libraries, and provide the libraries bundled in what you distribute (though you'll have to make sure it obeys licensing requirements)
I would not depend on certain PECL extensions being installed unless you're doing something particularly related to one (say an XDebug web-frontend or something), the majority of installs will be carrying a fairly vanilla set of extensions.
My suggestion is to start with assuming PEAR/PECL modules, and get the rest of the code done. Then, once you've got most of your code working the way you want, you can evaluate going back and piece by piece replacing the outside code with your own. Plus, by then you'll have a better idea of the impact using those has on your userbase.
Code it initially using PEAR/PECL and if you get people asking for a non PEAR/PECL version, start coding your own alternatives then for such a version.
The initial development will go much faster with this, and you may find that no-one cares about requiring 3rd party libraries once you have started releasing apps.
Use PEAR but allow for including the PEAR packages inside your project. All PEAR packages can be separately downloaded from http://pear.php.net/ and can be put anywhere. Depending on convenience and licensing issues you could then package all the required PEAR files with your project or tell users how to download and "install" them.
What I do most times is I'll never use PEAR installed globally on a server. Versions can change and affect your application.. Instead I have a config file (in my case XML) that lists all the packages required and their versions. The installer connects to my personal FTP repository and downloads and installs all the PEAR packages locally in $PROJECTBASE/lib/pear/ .. And PEAR is run locally instead of globally. Something you may want to consider.
Using PEAR is no problem, if users do not have root access to their webserver, they can simply download the PHP files from pear.php.net and add it to their include path. PECL's a little more tricky to work around, since there's often no way to install new modules without root access.
You need to watch out because a lot of modules in pear are really of pretty low quality.
Some are great, don't get me wrong, but don't assume that anything in pear, by virtue of being in pear, is at any given quality. Which means you need to at least skim the source of a pear module before deciding to use it, which for simple enough tasks may take more time than going without pear.
pecl is different, however. Extensions tend to be better vetted and tested, else they'd crash php.
Reiterating much of what's already been said: http://www.codinghorror.com/blog/archives/001145.html
Does anybody have experience working with PHP accelerators such as MMCache or Zend Accelerator? I'd like to know if using either of these makes PHP comparable to faster web-technologies. Also, are there trade offs for using these?
Note that Zend Optimizer and MMCache (or similar applications) are totally different things. While Zend Optimizer tries to optimize the program opcode MMCache will cache the scripts in memory and reuse the precompiled code.
I did some benchmarks some time ago and you can find the results in my blog (in German though). The basic results:
Zend Optimizer alone didn't help at all. Actually my scripts were slower than without optimizer.
When it comes to caches:
* fastest: eAccelerator
* XCache
* APC
And: You DO want to install a opcode cache!
For example:
alt text http://blogs.interdose.com/dominik/wp-content/uploads/2008/04/opcode_wordpress.png
This is the duration it took to call the wordpress homepage 10.000 times.
Edit: BTW, eAccelerator contains an optimizer itself.
MMCache has been deprecated. I recommend either http://pecl.php.net/package/APC or http://xcache.lighttpd.net/, both of which also give you variable storage (like Memcache).
Both are interesting and will provide speed boost since they compile source code into binary representation which is then executed by the PHP engine.
Any huge web site running with PHP (Facebook for example) is running some sort of opcode cache system like MMCache.
The problem is that they are not very easy to set up depending on your system.
Depending on how much of your PHP code is actually executed and how long that execution takes they can be a really big win. It certainly isn't going to hurt, but the gain you see will very much depend on where your time is currently spent.
btw mmcache has been rolled into a different project now, I forget the name but Google will tell you.
I use APC on my production servers and it works pretty well out of the box. Compile it and add it to PHP and there isn't much tweaking left to do for it. I check it every once in a while just to review stats but since I use MVC a lot all of the main files (routers, controllers, etc) rarely change on a day-to-day basis so that code stays compiled and runs pretty efficiently.
currently we use apc, free and was just a simple plug and play on our live servers. Provided a huge performance increase for our site, especially as the project size increased. I also have the apc.stat disabled so it doesn't check if the code has been updated, so whenever we need to update the code on the live site we restart apache.
I use APC, and can attest that it can dramatically reduce the CPU and I/O load on an app server if you maintain a high cache-hit rate. It not only saves you from having to compile, it can save you from having to read the php files from disk at all. (i.e. the bytecodes are served directly from main memory, so it's super fast) It lowers the speed to render a single page, and increases the requests per second your server can handle.
If you use RedHat or CentOS, installing APC is super simple:
yum install php-devel httpd-devel php-pear
pecl install apc
echo "extension=apc.so" > /etc/php.d/apc.ini
# if you're using SELinux:
chcon "system_u:object_r:textrel_shlib_t" /usr/lib/php/modules/apc.so
/etc/init.d/httpd restart
You asked about downsides. The only downside is that it requires some memory. The default on APC is 30MB, but it can be adjusted, and the cost of a little bit of memory more than pays for itself with the increased speed and response rate.
BlaM's testing included all the DB calls made by WordPress. When you're making fewer DB calls, you'll see the performance gain of opcode caches be even more dramatic.
I used Zend Accelerator a little back in the day (2004-ish). It certainly gave some significant performance wins on code it could work with, but unfortunately the system I was using was designed to quite often dynamically load code and then eval it, which Zend Accelerator couldn't do much with at the time (and I'd guess still can't).
On the down side, we certainly saw some caching issues (where the code would be changes, but the compiled version sync with the change for one reason or another). I imagine those problems have likely been ironed out by now.
Anyway, I don't have any hard comparison numbers, and certainly didn't write the same system in different environments for comparison, but for the vast majority of systems, PHP isn't going to kill you performance wise.
Have you checked out Phalanger? It compiles PHP to .NET code. Here are some benchmarks which show that it can dramatically improve performance.