I've tried opcache but it even makes it lower, in some cli programs there is a watch option, the cli keeps running that on changes it will redo something, I couldn't find that for phpunit.
Just to be clear, the tests are pretty fast, what's slow is the bootstrap because the codebase is rather big, the code base runs fine on a webserver thanks to opcache, but for phpunit opcache (which only should work with file cache I'd suppose?) seems to be hopeless...
I'm using PHP 8 and PHPUnit 9
Related
Currently we have written some unit test for our php laravel 5.5 application using codeception. just for additional info, our laravel code base have around 200k LOC. For normal unit test run, we noticed that it is really fast in which we can get more than 200 tests to finish within 1 hour.
But the main issue is that when we enabled codecoverage in codeception which is using xdebug by default, we noticed the execution time increased drastically.
Now it already took 1 week but the whole codecoverage execution not even finished yet.
I am not sure whether this is the problem from codeception or xdebug itself but if anybody have experiences running php codecoverage on a huge codebase, it would be nice if you can share how you achieve it. Would appreciate it also if somebody can suggest any other tools to look into. Currently we are considering switching to phpunit but are still open to other tools to explore.
Replacing Codeception with PHPUnit will be a lot of work for little gain, because Codeception uses PHPUnit and its PHP-Code-Coverage library under the hood.
There is a new code coverage extension, called pcov which is supposedly much faster than xdebug.
https://github.com/krakjoe/pcov/blob/develop/INSTALL.md
I haven't tried to use it, but be aware that it requires PHPUnit 8, which is only available on PHP 7.2 or later versions.
Recently I have seen code coverage sped up by replacing xdebug with phpdbg - I can't give exact numbers as the code base has extensive functional tests in its test run (and the speed-up was only for unit tests), but a 2+ hour test and coverage run has been reduced to around 50 minutes.
Note that xdebug and phpdbg can differ in their code coverage (it looked like xdebug better dealt with opcache optimisations).
edit:
Since replacing xdebug with phpdbg, I have seen further speed improvements by replacing phpdbg with pcov.
I want to allow my testers to use the development website "as usual" and collect code coverages of every "run", combined everything and be able to say "after 4 hours of tests, here are the 75% of the code that were executed".
I use the php-code-coverage library (https://github.com/sebastianbergmann/php-code-coverage) and everything is working fine except that with xdebug as the tool used for code coverage it's way too slow (10 times slower that without activating php-code-coverage).
I've compiled my own version of php 7.2 with "--enable-phpdbg" and with the help of the command "update-alternatives" I'm able to run in cli :
$ php index.php
and get the code coverage I need, and it's only two times slower (every call to "php" is calling "phpdbg").
But I can't find a way to make it work with apache so that when I'm loading my website it's the executable "phpdbg" and not "php" that is used.
Even if I'm compiling my own ".so" it will still be "php" that will be executed.
I have the latest PHPUnit as a phar, placed in /usr/local/bin/phpunit (4.1.3). When I execute this file on my vagrant host (ubuntu 12.04, php 5.3.10), it takes what seems to be 30s to 60s before it actually starts performing the unit tests. I cannot figure out why.
Any ideas?
I ran into a similar problem when running phars in production (AWS's phar. Unfortunately, PHP doesn't do any caching around phar archives, even when using APC as an OpCode cache. So, on each request, PHP is unarchiving and parsing the entire phar. My workaround has been to avoid phars in production unless the archive is small.
If you have the option to upgrade PHP 5.5 w/ OpCode caching, you shouldn't have this problem.
I am going to close this for now as I believe the primary issue is that I am working with a shared folder, and I cannot update to 5.5 like monte suggested. I appreciate your response! I will just have to deal with it for now. If I get a moment I will try a vagrant with 5.5 and OpCode like you suggested, and just run phpunit to see if it is faster, even in a shared folder. If so, I will change my accepted answer.
I am building a framework where product instances use the main framework files, until there is a copy of it's own version of that file. To achieve this I have done the following:
set_include_path(MY_PRODUCT_ROOT.'/' . PATH_SEPARATOR . MY_FRAMEWORK_ROOT.'/');
So if I call include('view-users.php'); it will first look in MY_PRODUCT_ROOT for /view-users.php and if that's not found, it will then look to MY_FRAMEWORK_ROOT/view-users.php.
This procedure is working very nicely until I add files to the product root. I know that PHP/Apache is caching the includes and one would think to run clearstatcache(true); to clear any status caching. PHP likely uses file_exists inside it's include(); and thinks the new file still does not exist. I have tried restarting Apache with no effect.
Unfortunately running clearstatcache(true); does not help either. Only once I have deleted MY_FRAMEWORK_ROOT/file does it think to clear cache and try again, thus finding MY_PRODUCT_ROOT/file.
Im a little stumped, I know we need to refresh PHP/Apache's understanding of whether the file(s) exist or not, but clearstatcache(true); is not helping...
Any ideas?
UPDATE: Correction, restarting Apache seems to help now. I reiterate that this only occurs when trying to ADD a file to MY_PRODUCT_ROOT, to overlap an existing MY_FRAMEWORK_ROOT file, for customization
UPDATE: Development environment is Zend Server CE PHP 5.3.14 on Windows, Production environment Centos linux httpd, PHP 5.3+. The fact that Zend optimizer is enabled on my dev environment could have an effect, Also not using APC or any other caching scripts
Zend Optimizer+ speeds up PHP execution by opcode caching and optimization. It stores precompiled script bytecode in shared memory. This eliminates the stages of reading code from the disk and compiling it on future access. For further performance improvements, the stored bytecode is optimized for faster execution.
This is caching the file contents found in the includes, thus clearstatcache does not work. I have disabled my Zend Optimizer and it works now.
How can I test performance of a PHP app using Apache Benchmark?
My environment is Ubuntu Linux - are there packages I can install?
If you have Apache 2 installed, Apache Benchmark is already installed. See man ab on how to use it. In most cases its just something like
ab -n 1000 -c 10 http://localhost/path/to/app
Where -n is the number of all requests, that should be performed and -c is the number of requests, that should be performed in concurrency.
Note, that you don't test the performance of your php project this way, but test everything, that is affected, beginning with the webserver, PHP, your application, the database, your filesystem, and so on. This means, that if you got poor results, that can also be caused by low memory, or you have just many stuff running in the background, or such. Use a profiler to analyze the performance of a php application. A profiler is built-in within xdebug.