We're having an issue with PHPUnit sometimes failing to correctly run our test suite in CI. The behaviour manifests as roughly 1/3 tests failing to start and the CI step timing out. Interestingly enough, we don't see this behaviour:
Locally on our machines
Every time we run the tests
Occur with the dusk tests
Our setup is to run the tests using docker compose. We specifically start the dependencies of the application container first and pause for 30 seconds to wait for them to come up (i.e. the database).
Some things I have tried to fix this that haven't made a difference:
Running the tests with process isolation on.
Splitting the test suite into two to run less tests (both splits have the same problem)
The tests are running in Github Actions, this makes around ~7 gigs of ram available to the machines, along with multiple cores.
What is also interesting is that the dusk based tests have never stalled like this, despite them using the same docker-compose file but with additional services (like a Selenium).
I'm not sure if this has a clear answer, but I'm at the point where I'm out of ideas of what to try next (short of hosting the GitHub action runners ourselves).
Output from a failed test looks like:
+ sleep 30
+ docker-compose -f docker-compose.ci.yml run --rm postgres pg_isready -h postgres -d testing -U testing -t 30
postgres:5432 - accepting connections
+ docker-compose -f docker-compose.ci.yml run --no-deps backend phpunit '--filter=/::test[J-Z|j-z]/' --testdox
Pulling backend (***.dkr.ecr.ap-southeast-2.amazonaws.com/builder:acb548c6125ad97e57e8070c414644c7e6f385f5)...
acb548c6125ad97e57e8070c414644c7e6f385f5: Pulling from builder
Digest: sha256:c84a53dbb201b809a2325a93c52d4a362eefa6de64ab98f759e43c5dd363aac2
Status: Downloaded newer image for ***.dkr.ecr.ap-southeast-2.amazonaws.com/builder:acb548c6125ad97e57e8070c414644c7e6f385f5
PHPUnit 9.1.5 by Sebastian Bergmann and contributors.
##[error]The operation was canceled.
A successul run:
+ sleep 30
+ docker-compose -f docker-compose.ci.yml run --rm postgres pg_isready -h postgres -d testing -U testing -t 30
postgres:5432 - accepting connections
+ docker-compose -f docker-compose.ci.yml run --no-deps backend phpunit '--filter=/::test[A-I|a-i]/' --testdox
Pulling backend (***.dkr.ecr.ap-southeast-2.amazonaws.com/builder:acb548c6125ad97e57e8070c414644c7e6f385f5)...
acb548c6125ad97e57e8070c414644c7e6f385f5: Pulling from builder
Digest: sha256:c84a53dbb201b809a2325a93c52d4a362eefa6de64ab98f759e43c5dd363aac2
Status: Downloaded newer image for ***.dkr.ecr.ap-southeast-2.amazonaws.com/builder:acb548c6125ad97e57e8070c414644c7e6f385f5
PHPUnit 9.1.5 by Sebastian Bergmann and contributors.
Audit Controller (Tests\Feature\Admin\AuditController)
✔ Audits index
(I've truncated the full test output)
Sometimes I will see it reach the first test before stalling.
I had similar (but not the same) problem on Codeception framework for tests but I was not using Github Actions and docker.
The issue was in the way how database was deployed to the server during tests. Because of that database was deployed in long long time but after that, tests starts. We didn't have limit on server so there was no time out.
On Local machine it was working fine.
As I see you are running functional/acceptance tests with database, so I propose to check module for database.
Enable verbose with -v --debug option for phpunit first. You will see then when it hangs and you should be able to debug then easily.
Related
I'm in a multi developer program. The other developer installed PHPUnit via composer and successfully is running tests. I took the composer files, ran composer update, but when I try to run the tests, I am receiving this error:
C:\Repos\Project\TDD>php vendor/bin/phpunit
Output:
dir=$(cd "${0%[/\\]*}" > /dev/null; cd "../phpunit/phpunit" && pwd)
if [ -d /proc/cygdrive ]; then
case $(which php) in
$(readlink -n /proc/cygdrive)/*)
# We are in Cygwin using Windows php, so the path must be translated
dir=$(cygpath -m "$dir");
;;
esac
fi
"${dir}/phpunit" "$#"
This is the content of the phpunit shell script. The shebang first line #!/usr/bin/env sh is not being output, but this is the rest of the file. The problem appears to be that the PHP CLI is not executing the script, just reading it. I can't figure out how to fix that.
I am on a windows environment, and I am not running apache locally.
Things I have tried:
I have tried using full paths (eg, C:/php/php.exe) for both the php and the shell script, and that didn't matter.
I have tried reversing the slashes vendor\bin\phpunit
I can run php -v successfully, but php -f vendor/bin/phpunit still just outputs the file contents.
I have confirmed the PATH for php is accurate and restarted.
All expected vendor files do appear to be present.
Composer update is not throwing errors of any kind.
I've changed my terminal / CLI processor to use powershell instead of the default cmd.exe (same result as above)
I've confirmed encoding of the phpunit file is UTF-8 without BOM
I've tried making the php command execute the phpunit script in powershell and git bash directly with no luck --- it outputs the shell file. It seems to be that php is just not executing it, which makes me think there is a problem with my php.ini or something.
I CAN run the phpunit shell directly, such as phpunit --version, but I can't get it to run my tests.
I set the composer platform to match my cli php -v
"config": {
"platform": {
"php": "7.4.20"
}
}
Edit: Interestingly, I uploaded it all to my LAMP webserver and ran the same script there, and it is not executing there either. It's outputting the files...
There is a lot of stuff you're fiddling with while trouble-shooting your issue, so there can be only giving extended comments on it. Take it with a grain of salt and pick the pieces that make sense to you, it looks to me you're clever and should be close to the point where you connect the dots more soon than later (don't give up):
C:\Repos\Project\TDD>php vendor/bin/phpunit
This on your windows shell prompt command the php binary to execute the (php) script vendor/bin/phpunit.
And PHP does exactly this (everything else would be a surprise, right?), it executes that script.
The output you see confirms it. As php is the PHP CLI SAPI (PHP Command Line Binary), the first line starting with # is not output (you can find this behaviour described in the PHP documentation as well: https://www.php.net/manual/en/features.commandline.usage.php and on antoher place that I can't find right now that exactly describes this difference to the common mode https://www.php.net/manual/en/language.basic-syntax.phpmode.php).
You then correctly analyze:
This is the content of the phpunit shell script.
and make the note on the shebang line:
#!/usr/bin/env sh
So this looks to me that this shell script vendor/bin/phpunit is not to be executed with the given commandline:
php vendor/bin/phpunit
but instead just by:
C:\Repos\Project\TDD>vendor/bin/phpunit
Which won't (or might not) work on your windows system.
Some more background information (this applies to phpunit bin-stubs as well as other when installed as composer(1) dependencies):
composer manages these scripts (for you)
composer handles them for POSIX compatible systems as well as for others (compare the /usr/bin/env sh which is stupid, it's /bin/sh, always, may need a report on the composer project) incl. windows or windows with cygwin or WSL running on Windows, Vagrant box setups etc. pp. (yes they really care since years).
composer handles this at the time of install/update.
for windows (which I can't compare are not using it but from what I remember) composer is adding .bat or .cmd files next to the commands (as that is how Windows handles executables).
So mabye using:
C:\Repos\Project\TDD>composer exec phpunit
already solves your invocation problem.
Or
C:\Repos\Project\TDD>vendor\bin\phpunit.bat
Or
C:\Repos\Project\TDD>vendor\bin\phpunit.cmd
Or
C:\Repos\Project\TDD>vendor\bin\phpunit
does. Point in case is this: https://getcomposer.org/doc/articles/vendor-binaries.md
All in all this looks to me you got instructions from one developer working on a unixoide sytem (Mac, Linux) and you're running on Windows. This should not pose a problem on that level, however the onboarding in your team might be low or it's just the knowledge management (unfortunately after a decade or more of Stackoverflow this got worse) and you've been left alone with the trouble shooting.
It's not a programming problem, but merely getting the tooling to run.
Perhaps there is some regime about composer and running it which prevents you from looking there first. But this is hard to say on Stackoverflow.
I'd start fresh with the project, remove it from disk and install it anew. This must work, always. To not sink the current project you can do this a-new in a second directory (the project might support composer create-project to give it a quick start - that is cloning new and doing the composer install).
Similar it is that you could do a composer update in the existing project (which is intended to modify it and depending on how well the project you work with is integrated with composer and your development platform will enable or break things).
At the end of the day the common workflow is:
> rmdir vendor
(remove the vendor directory, that is the blank slate)
> composer install
(install the vendor directory, that is all PHP dependencies of the project)
> composer exec phpunit
(execute the phpunit test-runner)
Edit: Interestingly, I uploaded it all to my LAMP webserver and ran the same script there, and it is not executing there either. It's outputting the files...
Never install phpunit on a webserver system. Run it in development or CI, but not on the webserver. Period. (this has security implications, and it won't help you to execute it there, you need to run it where you develop when you're doing TDD - keep this focus for troubleshooting)
The true path to the PHP script that is represented by vendor/bin/phpunit is:
vendor/sebastianbergmann/phpunit/phpunit
(you can find it in the composer.json of phpunit https://github.com/sebastianbergmann/phpunit/blob/master/composer.json#L66)
Given your original command-line and from the various other information you give, this should work:
php vendor/sebastianbergmann/phpunit/phpunit
My current setup involve PhpStorm IDE in which I have imported Symfony 3 projects which is basically CLI tool. On the host machine I don't have PHP installed so I'm running the application from Docker container which has PHP and Xdebug installed.
I don't have issues to debug web applications from Docker containers but with Symfony and this CLI tool it seems a little bit more tricky.
My question is how to properly set this up and debug it from PhpStorm? I tried to create a new debug configuration (PHP Remote Debug) but breakpoints are not trigged.
Suppossing you have followed into the instructions mentioned into the following links:
Can't connect PhpStorm with xdebug with Docker
How to setup Docker + PhpStorm + xdebug on Ubuntu 16.04
Or similar questions
Then you need to follow theese steps:
Step1:
Get shell access to your container via running:
docker exec -ti ^container_id^ /bin/sh
Or if running a debian/ubuntu based one (or you installed manually bash):
docker exec -ti ^container_id^ /bin/bash
The ^container_id^ can be found via docker ps command it is the first column of the table. If running on a small window just pipe int into less -S resulting the command:
docker ps | less -S
Then export the following enviromental variables:
export PHP_IDE_CONFIG="serverName=0.0.0.0:5092"
export XDEBUG_CONFIG="idekey=PHPSTORM"
Please keep in mind to setup the correct value specified into Servers section as you see in the image:
It is important in order not to run into the problem specified in this question.
Then just enable debugger listentin into the phpstorm and spawn the cli as you do when you run a symfony application.
I'm following a docker web application setup tutorial from here. As described in the docker-compose.yml file the MySQL container is linked to the app and it is accessible via both index.php and phpMyAdmin. No problem so far.
When I opened an interactive shell for my application using docker exec -it container-id bash and tried to access the MySQL service but I couldn't. What I'm I missing here?
Edit1: When I type MySQL, I get bash: mysql: command not found.
Edit2: Output of docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1643f0ba5637 dockertutorial_php "docker-php-entryp..." 9 seconds ago Up 6 seconds 9000/tcp dockertutorial_php_1
610f2c8bf4c4 phpmyadmin/phpmyadmin "/run.sh phpmyadmin" 9 seconds ago Up 6 seconds 0.0.0.0:8080->80/tcp dockertutorial_phpmyadmin_1
29c552da473b mysql:latest "docker-entrypoint..." 10 seconds ago Up 8 seconds 3306/tcp dockertutorial_mysql_1
Edit3: Perhaps I should have started with this. I apologise for all the confusion caused/would cause (Especially to #Jay Blanchard & #mkasberg). I want to run a bash script inside the nginx container which creates a MySQL database. It works if I have installed MySQL in the nginx container but not if it is a separate container. Is it possible to achieve with the current approach? If not how should I correct it?
mysql: command not found means that the mysql client binary is not available in whichever container you're logged in to. I suspect you're running docker exec on a container other than the MySQL one. The binary would be there in the MySQL container. While those other containers can connect do MySQL, they don't have the MySQL command line application available (to keep image size small).
Make sure you're connecting to the MySQL container with your docker exec command and it should work. In fact, docker-compose provides a way for you to do this. With your services running (after doing docker-compose up), try this:
$ docker-compose exec mysql /bin/bash
That should get you to a bash prompt in the mysql container. Before, you were getting to a bash prompt in the nginx container. Which can't run mysql on the command line.
I am using docker container on Linux Mint host machine to run PHPUnit tests, like this:
docker run -t -i --volume=$PWD:/var/www username/phpunit:v1 --stderr tests/
PHPUnit 4.8.3 by Sebastian Bergmann and contributors.
..........
Time: 1.18 seconds, Memory: 11.25Mb
So, this works fine. My next step was to create File Watcher inside IDE(PHPStorm), so that this docker command is run every time some of the test files change. However, when IDE runs the command, instead of getting the output like above, what I get is this error mesage:
cannot enable tty mode on non tty input
Looks like this is happening a lot on Windows machines, but I couldn't find what causes this issue on Linux. Is there a way to fix this ?
You are running the command from an IDE, which most probably isn't a terminal. So you might want to remove the -t parameter from the command:
docker run -i --volume=$PWD:/var/www username/phpunit:v1 --stderr tests/
Find a more detailed answer here.
I am reading the book "Learn ZF2: Learning By Example" by Slavey Karadzhov. Now I am at pages 103-107. I am trying to run PHPUnit test as described, but nothing happens... Where is the problem and how to fix it?
To go into the same situation you would have to:
git clone https://github.com/slaff/learnzf2.git .
composer.phar self-update
composer.phar install
git stash
git checkout 'ch-unittest'
This should bring you the same files as I have (=as in the book).
Now, I try to run this command:
ZF2_PATH=`pwd`/vendor/zendframework/zendframework/library \php vendor/bin/phpunit -c module/User/tests/
And the command does not work as expected...
I would expect some test result output or so, but all I get is the output witch is similar to the content I have in vendor/bin/phpunit file. It is:
#!/usr/bin/env sh
SRC_DIR="`pwd`"
cd "`dirname "$0"`"
cd "../phpunit/phpunit/composer/bin"
BIN_TARGET="`pwd`/phpunit"
cd "$SRC_DIR"
"$BIN_TARGET" "$#"
To be more precise, the first line (#!/usr/bin/env sh) does not show up in the output. The rest do show up.
**
I am running the code on Windows 8.1 Pro OS, using Git Bash (git version 1.9.4.msysgit.1). php keyword seems to work well as writing php -v works as expected (outputs PHP 5.5.12 (cli) (built: Apr 30 2014 11:20:58) ...)
My own guess would be that Git bash cannot resolve the code it finds in vendor/bin/phpunit so it fails there... But this is the guess only...
EDIT #1:
Just tried on shared hosting with LAMP set. Everything went fine. It seems that Zend Framework 2 and Windows are not so friendly at some points (e.g. at the just mentioned unit testing case). Looks like the very good book (no sarcasm, it is really the good one) would be better if it would have one more chapter on something like VirtualBox + Vagrant + PuPHPet (or something close to that).
The line of composer.json file used in the book: phpunit/phpunit" : "3.7.*
I have changed phpunit/phpunit" : "3.7.* to phpunit/phpunit" : "4.5.* as 4.5 version of PHPUnit is the current stable.
Then, of course: php composer.phar update
Then this one will work nicely:
ZF2_PATH=`pwd`/vendor/zendframework/zendframework/library \php vendor/phpunit/phpunit/phpunit -c module/User/tests/
The command above is not the same as in the book.
Tested on Windows 8.1, Ubuntu Trusty 14.04 LTS x64, the isolated development environment provide here. Works well.
There is also a way to make PHPUnit 3.7.* work on all systems (same systems tested by me), but the command must be changed to:
ZF2_PATH=`pwd`/vendor/zendframework/zendframework/library \php vendor/phpunit/phpunit/composer/bin/phpunit -c module/User/tests/
The command above is not the same as in the book.
So... Here we have two ways mentioned to go with PHPUnit tests to make them work. It is up to us to decide witch of two is better. I would prefer the one with increasing PHPUnit version to 4.5 at the moment.
Many thanks go to Slavey Karadzhov - the author of the book who made isolated development environment for programming with Zend Framework 2(ZF2) available.
Also, thanks to #jamaldo, #steeler and #malte whose activity (post, comments) brought me on the right track. The post I am talking about is here: https://stackoverflow.com/questions/23881289/php-vendor-bin-phpunit-is-printing-in-console-a-text