Test multiple urls (files) with codeception and log separate results - php

I'm trying to create a small test suite for a homework assignment, using Codeception. My idea is to apply a single Cest test scenario to multiple URLs and get the different results, which I can then log by some identification (for example name of the file).
There are two main things I can't figure out:
How to run a single test with different URL and log failed assertions/passed assertions for every test?
How to check every assertion, even if some of them fail?

First part:
I assume you are using Codeception with Selenium.
You can configure this in acceptance.suite.yml:
class_name: AcceptanceTester
modules:
enabled:
- WebDriver
- AcceptanceHelper
config:
WebDriver:
url: 'http://localhost.dev:8000/'
browser: 'firefox'
env:
url1:
modules:
config:
WebDriver:
url: 'http://url-1.dev/'
browser: 'chrome'
url2:
modules:
config:
WebDriver:
url: 'http://url-2.dev/'
browser: 'chrome'
url3:
modules:
config:
WebDriver:
url: 'http://url-3.dev/'
browser: 'chrome'
Then invoke the tests like this:
$ php codecept.phar run acceptance --env url1 --env url2 --env url3
For more details: http://codeception.com/docs/07-AdvancedUsage#Environments
Second part:
If an assertion fails, testing still continues with the subsequent environments. You can also use conditional assertions (those that start with canSee and cantSee), which will allow the test to continue even if the assertion fails.
For more details: http://codeception.com/docs/03-AcceptanceTests#Conditional-Assertions

Related

Codeception acceptance tests report that coverage is 0%

I'm trying to test my Yii2 application by codeception.
I want to see coverage of my code.
I'm running selenium server:
java -jar -Dwebdriver.chrome.driver=chromedriver /path/to/selenium-server-standalone-3.141.59.jar
And i'm running my test in the following way:
./vendor/bin/codecept run `tests/acceptance/callIn/FinishCallCest.php:submitBlank --coverage`
When I'm runinng codeception acceptance test without --coverage everything works fine. And if I'm running codeception unit tests with --coverage I get correctly coverage report.
But, if I run codeception acceptance test with --coverage I seeing following:
localhost:8080 link opens in my browser. And i see "NOT FOUND 404" page from Yii2.
Then redirect to http://localhost:8080/index-test.php?r=call-in%2Ffinish-call&callId=1
happens, and I see the page and test is successfully done.
Why does the localhost:8080 page load first? And how can I fix the error with 0% coverage?
Here's my codeception.yml
actor: Tester
paths:
tests: tests
log: tests/_output
data: tests/_data
helpers: tests/_support
settings:
bootstrap: _bootstrap.php
memory_limit: 1024M
colors: true
modules:
config:
Yii2:
configFile: 'config/test.php'
cleanup: false
coverage:
c3_url: http://127.0.0.1:8080/index-test.php/
enabled: true
whitelist:
include:
- models/*
- controllers/*
config:
test_entry_url: http://localhost:8080/idnex-test.php
Here's my acceptance.suite.yml
class_name: AcceptanceTester
modules:
enabled:
- Yii2:
part: [orm] # allow to use AR methods
cleanup: false
- WebDriver:
url: 'http://test:test#localhost:8080'
auth: ['test', 'test']
browser: chrome
restart: true
window_size: 1024x768
And here's a codeception test log:
Acceptance Tests (1) ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Modules: Yii2, WebDriver
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
FinishCallCest: Submit blank finish form
Signature: calIn\FinishCallCest:submitBlankFinishForm
Test: tests/acceptance/callIn/FinishCallCest.php:submitBlankFinishForm
Scenario --
Destroying application
Starting application
[ConnectionWatcher] watching new connections
[Fixtures] Loading fixtures
[Fixtures] Done
[GET] http://test:test#localhost:8080/
[Cookies] [{"domain":"localhost","httpOnly":true,"name":"_csrf","path":"/","secure":false,"value":"c99eee70d806de0660302db950c436a04f5a6316b5847caed6de7d1d2e2d896aa%3A2%3A%7Bi%3A0%3Bs%3A5%3A%22_csrf%22%3Bi%3A1%3Bs%3A32%3A%22PXMEKYNF-k-StsR38slXolelVVLuFKTK%22%3B%7D"},{"domain":"localhost","expiry":1543848123.606772,"httpOnly":true,"name":"_identity","path":"/","secure":false,"value":"dacb46519512cd901d6c8ceafb494a3d2893b6cf41a196e111c9c522aa4605c9a%3A2%3A%7Bi%3A0%3Bs%3A9%3A%22_identity%22%3Bi%3A1%3Bs%3A77%3A%22%5B%22corebofs000080000ktnhd8g9vnbio68%22%2C%22corebofs000080000ktnhd8g9vnbio68%22%2C18000%5D%22%3B%7D"},{"domain":"localhost","httpOnly":true,"name":"PHPSESSID","path":"/","secure":false,"value":"06okm3re6q6ncjj6q878sdq88j"}]
I am on page "/index-test.php?r=call-in%2Ffinish-call&callId=1"
[GET] http://test:test#localhost:8080/index-test.php?r=call-in%2Ffinish-call&callId=1
I wait 5
I click "//*[#id="w3"]/div[1]/div[1]/h4/a"
I wait 5
I see "Завершение звонка","//*[#id="w1"]/div[4]/button"
I click "//*[#id="w1"]/div[4]/button"
PASSED
Destroying application
[ConnectionWatcher] no longer watching new connections
[ConnectionWatcher] closing all (0) connections
Suite done, restoring $_SERVER to original
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Time: 24.18 seconds, Memory: 14.00MB
OK (1 test, 1 assertion)
Code Coverage Report:
2018-12-03 09:42:01
Summary:
Classes: 0.00% (0/84)
Methods: 0.00% (0/389)
Lines: 0.00% (0/2706)
Remote CodeCoverage reports are not printed to console
HTML report generated in coverage

Codeception Acceptance Tests not working

everyone!
I have been trying to configure Codeception 2.3.6 with Laravel 5.3.30 running on PHP 7.0.23 powered by WAMP 3.1.0. My functional test cases are running fine, but when I try to run my acceptance test cases, a new chrome window opens and then closes without doing anything.
The output in the HTML Report is Codeception Results OK(0s), while the output on the command line is:
WelcomeCept: Perform actions and see result (0.00s)
Time: 3.24 seconds, Memory: 22.75MB
OK (1 test, 0 assertions)
HTML report generated in file://D:\wamp\www\myApp\tests/_output\report.html
First I start ChromeDriver with the command
chromedriver --url-base=/wd/hub
Then I start Selenium Standalone Server 3.13.0 with the command:
java -Dwebdriver.chrome.driver="chromedriver" -jar selenium-server-standalone-3.13.0.jar -port 4445
Then I run my acceptance test suite which contains a single test file, with the command:
call vendor/bin/codecept run acceptance --html
My acceptance.suite.yml is:
class_name: AcceptanceTester
modules:
enabled:
- WebDriver:
url: http://lcms.com/
window_size: false # disabled in ChromeDriver
port: 9515
browser: 'chrome'
restart: true
wait: 200
capabilities:
unexpectedAlertBehaviour: 'accept'
webStorageEnabled: true
javascriptEnabled: true
- Laravel5:
part: ORM
cleanup: false # can't wrap into transaction
environment_file: .env
- \Helper\Acceptance
My WelcomeCept.php file, just for testing the configuration, is:
<?php
class WelcomeCept
{
public function welcomeTest(AcceptanceTester $I)
{
$I->wantTo('perform actions and see result');
}
}
Please review my workflow and let me know if I'm doing things incorrectly or am missing something.
Thanks!
Update: Same thing is happening using GeckoDriver or PhantomJS in WebDriver mode. The tests are passing with OK but not performing any actions.
Solved! I tried WelcomeCest instead of WelcomeCept and things have fallen into place.

How do I configure REST module in Codeception to use browserstack?

I am writing acceptance codeception tests on a web application. It will require me to check headers, specifically response codes. I have found that the codeception REST module can be used, but it appears to require phpbrowser. I am required though to use browserstack. How would the configuration look using browserstack as aposed to phpbrowser.
My acceptance.suite.yml is currently set up like:
chrome-dev:
modules:
enabled:
- WebDriver
- \Helper\
config:
WebDriver:
url: 'https://XXXXXXXXXXXXXXXXXX.com'
host: 'XXXXXXXX#XXXXXXXX.com:XXXXXXXXXX.browserstack.com'
port: 80
browser: 'chrome'
window_size: maximize
capabilities:
'os': 'Windows'
'os_version': '10'
'browser_version': '57'
'acceptSslCerts': true
'resolution': '1600x1200'
'browserstack.chrome.enablePopups': true
'browserstack.chrome.allowAllCookies': true
My question here is how would I insert the REST module without using phpbrowser? It appears that it may be required, but does anybody know otherwise?
It is impossible - WebDriver and REST modules are incompatible.

Codeception\Extension\RunProcess not starting processes

PROBLEM:
Yesterday Codeception introduced new extension which should help with starting/stopping Selenium Server, Chrome Driver, etc (more here). However, even it shows that processes are starting - they're actually not.
acceptance.suite.yml
class_name: AcceptanceTester
modules:
enabled:
- WebDriver:
url: http://127.0.0.1:8080/
browser: chrome
- Yii2:
part: orm
entryScript: index-test.php
cleanup: false
extensions:
enabled:
- Codeception\Extension\RunProcess:
- java -jar /home/tajgeer/.executables/bin/selenium-server.jar
- php /home/tajgeer/Repozytoria/Yii2/yii serve
output:
Acceptance Tests (5)
[RunProcess] Starting java -jar /home/tajgeer/.executables/bin/selenium-server.jar
[RunProcess] Starting php /home/tajgeer/Repozytoria/Yii2/yii serve
E AboutCest: Ensure that about works
E ContactCest: Ensure that contact page works
E ContactCest: Contact form can be submitted
E HomeCest: Ensure that home page works
E LoginCest: Ensure that login works
[RunProcess] Stopping php /home/tajgeer/Repozytoria/Yii2/yii serve
[RunProcess] Stopping java -jar /home/tajgeer/.executables/bin/selenium-server.jar
example error:
[ConnectionException] Can't connect to Webdriver at http://127.0.0.1:4444/wd/hub. Please make sure that Selenium Server or PhantomJS is running.
I've already tried to set o+x permissions on both execs - didn't change anything. When I'm trying to run both tools by using entered commands - everything works well.
Could anyone point out what am I doing wrong?
SOLUTION:
Actually... I've solved it on my own. I've forgot about sleep (processes didn't even had a chance to run as tests were very quick). I've changed acceptance.suite.xml to following:
class_name: AcceptanceTester
modules:
enabled:
- WebDriver:
url: http://127.0.0.1:8080/
browser: chrome
- Yii2:
part: orm
entryScript: index-test.php
cleanup: false
extensions:
enabled:
- Codeception\Extension\RunProcess:
0: java -jar /home/tajgeer/.executables/bin/selenium-server.jar
1: php /home/tajgeer/Repozytoria/Yii2/yii serve
sleep: 5

Webception / webdriver / ftp

I use Codeception with webdriver and use command line to start tests. I try to use webception to show it on a web-application for my colleagues so I deployed webception on my FTP but i have problems with piloting selenium. I didnt succeed to run the test from my ftp with webcept.
[WebDriverCurlException]
Curl error thrown for http POST to http://127.0.0.1:4444/wd/hub/session with params: {"desiredCapabilities":{"unexpectedAlertBehaviour":"accept","browserName":"firefox"}}
couldn't connect to host
Here is my acceptance.suit.yml :
class_name: WebGuy
modules:
enabled:
#- PhpBrowser
- WebHelper
- REST
- WebDriver
config:
REST:
url: 'http://theAddress/'
timeout: 90
PhpBrowser:
url: 'http://theAddress/'
curl:
CURLOPT_RETURNTRANSFER: true
CURLOPT_FOLLOWLOCATION: true
WebDriver:
url: 'http://theAddress/'
browser: firefox
capabilities:
unexpectedAlertBehaviour: 'accept'
Any suggestions will be welcome. thank you
Try to run Selenium manually. Download Selenium Server here Selenium Downloads , then in the command line, navigate to the folder where the file has been downloaded and type:
java -jar selenium-server-standalone-W.XY.Z.jar
Now, Selenium should be running on the 4444 port. After that, try to run your test again.

Categories