Detecting slow tests with PHPUnit

Edit: As Sebastian Bergmann told me via Twitter, PHPUnit already generate timing information into the JUnit XML logs, but the idea of this post is the earlier detection of slowness in tests, i mean, you are developing tests and you are executing them, thus you will detect quickly if the tests are exceeding a time limit.

The tests, in most of the cases, are written by developers. Sometimes, they don’t take care of the speed of those tests.

We, as test engineers, should detect, or even prevent, the slowness in tests.

PHPUnit let you use listeners to add logic in some parts of the test execution, therefore, you can detect when a test starts and finish.

The listeners are PHP classes that must implement the PHPUnit_Framework_TestListener interface, to use them you will need to create a XML configuration file passed as a param to the PHPUnit runner.

Type in your shell:

phpunit --configuration MyConfiguration.xml testsDirectory

Continue reading

PHPUnit and the Fatal error: Call to undefined method

PHPUnit+Selenium+Fatal errors:

Once upon a time i was writing and executing some Selenium acceptance tests using PHPUnit and i realized that PHPUnit detects fatal errors due to an undefined method and convert them into exceptions. Good feature! i though, if you have a fatal error in your test the process execution  doesn’t stop, the log is generated and the rest of the tests can finalize. But i started to investigate the PHPUnit code and i saw that the detection of the fatal error might be done by sheer chance.

The PHPUnit SeleniumTestCase driver has a well known list of valid commands to send to Selenium, if you send an invalid one from your test it will be detected and the exception is thrown. How PHPUnit detects it? using the magic function “__call”, each command the driver receives goes through this magic function and switch it to see if the command is contained in the list of commands.

Although i don’t like the PHP magic functions and i think this conversion from the fatal error to the exception was not on purpose, the result is good and gives us a better way to handle the “undefined method” fatal errors because the process execution is not stopped and the test result follows the PHPUnit format therefore is easily readable.

But the behaviour is not always like this, what will happen if you are delegating test responsabilities to other external classes? maybe because you are using the Page Object pattern or just using helpers.

Continue reading