Testing in the PHP w orld Marcus Börger PHP Québec Conference 2007
The need for Testing � Why Test? � Introduction to phpt Testing Marcus Börger Testing in the PHP World 2
Why Test? � Programming often comes along with code re-use � Code re-use comes along with code changes � Code changes are changes � Even for a few code lines - looking is not enough � Names can mislead � Code may have non obvious side effects � Sometimes code is designed for a limited domain � Increasing/ Changing that domain is error prone � Code interaction is often underestimated � A bug fix in one function may affect other functions Marcus Börger Testing in the PHP World 3
How to test � Testing after test log � Record problematic input actions and replay them � Automated testing � Integration/ System testing � Function testing � Unit testing � Acceptance/ Requirements testing � Regression Testing Marcus Börger Testing in the PHP World 4
Integration testing � Not only particular pieces but the whole � Major is to verify all parts work together � When working on real data it can detect system issues � Often requires multiple test systems � A manual or automated log is required � Usually performed/ organized by QA Does the system work? Marcus Börger Testing in the PHP World 5
Function testing � Execute parts of API � Use common data (domain API is designed for) � Use code from observed bugs Does the API work? Marcus Börger Testing in the PHP World 6
Unit testing � Execute testing on code � From single routines, to parts (usually not the whole) � Test private stuff � Analyze untouched code to write more tests � Analytically find test data � Use code from observed bugs Does the code work? Marcus Börger Testing in the PHP World 7
Acceptance testing � Requirements engineering � Develop tests from requirements Does it do what the customer wants? Marcus Börger Testing in the PHP World 8
Regression testing � Backwards compatibility test � Verify input against expected output Does it still work as expected? Marcus Börger Testing in the PHP World 9
Non-functional testing � Performance � Stability � Usability � Stress-Testing Marcus Börger Testing in the PHP World 10
Test-driven development � Think what you want or review specs � Write tests � Develop code and test � Write more tests if you figure any weakness Marcus Börger Testing in the PHP World 11
phpt Testing Marcus Börger Testing in the PHP World 12
What is phpt Testing? � Easy 1 PHP script test system (run-tests.php) � Everything goes into one file (* .phpt) � Capable of testing any aspect of PHP � Regression testing with pattern & regex matching � Integrates with memcheck � Used on http: / / gcov.php.net Marcus Börger Testing in the PHP World 13
http: / / gcov.php.net Marcus Börger Testing in the PHP World 14
http: / / gcov.php.net Marcus Börger Testing in the PHP World 15
http: / / gcov.php.net Marcus Börger Testing in the PHP World 16
http: / / gcov.php.net Marcus Börger Testing in the PHP World 17
Test file names � Tests for bugs bug< bugid> .phpt bug17123.phpt � Tests for functions < functionname> .phpt dba_open.phpt � General tests for extensions < extname> _< num> .phpt dba_003.phpt � Do not use any .php files for includes or alike Marcus Börger Testing in the PHP World 18
Getting started with phpt � Each test consists of several sections � Name � Input � Expected output - - TEST- - Hel l o W or l d - - FI LE- - Hel l o W or l d - - EXPECT- - Hel l o W or l d Always output something that can be verified. Marcus Börger Testing in the PHP World 19
Getting started with phpt � Each test consists of several sections � The input is usually a php snippet � An additional empty line makes cvs happy - - TEST- - Hel l o W or l d - - FI LE- - <?php echo " Hel l o W or l d" ; ?> - - EXPECT- - Hel l o W or l d Use only the long version of the php script tag. Marcus Börger Testing in the PHP World 20
Getting started with phpt � Each test consists of several sections � The input is usually a php snippet � The expected out must not be fixed � Scanf-like expressions - - TEST- - Hel l o W or l d - - FI LE- - <?php echo " Hel l o W or l d - - EXPECTF- - Par se er r or : synt ax er r or , unexpect ed $end i n % s. php on l i ne % d Do not check directories in error messages. Marcus Börger Testing in the PHP World 21
Getting started with phpt � Each test consists of several sections � The input is usually a php snippet � The expected out must not be fixed � Scanf-like expressions - - TEST- - Hel l o W or l d - - FI LE- - <?php echo " Hel l o W or l d - - EXPECTF- - Par se er r or : synt ax er r or , unexpect ed $end i n % s. php on l i ne % d When executed, the test file has .php ending. Marcus Börger Testing in the PHP World 22
Getting started with phpt � Each test consists of several sections � The input is usually a php snippet � The expected out must not be fixed � Scanf-like expressions � Regular expressoins - - TEST- - Hel l o W or l d - - FI LE- - <?php echo " Hel l o W or l d" - - EXPECTREG EX- - Par se er r or : ( par se| synt ax) er r or , unexpect ed $end i n . * on . * You can - but don't drop too much: It is "on line". Marcus Börger Testing in the PHP World 23
Use var_dump() � Usually output variables are verified by var _dum var _dum p p � Allows to check for exact type � Allows to check for private/ protected properties - - TEST- - Var _dum p - - FI LE- - <?php var _dum p( NULL) ; Var _dum p( 0) ; Var _dum p( f al se) ; Var _dum p( " " ) ; ?> - - EXPECT- - When checking object NULL IDs, use scanf/regex. i nt ( 0) bool ( f al se) st r i ng( 0) " " Marcus Börger Testing in the PHP World 24
More scanf matching � Allows matching blocks of output % s Any string % i Integers (includes "–") % d Numbers % f Floating point values % c Single characters % x Hexadecimal values % w Any amount of Whitespace % e DIRECTORY_SEPARATOR ('\ ' or '/ '). � Cannot verify complex output - - TEST- - M or e Test i ng - - FI LE- - <?php $s = ' 123' ; var _dum p( st r _shuf f l e( $s) ) ; var _dum p( $s) ; ?> Do not use %d for string - - EXPECTF- - length, unless you have to. st r i ng( 3) " % s" st r i ng( 3) " 123" Marcus Börger Testing in the PHP World 25
More regex matching � Regex matching requires escaping � Full regex support - - TEST- - M or e Test i ng - - FI LE- - <?php $s = ' 123' ; var _dum p( st r _shuf f l e( $s) ) ; var _dum p( $s) ; ?> Be as precise as possible - - EXPECTREG EX- - in matching expressions. st r i ng\ ( 3\ ) " [ 123] { 3} " st r i ng\ ( 3\ ) " 123" Marcus Börger Testing in the PHP World 26
More output matching � Huge output can be verified indirectly using md5 � When using files delete them before and after - - TEST- - O ut put val i dat i on usi ng m d5 - - FI LE- - <?php $dest = di r nam e( __FI LE__) . ' / bug22544. png' ; @ unl i nk( $dest ) ; i m agePng( i m ageCr eat eTr uecol or ( 640, 100) , $dest ) ; Var _dum p( m d5_f i l e( $dest ) ) ; Use dirname(__FILE__) @ unl i nk( $dest ) ; as temporary directory. ?> - - EXPECT- - St r i ng( 32) " 10a57d09a2c63f ad87b85b38d6b258d6" Marcus Börger Testing in the PHP World 27
More output matching � Huge output can be verified indirectly using md5 � When using files delete them before and after � Move clean-up code into a special section - - TEST- - O ut put val i dat i on usi ng m d5 - - FI LE- - <?php $dest = di r nam e( __FI LE__) . ' / bug22544. png' ; @ unl i nk( $dest ) ; i m agePng( i m ageCr eat eTr uecol or ( 640, 100) , $dest ) ; Var _dum p( m d5_f i l e( $dest ) ) ; Hide potential notices ?> using the @ operator. - - CLEAN- - <?php @ unl i nk( di r nam e( __FI LE__) . ' / bug22544. png' ) ; ?> - - EXPECT- - St r i ng( 32) " 10a57d09a2c63f ad87b85b38d6b258d6" Marcus Börger Testing in the PHP World 28
When tests get bigger � The special section = = = DONE= = = ends the test � Only available in --FILE-- � Anything below that will be ignored - - TEST- - M or e Test i ng - - FI LE- - <?php $s = ' 123' ; var _dum p( st r _shuf f l e( $s) ) ; var _dum p( $s) ; ?> With exit() in tests, no ===DO NE=== memleaks get reported. <?php exi t ( 0) ; ?> - - EXPECTF- - st r i ng( 3) " % s" st r i ng( 3) " 123" Marcus Börger Testing in the PHP World 29
Recommend
More recommend
