L A T EX3 Project Team A Modern Regression Test Suite for T EX Programming Frank Mittelbach, Joseph Wright, Will Robertson 2014-07-28, TUG 2014 Portland, Oregon
Outline History The Needs Approach A Time Line The New Needs Today’s Issues The New System Live Demo
How it began A T add tests for each bug fjx EX 2.09 correctly A T functionality of L EX2 maintains (most) of the typesetting A T ensure L EX Ideas for regression tests for L doc.sty and and later docstrip.tex Literate Programming: — My takeaway from that get into a devilish mindset EX Trip test for T Tangle and Weave Literate Programming: EX add tests for each interface (changed or unchanged) — Don’s approach when developing T
How it began A T add tests for each bug fjx EX 2.09 correctly A T functionality of L EX2 maintains (most) of the typesetting A T ensure L EX Ideas for regression tests for L doc.sty and and later docstrip.tex Literate Programming: — My takeaway from that EX EX add tests for each interface (changed or unchanged) — Don’s approach when developing T ▶ Literate Programming: ▶ Tangle and Weave ▶ Trip test for T ▶ get into a devilish mindset
How it began A T add tests for each bug fjx EX 2.09 correctly A T functionality of L EX2 maintains (most) of the typesetting A T ensure L EX Ideas for regression tests for L — My takeaway from that EX EX add tests for each interface (changed or unchanged) — Don’s approach when developing T ▶ Literate Programming: ▶ Tangle and Weave ▶ Trip test for T ▶ get into a devilish mindset ▶ Literate Programming: ▶ doc.sty and and later docstrip.tex
How it began functionality of L A T EX 2.09 correctly EX A T — My takeaway from that EX EX A T — Don’s approach when developing T ▶ Literate Programming: ▶ Tangle and Weave ▶ Trip test for T ▶ get into a devilish mindset ▶ Literate Programming: ▶ doc.sty and and later docstrip.tex ▶ Ideas for regression tests for L ▶ ensure L EX2 ε maintains (most) of the typesetting ▶ add tests for each bug fjx ▶ add tests for each interface (changed or unchanged)
Excursion on doc and docstrip EX works — Initial ideas ( doc ): Use a format that works both directly (as a L A T EX package) But could also be automatically formatted (with a suitable setup) — Extensions ( docstrip ): Strip out documentation lines to speed up loading Provide features for generating several fjles from one source Provide features for reorganizing code, adding licenses, etc. Provide installation support into difgerent directories — Requirements ▶ It should be easily available ▶ It should work on any platform T
Excursion on doc and docstrip EX works — Initial ideas ( doc ): A T EX package) setup) — Extensions ( docstrip ): Strip out documentation lines to speed up loading Provide features for generating several fjles from one source Provide features for reorganizing code, adding licenses, etc. Provide installation support into difgerent directories — Requirements ▶ It should be easily available ▶ It should work on any platform T ▶ Use a format that works both directly (as a L ▶ But could also be automatically formatted (with a suitable
Excursion on doc and docstrip EX works — Initial ideas ( doc ): A T EX package) setup) — Extensions ( docstrip ): — Requirements ▶ It should be easily available ▶ It should work on any platform T ▶ Use a format that works both directly (as a L ▶ But could also be automatically formatted (with a suitable ▶ Strip out documentation lines to speed up loading ▶ Provide features for generating several fjles from one source ▶ Provide features for reorganizing code, adding licenses, etc. ▶ Provide installation support into difgerent directories
How it continued ( Validating L 2. Exhaustive tests of special environments/modules such as details will be furnished to anyone interested in this task. A procedure for processing validation fjles has been devised; about their status). be fjxed (as well as those that are not fjxed, with comments 3. Bug fjles containing tests of all bugs that are supposed to pictures, etc. tables, displayed equations, theorems, fmoating fjgures, 1. General documents. A T like three kinds of validation fjles: document styles, style options, and environments. We would they were intended to correct; testing interaction with various efgects; making sure that bug fjxes really correct the problem improvements to verify that they don’t have undesirable side Writing test fjles for regression testing: checking bug fjxes and EX 2.09 ) Estimated time required: 2 to 3 weeks, could be divided up.
Outline History The Needs Approach A Time Line The New Needs Today’s Issues The New System Live Demo
What was needed (back then in the ‘90s)? — Verifjcation of coding (interfaces, functionality) of documentation — Assembling a complex distribution unpacking sources fjles and generating production fjles typesetting and verifying documentation adding license information — Installation independence several developers, difgerent OSes, difgerent installations — Full automation as few manual steps as possible
What was needed (back then in the ‘90s)? — Verifjcation — Assembling a complex distribution unpacking sources fjles and generating production fjles typesetting and verifying documentation adding license information — Installation independence several developers, difgerent OSes, difgerent installations — Full automation as few manual steps as possible ▶ of coding (interfaces, functionality) ▶ of documentation
What was needed (back then in the ‘90s)? — Verifjcation — Assembling a complex distribution — Installation independence several developers, difgerent OSes, difgerent installations — Full automation as few manual steps as possible ▶ of coding (interfaces, functionality) ▶ of documentation ▶ unpacking sources fjles and generating production fjles ▶ typesetting and verifying documentation ▶ adding license information
What was needed (back then in the ‘90s)? — Verifjcation — Assembling a complex distribution — Installation independence — Full automation as few manual steps as possible ▶ of coding (interfaces, functionality) ▶ of documentation ▶ unpacking sources fjles and generating production fjles ▶ typesetting and verifying documentation ▶ adding license information ▶ several developers, difgerent OSes, difgerent installations
What was needed (back then in the ‘90s)? — Verifjcation — Assembling a complex distribution — Installation independence — Full automation ▶ of coding (interfaces, functionality) ▶ of documentation ▶ unpacking sources fjles and generating production fjles ▶ typesetting and verifying documentation ▶ adding license information ▶ several developers, difgerent OSes, difgerent installations ▶ as few manual steps as possible
Outline History The Needs Approach A Time Line The New Needs Today’s Issues The New System Live Demo
What to test? & How to test? How do you verify correctness of typography (other than Provide a mechanism to hide irrelevant details during Provide commands that add suitable data to the .log fjle Use verifjed .log fjles for comparison — Approach How do you avoid generating false positives? How do you verify correctness of interfaces? by looking at the .dvi or .pdf )? — Questions EX A T EX code A T comparison — Typical problems with L ▶ Many hidden dependencies ▶ Packages that hook into various layers of L ▶ Packages that overlay/replace macros
What to test? & How to test? by looking at the .dvi or .pdf )? Provide a mechanism to hide irrelevant details during Provide commands that add suitable data to the .log fjle Use verifjed .log fjles for comparison — Approach — Questions EX A T EX code A T comparison — Typical problems with L ▶ Many hidden dependencies ▶ Packages that hook into various layers of L ▶ Packages that overlay/replace macros ▶ How do you verify correctness of typography (other than ▶ How do you verify correctness of interfaces? ▶ How do you avoid generating false positives?
What to test? & How to test? A T — Approach by looking at the .dvi or .pdf )? EX — Questions A T EX code comparison — Typical problems with L ▶ Many hidden dependencies ▶ Packages that hook into various layers of L ▶ Packages that overlay/replace macros ▶ How do you verify correctness of typography (other than ▶ How do you verify correctness of interfaces? ▶ How do you avoid generating false positives? ▶ Use verifjed .log fjles for comparison ▶ Provide commands that add suitable data to the .log fjle ▶ Provide a mechanism to hide irrelevant details during
Output “relevant” data to the .log — In general limit output to a suitable minimum — Use \typeout , \showthe , etc. for “results” — Avoid using \tracingall or other macro expansion tracing settings (like \show\somecommand ) as this displays internal implementation details that we should not be concerned with (normally) — A few \tracing... parameters may be useful, e.g., \tracingparagraphs or \tracingpages — For typesetting verifjcation try \showlists , \showbox or \showoutput but be careful that they do not generate too much output that is diffjcult to verify — In some cases you may end up visually verifying the printed page and then freezing its symbolic representation via \showoutput or \tracingoutput
Recommend
More recommend
