no silver bullet silver buckshot may work
play

"No Silver Bullet? Silver Buckshot May Work" Presented - PDF document

KT3 Keynote 11/10/2011 12:45:00 PM "No Silver Bullet? Silver Buckshot May Work" Presented by: Gregory Pope Law rence Livermore National Laboratory Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888 268


  1. KT3 Keynote 11/10/2011 12:45:00 PM "No Silver Bullet? Silver Buckshot May Work" Presented by: Gregory Pope Law rence Livermore National Laboratory Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888 ‐ 268 ‐ 8770 ∙ 904 ‐ 278 ‐ 0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

  2. Gregory Pope Law rence Livermore National Laboratory With more than forty years of experience developing software in the commercial and government sectors, Gregory Pope currently works for the Lawrence Livermore National Laboratory as a software quality engineering group leader, and verification and validation project leader for advanced simulation. Previously, Greg founded and ran a software testing company, patented automated software testing tools, and held management and technical positions involving mission-critical testing of military systems and development of software code for avionics and aerospace uses. Greg has given industry keynote addresses, written technical papers, taught on software quality internationally, and been a consultant.

  3. 9/28/2011 �������������������������������������� ������������������������������������������ Gregory Pope SQE Group Leader This work performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE$AC52$07NA27344 ��������������� � The proverbial Silver Bullet � Most common request � Definition of Better � Good ideas reincarnate � Some modern challenges � Buckshot – Common problems and solutions �������������������������������������� � LLNL$PRES$493892 $ Draft 1

  4. 9/28/2011 ����������������� The phrase typically appears with an expectation that some new technological development or practice will easily cure a major prevailing problem. �������������������������������������� � LLNL$PRES$493892 $ Draft ���������������������� High Level Language $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ c. 1968 Top Down Structured Programming $$$$$$$$$$$$$$$$$$$$c. 1974 Waterfall Lifecycle $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ c. 1976 DoD Standard 2167A $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ c. 1988 Computer Aided Software Engineering (CASE)$$$$$ c. 1990 Object Oriented Design and Programming $$$$$$$$$$ c. 1992 CMM/CMMI $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ c. 1993 Rational Unified Process $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ c. 1996 Automated Testing Tools $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ c. 1995 Continuous Integration $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ c. 1999 �������������������������������������� � LLNL$PRES$493892 $ Draft 2

  5. 9/28/2011 ��������������������� �������� Increased productivity Less spaghetti code In process error detection, iterative, incremental spins More documentation is not the answer $ RUP, artifacts Frameworks and plug ins $ Eclipse, Team Forge Reusability $ Java, Python, Ruby, C# Regulated vendor minimum competencies, Outsource Expect requirements to change $ Agile Nightly regression testing – Junit, Subversion, QTP Defects detected earlier – CMAKE, BuildBot, Jenkins �������������������������������������� � LLNL$PRES$493892 $ Draft !������"������� What can we do to make our process better? �������������������������������������� � LLNL$PRES$493892 $ Draft 3

  6. 9/28/2011 �������#������ A. Make the software better $$$$$$$$$$$$$$$$$$$$ c. 1970 B. Make the user experience better$$$$$$$$$$$$c. 1990 C. Make the developer experience better$$$$$c. 2005 Better = A & B & C �������������������������������������� � LLNL$PRES$493892 $ Draft ������ � If the improvements do not make the developers life better, they will probably not be easily adopted. Dah � Examples: � Integrated Development Environments (IDEs) � Networking � Static analysis � Continuous integration � Distributed Code repositories �������������������������������������� � LLNL$PRES$493892 $ Draft 4

  7. 9/28/2011 �����!������$�������%�����������&�������'( � Good idea, but we are busy, we will do it later. �������������������������������������� � LLNL$PRES$493892 $ Draft )��*����������+������ � The “land of later” is a mythical place where nothing ever happens. � Why? � Because if you are successful you will be whisked off to a new project. � If you unsuccessful you will be whisked off. �������������������������������������� �� LLNL$PRES$493892 $ Draft 5

  8. 9/28/2011 �������� �,����-����� � Debug and try it � Write unit test drivers and stubs manually � Stand alone framework to write unit tests � Automated tool (Automated Regression Tests to run periodically) � Automated the Automated tool, run unit tests when a code change is detected �������������������������������������� �� LLNL$PRES$493892 $ Draft �������� �!���� ������ � Leave coding styles up to individuals � Ask team to follow a coding standard � Have a plug in tool that formats the code to the correct style (i.e. indents) in the IDE � Have a static analyzer that checks for style violations �������������������������������������� �� LLNL$PRES$493892 $ Draft 6

  9. 9/28/2011 �������� �$����&������ � No peer review of code � Meeting to peer review code � Meeting to discuss findings in code � Collaborative tool to allow code review � Collaborative tool with built$in diff and tracker interface (to requirements and bugs) and CM tool �������������������������������������� �� LLNL$PRES$493892 $ Draft �������� ��������)������� � Compiler, debugging, and testing to find bugs � Static analysis on integrated code � New rules added to reduce false alarms � Static analysis on code as it is built � Automated static analysis on check in and nightly � Automate the automated static analysis emailing only new issues found �������������������������������������� �� LLNL$PRES$493892 $ Draft 7

  10. 9/28/2011 %���������!�����������.������ � Cost to manually test – 4 hours per bug* � Cost of automated test – 1 hour per bug* � Cost of static analysis – 10 minutes per bug** * Cost to design tests (scripts) and execute ** Cost to build and triage code Source: William Oliver LLNL “Quantifying the Value of Static Analysis”, Starwest 2011 �������������������������������������� �� LLNL$PRES$493892 $ Draft �������)������� System All bugs are not 16% structuralLL 15% But some structural bugs cause system, functional, and data Functional Structural 15% bugs. Bugs 27% Bugs 27% 15% Data, Code, Other 30% Source: Boris Beizer Software System Testing and Quality Assurance, page 34 �������������������������������������� �� LLNL$PRES$493892 $ Draft 8

  11. 9/28/2011 �����������##��� � Study from 1927 to 1932 at a Western Electric Company Plant in Cicero, Illinois by Harvard Researcher Elton Mayo. � One reasonable conclusion is that the workers were pleased to receive attention from the researchers who expressed an interest in them. � Any new tool or process can cause process improvement. �������������������������������������� �� LLNL$PRES$493892 $ Draft �����%������������� ���&����������� � Built$in$test for avionics $$$$$$ c. 1972 � Design by Contract � Assertions �������������������������������������� �� LLNL$PRES$493892 $ Draft 9

Recommend


More recommend