Reuse and Beyond: Innovative Software Retrieval Approaches Oliver - PowerPoint PPT Presentation

Palladio Days 2012 Reuse and Beyond: Innovative Software Retrieval Approaches Oliver Hummel SOFTWARE DESIGN AND QUALITY GROUP sdq.ipd.kit.edu INSTITUTE FOR PROGRAM STRUCTURES AND DATA ORGANIZATION, FACULTY OF INFORMATICS KIT – University of the State of Baden-Wuerttemberg and www.kit.edu National Research Center of the Helmholtz Association

Three Problems I ssdfsd „I was having three problems: No job, no money, and no clue how to continue. Otto – The Movie (1986) Photo: [Andreas Reiner, Wikipedia] Software Retrieval Beyond Reuse Software Design and Quality Group 2 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

Three Problems II “Software reuse is the process of creating software systems from existing software rather than building software systems from scratch.” [Krueger 92] “A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third parties .” [Szyperski 02] „[…] And there they were again, my three problems …“ � No repository structure, no reusable software, and no clue how to retrieve something. The Reuse Community (1999) Software Retrieval Beyond Reuse Software Design and Quality Group 3 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

Overview I’m going to share some insights on software search & retrieval with you – Using the web as a reuse repository 1. How to build a software search engine with Lucene 2. and how to use it • How to add functional semantics to software searches 3. A Word on proactive reuse recommendations in Eclipse 4. Can we utilize the wisdom of the crowd in software development? 5. Outlook, summary and conclusion 6. Software Retrieval Beyond Reuse Software Design and Quality Group 4 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

Solution Idea I Using the Web (and Google ) as a Reuse Repository (2006) with some additional keywords Stack filetype:java „class stack“ „void push“ +push(o:Object):void +pop():Object „Object pop“ + simple, easy to use + no overhead + precise for simple requests - still too imprecise - „dark web“ not covered anymore - no reliable API for IDE integration Software Retrieval Beyond Reuse Software Design and Quality Group 5 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

Solution Idea II Build a specialized software search engine (2008) with google-like interface and IDE integration and suport for programming languages, signatures, interfaces etc. Use Database or Lucene or both? relational model vs. field-based approach � „De-Normalizing“ allows effective software searches in Lucene name stack Stack method push method pop msign 1_pt:object_rt:void +push(o:Object):void msign 1_pt:void_rt:object +pop():Object minter 1_mn:push_pt:object_rt:void minter 1_mn:pop_pt:void_rt:object 10 million files (~40 GB) in < 3 seconds with .com Software Retrieval Beyond Reuse Software Design and Quality Group 6 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

Simple Applications What can we do with such a software search engine? Calculator lang:java Keyword Searches method:push Name Matching $(float,float):float; Signature Matching isLeapYear(int):boolean; Interface Matching java.util.Stack Open Source Lookup findjar:org.apache.lucene.search.Query Library Identification Software Retrieval Beyond Reuse Software Design and Quality Group 7 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

Simple Applications II Searches for full object interfaces are also possible … even in Java/C# syntax public class Matrix { public Matrix add(Matrix) {} public Matrix multiply(Matrix) {} } Only test cases „refuse“ to deliver test cases..? import junit.framework.TestCase; public class CalculatorTest extends TestCase { public void testAdd() { Calculator calc = new Calculator(); assertEquals(calc.add(2, 1), 3); } } Software Retrieval Beyond Reuse Software Design and Quality Group 8 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

Precision from a Reuse Perspective Preliminary comparison of retrieval approaches has shown limited top 25 precision results for 13 method queries � even a full syntax matching is not precise enough how can we support semantics in searches? Software Retrieval Beyond Reuse Software Design and Quality Group 9 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

Semantic Software Retrieval [Hummel, 2008] And what about „contractually specified interfaces“? and a seamless integration in modern development processes and tools? test-driven development � Test-Driven Reuse � candidate selection based on interface of C.U.T. � assessment of semantics based on test cases � very precise � more candidates by ignoring names in interface � expensive, but still precise Software Retrieval Beyond Reuse Software Design and Quality Group 10 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

Success Examples & Discussion [Hummel, 2008] Software Retrieval Beyond Reuse Software Design and Quality Group 11 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

Eclipse Integration Precise reuse recommendations on the fly… [CodeConjurer.org] Software Retrieval Beyond Reuse Software Design and Quality Group 12 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

The Wisdom of the Crowd Book Recommendations @ Amazon API Recommendations with ParseWeb (Thummalapenta & Xie, 2007) Software Retrieval Beyond Reuse Software Design and Quality Group 13 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

The Wisdom of the Crowd „Developers who have coded a Stack have implemented a push, pop and isEmpty method…“ � Towards crowd-based design recommendations (2010) via intersecting interfaces of search result or attributes or dependencies etc. Software Retrieval Beyond Reuse Software Design and Quality Group 14 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

Ongoing & Future Work Component recognition tracing dependencies -> e.g. delivered a full-grown FTP Client goal is to get this into the Merobase index [Kakarontzas & Stamelos] have been working on Facade detection in Open SME EU project [Janjic & Atkinson subm.] Test case recommendation Software Retrieval Beyond Reuse Software Design and Quality Group 15 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

Conclusion Numerous applications for software retrieval beyond reuse Code Recommendations Design Recommendations explorative searches Snippet Reuse Component Reuse concrete searches Library Reuse Back-to-Back Testing Test Recommendation Libray Lookup Open Source Lookup Defect information Impact Analysis Program Understanding [Janjic et al., 2010] Software Retrieval Beyond Reuse Software Design and Quality Group 16 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization

The End. Thank you for your attention! Time for some questions? … or maybe you want to contact me later Software Retrieval Beyond Reuse Software Design and Quality Group 17 2012-10-16 Jun.-Prof. Dr. Oliver Hummel Institute for Program Structures and Data Organization