Because you cant fix what you dont know is broken About me - PowerPoint PPT Presentation

Because you can’t fix what you don’t know is broken…

About me Initiator of the Automated Error Reporting Initiative • Project Lead of Eclipse Code Recommenders • Eclipse Committer since 2010 • Plug-in Developer for 10+ years (Eclipse 2.1) • Member of the Eclipse Architecture Council • Co-Lead of Java User Group Darmstadt • CEO of Codetrails • marcelbruch @marcelbruch Passion to improve developers day-to-day work with • intelligent and practical tools - mostly using Data Mining on Big (Software Engineering) Data. 2

Adding a new character to the plot { — — – } Product Software 
 Application 
 Manager User Developer 3

Imagine the following situation… { — — – } Product Software 
 Application 
 Manager User Developer 4

But when the user starts the app… 5

This image is not available in your country Can you feel their pain…? 6

In the logs you’d find… 7

The tragedy of manual error reports Release SR 1 SR 2 ! ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ production maintenance 8

The tragedy of manual error reports Release SR 1 SR 2 ! ✘ ✘ ✘ ✘ production maintenance 9

This image is not available in your country Not too happy but feeling okay… 10

? What can we do? { — — – } Application Developer 11

The Automated Error Reporting Initiative 12

Live Demo 13

„Keep users informed about your progress…“ About server responses, 
 Bugzilla, and immediate feedback 14

Systems Overview Eclipse IDEs AERI Web UI Eclipse Bugzilla „Direct Reporter Feedback“ 15

Not yet tracked problems… 16

Committers needinfo… Bug 442500 – …. Status: – – – Resolution: – – – 17

Fixed… Bug 442500 – …. Status: – – – Resolution: – – – 18

Won’t Fix… Bug 442500 – …. Status: – – – Resolution: – – – 19

Invalid (aka log messages) Bug 442500 – …. Status: – – – Resolution: – – – 20

„What goes to eclipse.org?“ About caching, filtering, 
 anonymization, and other shortcuts… 21

really Do we send all errors to Eclipse…? 22

The error reporting funnel ✘ structural filter 23

#1: Structural Filtering Error connecting to ‘ http://marketplace.eclipse.org/rss.xml’ . java.net.UnknownHostException: marketplace.eclipse.org at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184) at java.net.Socket.connect(Socket.java:538) at com.example.HttpClient.openServer(HttpClient.java:527) at com.example.http.HttpClient.<init>(HttpClient.java:211) at com.example.HttpURLConnection.connect(HttpURLConnection.java:932) at org.eclipse.recommenders.internal.news.rcp.PollFeedJob.run(PollFeedJob.java:69) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55) 24

#2: Did I already sent this? ✘ structural filter personal filter 25

#3: Did anyone else sent this? ✘ structural filter personal filter global hist. 26

If you hit a known issue… ✘ structural filter personal filter global hist. 27

#4: Review ✘ structural filter personal filter global hist. review 28

#4: Review 29

Walkthrough through the committer front-end The server side 30

The core concepts 31

The Core Concepts ✘ ✘ ✘ ✘ ✘ ✘ ✘ Incidents ✘ ✘ Problems Projects 32

The committer front-end 33

Creating bug reports 34

Creating bug reports 35

Notifying committers 36

Project Alerts 37

Weekly or daily email digests 38

e = mc 2 —> errors = (more code) 2 Some fun facts 39

Disclaimer Don’t draw preliminary conclusions! Please consult your personal statistican…

How many milestones testers? 450 simrel committers 1200 eclipse committers ~7000 error reporters Testers?

Milestones adoption Do new milestones introduce new problems? 600 560 1000 650 400 450 RC3 RC2 RC1 # reporters one week high M7 M6 220 160 M5 M4 70 M3 M2 10 Mars M1 Luna SR2 SR1 R 42

Milestones adoption 1400 RCs Addition to all EPP packages 1050 700 Christmas Addition to first EPP packages 350 0 4.4.0 4.4.1 4.4.2 M1 M2 M3 M4 M5 M6 M7 RC1 RC2 RC3 43

How hard do they suffer? 1-2 reports 3-50 reports 50-300 reports 300 - 3000 reports 3000 testers 3600 testers 330 testers 25 testers 44

What do they report? 3000 3rd Party Involved Stack Overflows Freezes Out of Memory ‚Normal‘ Errors 2250 1500 Me 750 0 Top 100 of 7000 45

What kind of errors are send? Out Of Memory 1 % Freezes 9 % 'Normal' Errors 74 % Stack Overflows 0 % 3rd Party Involved 16 % 46

What are the most popular exceptions? 18 % 2 % 8 % 3 % 32 % 25 % 12 % 47

What are the most popular exceptions? {File} IOException 18 % ClassCastException 2 % IllegalArgumentException 8 % IndexOutOfBoundsException 3 % Other 32 % NullPointerException 25 % No {Class | Method | Field} DefFoundError 12 % 48

NullPointerExceptions by Projects Statisticians, note: Projects automatically marked as „being responsible“ fixing these NullPointerExceptions but does not mean the NPE happened in their code. 3000 2000 Code Recommenders 1000 0 65 NullPointerException 49

Tip: Enable null analysis @Override protected void run(@Nullable List<Incident> slice) { for (Incident incident : slice) { 50

JDT offers a lot… 51

Tip: Never return null Use java.util.Optional or com.google.common.base.Optional instead @Override public Optional<String> getReason() { return Optional.ofNullable(cause); } 52

Oh Class, Were Art Thou? Use proper import ranges and define uses on your exports to get rid of 
 { Class | Method | Field } DefNotFoundErrors , LinkageErrors , and some ClassCastExceptions . OSGI needs that information to work properly… Import-Package: com.google.common.annotations;version="[15.0.0,16.0.0)", Export-Package: org.eclipse.recommenders.utils; uses:="org.osgi.framework, org.eclipse.core.runtime, com.google.common.base, com.google.common.collect" 53

Index out of bounds? a b c d e -1 and array.length do not work. No matter how often you try… There is no tool support for this. Just unit tests and code that tries to fail gracefully. 54

Tip: If error messages are not helpful… 21% of all logged errors do not contain stack traces. try { // … } catch (Exception e) { log(„An error occurred“, e); } try { // … } catch (Exception e) { log(„Invalid argument {}“, value, e); } 55

Are your users ready for Java 8? Which VM do users run the Eclipse IDE with? Java 1,8 
 64% both 
 2,5% Java 1.7 33% Java 1.9 0,3% 56

Do people anonymize their messages? Are you concerned that your exception message may reveal sensitive data? No 81 % Sometimes 3 % Yes 16 % 57

Speak plain: Does it work? Bugfixing Rates 58

Bugzilla Summary 1170 bugs 467 in progress (40%) 275 fixed (24%) 172 duplicates (15%) 69 works 4 me 100 invalid 28 wont fix 70 not eclipse Bugzilla 59

Fixed bugs by projects Recommenders Xtext 41 42 Papyrus 19 EGit Oomph 15 58 --- 11 More (35) 84 60

In progress & fixed bugs 120 97 90 20 29 60 58 17 42 41 30 19 14 19 15 11 0 Oomph TMF Recommenders Papyrus EGit --- In Progress Fixed 61

Future Work? Feature requests are tracked in bugzilla. Make sure your request is in there and vote for it. Client: • Separate view to „review, analyze and send later“ • Making client reusable in other (Eclipse- or SWT-based) applications • Additional client-side automated analysis Server: • Ensure scalability of the system • Improve duplicate detections • Go Neon if feedback is good. Hosting an error reporting service for Java software? • Need one? TTML. 62

How to reach EPP Logging • Code: Check out the EPP Logging Oomph profile • Wiki : https://wiki.eclipse.org/EPP/Logging • Manual: https://dev.eclipse.org/recommenders/community/confess/#/about • Forum: https://www.eclipse.org/forums/index.php/f/69/ • Bugzilla: https://bugs.eclipse.org/bugs/buglist.cgi?component=logging&product=EPP • Mail: error-reports-inbox@eclipse.org 63

Credits • Daniel Haftstein, Codetrails – Eclipse client conception and implementation • Marcel Bruch, Codetrails – Server-side conception and implementation • Webmasters, Eclipse Foundation – Eclipse server infrastructure & integration • Thanks to many early adopters and committers providing feedback 64

Feedback 65

Contact Us Codetrails GmbH Dr. Marcel Bruch Robert-Bosch-Str. 7 CEO 64293 Darmstadt, Germany www: codetrails.com phone: 06151 / 2767 092 mobile: 0179 / 1317 721 email: info@codetrails.com email: marcel.bruch@codetrails.com 66