WP5: Wikipedia Use Case Angelika Mhlbauer (Wikimedia), Kai Nissen - PowerPoint PPT Presentation

WP5: Wikipedia Use Case Angelika Mühlbauer (Wikimedia), Kai Nissen (Wikimedia) 3 nd year review 21 November 2013 Karlsruhe Institute of Technology, Karlsruhe - Germany

Main contributions in Y3 o Evaluation and improvement of the supporting tools • 2 evaluation phases • Improvement of the software components according to evaluation results and feedback comments • Documentation in D5.1.4 21/11/2013 www.render-project.eu 2

Main contributions in Y3 – Article Monitor 21/11/2013 www.render-project.eu 3

Main contributions in Y3 – Article List Generator 21/11/2013 www.render-project.eu 4

Main contributions in Y3 o Analysis of the influences on Wikipedia • Documented in D5.1.3 o Implementation of the Interactive Limes Map • result of a cooperation the German Archaeological Institute, WikiData and the Wikipedia project “ Römischer Limes” 21/11/2013 www.render-project.eu 5

Evaluation of the tools D5.1.4 o Evaluation: In two phases • First phase Feb/Mar 2013 • Final phase Jul/Aug 2013 o First Phase: • 9 participants: 5 Article Monitor, 4 Article List Generator • Thinking Aloud Tests – 3 Tasks • Questionnaires – Assess understandability, benefit, performance, efficiency and further needs → Improvement according to results and feedback 21/11/2013 www.render-project.eu 6

Results of First Evaluation – Article Monitor Component Requirements Priority * Realisation * Article Monitor Renaming of ASQM to Article Monitor high yes Restructuring the result box and renaming sections high yes Parameters are visible if results are available medium yes Statistics Visualisation of editor numbers: Author (+ IP) and medium yes linking to user page Visitors today → Visitors yesterday low yes Visitors last 30 days → Visitors last month Number of authors within a time period low no Analysis Approaches Improvement of description texts for Link Extractor, high yes Change Detector and WikiGini Improve the Link Extractor page: Optical separation of high yes the example section and form section General Performance problems → Moving to Wikimedia Labs high yes * For the final release 21/11/2013 www.render-project.eu 7

Results of First Evaluation – Article List Generator Component Requirements Priority * Realisation * ALG Renaming TLG (Task List Generator) → ALG (Article List low yes Generator) Optimisation for the progress bar medium no Further language versions medium yes No underlines in result list visualisation in WikiSyntax low yes Presentation of number of results high yes Search terms Simpler search term insertions function high yes Auto completion high yes Allow the individual watch list as search terms medium yes Email function Fixing of bugs: no error message/ results if no email address high yes was inserted Filters More precise description texts (tool tips) high yes Additional Filters No links to an article medium yes Pending article (more than 12 hours) medium yes Missing sources medium no Abnormalities in article structure, style of speech and further medium no formal parameters General Performance problems → Moving to Wikimedia Labs high yes * For the final release 21/11/2013 www.render-project.eu 8

Supporting tools o Final versions of tools released at the end of June 2013 o Available at: http://tools.wmflabs.org/render/stools/ o Demo of the Article Monitor and the Article List Generator 21/11/2013 www.render-project.eu 9

Results of Final Evaluation o Final Phase - Jul/ Aug 2013: • Online Questionnaires • Assess understandability, benefit, performance, efficiency, hints to contribute and how, further needs • 25 participants:  16 Article Monitor  9 Article List Generator 21/11/2013 www.render-project.eu 10

Results of Final Evaluation o Article Monitor: • 64 installations between 21.6. and 26.8.2013 • 675 articles requested o Article List Generator: • 447 requests Number requests per filter 160 • 380 DE, 67 EN 140 120 100 80 60 40 20 0 21/11/2013 www.render-project.eu 11

Evaluation Results – Article Monitor Article Monitor: User Ratings (EP1 - EP2) 1 1,5 2 ratings 2,5 3 3,5 4 Understand Benefit Performance Efficiency EP1 EP2 o Better assessment for understandability and benefit in EP2 o Performance problems were caused by initial difficulties of Wiki Labs 21/11/2013 www.render-project.eu 12

Evaluation Results – Article List Generator ALG: User Ratings (EP1 - EP2) 1 1,5 2 ratings 2,5 3 3,5 4 Understand Benefit Performance Efficiency EP1 EP2 o Better assessment for performance and understandability o Assessment of benefit is slightly worse but still positive o Similar assessment results for efficiency in both phases 21/11/2013 www.render-project.eu 13

Evaluation Results - Summary o The supporting tools • provide valuable information for the users for improving the articles • have the potential to become highly productive instruments to support Wikipedia users o Some adaptions could increase the number of users extremely o We identified several points and requirements for the future → Next steps will go close together with community 21/11/2013 www.render-project.eu 14

Influence on Wikipedia D5.1.3 o Methodology of analysis • Logging of requests between 21.06. – 26.08. • Reproduction of requests • Questionnaires: additional questions concerning the rating of the outcomes and their usefulness to contribute In all cases we had to respect privacy aspects! 21/11/2013 www.render-project.eu 15

Influence on Wikipedia o We found first results that the RENDER supporting tools influence the diversity indirectly o Many-Eyes-Principle: By increasing the visibility of articles we increase the quality and diversity 21/11/2013 www.render-project.eu 16

Contribution to the project o Bring diverse information throughout different language versions of Wikipedia articles closer together o Summarizing different metrics about an article or providing suggestions about what may be missing → Attract people to edit in Wikipedia → Keep raising the number of active authors → Wider diversity of authors 21/11/2013 www.render-project.eu 17

Future Steps o Further maintaining and expansion together with community • Integrated versions of the tools • Expansion for further language versions • First discussions for pilot project together for Wikimedia India • WikiCon2013 - 22.- 24.11. in Karlsruhe (Workshop) o Research partners to re-use and individualize tools for further analysis; inclusion of further analysis approaches o All component are open source available on Wikimedia Labs and Github 21/11/2013 www.render-project.eu 18

The Limes Map o Collaboration of RENDER with the German Archaeological Institute, WikiData and the Wikipedia Project “ Römischer Limes” o Symposium “Wikidata trifft Archäologie“, 16 March 2013, Berlin http://tools.wmflabs.org/render-tests/limes/web / 21/11/2013 www.render-project.eu 19

Thank you