Controversy in Web Data Todays schedule 09:00 - 09:10 Coffee & - PowerPoint PPT Presentation

Controversy in Web Data

Today’s schedule 09:00 - 09:10 Coffee & Introduction & Chair by Lora Aroyo, Web & Media group, VU Computer Science 09:10 - 09:20: Understanding Controversy Using Collective Intelligence - Benjamin Timmermans, VU Computer Science 09:20 - 09:30: Detecting Controversies in Online News Media - Kaspar Beelen, UvA Faculty of Humanities 09:30 - 09:45: Crowdynews deploying ControCurator - Gerben van Eerten, CrowdyNews 09:45 - 10:00: Identifying High-quality Opinions on the Web - Davide Ceolin, VU Computer Science 10:00 - 10:15: Political Filter Bubbles - Damian Trilling, UvA Faculty of Social and Behavioural Sciences 10:15 - 10:30: Making & Breaking Filter Bubbles - Daan Odijk, Blendle 10:30 - 10:45: Skewing the data - Andy Tanenbaum 10:45 - 11:00: Q&A + Coffee

What is Controversy? Controversy noun | con·tro·ver·sy | \ ˈ kän-tr ə - ˌ v ə r-sē An argument that involves many people who strongly disagree about something Merriam-Webster Prolonged public disagreement or heated discussion Oxford English Dictionary When people often disagree and debate with opposing viewpoints Mejova, Y., Zhang, A. X., Diakopoulos, N., & Castillo, C. (2014).

Why this is relevant Unbiased source for journalists, government ● Preventing filter bubbles ● Identify fake news ● Fact checking explorations ● Understand how issues develop ●

Vaccination Pro Against Vaccines save lives Vaccines have side effects ● ● Ingredients are safe Ingredients are harmful ● ● Eradicated smallpox and polio Freedom of choice ● ● Vaccines prevent medical cost It is unnatural ● ●

Climate Change Pro Against Humans are responsible Humans are not responsible ● ● Greenhouse effect Natural fluctuation ● ● CO2 rising rate unnatural Not link with CO2 rise ● ● No change in Sun heat Caused by Sun activity ● ●

What are the challenges How to explain controversy scores ● How to compare different controversial documents ● How to present, visualize & interact with controversy ● How to understand the utility of knowing something is controversial ● How to combine methods & sources ● How to identify the influence & relation to quality, varsity, sentiment ●

ControCurator

ControCurator.org

Goal of ControCurator Enable the discovery and understanding of controversial issues and events by bringing together humans & machine in active learning workflows

The Five Aspects of Controversy Actors → many participating actors Polarity → clustered opposing viewpoints Openness → plays out in an open public space Time-Persistence → persists over longer stretches of time Emotion → triggers emotional responses

Dataset Issues: 12k Guardian articles ● Discussions: 700k related comments ● Comments Trump seeking quickest way to quit Paris climate agreement, says report “Forget WW3, dumping the Paris agreement will be the equivalent of Donald Trump is looking at quick ways of withdrawing from the Paris climate agreement in defiance declaring war on humanity.” of widening international backing for the plan to cut greenhouse gas emissions, Reuters has “If true then @realDonaldTrump is an idiot because #climatechange is for reported. real and we need #renewableenergy over #fossilfuels” Since the US president-elect was chosen, governments ranging from China to small island states “It will be a countdown to see whether Trump destroys humanity via climate have reaffirmed support for the 2015 Paris agreement at 200-nation climate talks running until 18 change or thermonuclear war.” November in Marrakesh, Morocco. “I always suspected Donald Trump would end humanity if he became president, I never thought he would find a way to do it quite so quickly” The Guardian - November 13th 2016 “Absolutely horrifying. Have your day defying the "elites", people, but nobody can deny reality."

Crowdsourcing Task Input Data: 5k Guardian articles + 5 representative comments per article ● Targets the five controversy aspects ● 6 annotators per article ● $ 0.01 reward ●

Results 30k annotations, 518 annotators ● Emotion and polarity are strongest indicators of controversy ● Significant positive effect (adjusted R 2 of 59%) ● Emotion: 10% ○ Polarity: 5% ○ Other: < 2% ○

ControCurator Barometer controcurator.org/browse

Summary Identifying controversy is difficult because multiple aspects affect it ● It is not feasible for a single annotator to fully review a discussion ● All aspects were found to contribute to controversy ● ControCurator.org