Enticing Notification Text & the Impact on Engagement Kieran - PowerPoint PPT Presentation

UbiTtention 2020 Enticing Notification Text & the Impact on Engagement Kieran Fraser & Owen Conlan

Overview 1. Introduction 2. Related Work 3. Information-Gap Theory 4. Capturing Mobile Push-notifications 5. Enticing Push-notifications 6. Push-notification Delivery Agents 7. Capturing Web Push-notifications 8. Enticing Titles vs Summaries 9. Limitations & Future Work 10. Final Thoughts 1

Introduction 2

Related Work Growing number of Notification notifications pushed delivery not smart at users (Pielot, M. (Mehrotra, A. et al, et al, 2014). 2016). Unnecessary Large no. of notifications may incoming dramatically notifications = decrease negative user productivity (Iqbal, S. emotions (Sahami T. et al, 2010). Shirazi, A. et al, 2014). 3

Information-Gap Theory “Curiosity happens when we feel a gap in our knowledge” - Loewenstein Previous “Zone of Something Knowledge or Curiosity” New Experience Curiosity peaks when a small amount of information is revealed, but key parts are withheld 4

Information-Gap Theory John Caples, 1926 WordStream, 2017 5

Capturing Mobile Push-notifications WeAreUs Android App  Experience Sampling Method  Moments of notification interest, moments of phone usage interest  Privacy-preserving text feature 6

Capturing Mobile Push-notifications 35,314 18 participants notifications over 3 months logged 309 7,123 questionnaire smartphone responses usage logs 7

Enticing Push-notifications 8

Enticing Push-notifications Neutral Enticing Notification Notification Unigrams Unigrams 9

Enticing Push-notifications 10

Enticing Push-notifications 11

Enticing Push-notifications Enticing Vs Neutral Notification Click-Through Rate 80 67.51 70 60 48.44 50 CTR (%) 40 30 20 10 0 Neutral Enticing Exact Sign Test: p = .012 12

Push-notification Delivery Agents OpenAI Gym Open source toolkit for “developing and comparing reinforcement learning algorithms” 1 Gym-Push Custom OpenAI Gym environment simulating push-notification overload on mobile device users 1. https://gym.openai.com/ 13

Push-notification Delivery Agents Gym-Push Custom OpenAI Gym environment simulating push-notification overload on mobile device users State Action Privacy-preserving Open / Dismiss Text Embedding the notification 1. https://gym.openai.com/ 14

Push-notification Delivery Agents 15

Push-notification Delivery Agents 16

Capturing Web Push-notifications Subscribed Topics: • Breaking • Entertainment Title • Health • Beauty Content • Style • Science • Sport • Tech • World 17

Capturing Web Push-notifications 18

Enticing Titles vs Summaries 19

Limitations & Future Work Limitations • Granularity of notification text in WeAreUs study • Lack of engagement action in Google Alert data set • Enticing but useful, distinguishing nuance • Offline simulations Future Work • Larger online in-the-wild study • Link toward inherent participant traits e.g. curiosity or personality • How can we train intelligent systems to be better than us 20

Final Thoughts • Click-through Rate larger for notifications with enticing text • Enticement is an important feature for predicting subscriber engagement action • Summarisation has potential to reduce the influence of enticing notification text This research is supported by the ADAPT Centre for Digital Content Technology under the SFI Research Centre Program (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund. 21

Thank you! Email: kieran.fraser@adaptcentre.ie www.adaptcentre.ie 22