The Data Interaction Game Ben McCamish, Vahid Ghadakchi, Arash - PowerPoint PPT Presentation

The Data Interaction Game Ben McCamish, Vahid Ghadakchi, Arash Termehchy, Behrouz Touri, Liang Huang I nformation & D ata Manag e ment and A nalytics Laboratory (IDEA) 1

The User and the Database Grades First_Name Last_Name Dept. Grade … … … … Sarah Smith CE A John Smith EE B Kerry Smith CS D … … … … • Users wish to find information from the database. 2

The intent is what the user is looking for in the database Intents they wish to find Grades Kerry Smith First_Name Last_Name Dept. Grade in CS … … … … Sarah Smith CE A John Smith EE B Kerry Smith CS D … … … … • The user wishes to find Kerry Smith from the CS department in the database 3

Intents are expressed using queries Use Queries to Grades express intents Kerry Smith First_Name Last_Name Dept. Grade in CS SELECT * … … … … WHERE Sarah Smith CE A First_Name=‘Kerry’ and Last_Name=‘Smith’ and John Smith EE B Dept.=‘CS’ Kerry Smith CS D … … … … • The user expresses their intent with a SQL query 4

Most users do not know structure and content of database or SQL Intents they wish to find Use Queries to Grades express intents Kerry Smith First_Name Last_Name Dept. Grade in CS SELECT * … … … … WHERE Sarah Smith CE A First_Name=‘Kerry’ and Last_Name=‘Smith’ and John Smith EE B Dept.=‘CS’ Kerry Smith CS D … … … … • Normal users such as scientists prefer to use keyword queries 5

Users prefer to use keyword queries as they are easier to use Intents they wish to find Use Queries to Grades express intents Kerry Smith First_Name Last_Name Dept. Grade in CS … … … … Smith Sarah Smith CE A John Smith EE B Kerry Smith CS D … … … … • Don’t need to know the structure or content of the database • No need to know SQL or other structured query language 6

Database struggles with keyword queries Intents they wish to find Use Queries to Grades express intents Kerry Smith First_Name Last_Name Dept. Grade in CS … … … … Smith Sarah Smith CE A John Smith EE B Kerry Smith CS D … … … … Results First_Name Last_Name Dept. Grade • Since keyword queries are Sarah Smith CE A imprecise, database system struggles to satisfy the user John Smith EE B 7

Users learn by interacting with database systems Grades Kerry Smith First_Name Last_Name Dept. Grade in CS … … … … Smith CS Sarah Smith CE A John Smith EE B Kerry Smith CS D … … … … Results First_Name Last_Name Dept. Grade • Learning and reformulating query allowed the user to find Kerry Smith CS D the desired student 8

Database system can also learn from interactions Grades Kerry Smith in CS First_Name Last_Name Dept. Grade … … … … Smith Sarah Smith CE A John Smith EE B Kerry Smith CS D … … … … • Results User gives feedback to database through clicks First_Name Last_Name Dept. Grade • Kerry Smith CS D Database system has learned to return Kerry Smith in CS department 9

Naturally data interaction is a game between two rational agents Kerry • Two Players: user and database system • Agents learn and adapt • Final goal: user to get desired information ‣ Database system understands the intent behind users queries ‣ User expresses intent in a way that DBMS understands • User Strategy: How intents are expressed using queries • DBMS Strategy: How to map imprecise queries to desired queries • Payo ff : The amount of desired information the user receives. 10

User thinks of what they want to find in DBMS Intent # Intent Sarah e 1 John Smith in EE e 2 Sarah Smith in CE e 3 Kerry Smith in CS Query # Query ? Grades q 1 “Smith CE” First_Name Last_Name Dept. Grade q 2 “Smith” … … … … Sarah Smith CE A • The intent can be multiple tuples John Smith EE B • They need to decide how to Kerry Smith CS D express their intent to DBMS … … … … 11

User strategy is mapping of intents to queries User Strategy Intent # Intent Sarah q 1 q 2 e 1 John Smith in EE e 1 0 1 e 2 Sarah Smith in CE e 2 0.5 0.5 e 3 Kerry Smith in CS e 3 0 1 Query # Query Grades q 1 “Smith CE” First_Name Last_Name Dept. Grade q 2 “Smith” … … … … • Use keyword queries Sarah Smith CE A John Smith EE B • Row-stochastic mapping Kerry Smith CS D from intents to queries. … … … … 12

User may use a single query for multiple intents User Strategy Intent # Intent Sarah q 1 q 2 e 1 John Smith in EE e 1 0 1 e 2 Sarah Smith in CE e 2 0.5 0.5 e 3 Kerry Smith in CS e 3 0 1 Query # Query Grades q 1 “Smith CE” First_Name Last_Name Dept. Grade q 2 “Smith” … … … … • Due to the lack of knowledge, Sarah Smith CE A saving time, … John Smith EE B • Makes it hard to interpret the Kerry Smith CS D exact intent behind the query. … … … … 13

DBMS receives query and needs to decide what user wants Query # Query q 1 “Smith CE” q 2 “Smith” Intent # Intent e 1 ans(y) ← Grades(x,’Smith’, ‘EE’, y) ? Grades e 2 ans(y) ← Grades(x,’Smith’, ‘CE’, y) First_Name Last_Name Dept. Grade e 3 ans(y) ← Grades(x,’Smith’, ‘CS’, y) … … … … Sarah Smith CE A • How should it map the John Smith EE B received keyword queries Kerry Smith CS D to the user’s actual information needs? … … … … 14

DBMS receives query and needs to decide what user wants Database System Strategy Query # Query e 1 e 2 e 3 q 1 “Smith CE” q 1 0 1 0 q 2 “Smith” q 2 0.5 0 0.5 Intent # Intent Sarah Smith in CE e 1 ans(y) ← Grades(x,’Smith’, ‘EE’, y) Grades e 2 ans(y) ← Grades(x,’Smith’, ‘CE’, y) First_Name Last_Name Dept. Grade e 3 ans(y) ← Grades(x,’Smith’, ‘CS’, y) … … … … Sarah Smith CE A John Smith EE B • Row-stochastic mapping Kerry Smith CS D from queries to intents … … … … 15

Payoff: expected effectiveness of communicating every intent User Strategy Intent # Intent q 1 q 2 e 1 John Smith in EE e 1 0 1 e 2 Sarah Smith in CE e 2 1 0 e 3 Kerry Smith in CS e 3 0 1 Query # Query Database Strategy q 1 “Smith CE” e 1 e 2 e 3 q 2 “Smith” q 1 0 1 0 q 2 0.5 0 0.5 m n o X X X r ( U, D ) = D j ` prec ( e i , e ` ) U ij π i • Prior on how often intents i =1 j =1 ` =1 are queried for 16

Payoff: expected effectiveness of communicating every intent User Strategy Intent # Intent q 1 q 2 e 1 John Smith in EE e 1 0 1 e 2 Sarah Smith in CE e 2 1 0 e 3 Kerry Smith in CS e 3 0 1 Query # Query Database Strategy q 1 “Smith CE” e 1 e 2 e 3 q 2 “Smith” q 1 0 1 0 q 2 0.5 0 0.5 m n o • Computed using user X X X r ( U, D ) = D j ` prec ( e i , e ` ) U ij π i feedback, such as clicks i =1 j =1 ` =1 • Any user satisfaction metric can be used 17

What algorithms model user learning? • Research in psychology and empirical game theory shows that humans exhibit reinforcement learning behavior • Components of reinforcement learning: ‣ Select a query based on its past success, i.e., exploitation. ‣ Explore and try new/ less successful queries to gain new knowledge, i.e., exploration. ‣ Sacrifice immediate success for more success in the long run. 18

We evaluate user learning using human learning algorithms from empirical game theory. • These algorithms generally di ff er in ‣ How much they use past interactions ✦ Short-Term Memory - Only remembers most recent interaction ✦ Long-Term Memory - Remembers all of previous interactions ‣ The degree of exploration versus exploitation ‣ Reinforcement formula: e.g., use payo ff versus discounted payo ff . 19

Empirical evaluation of user learning methods • Dataset ‣ Yahoo! interaction history of ~200,000 interactions (101 hours) ‣ Each interaction record contains: Query entered, Timestamp, User ID, Returned urls, which results were clicked, and which clicks are not noise. ‣ It can model database users as our users do not know the schema. • Experiment Design ‣ Train and test the algorithms on how accurately they predict what the user will do next, given the previous interactions 20

Roth and Erevs Method closely resembles user learning • Reinforces a query based on its payo ff . • Picks a query randomly to express an intent with a probability proportional to its accumulated success (exploration) Method Mean Squared Error Win-Stay/Lose-Randomize 0.0713 Latest Reward 0.3421 Bush and Mosteller’s 0.0673 Cross’s 0.0686 Roth and Erev 0.0666 Roth and Erev Modified 0.0666 UCB-1 0.1624 21

Roth and Erevs Method closely resembles user learning • As it picks queries randomly, it may use new/ less frequently used queries once in a while ( exploration). Method Mean Squared Error Win-Stay/Lose-Randomize 0.0713 Latest Reward 0.3421 Bush and Mosteller’s 0.0673 Cross’s 0.0686 Roth and Erev 0.0666 Roth and Erev Modified 0.0666 UCB-1 0.1624 22