Recommendation Systems Recommendation systems aim to identify items - PowerPoint PPT Presentation

Improving Package Recommendations through Query Relaxation M A T T E O B R U C A T O U N I V E R S I T Y O F M A S S A C H U S E T T S , A M H E R S T , U S A A Z Z A A B O U Z I E D N E W Y O R K U N I V E R S I T Y , A B U D H A B I , U A E A L E X A N D R A M E L I O U U N I V E R S I T Y O F M A S S A C H U S E T T S , A M H E R S T , U S A P R E S E N T E D B Y : MATTEO BRUCATO m a t t e o @ c s . u m a s s . e d u

Recommendation Systems — Recommendation systems aim to identify items of interest to users Recommended to me by Amazon before traveling to Hangzhou: 1 Improving Package Recommendations through Query Relaxation

“Package” Recommendations — But sometimes items are actually bundled together in packages of items Example 1 — Amazon bundles 2 Improving Package Recommendations through Query Relaxation

“Package” Recommendations — But sometimes items are actually bundled together in packages of items Example 2 — A flight package: * 3 * Recommended by Google Flights Improving Package Recommendations through Query Relaxation

“Package” Recommendations — But sometimes items are actually bundled together in packages of items Example 3 — A meal plan: 4 Improving Package Recommendations through Query Relaxation

A “Package” Query — All recipes should have less than 25 g of fat — The entire meal plan should have: ¡ At least 1700 kcal in total ¡ Between 3 and 5 meals per day — The meal plan should minimize the total preparation time 5 Improving Package Recommendations through Query Relaxation

Many Feasible Solutions… 6 Improving Package Recommendations through Query Relaxation

Too Many Feasible Solutions… 1704 feasible meal plans, with only 15 recipes 7 Improving Package Recommendations through Query Relaxation

A New “Big Data” Challenge — Usually we talk about: ¡ Lots of data ¡ Lots of features — But what about: ¡ More combinations! — Practical challenges of “more combinations”: ¡ Computational complexity ¡ Usability 8 Improving Package Recommendations through Query Relaxation

What Could Systems Do? Query • ≥ 1700 kcal in total • Minimal prep. time Top-1 meal plan 30 30 571 571 25 25 298 298 417 417 50 50 424 424 80 80 1,710 kcal 3 hrs 5 min The dietitian might be willing to accept lower calories for lower preparation time 9 Improving Package Recommendations through Query Relaxation

What Could Systems Do? Top-1 meal plan 30 30 571 571 298 298 25 25 50 50 417 417 Out 424 424 80 80 1,710 kcal 3 hrs 5 min Infeasible, but perhaps better than top-1 1 hr less! 571 571 30 30 298 298 25 25 417 417 50 50 60 60 In 364 364 16.2 16.2 21 21 10.7 10.7 20 20 1,650 < 1,700 kcal 2 hrs 5 min 10 Improving Package Recommendations through Query Relaxation

Our Approach — We propose a new use of query relaxation: — Usually we relax when: ¡ The query does not produce any result ¡ The query does not produce enough results — Here, we relax to: ¡ Improve upon some aspect of the query result 11 Improving Package Recommendations through Query Relaxation

Package Query Relaxations — What is a relaxation of a package query? Package Query • Base Constraints – Each meal: Fine-grained • ≤ 25 g of fat in each meal 30 Relaxation • Global Constraints – The meal plan: Constraints • ≥ 1700 kcal in total • Cardinality Constraints : Coarse-grained • 3 to 5 meals Relaxation Objective • Minimal preparation time 12 Improving Package Recommendations through Query Relaxation

Criteria for Relaxations — Relaxations modify the query and thus produce a different result than the original query — How do we pick a good relaxation? ¡ Relaxations should improve the result ÷ In some aspects specified by the query ÷ As much as possible ¡ But they may cause some error ÷ The total error should be as low as possible 13 Improving Package Recommendations through Query Relaxation

Impact of Coarse Relaxations — How much should we relax? 10 5 Diminishing returns 10 4 % change Relaxing only a 10 3 few constraints provides the 10 2 highest impact 10 1 Improvement Error 0 0 20 40 60 80 100 % relaxation More relaxed 14 Improving Package Recommendations through Query Relaxation

Review — Summary so far: ¡ Package recommendations ¡ Package query relaxations ¡ Relaxation trade-off — Rest of the talk: ¡ How do users react to relaxations? user study ¡ Lessons and future work 15 Improving Package Recommendations through Query Relaxation

How do users react to relaxations? — Two Research Questions: ① Are users willing to accept relaxations? ② Do they have preferences regarding the type of constraints to be removed? Let’s ask the crowd! 16 Improving Package Recommendations through Query Relaxation

Dataset Description — Dataset ¡ 7,955 (arguably) tasty recipes extracted from allrecipes.com 17 Improving Package Recommendations through Query Relaxation

Task Instructions We automatically generated 50 unique task configurations: Cardinality Constraint Objective 2 Base Constraints We varied these 4 constraints 2 Global Constraints 18 Improving Package Recommendations through Query Relaxation

Task Choices — For each of the 50 configurations, we showed 5 different meal plans, each removing one constraint only: 1. O RIGINAL No Relaxation Removing the 2. C ARDINALITY R ELAX cardinality constraint Removing one base 3. B ASE R ELAX constraint Removing one 4. G LOBAL R ELAX global constraint 5. R ANDOM A random package — We used colors to indicate constraints adherence or violation — Results were presented sorted by preparation time 19 Improving Package Recommendations through Query Relaxation

Task Screenshots G LOBAL R ELAX Objective is highlighted Global constraint violation and amount of violation 20 Improving Package Recommendations through Query Relaxation

Task Screenshots C ARDINALITY R ELAX Objective got worse 21 Improving Package Recommendations through Query Relaxation

Task Screenshots B ASE R ELAX 22 Improving Package Recommendations through Query Relaxation

Task Screenshots O RIGINAL Objective got even worse! 23 Improving Package Recommendations through Query Relaxation

Collected Data — Run on crowdflower.com — Each configuration completed by 10 unique workers — No worker allowed to complete more than 5 configurations — We removed obvious spammers a posteriori: ¡ Same explanations in every task ¡ Random explanations ¡ Inconsistent explanations — Resulting in 115 unique workers and 306 unique task instances 24 Improving Package Recommendations through Query Relaxation

Evaluation ① Are users willing to accept relaxations? ① Are users willing to accept relaxations? ② Do they have preferences regarding the ② Do they have preferences regarding the type of constraints to be removed? type of constraints to be removed? — The O RIGINAL plan was rejected 30% of the time We need to provide users with alternatives! 25 Improving Package Recommendations through Query Relaxation

Evaluation ① Are users willing to accept relaxations? ① Are users willing to accept relaxations? ② Do they have preferences regarding the ② Do they have preferences regarding the type of constraints to be removed? type of constraints to be removed? — Relaxed plans were chosen 76% of the time More likely to choose a relaxed plan than the original! ■ When O RIGINAL is recommended ■ When O RIGINAL is not recommended 70% 91% 26 Improving Package Recommendations through Query Relaxation

Evaluation ① Are users willing to accept relaxations? ① Are users willing to accept relaxations? ② Do they have preferences regarding the ② Do they have preferences regarding the type of constraints to be removed? type of constraints to be removed? ■ Overall ■ When O RIGINAL is recommended ■ When O RIGINAL is not recommended B ASE R ELAX G LOBAL R ELAX C ARDINALITY R ELAX 27 Improving Package Recommendations through Query Relaxation

Why Relaxations? — Lower preparation time was often the reason: 28 Improving Package Recommendations through Query Relaxation

Additional Lessons — Good explanations for the bias toward B ASE R ELAX : (The plans had to contain 4 meals) The workers relaxed base constraints by transforming them into global constraints! 29 Improving Package Recommendations through Query Relaxation

Future Work — What dictates user’s sensitivity toward different kinds of constraints? — Impact of fine-grained relaxations — Reverse relaxations ¡ Tightening the constraints — Additional relaxation methods ¡ Including the type of relaxation workers spontaneously applied 30 Improving Package Recommendations through Query Relaxation

Summary of Contributions — Novel application of query relaxation 10 5 10 4 % change 10 3 10 2 — Impact of coarse relaxations 10 1 Improvement Error 0 0 20 40 60 80 100 % relaxation — User reaction to package relaxations Thank you! 31 Improving Package Recommendations through Query Relaxation