Deep Learning for Semantic Search in E-commerce Somnath Banerjee - PowerPoint PPT Presentation

Deep Learning for Semantic Search in E-commerce Somnath Banerjee Head of Search Algorithms at Walmart Labs https://www.linkedin.com/in/somnath-banerjee/ March 19, 2019

Walmart E-commerce search problem E-commerce Search Store Associate provides the functionality of a human but at scale 100M+ 100M+ 100M+ Items Customers Queries 2

???? Flash Drive Miss isspelled Qu Queries USB Drive Flush Drive Thumb Drive USC Drive Jump Drive Thamb Drive Pen Drive Jmp Drive Zip Drive Pin Drive Memory Stick Zap Drive USB Stick Memory Steak USB Flash Drive USB Stock USB Memory USB Flash Drve USB Storage Device 3

Horse Saddle saddle Bike Saddle 4

Outline • Core problems of e-commerce search • Semantic search in e-commerce • Deep Learning for semantic search – Query classification – Query token tagging – Neural IR – Image understanding (sneak peek) 5

Core of E-commerce Search Text Query Text Find Items Query Catalog 6

Core problems of E-commerce search Learning book Ziploc Tide 100 oz Ambiguity Tide 100 fl oz Tide 100 ounce Neck style? Levi’s Fabric? Levi Strauss No. of pockets? Signature by Levi Strauss and Co. Open vocabulary in query and catalog Missing catalog values 7

Buying decision is influenced by item attractiveness pump shoes $300!!! Image quality Presence of expensive items Tags 8

Core technical problems of e-commerce search Matching Ranking query to items items ✅ Position 1 Pump shoes ❌ Position 2 ✅ 9

Text matching is not enough Nivea 16oz Lemon Tire sealant Nivea 15.5oz Lemon Fruit Lemon Balm 16oz 10

Sematic Search Query understanding Matching query and item • Attribute understanding Ranking Items • Text matching • Attribute matching 11

Deep learning for semantic search Matching query and item Deep Learning for Ranking Items Query understanding • Text matching • Attribute matching 12

Deep learning for semantic search Matching query and item Deep Learning for Ranking Items Query understanding • Text matching • Attribute matching Neural IR End-to-end matching and ranking Image understanding Not just text search 13

Outline • Core problems of e-commerce search • Semantic search in e-commerce • Deep Learning for semantic search – Query classification – Query token tagging – Neural IR – Image understanding (sneak peek) 14

Query Classification product type 1 : confidence level product type 2 : confidence level Text query product type 3 : confidence level Product Type • A predefined list • Indicates a specific product in the catalog • Every item in the catalog is tagged with a product type 15

Query classification examples Food Storage Bags: 1.0 ziploc bags Computer Video Cards : 0.85 nvidia gpu Laptop Computers: 0.08 Desktop Computers: 0.06 Hard to balance Large number of product bedroom furniture types precision vs recall 16

Query classification challenges Large scale Multi-class, multi- Needs to respond Unbalanced class Short text classification label problem in few milliseconds distribution • Queries are of 2-3 • Thousands of • Same query can • Classifies queries • Some product tokens product types have multiple at runtime types are much (classes) product types more popular 17

Softmax/sigmoid Data and Model Output Layer <query, product type ordered> <query, item ordered> word2vec Historical Search Log BiLSTM https://guillaumegenthial.github.io/sequence-tagging-with-tensorflow.html 18

Usage of query classification lemon 20% reduction of irrelevant items in certain query segments Without Query Classification After we understand the query “ lemon ” as a fruit 19

Key Learnings Accuracy Training Time Logistic 8 Core CPU Regression Deep 1 K80 GPU Learning 6X 6% faster higher accuracy CNN 1 K80 GPU 48 Core CPU BiLSTM Equal More accurate 20

Key Learnings - instability in Prediction Hard Drives : 1.00 Old model samsung 850 evo 250gb 2.5 inch Television Stands : 0.32; Laptop Computers : 0.27 Hard Drives : 0.11 New model 21

Instability in Prediction 2.5% difference Training data (N + 1) Training data N in training data Model (N + 1) Model N Top predicted class is different for 10% of the test set 22

Instability in prediction – different seeds Training data N Different tensorflow and numpy seeds Seed 1 Seed 2 Model N Model N’ Top predicted class is different for 7% of the test set 23

Sources of Instability Overfitting Sigmoid (1-vs-all) classifier is more unstable • Deep Learning model has high variance, particularly on the low • Softmax scores are traffic queries interdependent across classes • Simpler models could be more and less stable stable but less accurate Noisy training data Rounding errors in the arithmetic operations • Item order data is less noisy than click • CPU is more stable than GPU 24

Reduction of Instability Softmax Sigmoid Clicks Orders CNN BiLSTM 40% reduction of instability 25

Attributes to match • Product Type • Brand Not ot Fea easible le – Sep eparate cla classifie ier for or • Color each attri ea tribute • Gender • Too many classes (e.g. 100K+ brand values) • Age Group • Size (value & unit) • Sparse attributes; most attribute prediction – Pack Size should be NA – Screen Size – Shoe Size • Creating training data of <query, attribute> is – … more noisy and inaccurate • Character • Style • Material • … 26

Query token tagging Query tokens tagged with Query Attribute Names Brand Faded Glory Sleeve length Long sleeve faded glory long sleeve shirts for women shirts Product type for NULL women Gender 27

Training data Human curated data blue women levis jeans It is a hard task for human Color Gender Product Brand Type • Is "outside ” a product type token in the query, “ canopy tents for outside ”? toys for girls 3 – 6 years Disagreement between taggers are high (~30%) Age Product Gender Age Unit Type Value Fortunately 10K training data is a good start 28

Model – BiLSTM-CRF 𝑄(𝑢𝑏𝑕 1 , … , 𝑢𝑏𝑕 𝑜 ) Linear Chain CRF Features for CRF Char embedding word2vec query tokens https://guillaumegenthial.github.io/sequence-tagging-with-tensorflow.html 29

Char Embeddings word2vec word2vec type learnt on character BiLSTM-CRF Network sequence U G P Character embedding network 30

Char Embedding • Maps a sequence of characters to a fixed size vector • Handles out of vocabulary words • Handles misspellings sansung tv sansung tv Product NULL Product Brand Type Type Without Char Embedding With Char Embedding 31

Improving search results using query tagging Women citizen eco drive watch Reg egex match will ll be be incorrect for or qu queries s lik ike pioneer women dinnerware wonder women bedding spider man car seats After understanding the Gender token Before 32

Other use cases of query tagging Customer Demand Analysis samsung tv 32 in • Most searched brand of TV 32 in vizio tv TV queries sanyo flat screen tv led tv sony 55” Attribute filter suggestion samsung tv stand Not • Suggest top attributes (e.g. brand, TV queries sony tv remote screen size) that customers look for for in a product type query (e.g. TV) Search query log 33

Neural IR Traditional Semantic Neural IR IR Search • Attribute extraction Token and synonym End-to-end matching • Token, synonym and match and ranking attribute match Learning to Rank • Learning to rank 34

Neural IR – Design 1 • Runtime computation • Not scalable for large number of items Query Item Title Transformed Relevance Neural Input Concatenation feature Score Transformation Embedding 35

Neural IR – Design 2 Query shared weights item embeddings can be Item computed offline and Title indexed Relevance Input Neural Query, item Score Embedding Transformation embeddings 36

Input Embedding token 1 token 1 Query Query … … CNN or … or … AVG Item Title Item Title token n token n Input Input word2vec Embedding word2vec Embedding Comparable Accuracy 37

Training Data query, item title, click through rate (ctr)* Historical search log *Position bias correction for ctr of a query, item pair 𝑑𝑢𝑠 = σ 𝑠 𝑑𝑚𝑗𝑑𝑙𝑡_𝑑𝑝𝑠𝑠𝑓𝑑𝑢𝑓𝑒 𝑠 σ 𝑠 𝑗𝑛𝑞𝑠𝑓𝑡𝑡𝑗𝑝𝑜𝑡 𝑠 𝑑𝑚𝑗𝑑𝑙𝑡_𝑑𝑝𝑠𝑠𝑓𝑑𝑢𝑓𝑒 𝑠 = 𝑑𝑚𝑗𝑑𝑙𝑡 𝑠 + 𝑗𝑛𝑞𝑠𝑓𝑡𝑡𝑗𝑝𝑜𝑡 𝑠 − 𝑑𝑚𝑗𝑑𝑙𝑡 𝑠 ∗ 𝑄 𝑑𝑚𝑗𝑑𝑙 𝑠) 𝑠 = 𝑠𝑏𝑜𝑙 𝑏𝑢 𝑥ℎ𝑗𝑑ℎ 𝑢ℎ𝑓 𝑗𝑢𝑓𝑛 𝑥𝑏𝑡 𝑒𝑗𝑡𝑞𝑚𝑏𝑧𝑓𝑒 38