search discovery
play

Search + Discovery Peter Bourgon Evolution of search Relevance - PowerPoint PPT Presentation

Oct 06, 2023 •266 likes •916 views

Search + Discovery Peter Bourgon Evolution of search Relevance ranking A bit about SOA EVOLUTION OF SEARCH KATAMARI DAMACY, NAMCO HOW TO FIX? HOW TO FIX? ElasticSearch HOW TO FIX? ElasticSearch Greenfield development HOW TO FIX?

  1. Search + Discovery Peter Bourgon

  2. Evolution of search Relevance ranking A bit about SOA

  3. EVOLUTION OF SEARCH

  8. KATAMARI DAMACY, NAMCO

  10. HOW TO FIX?

  11. HOW TO FIX? ElasticSearch

  12. HOW TO FIX? ElasticSearch Greenfield development

  13. HOW TO FIX? ElasticSearch Greenfield development Parallel dark launch

  14. HOW TO FIX? ElasticSearch Greenfield development Parallel dark launch Migrate traffic slowly

  15. HOW TO FIX? ElasticSearch Greenfield development Parallel dark launch Migrate traffic slowly Scale out

  17. WHY ELASTICSEARCH

  18. WHY ELASTICSEARCH Good API

  19. WHY ELASTICSEARCH Good API Clear path for growth

  20. WHY ELASTICSEARCH Good API Clear path for growth Batteries included

  21. WHY ELASTICSEARCH Good API Clear path for growth Batteries included Works like you expect it to

  22. COMPONENTS

  23. RELEVANCE RANKING

  24. PageRank

  25. DiscoRank

  27. DISCORANK DETAILS

  28. DISCORANK DETAILS Calculations done offline

  29. DISCORANK DETAILS Calculations done offline Serialize to compact array

  30. DISCORANK DETAILS Calculations done offline Serialize to compact array Load in ES custom scorer

  31. DISCORANK DETAILS Calculations done offline Serialize to compact array Load in ES custom scorer “Hybrid” 2-phase queries

  32. DISCORANK DETAILS Calculations done offline Serialize to compact array Load in ES custom scorer “Hybrid” 2-phase queries Recalculate/reload every N

  34. DISCORANK++

  35. DISCORANK++ + Time component → “Hotness”

  36. DISCORANK++ + Time component → “Hotness” + User behavior → “Trust”

  37. THE BIGGER PICTURE

  40. INDEXING Single-purpose application(s) Stateless and idempotent Build, iterate, iterate, iterate Full catalog ~= 1 hour

  42. SEARCHING Single-purpose application Stateless and idempotent SC/ES DSL translation Open-source ES library

  44. EXPLORING Single-purpose application “Stateless” and idempotent Special, complex searches Cached on regular interval

  45. SERVICE ORIENTED ARCHITECTURE

  47. SOA AT SOUNDCLOUD

  48. SOA AT SOUNDCLOUD Bazooka platform

  49. SOA AT SOUNDCLOUD Bazooka platform API as firewall

  50. SOA AT SOUNDCLOUD Bazooka platform API as firewall 12 Factor applications

  52. 12 FACTOR APPS

  53. 12 FACTOR APPS Single codebase (repository)

  54. 12 FACTOR APPS Single codebase (repository) Config stored in environment

  55. 12 FACTOR APPS Single codebase (repository) Config stored in environment Backing services as resources

  56. 12 FACTOR APPS Single codebase (repository) Config stored in environment Backing services as resources Stateless

  57. 12 FACTOR APPS Single codebase (repository) Config stored in environment Backing services as resources Stateless Horizontal scaling with processes

  59. WHY SOA?

  60. WHY SOA? Velocity

  61. WHY SOA? Velocity Developer happiness

  62. WHY SOA? Velocity Developer happiness Distributed systems are complex

  63. ElasticSearch core + SOA infrastructure

  64. THANKS! (ALSO WE ARE HIRING) Peter Bourgon @peterbourgon http://peter.bourgon.org

Recommend

From Search to Discovery in our Future Library From Search to Discovery W e see a spectrum of

From Search to Discovery in our Future Library From Search to Discovery W e see a spectrum of

From Search to Discovery in our Future Library From Search to Discovery W e see a spectrum of di ff erence between Search & Discovery, Search being more targeted and specific and Discovery being more open and vague. This slide also

650 views • 53 slides
RNA Search and Whirlwind tour of ncRNA search & discovery Motif Discovery RNA motif

RNA Search and Whirlwind tour of ncRNA search & discovery Motif Discovery RNA motif

Outline RNA Search and Whirlwind tour of ncRNA search & discovery Motif Discovery RNA motif description (Covariance Model Review) Algorithms for searching Rigorous & heuristic filtering Motif discovery Lecture

809 views • 27 slides
Search/Discovery Under the Hood Tricia Jenkins and Sean Luyk | Spring Training 2019

Search/Discovery Under the Hood Tricia Jenkins and Sean Luyk | Spring Training 2019

Search/Discovery Under the Hood Tricia Jenkins and Sean Luyk | Spring Training 2019 Outline - Search in libraries - Search trends - Search under the hood 2 The Discovery Technology Stack - Open Source Apache Project since 2007

618 views • 46 slides
Web Mining for Knowledge Discovery Current Search Engine Search engines are doing good jobs

Web Mining for Knowledge Discovery Current Search Engine Search engines are doing good jobs

Web Mining for Knowledge Discovery Current Search Engine Search engines are doing good jobs so far The idea is to use any of the popular search engine like Yahoo or Google Building a web crawler that feeds on the results of the

439 views • 16 slides
Discovery Search Searches most of the our subscriptions and open access platforms from a single

Discovery Search Searches most of the our subscriptions and open access platforms from a single

Search Options Discovery Search Searches most of the our subscriptions and open access platforms from a single search box 1. Discover Everything (references, abstracts & full-text) 2. Full-text Search (Bento Search) useful for

542 views • 24 slides
Wowd distributed search engine Computers in Scientific Discovery 5 Aleksandar Ili d

Wowd distributed search engine Computers in Scientific Discovery 5 Aleksandar Ili d

Wowd distributed search engine Computers in Scientific Discovery 5 Aleksandar Ili d aleksandari@gmail.com University of Ni , Serbia Sheffield, July 2010 Wowd Distributed P2P real-time discovery & search engine

846 views • 21 slides
Top- k Processing for Search and Information Discovery in Social Applications Lecture 2:

Top- k Processing for Search and Information Discovery in Social Applications Lecture 2:

Top- k Processing for Search and Information Discovery in Social Applications Lecture 2: Network-Aware Search in Social Tagging Sites Sihem Amer-Yahia Julia Stoyanovich Social top- k @ Joint RuSSIR/EDBT Summer School 2011 Summary of last

622 views • 37 slides
RNA Search and Motif Discovery Lecture 9 CSEP 590A Summer 2006 Outline Whirlwind tour of

RNA Search and Motif Discovery Lecture 9 CSEP 590A Summer 2006 Outline Whirlwind tour of

RNA Search and Motif Discovery Lecture 9 CSEP 590A Summer 2006 Outline Whirlwind tour of ncRNA search & discovery Covariance Model Review Algorithms for Training Mutual Information Algorithms for searching Rigorous &

829 views • 72 slides
Watson Discovery Spring 2020 Discovery pipeline Using NLU, document conversion, and UI tools

Watson Discovery Spring 2020 Discovery pipeline Using NLU, document conversion, and UI tools

Watson Discovery Spring 2020 Discovery pipeline Using NLU, document conversion, and UI tools Watson Discovery use cases The need to search thousands of product reviews at once: Create a Discovery collection and build a UI to query the

657 views • 9 slides
Large Scale Search, Discovery and Analytics with Solr, Mahout and Hadoop Grant Ingersoll Chief

Large Scale Search, Discovery and Analytics with Solr, Mahout and Hadoop Grant Ingersoll Chief

Search Discover Analyze Large Scale Search, Discovery and Analytics with Solr, Mahout and Hadoop Grant Ingersoll Chief Scientist Lucid Imagination | | 1 1 Search is Dead, Long Live Search User Interaction User Interaction

452 views • 17 slides
English 120 INFORMATION SEARCH PROCESS Search Process Book Resources (searching LIBROS-WorldCat

English 120 INFORMATION SEARCH PROCESS Search Process Book Resources (searching LIBROS-WorldCat

English 120 INFORMATION SEARCH PROCESS Search Process Book Resources (searching LIBROS-WorldCat Discovery ) Periodical Articles (searching online databases) Films on Demand World Wide Web 1 Accessing LIBROS (WorldCat Discovery) Access the

157 views • 5 slides
In Search of Discovery: Results from the Tevatron Chris Hays, Oxford University UK HEP Forum,

In Search of Discovery: Results from the Tevatron Chris Hays, Oxford University UK HEP Forum,

In Search of Discovery: Results from the Tevatron Chris Hays, Oxford University UK HEP Forum, Coseners House, Abingdo n Fundamental Particle Discoveries ~ 15 years since last co ider > 30 years since last surpris e particle discovery

632 views • 35 slides
Benefits Local catalog Seamless discovery and delivery: find it, click it, get it Consortial

Benefits Local catalog Seamless discovery and delivery: find it, click it, get it Consortial

8/26/2019 Search multiple Illinois library catalogs at once. Anyone can search. Patrons and staff of participating libraries can log in to place and manage requests. Benefits Local catalog Seamless discovery and delivery: find it,

434 views • 5 slides
Fall 2008 RNA Function, Secondary Structure Prediction, Search, Discovery The

Fall 2008 RNA Function, Secondary Structure Prediction, Search, Discovery The

CSE P 590A Fall 2008 RNA Function, Secondary Structure Prediction, Search, Discovery The Message noncoding RNA Cells make lots of RNA Functionally important, functionally diverse Structurally complex New

1.58k views • 102 slides
CLASS OF 2019 Strategies for Success in the College Search The College Search is: A process

CLASS OF 2019 Strategies for Success in the College Search The College Search is: A process

ST. JOSEPH HIGH SCHOOL CLASS OF 2019 Strategies for Success in the College Search The College Search is: A process of self-assessment, reflection, and discovery This is YOUR search (parents included) About 3 decisions, two of which

357 views • 19 slides
SEMANTIC SEARCH IN E-DISCOVERY: AN INTERDISCIPLINARY APPROACH DESI V Workshop UvA: David Graus,

SEMANTIC SEARCH IN E-DISCOVERY: AN INTERDISCIPLINARY APPROACH DESI V Workshop UvA: David Graus,

SEMANTIC SEARCH IN E-DISCOVERY: AN INTERDISCIPLINARY APPROACH DESI V Workshop UvA: David Graus, Zhaochun Ren, Maarten de Rijke HvA: David van Dijk, Hans Henseler Leiden University & Fox-IT: Nina van der Knaap ICAIL 2013 Conference, Rome,

115 views • 7 slides
RNA Search and Motif Discovery Lectures 18-19 CSE 527 Autumn 2007 The Human Parts List, circa

RNA Search and Motif Discovery Lectures 18-19 CSE 527 Autumn 2007 The Human Parts List, circa

RNA Search and Motif Discovery Lectures 18-19 CSE 527 Autumn 2007 The Human Parts List, circa 2001 1 gagcccggcc cgggggacgg gcggcgggat agcgggaccc cggcgcggcg gtgcgcttca 61 gggcgcagcg gcggccgcag accgagcccc gggcgcggca agaggcggcg ggagccggtg 121

1.03k views • 91 slides
CSEP 527 Computational Biology RNA: Function, Secondary Structure Prediction, Search, Discovery

CSEP 527 Computational Biology RNA: Function, Secondary Structure Prediction, Search, Discovery

CSEP 527 Computational Biology RNA: Function, Secondary Structure Prediction, Search, Discovery The Message noncoding RNA Cells make lots of RNA Functionally important, functionally diverse Structurally complex New tools required alignment,

811 views • 62 slides
An Amphibious Search and Discovery Rover Intel-Cornell Cup Final Presentation Advised By: Team:

An Amphibious Search and Discovery Rover Intel-Cornell Cup Final Presentation Advised By: Team:

An Amphibious Search and Discovery Rover Intel-Cornell Cup Final Presentation Advised By: Team: Professors Taskin Padir, Sonia Chernova Brian Eccles, Brendan McLeod & Kenneth Stafford Tim Murcko, TJ Watson, Mitch Wills 2 Presentation

647 views • 20 slides
Top- k Processing for Search and Information Discovery in Social Applications Sihem Amer-Yahia

Top- k Processing for Search and Information Discovery in Social Applications Sihem Amer-Yahia

Top- k Processing for Search and Information Discovery in Social Applications Sihem Amer-Yahia Julia Stoyanovich Social top- k @ Joint RuSSIR/EDBT Summer School 2011 Why top- k ? Practical many applications, with new ones emerging every

700 views • 37 slides
RNA Search and Motif Discovery CSEP 527 Computational Biology Previous Lecture

RNA Search and Motif Discovery CSEP 527 Computational Biology Previous Lecture

RNA Search and Motif Discovery CSEP 527 Computational Biology Previous Lecture Many biologically interesting roles for RNA RNA secondary structure prediction 1 2 Approaches to Structure Prediction Maximum Pairing + works on

840 views • 71 slides
In Search of the Root: Discovery of a Highly Divergent Y Chromosome Lineage Bonnie Schrack

In Search of the Root: Discovery of a Highly Divergent Y Chromosome Lineage Bonnie Schrack

In Search of the Root: Discovery of a Highly Divergent Y Chromosome Lineage Bonnie Schrack Haplogroup A Project Admin Family Tree DNA 8th International Conference on Genetic Genealogy, 2012 Bonnie Schrack Have you heard from your project

243 views • 22 slides
Efficient Model Evaluation in the Search-Based Approach to Latent Structure Discovery Tao Chen,

Efficient Model Evaluation in the Search-Based Approach to Latent Structure Discovery Tao Chen,

Efficient Model Evaluation in the Search-Based Approach to Latent Structure Discovery Tao Chen, Nevin L. Zhang and Yi Wang Department of Computer Science & Engineering The Hong Kong University of Science & Technology 1 Latent Tree

595 views • 25 slides
Statistical Quantification of Discovery in Neutrino Physics David A. van Dyk Statistics Section,

Statistical Quantification of Discovery in Neutrino Physics David A. van Dyk Statistics Section,

Motivating Problems Statistical Criteria for Discovery Examples: Mass Hierarchy, CP-violation, Higgs Search Advice Statistical Quantification of Discovery in Neutrino Physics David A. van Dyk Statistics Section, Imperial College London

944 views • 34 slides

More recommend