Metric-Optimized Example Weights Sen Zhao , Mahdi Milani Fard, - PowerPoint PPT Presentation

Feb 21, 2024 •306 likes •454 views

Metric-Optimized Example Weights Sen Zhao , Mahdi Milani Fard, Harikrishna Narasimhan, Maya Gupta Google Research Motivation: Building a Ranking Model Goal : positive precision@3 globally, and not negative in any specific locales. Training Data :

Metric-Optimized Example Weights Sen Zhao , Mahdi Milani Fard, Harikrishna Narasimhan, Maya Gupta Google Research
Motivation: Building a Ranking Model Goal : positive precision@3 globally, and not negative in any specific locales. Training Data : Jan - Oct Testing Data : Nov - Dec Train with pairwise hinge loss.
Motivation: Building a Ranking Model Attempt 1: Train a ranking model on global data. Good global precision@3, but negative in Japan and Brazil . ●
Motivation: Building a Ranking Model Attempt 1: Train a ranking model on global data. Good global precision@3, but negative in Japan and Brazil . ● Attempt 2: Upweight Japan and Brazil training data. Good metric in Japan and Brazil , but negative in UK and India . ●
Motivation: Building a Ranking Model Attempt 1: Train a ranking model on global data. Good global precision@3, but negative in Japan and Brazil . ● Attempt 2: Upweight Japan and Brazil training data. Good metric in Japan and Brazil , but negative in UK and India . ● Attempt 3: Upweight UK and India training data. US turns negative…. ●
Motivation: Building a Ranking Model Attempt 1: Train a ranking model on global data. Good global precision@3, but negative in Japan and Brazil . ● Attempt 2: Upweight Japan and Brazil training data. Good metric in Japan and Brazil , but negative in UK and India . ● Attempt 3: Upweight UK and India training data. US turns negative…. ● Attempt 4: ...
A Practitioner’s Challenge Training Evaluation Training Distribution Testing Distribution (Jan - Oct) (Holiday Season) Training Loss Testing Metric (Pairwise Hinge) (Precision@3)
Metric-Optimized Example Weights (MOEW) MOEW learns the optimal weighting on training examples to maximize the testing metric . Suitable for any loss and any (black-box, non-differentiable) metrics. ● Accompanied by theoretical analysis (generalization bounds etc.). ●
Formulation The main model θ is an ERM problem with weighted loss: The weighting model ⍵ has one parameter ɑ that is learned to maximize validation metric: Iteratively optimize...
A Sneak Peek of MOEW
A Sneak Peek of MOEW
A Sneak Peek of MOEW
A Sneak Peek of MOEW
Poster Tonight 06:30 -- 09:00 PM @ Pacific Ballroom #122

Recommend

Welcome back... Metric spaces. Approximate metric using a tree. Tree metric: 16 16 A metric

Welcome back... Metric spaces. Approximate metric using a tree. Tree metric: 16 16 A metric space X , d ( i , j ) where d ( i , j ) d ( i , k )+ d ( k , j ) , X is nodes of tree with edge weights d ( i , j ) = d ( j , i ) , and d ( i , j )

376 views • 4 slides

A Formally Verified, Optimized Monitor for Metric First-Order Dynamic Logic David Basin, Thibault

A Formally Verified, Optimized Monitor for Metric First-Order Dynamic Logic David Basin, Thibault Dardinier, Lukas Heimes, Sran Krsti , Martin Raszyk , Joshua Schneider and Dmitriy Traytel Department of Computer Science 1 Dmitriy Joshua

1.01k views • 83 slides

Metric Spaces Definition If d is a metric on X , then the metric topology on X induced by d is

Metric Spaces Definition If d is a metric on X , then the metric topology on X induced by d is the topology generated by the basis = { B d ( X , ) : x X and > 0 } . The set X endowed with this topology is called the metric space (

52 views • 4 slides

Constructing Inverse Probability Weights for Static Constructing Inverse Probability Weights for

Constructing Inverse Probability Weights for Static Constructing Inverse Probability Weights for Interventions Selection Bias Kunjal Patel, DSc MPH Senior Research Scientist Harvard T.H. Chan School of Public Health Acknowledgement Slides

928 views • 61 slides

ZIVD, LLC 1 Laboratory Optimized patient care Clinician Optimized patient care 2

Marcia L. Zucker, Ph.D. ZIVD, LLC 1 Laboratory Optimized patient care Clinician Optimized patient care 2 Laboratory Clinician Accuracy Speed High Quality Precision Minimal disruption of Test results routine Compliance

316 views • 20 slides

Multiwavelength UV-metric and pH-metric determination of the dissociation constants of the

Multiwavelength UV-metric and pH-metric determination of the dissociation constants of the hypoxia-inducible factor prolyl hydroxylase inhibitor Roxadustat * Milan Meloun 1 , Lucie Pilaov 1 , Milan Javrek 2 and Tom Pekrek 3 1

178 views • 15 slides

Moving Shadow Tracking in VR Interaction A novel optimized approach A novel optimized approach

Moving Shadow Tracking in VR Interaction A novel optimized approach A novel optimized approach Haipeng Cai Outline Moving Shadow Tracking - the generals The two-step discriminant Improve the classical GMM (Gaussian Mixture Model)

509 views • 16 slides

Multiplicative Weights Algorithms CompSci 590.03 Instructor: Ashwin Machanavajjhala Lecture 13 :

Multiplicative Weights Algorithms CompSci 590.03 Instructor: Ashwin Machanavajjhala Lecture 13 : 590.03 Fall 12 1 This Class A Simple Multiplicative Weights Algorithm Multiplicative Weights for privately publishing synthetic data

464 views • 34 slides

Metric Conversions Ladder Method T. Trimpe 2008 http://sciencespot.net/ Metric System The

Metric Conversions Ladder Method T. Trimpe 2008 http://sciencespot.net/ Metric System The metric system is based on a base unit that corresponds to a certain kind of measurement Length = meter Volume = liter Weight (Mass) =

293 views • 11 slides

Plane partitions with two-periodic weights Sevak Mkrtchyan University of Rochester GGI June 15,

The model Homogeneous weights Inhomogeneous weights Other regimes Plane partitions with two-periodic weights Sevak Mkrtchyan University of Rochester GGI June 15, 2015 The model Homogeneous weights Plane partitions Inhomogeneous weights

716 views • 49 slides

AIRS DETECTOR A/B WEIGHTS - UPDATE Background on degradation in channel noise properties for a

AIRS DETECTOR A/B WEIGHTS POSSIBLE CHANGES FOR IMPROVED NOISE PERFORMANCE Margie Weiler ATK Space Consultant to JPL AIRS Calibration Team NASA Sounder Science Meeting Pasadena, CA May 4, 2009 1 AIRS DETECTOR A/B WEIGHTS - UPDATE

352 views • 12 slides

Dynamical Systems Continuous maps of metric spaces We work with metric spaces, usually a

Dynamical Systems Continuous maps of metric spaces We work with metric spaces, usually a subset of R n with the Euclidean norm. A map of metric spaces F : X Y is continuous at x X if it preserves the limits of convergent sequences,

409 views • 17 slides

Some remarks about metric spaces, 1 Stephen William Semmes Of course various kinds of metric

Some remarks about metric spaces, 1 Stephen William Semmes Of course various kinds of metric spaces arise in various contexts and are viewed in various ways. In this brief survey we hope to give some modest indications of this. In particular,

566 views • 10 slides

Today. Notes. The multiplicative weights framework. Quick Review: experts

Today. Notes. The multiplicative weights framework. Quick Review: experts framework/multiplicative weights algorithm Got to definition of Approximate Equilibrium for zero sum Finish: games. randomized multiplicative weights algorithm for

63 views • 5 slides

The A class of weights and some of its extensions Carlos P erez University of the

The A class of weights and some of its extensions Carlos P erez University of the Basque Country and BCAM Probability and Analysis 2019 Banach Center for Mathematics Be dlewo, May 22, 2019 The C p class of weights

1.6k views • 142 slides

Optimized geothermal Optimized geothermal binary power cycles binary power cycles Kontoleontos

ENhanced hanced G Geothermal eothermal I Innovative nnovative N Network for etwork for E Europe ( urope (ENGINE ENGINE) ) EN Optimized geothermal Optimized geothermal binary power cycles binary power cycles Kontoleontos E.,

412 views • 13 slides

Shortest Path Similar Routing 2 A New Metric A new metric path- based metric that can use used

R outing S tate D istance: A Path-based Metric for Network Analysis Gonca Grsun joint work with joint work with Natali Ruchansky, Evimaria Terzi, Mark Crovella Distance Metrics for Analyzing Routing Shortest Path Similar Routing 2 A New

1.43k views • 48 slides

Information Theoretic Metric Learning Instructor: Sham Kakade 1 Metric Learning In k -nearest

CSE 547/Stat 548: Machine Learning for Big Data Lecture by Adam Gustafson Information Theoretic Metric Learning Instructor: Sham Kakade 1 Metric Learning In k -nearest neighbors ( k -nn) and other classification algorithms, one crucial choice

221 views • 8 slides

Gaussian Quadrature We use the following formula with weights w i and nodes x i 1 n f ( x

Gaussian Quadrature We use the following formula with weights w i and nodes x i 1 n f ( x ) dx w i f ( x i ) 1 i =1 The nodes and weights are of the form: n (# of points) x i w i 2 0.5773502691896257 1.0000000000000000

384 views • 5 slides

Companies Need to Know Bureau of Weights and Measures E15 Presentation Overview Presenter: Judy

E15: What Retailers and Service Companies Need to Know Bureau of Weights and Measures E15 Presentation Overview Presenter: Judy Cardin Director, Bureau of Weights and Measures E15 Overview The Bureau of Weights and Measures has developed

715 views • 55 slides

CTA WEIGHTS AND CTA WEIGHTS AND DIMENSIONS DIMENSIONS INITIATIVES INITIATIVES Meeting of the

CTA WEIGHTS AND CTA WEIGHTS AND DIMENSIONS DIMENSIONS INITIATIVES INITIATIVES Meeting of the National Task Force on Meeting of the National Task Force on Vehicle Weights and Dimensions Policy Vehicle Weights and Dimensions Policy Toronto,

1.07k views • 21 slides

Optimized design and analysis of Optimized design and analysis of sparse-sampling fMRI

Optimized design and analysis of Optimized design and analysis of sparse-sampling fMRI experiments sparse-sampling fMRI experiments of speech and hearing of speech and hearing Tyler Perrachione John Gabrieli Satrajit Ghosh Overview

401 views • 15 slides

GENERAL RULES OF MEASUREMENT All measurements are METRIC! Use the correct piece of

November 05, 2015 GENERAL RULES OF MEASUREMENT All measurements are METRIC! Use the correct piece of equipment All measurements are METRIC! Make all readings at eye level All measurements are METRIC! If your measurement

287 views • 6 slides

Phrase Weights Statistical NLP Spring 2011 Lecture 10: Phrase Alignment Dan Klein UC

Phrase Weights Statistical NLP Spring 2011 Lecture 10: Phrase Alignment Dan Klein UC Berkeley 1 Phrase Scoring Phrase Size Phrases do help Learning weights has been tried, several times: But they dont need [Marcu

77 views • 6 slides