Domain Adaptation with Asymmetrically Relaxed Distribution Alignment - PowerPoint PPT Presentation

Nov 01, 2023 •244 likes •332 views

Domain Adaptation with Asymmetrically Relaxed Distribution Alignment Yifan Wu , Ezra Winston, Divyansh Kaushik, Zachary Lipton Carnegie Mellon University ICML 2019 1 / 8 Background - Unsupervised Domain Adaptation Unsupervised Domain

Domain Adaptation with Asymmetrically Relaxed Distribution Alignment Yifan Wu , Ezra Winston, Divyansh Kaushik, Zachary Lipton Carnegie Mellon University ICML 2019 1 / 8
Background - Unsupervised Domain Adaptation Unsupervised Domain Adaptation: Labeled data from source domain: { ( x i , y i ) } i =1 ,..., n ∼ p S · p y | x . Unlabeled data from target domain: { x i } i =1 ,..., m ∼ p T Goal: learn a good target domain classifier ˆ y x = argmax y p y | x ( y | x ) for x ∼ p T . 2 / 8
Background - Domain Adversarial Training Domain Adversarial Training (Ganin et al., 2016): Learn a predictor ˆ y x = h ( φ ( x )) by optimizing: φ, h E S ( φ, h ) + λ D ( p φ S , p φ min T ) + Ω( φ, h ) . source domain prediction error distance between feature dis- tributions in the latent space 3 / 8
Contribution Problems with domain adversarial training: Fails under label distribution shift. We propose to use relaxed distribution alignment. Not clear how to prevent cross-label matching. We drive a general error bound which explains under what assumptions this CANNOT happen. Latent Space Z Latent Space Z Latent Space Z + − Source Source Source Target Target Target φ : X �→ Z φ : X �→ Z φ : X �→ Z + + + + + + − − − − − − Source Target Source Target Source Target Input Space X Input Space X Input Space X 4 / 8
Relaxed Distances between Distributions Our approach: replace the standard distance between distributions with a relaxed distance: φ, h E S ( φ, h ) + λ D β ( p φ S , p φ min T ) + Ω( φ, h ) . Relaxed Jensen-Shannon Divergence: � � � � �� log g ( z ) 1 − g ( z ) D ¯ f β ( p , q ) = sup + E z ∼ p log . E z ∼ q 2 + β 2 + β g : Z�→ (0 , 1] Relaxation for any f -divergence, Wasserstein distance, etc. 5 / 8
Experiments - Handwritten Digits target [0-4] [5-9] [0-9] target [0-4] [5-9] [0-9] labels Shift Shift No-Shift labels Shift Shift No-Shift Source 74.3 ± 1.0 59.5 ± 3.0 66.7 ± 2.1 Source 69.4 ± 2.3 30.3 ± 2.8 49.4 ± 2.1 DANN 50.0 ± 1.9 28.2 ± 2.8 78.5 ± 1.6 DANN 57.6 ± 1.1 37.1 ± 3.5 81.9 ± 6.7 fDANN-1 71.6 ± 4.0 73.7 ± 1.5 fDANN-1 80.4 ± 2.0 40.1 ± 3.2 75.4 ± 4.5 67.5 ± 2.3 fDANN-2 74.3 ± 2.5 61.9 ± 2.9 72.6 ± 0.9 fDANN-2 41.7 ± 6.6 70.0 ± 3.3 86.6 ± 4.9 fDANN-4 75.9 ± 1.6 64.4 ± 3.6 72.3 ± 1.2 fDANN-4 77.6 ± 6.8 34.7 ± 7.1 58.5 ± 2.2 sDANN-1 71.6 ± 3.7 49.1 ± 6.3 81.0 ± 1.3 sDANN-1 68.2 ± 2.7 78.8 ± 5.3 45.4 ± 7.1 sDANN-2 76.4 ± 3.1 48.7 ± 9.0 81.7 ± 1.4 sDANN-2 78.6 ± 3.6 36.1 ± 5.2 77.4 ± 5.7 sDANN-4 81.0 ± 1.6 60.8 ± 7.5 82.0 ± 0.4 sDANN-4 83.5 ± 2.7 41.1 ± 6.6 75.6 ± 6.9 Table: MNIST → USPS Table: USPS → MNIST 6 / 8
Thank You Poster 177 7 / 8
References Ganin, Yaroslav, Ustinova, Evgeniya, Ajakan, Hana, Germain, Pascal, Larochelle, Hugo, Laviolette, Fran¸ cois, Marchand, Mario, and Lempitsky, Victor. Domain-adversarial training of neural networks. The Journal of Machine Learning Research , 17(1):2096–2030, 2016. 8 / 8

Recommend

What is domain adaptation?

A Two-Stage Approach to Domain Adaptation for Statistical Classifiers Jing Jiang & ChengXiang Zhai Department of Computer Science University of Illinois at Urbana-Champaign What is domain adaptation?

523 views • 20 slides

discrepancy for unsupervised domain adaptation Hongliang Yan 2017/06/21 Domain Adaptation DA

Mind the class weight bias: weighted maximum mean discrepancy for unsupervised domain adaptation Hongliang Yan 2017/06/21 Domain Adaptation DA Training Test Problem: (Source (Target) Training and test sets are related but under

495 views • 21 slides

Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptation 1,2 1,4 Xiang

Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptation 1,2 1,4 Xiang Jiang Qicheng Lao 1,3 1 Stan Matwin Mohammad Havaei 1 Imagia 2 Dalhousie University 3 Polish Academy of Sciences 4 Mila, Universit e de Montr

1.02k views • 43 slides

Towards Assumption-free Unsupervised Domain Adaptation for Visual recognition

Towards Assumption-free Unsupervised Domain Adaptation for Visual recognition Outline Background and Motivations Unsupervised Domain Adaptation - Estimating label information in

1.51k views • 47 slides

Instance Weighting for Domain Adaptation in NLP Jing Jiang & ChengXiang Zhai University of

Instance Weighting for Domain Adaptation in NLP Jing Jiang & ChengXiang Zhai University of Illinois at Urbana-Champaign June 25, 2007 Domain Adaptation Many NLP tasks are cast into classification

582 views • 20 slides

Domain adaptation model for retinopathy detection from cross-domain OCT images Jing Wang 1;2 ,

Domain adaptation model for retinopathy detection from cross-domain OCT images Jing Wang 1;2 , Yiwei Chen 2 , Wanyue Li1; 2 , Wen Kong 1;2 , Yi He 2 , Chuihui Jiang *3 , Guohua Shi *2;4 1 University of Science and technology of China, Hefei 230026,

348 views • 9 slides

Adaptation Philipp Koehn 27 October 2020 Philipp Koehn Machine Translation: Adaptation 27

Adaptation Philipp Koehn 27 October 2020 Philipp Koehn Machine Translation: Adaptation 27 October 2020 Adaptation 1 Better quality when system is adapted to a task Domain adaptation to a specific domain, e.g., information technology

962 views • 46 slides

Few-shot Domain Adaptation 1/12 by Causal Mechanism Transfer Domain adaptation Causal mechanism

Few-shot Domain Adaptation 1/12 by Causal Mechanism Transfer Domain adaptation Causal mechanism Q. When is it possible? A. Common causal Transfer assumption (TA)? mechanism as the relation. Summary Common data generating (causal) mechanism

1.03k views • 15 slides

Theoretical Analysis of Domain Adaptation Current state of the art Shai Ben-David September 14,

Theoretical Analysis of Domain Adaptation Current state of the art Shai Ben-David September 14, 2012 Domain Adaptation Most of the statistical learning guarantees are based on assuming that the learning environment is unchanged throughout the

685 views • 64 slides

Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction Rahaf Aljundi,

Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction Rahaf Aljundi, Tinne Tuytelaars Unsupervised Domain Adaptation When you expect the test data (Target) to be different from your training data (Source) DA in the

683 views • 39 slides

Unsupervised Clustering Approaches for Domain Adaptation in Speaker Recognition Systems Stephen

Unsupervised Clustering Approaches for Domain Adaptation in Speaker Recognition Systems Stephen H. Shum Douglas A. Reynolds Daniel Garcia-Romero Alan McCree Unsupervised Clustering Approaches for Domain Adaptation in Speaker Recognition

437 views • 25 slides

Active Online Domain Adaptation Yining Chen (Stanford) , Haipeng Luo (USC), Tengyu Ma (Stanford),

Active Online Domain Adaptation Yining Chen (Stanford) , Haipeng Luo (USC), Tengyu Ma (Stanford), Chicheng Zhang (University of Arizona) Domain shift: challenge for ML system Time Online learning: a classical framework for domain shift At

390 views • 7 slides

for Domain Adaptation in Chest X-ray Classification Matthias Lenga, Heinrich Schulz, Axel

MIDL 2020 Continual Learning for Domain Adaptation in Chest X-ray Classification Matthias Lenga, Heinrich Schulz, Axel Saalbach Philips Research Hamburg arxiv.org/abs/2001.05922 Domain shift in CXR classification For Chest X-ray

445 views • 7 slides

Unsupervised Domain Adaptation Based on Source-guided Discrepancy 23 th Sep. Han Bao (The

Unsupervised Domain Adaptation Based on Source-guided Discrepancy 23 th Sep. Han Bao (The University of Tokyo / RIKEN AIP) Research interests 2 Classification from Pairwise Similarity and Unlabeled Data [BNS18] (ICML2018) Weak

919 views • 30 slides

Transfer learning/Domain adaptation: Methods and

Transfer learning/Domain adaptation: Methods and Application Lei Zhang ( ) Website:

1.19k views • 104 slides

Unsupervised Domain Adaptation by Backpropagation Chih-Hui Ho, Xingyu Gu, Yuan Qi Outline

Unsupervised Domain Adaptation by Backpropagation Chih-Hui Ho, Xingyu Gu, Yuan Qi Outline Introduction Related works Proposed solution Experiments Conclusions Problems Deep network : requires massive labeled

457 views • 35 slides

Relaxed Separation Logic Tutorial @ POPL14 Viktor Vafeiadis MPI-SWS 20 January 2014

Relaxed Separation Logic Tutorial @ POPL14 Viktor Vafeiadis MPI-SWS 20 January 2014 Tutorial outline Part I. Weak memory models 1. Introduction to relaxed concurrency 2. The C11 relaxed memory model Part II. Relaxed program logics 3.

699 views • 34 slides

Domain adaptation with optimal transport from mapping to learning with joint distribution R.

Domain adaptation with optimal transport from mapping to learning with joint distribution R. Flamary - Lagrange, OCA, CNRS, Universit e C ote dAzur Joint work with N. Courty, A. Habrard, A. Rakotomamonjy and B. Bushan Damodoran Data

618 views • 51 slides

5th STL Workshop, June 2005 Title: Relaxed weak queues: an alternative to run-relaxed heaps

5th STL Workshop, June 2005 Title: Relaxed weak queues: an alternative to run-relaxed heaps Speaker: Jyrki Katajainen Co-workers: Amr Elmasry and Claus Jensen These slides as well as the underlying paper are available at

262 views • 22 slides

Action Segmentation with Jo Join int Self-Superv rvised Temporal Domain Adaptation Min-Hung

ghassanalregib.info/ Action Segmentation with Jo Join int Self-Superv rvised Temporal Domain Adaptation Min-Hung Chen 1 Baopu Li 2 Yingze Bao 2 Ghassan AlRegib 1 Zsolt Kira 1 1 Georgia Institute of Technology 2 Baidu USA June 17, 2020

416 views • 13 slides

Two Methods for Domain Adaptation of Bilingual Tasks: Delightfully Simple and Broadly Applicable

Two Methods for Domain Adaptation of Bilingual Tasks: Delightfully Simple and Broadly Applicable Viktor Hangya 1 , Fabienne Braune 1 , 2 , Alexander Fraser 1 , Hinrich utze 1 Sch 1 Center for Information and Language Processing LMU Munich,

791 views • 30 slides

Dictionaries, Manifolds and Domain Adaptation for Image and Video- based Recognition Rama

Dictionaries, Manifolds and Domain Adaptation for Image and Video- based Recognition Rama Chellappa University of Maryland Student and the teachers Major points Training and testing data come from different distributions.

780 views • 53 slides

Robust Causal Domain Adaptation in a Simple Diagnostic Setting Thijs van Ommen Ghent, July 4,

Robust Causal Domain Adaptation in a Simple Diagnostic Setting Thijs van Ommen Ghent, July 4, 2019 Thijs van Ommen (UU) Robust Causal Domain Adaptation July 4, 2019 1 / 8 Background Thijs van Ommen (UU) Robust Causal Domain Adaptation

551 views • 18 slides

Action Segmentation with Joint Self-Supervised Temporal Domain Adaptation Min-Hung Chen 1 Baopu

Action Segmentation with Joint Self-Supervised Temporal Domain Adaptation Min-Hung Chen 1 Baopu Li 2 Yingze Bao 2 Ghassan AlRegib 1 Zsolt Kira 1 1 Georgia Institute of Technology 2 Baidu USA Abstract Source Video Inputs Target Labels

136 views • 10 slides