Adversarial Contrastive Estimation ACL 2018 *AVISHEK (JOEY) BOSE, - PowerPoint PPT Presentation

Adversarial Contrastive Estimation Adversarial Contrastive Estimation ACL 2018 *AVISHEK (JOEY) BOSE, *HUAN LING, *YANSHUAI CAO Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 1 / 29

Adversarial Contrastive Estimation Contrastive Estimation Many Machine Learning models learn by trying to separate positive examples from negative examples. Positive Examples are taken from observed real data distribution (training set) Negative Examples are any other configurations that are not observed Data is in the form of tuples or triplets ( x + , y + ) and ( x + , y − ) are positive and negative data points respectively. Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 2 / 29

Adversarial Contrastive Estimation Easy Negative Examples with NCE Noise Constrastive Estimation samples negatives by taking p ( y − | x + ) to be some unconditional p nce ( y ). What’s wrong with this? Negative y − in ( x , y − ) is not tailored toward x Difficult to choose hard negatives as training progresses Model doesn’t learn discriminating features between positive and hard negative examples NCE negatives are easy !!! Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 3 / 29

Adversarial Contrastive Estimation Hard Negative Examples Informal Definition: Hard negative examples are data points that are extremely difficult for the training model to distinguish from positive examples. Hard Negatives result to higher losses and thus more more informative gradients Not necessarily closest to a positive datapoint in embedding space Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 4 / 29

Adversarial Contrastive Estimation Technical Contributions Adversarial Contrastive Estimation: A general technique for hard negative mining using a Conditional GAN like setup. A novel entropy regularizer that prevents generator mode collapse and has good empirical benefits A strategy for handling false negative examples that allows training to progress Empirical validation across 3 different embedding tasks with state of the art results on some metrics Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 5 / 29

Adversarial Contrastive Estimation Adversarial Contrastive Estimation Problem: We want to generate negatives that ... “fool” a discriminative model into misclassifying. Solution: Use a Conditional GAN to sample hard negatives given x + . We can augment NCE with an adversarial sampler, λ p nce ( y ) + (1 − λ ) g θ ( y | x ). Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 6 / 29

Adversarial Contrastive Estimation Conditional GAN Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 7 / 29

Adversarial Contrastive Estimation Adversarial Contrastive Estimation , Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 8 / 29

Adversarial Contrastive Estimation The ACE Generator The ACE generator defines a categorical distribution over all possible y − values Picking a negative example is a discrete choice and not differentiable Simplest way to train via Policy Gradients is the REINFORCE gradient estimator Learning is done via a GAN style min-max game min ω max V ( ω, θ ) = min ω max E p + ( x ) L ( ω, θ ; x ) (1) θ θ Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 9 / 29

Adversarial Contrastive Estimation Technical Contributions for effective training Problem: GAN training can suffer from mode collapse? What happens if the generator collapses on its favorite few negative examples? Solution: Add a entropy regularizer term to the generators loss: R ent ( x ) = max(0 , c − H ( g θ ( y | x ))) (2) H ( g θ ( y | x )) is the entropy of the categorical distribution c = log( k ) is the entropy of a uniform distribution over k choices Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 10 / 29

Adversarial Contrastive Estimation Technical Contributions for effective training Problem: The Generator can sample false negatives → gradient cancellation Solution: Apply an additional two-step technique, whenever computationally feasible. 1 Maintain an in memory hash map of the training data and Discriminator filters out false negatives 2 Generator receives a penalty for producing the false negative 3 Entropy Regularizer spreads out the probability mass Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 11 / 29

Adversarial Contrastive Estimation Technical Contributions for effective training Problem: The Generator can sample false negatives → gradient cancellation Solution: Apply an additional two-step technique, whenever computationally feasible. 1 Maintain an in memory hash map of the training data and Discriminator filters out false negatives 2 Generator receives a penalty for producing the false negative 3 Entropy Regularizer spreads out the probability mass Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 12 / 29

Adversarial Contrastive Estimation Technical Contributions for effective training Problem: The Generator can sample false negatives → gradient cancellation Solution: Apply an additional two-step technique, whenever computationally feasible. 1 Maintain an in memory hash map of the training data and Discriminator filters out false negatives 2 Generator receives a penalty for producing the false negative 3 Entropy Regularizer spreads out the probability mass Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 13 / 29

Adversarial Contrastive Estimation Technical Contributions for effective training Problem: REINFORCE is known to have extremely high variance. Solution: Reduce Variance using the self-critical baseline. Other baselines and gradient estimators are also good options. Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 14 / 29

Adversarial Contrastive Estimation Technical Contributions for effective training Problem: The generator is not learning from the NCE samples. Solution: Use Importance Sampling. Generator can leverage NCE samples for exploration in an off-policy scheme. The modified reward now looks like g θ ( y − | x ) / p nce ( y − ) . Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 15 / 29

Adversarial Contrastive Estimation Related Work Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 16 / 29

Adversarial Contrastive Estimation Contemporary Work GANs for NLP that are close to our work MaskGAN Fedus et. al 2018 Incorporating GAN for Negative Sampling in Knowledge Representation Learning Wang et. al 2018 KBGAN Cai and Wang 2017 Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 17 / 29

Adversarial Contrastive Estimation Example: Knowledge Graph Embeddings Data in the form of triplets (head entity, relation, tail entity) . For example { United states of America, partially contained by ocean, Pacific } Basic Idea: The embeddings for h , r , t should roughly satisfy h + r ≈ t Link Prediction: Goal is to learn from observed positive entity relations and predict missing links. Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 18 / 29

Adversarial Contrastive Estimation ACE for Knowledge Graph Embeddings Positive Triplet: ξ + = ( h + , r + , t + ) Negative Triplet: Either negative head or tail is sampled i.e. ξ − = ( h − , r + , t + ) or ξ − = ( h + , r + , t − ) Loss Function: L = max(0 , η + s ω ( ξ + ) − s ω ( ξ − )) (3) ACE Generator: g θ ( t − | r + , h + ) or g θ ( h − | r + , t + ) parametrized by a feed forward neural net. Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 19 / 29

Adversarial Contrastive Estimation ACE for Knowledge Graph Embeddings Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 20 / 29

Adversarial Contrastive Estimation Experimental Result: Ablation Study Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 21 / 29

Adversarial Contrastive Estimation ACE for Order Embeddings Hypernym Prediction : A hypernym pair is a pair of concepts where the first concept is a specialization or an instance of the second. Learning embeddings that are hierarchy preserving. The Root Node is at the origin and all other embeddings lie on the positive semi-space Constraint enforces the magnitude of the parent’s embedding to be smaller than child’s in every dimension Sibling nodes are not subjected to this constraint. Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 22 / 29

Adversarial Contrastive Estimation Order Embeddings (Vendrov et. al 2016) Avishek (Joey) Bose* , Huan Ling*, Yanshuai Cao* | Borealis AI, University of Toronto | August 2, 2018 23 / 29 1