A Fine-grained and Noise-aware Method for Neural Relation Extraction - PowerPoint PPT Presentation

1 A Fine-grained and Noise-aware Method for Neural Relation Extraction ADVISOR: JIA-LING, KOH SOURCE: CIKM 2019 SPEAKER: SHAO-WEI, HUANG DATE: 2020/05/04

2 OUTLINE ⚫ Introduction ⚫ Method ⚫ Experiment ⚫ Conclusion

3 INTRODUCTION ➢ Relation Extraction （ RE ）： Extracting semantic relations between two entities from the text corpus. (Ex): Donald Trump is the 45th President of United States. President of

4 INTRODUCTION ➢ Supervised RE ： ⚫ Heavily relies on human-annotated data to achieve outstanding performance. ⚫ Limited in size and domain specific, preventing large- scale supervised relation extraction.

5 INTRODUCTION ➢ Distant supervision RE ： ⚫ Automatically generates large-scale training data through knowledge base and plain texts. Relations Relations President_of (Donald Trump, United States) President_of (Donald Trump, United States) In KB In KB S1: Donald Trump is the 45th President of the United States. S1: Donald Trump is the 45th President of the United States. President of Sentences in Sentences in S2: Donald Trump was born in the United States. S2: Donald Trump was born in the United States. Plain texts Plain texts (A training bag) S3: Donald Trump believes the United States has incredible potential. S3: Donald Trump believes the United States has incredible potential.

6 INTRODUCTION ➢ Challenge in Distant supervision(1/2) ： ⚫ Multi-instance multi-label problem （ MIML ） . Place_of_birth(Donald Trump, United States) Relations Place_of_birth(Donald Trump, United Relations In KB President_of (Donald Trump, United States) States) In KB S1: Donald Trump is the 45th President of the President_of (Donald Trump, United States) Sentences Sentences United States. (President_of) in S3: Donald Trump believes the United in S2: Donald Trump was born in the United States. Plain texts Plain States has incredible potential. (-) texts (Place_of_birth) Multi-label problem Multi-instance problem

7 INTRODUCTION ➢ Challenge in Distant supervision(2/2) ： ⚫ The assigned relation labels are annotated at bag- level (a set of sentences) instead of sentence-level. Reinforcement Learning Model

8 INTRODUCTION The most informative ➢ Goal ： sentence for each relation Relatin 1 Relatin 2 A training bag ..... (Donald Trump, United States) Relatin l

9 OUTLINE  Introduction  Method  Experiment  Conclusion

REINFORCEMENT LEARNING 10 10 https://www.youtube.com/watch?v=vmkRMvhCW5c&t=1557s State Action; Agent Reward

REINFORCEMENT LEARNING 11 11 https://www.youtube.com/watch?v=vmkRMvhCW5c&t=1557s

FRAMEWORK 12 12

13 METHOD Notation and Problen definition ➢ Let B ={ ⟨ 𝑓 1 , 𝑓 2 ⟩ , ( 𝑠 1 , · · · , 𝑠 𝑚 ), { 𝑇 1 , · · · , 𝑇 𝑜 }} be a training bag. The sentences from corpus An entity pair which mention this entity pair The relations that link the entity pair in KB ➢ Problem definition ： Figuring out the most expressive sentence for each relation in B. （ in order to promote the extractor’s performance ）

14 METHOD State ➢ Embedding of target entity pair ： [ 𝑓 1 , ； 𝑓 2 ] ➢ Encoding for previously chosen sentence ： S 𝑞𝑠𝑓 , encoded by CNN. S 𝑞𝑠𝑓 S 𝑑𝑣𝑠

15 METHOD State Word Filters Embeddings Donald Trump 0.7 is 0.8 S 𝑞𝑠𝑓 the 0.9 45th 0.2 President 0.5 of 0.6 Max 0.1 United States pooling 0.1 Input Convolution

METHOD State 16 ➢ Confidence score for S 𝑞𝑠𝑓 ： p (r | S 𝑞𝑠𝑓 ; θ) , calculated by relation extractor. 𝑆 |𝑠| (r=53) ➢ Encoding for current sentence ： S 𝑑𝑣𝑠 ➢ Confidence score for S 𝑑𝑣𝑠 ： p (r | S 𝑑𝑣𝑠 ; θ) ➢ St ： [ 𝑓 1 ; 𝑓 2 ; S 𝑞𝑠𝑓 ; p (r | S 𝑞𝑠𝑓 ; θ) ; S 𝑑𝑣𝑠 ; p (r | S 𝑑𝑣𝑠 ; θ) ]

METHOD Action 17 17 ➢ First part of action ： U, decide whether to adopt the current sentence to replace the previously chosen. ⚫ U =1 → Yes. ⚫ U = 0 → No. S 𝑞𝑠𝑓 S 𝑑𝑣𝑠 Judge U=1? or U = 0?

METHOD Action 18 18 ➢ Reasonable assumptions for each bag ： ⚫ Expressed-at-least-once ⚫ Express-at-most-one ➢ Compete mechanism ： ⚫ When more relations simultaneously intend to update the S 𝑞𝑠𝑓 with the S 𝑑𝑣𝑠 . ⚫ Only update the relation which has highest Q( S 𝑢 ,U =1).

METHOD Action 19 19 ➢ Second part of action ： P, decide the relation whether to stop the search action. ⚫ P =1 → Yes （ believe has picked out the expressive sentence for this relation ） . ⚫ P = 0 → No.

20 20 METHOD Action ➢ Takes the action 𝑩 ∗ that equals 𝒃𝒔𝒉𝒏𝒃𝒚 𝑩 Q( Q( 𝑻 𝒖 ,A ,A) ： ⚫

METHOD Reward 21 21 ➢ Reward ： ⚫ When execute a action → Get a reward. ⚫ The objective of reinforcement learning ： By maximize total reward to learn Q function. (+)/(-) (+)/(-) (+) (-) (+) (-)

22 METHOD Initial states S𝒖 𝒋 𝟏 for all episodes 𝟏 for all relations of a bag are the same ： ➢ Initial states 𝑻𝒖 𝒋 ⚫ B ={ ⟨ 𝑓 1 , 𝑓 2 ⟩ , ( 𝑠 1 , · · · , 𝑠 𝑚 ), { 𝑇 1 , · · · , 𝑇 𝑜 }}, 𝑗 ∈{ 1, 2, · · · , 𝑚 } 0 = [ 𝑓 1 ; 𝑓 2 ; 0 ; 0 ; S 1 ; p (r | S 1 ; θ) ], for all 𝑗 . ⚫ 𝑇𝑢 𝑗 ⚫ The output values of Q-function are the same too.

METHOD Initial states S𝒖 𝒋 𝟏 for all episodes 23 ➢ heuristic initialization ： 0 = [ 𝑓 1 ; 𝑓 2 ; S 2 ; p (r | S 2 ; θ); 𝑇𝑢 1 𝑏𝑠𝑕𝑛𝑏𝑦 𝑠 p (r | S 1 ; θ) = 𝑠 𝑇 1 𝑠 3 1 S 3+1 ; p (r | S 3+1 ; θ) ] 0 = [ 𝑓 1 ; 𝑓 2 ; S 3 ; p (r | S 3 ; θ); 𝑇𝑢 2 𝑇 2 𝑠 𝑏𝑠𝑕𝑛𝑏𝑦 𝑠 p (r | S 2 ; θ) = 𝑠 2 1 S 3+1 ; p (r | S 3+1 ; θ) ] 0 = [ 𝑓 1 ; 𝑓 2 ; S 1 ; p (r | S 1 ; θ); 𝑇𝑢 3 𝑏𝑠𝑕𝑛𝑏𝑦 𝑠 p (r | S 3 ; θ) = 𝑠 𝑇 3 𝑠 2 3 S 3+1 ; p (r | S 3+1 ; θ) ]

24 24 METHOD ➢ Reinforcement learning algorithm for MIML ： ⚫ In paper Algorithm 1.

https://morvanzhou.github.io/tutorials/machine- METHOD Joint training learning/ML-intro/4-03-q-learning/ 25 ➢ Optimization of the extractor ： 𝑦 𝑗 is a sentence in TS ⚫ with relation 𝑧 𝑗 |TS TS| Maximize ➢ Optimization of the reinforcement learning ： R egarded as accurate ⚫ value of Q( 𝑇𝑢 𝑗 , 𝐵 𝑗 ) minimize ⚫

26 26 METHOD ➢ Joint training for extractor and reforcement learning ： ⚫ In paper Algorithm 2.

27 OUTLINE  Introduction  Method  Experiment  Conclusion

28 EXPERIMENT Dataset ➢ NYT+Freebase ： Aligning entities and relations in Freebase with the corpus of New York Times. ⚫ NYT in 2005-2006 → Training data ⚫ NYT in 2007 → Testing data

https://blog.csdn.net/u013249853/article/details/961 EXPERIMENT 32766 29 Performance with NN methods

EXPERIMENT 30 Performance with NN methods

https://blog.csdn.net/u013249853/article/details/961 EXPERIMENT 32766 31 Reasonability of model design

EXPERIMENT 32 Case study

EXPERIMENT 33 Case study

34 OUTLINE  Introduction  Method  Experiment  Conclusion

35 CONCLUSION ➢ Craft reinforcement learning to solve MIML problem,and generate the sentence-level annotated signal in distantlysupervised relation extraction. ➢ Then these chosen expressive sentences serve as training instances to feed the extractor. ➢ We conduct extensive experiments and the experimental results demonstrate that our model can effectively alleviate MIML problem and achieve the new state-of-the-art performance.