parsing contexts
play

Parsing Contexts Context Sensitive Grammars for Everyone Jan Kur - PowerPoint PPT Presentation

Mar 26, 2024 •6 likes •576 views

Parsing Contexts Context Sensitive Grammars for Everyone Jan Kur kurs@iam.unibe.ch Software Composition Group Indentation using Petit Parser print Hello! if keyword if (outOf(milk)): print We are out of milk! block of commands

  1. Parsing Contexts Context Sensitive Grammars for Everyone Jan Kurš kurs@iam.unibe.ch Software Composition Group

  2. Indentation using Petit Parser print “Hello!” if keyword if (outOf(milk)): print “We are out of milk!” block of commands order(milk) if (outOf(eggs)): eggs identifjer … 2

  3. Indentation using Petit Parser print “Hello!” if keyword if (outOf(milk)): print “We are out of milk!” block of commands order(milk) if (outOf(eggs)): eggs identifjer … 3

  4. Indentation using Petit Parser print “Hello!” if keyword if (outOf(milk)): print “We are out of milk!” block of commands order(milk) if (outOf(eggs)): eggs identifjer … ifKeyword := 'if' asParser. 4

  5. Indentation using Petit Parser print “Hello!” if keyword if (outOf(milk)): print “We are out of milk!” block of commands order(milk) if (outOf(eggs)): eggs identifjer … 5

  6. Indentation using Petit Parser print “Hello!” if keyword if (outOf(milk)): print “We are out of milk!” block of commands order(milk) if (outOf(eggs)): eggs identifjer … identifier := #letter, (#letter / #digit) star. 6

  7. Indentation using Petit Parser print “Hello!” if keyword if (outOf(milk)): print “We are out of milk!” block of commands order(milk) if (outOf(eggs)): eggs identifjer … 7

  8. Indentation using Petit Parser print “Hello!” if keyword if (outOf(milk)): print “We are out of milk!” block of commands order(milk) if (outOf(eggs)): eggs identifjer … block := indent, command plus, dedent. 8

  9. Indentation using Petit Parser print “Hello!” if keyword if (outOf(milk)) print “We are out of milk!” block of commands order(milk) if (outOf(eggs)) eggs identifjer … block := indent, command plus, dedent. 9

  10. Indentation using Petit Parser print “Hello!” if keyword if (outOf(milk)) print “We are out of milk!” block of commands order(milk) if (outOf(eggs)) eggs identifjer … block := indent, command plus, dedent. indent := ??? dedent := ??? 10

  11. Regular Context-Free Context-Sensitive Languages Languages Languages Simple State of the Art Complex 11

  12. Regular Context-Free Context-Sensitive Languages Languages Languages Expression Regular Simple State of the Art Complex 12

  13. Regular Context-Free Context-Sensitive Languages Languages Languages Expression Regular Simple State of the Art Machine Turing Complex 13

  14. Regular Context-Free Context-Sensitive Languages Languages Languages Expression Regular Simple State of the Art Machine Turing Complex 14

  15. Regular Context-Free Context-Sensitive Languages Languages Languages Expression Regular Simple State of the Art Machine Turing Complex 15

  16. Regular Context-Free Context-Sensitive Languages Languages Languages Expression Regular Simple State of the Art Machine Turing Complex 16

  17. Regular Context-Free Context-Sensitive Languages Languages Languages Expression Regular Simple State of the Art Machine Turing Complex 17

  18. Goal Context-Sensitive Turing Machine Context-Free Regular Regular Expression Complex 18 Simple

  19. Goal Context-Sensitive Turing Machine Parsing Parsing Contexts Contexts Context-Free Regular Regular Expression Complex 19 Simple

  20. Goal Context-Sensitive Turing Machine Parsing Parsing Contexts Contexts Context-Free Regular Regular Expression Complex 20 Simple

  21. PetitParser uses Parser Combinators 21

  22. PetitParser uses Parser Combinators #letter, (#letter / #digit) star 22

  23. PetitParser uses Parser Combinators #letter, (#letter / #digit) star 23

  24. PetitParser uses Parser Combinators , , # letter star #letter, (#letter / #digit) star / / # letter # digit 24

  25. String String 25

  26. String String 26

  27. String “Hello” #letter #letter String “ello” 27

  28. Indentation Example print “Hello!” if keyword if (outOf(milk)) print “We are out of milk!” block of commands order(milk) if (outOf(eggs)) eggs identifjer … block := indent, command plus, dedent. 28

  29. Indentation Example print “Hello!” if keyword if (outOf(milk)) print “We are out of milk!” block of commands order(milk) if (outOf(eggs)) eggs identifjer … block := indent, command plus, dedent. indent := ??? dedent := ??? 29

  30. Indentation Example print “Hello!” ▼ if keyword if (outOf(milk)) print “We are out of milk!” block of commands order(milk) if (outOf(eggs)) eggs identifjer … block := indent, command plus, dedent. 30

  31. Indentation Example print “Hello!” ▼ if keyword if (outOf(milk)) print “We are out of milk!” block of commands order(milk) if (outOf(eggs)) eggs identifjer … block := indent, command plus, dedent. indent := ??? dedent := ??? 31

  32. String String 32

  33. String String 33

  34. String & String Indentation Stack String String & Indentation Stack 34

  35. Parsing Parsing Contexts Contexts String String Parsing Parsing Contexts Contexts 35

  36. String Parsing Parsing Contexts Contexts String Other Data String Parsing Parsing Contexts Contexts 36

  37. Indentation Example print “Hello!” ▼ if keyword if (outOf(milk)) print “We are out of milk!” block of commands order(milk) if (outOf(eggs)) eggs identifjer … [ :context | [ :context | | column referenceIndentation stack | | column indentation stack | (context isBeginOfLine) ifFalse: [ (context isBeginOfLine) ifFalse: [ ^ Failure ^ Failure ]. ]. context consumeLeadingWhitespace. block := context consumeLeadingWhitespace. , command plus, column := context stream column. " Save the current column " " Restore previous column from the " column := context stream column. " stack and compare with current " stack := (context propertyAt: #indent). stack := (context propertyAt: #indent). indentation := stack top. stack pop. referenceIndentation := stack top. (column > indentation) ifTrue: [ stack push: column. (column == referenceIndentation) ifTrue: [ ^ #indent ^ #dedent ]. ]. ^ Failure ^ Failure ] 37 ]

  38. Context-Sensitive Turing Machine Parsing Parsing Contexts Contexts Context-Free Regular Regular Expression Complex 38 Simple

  39. Context-Sensitive Turing Parsing Parsing Machine Contexts Contexts Context-Free Regular Regular Expression Complex 39 Simple

  40. Context-Sensitive Turing Parsing Parsing Machine Contexts Contexts Context-Free Regular Regular Expression Complex 40 Simple

  41. Grammar Implementors 41

  42. Grammar Implementors Framework Implementors 42

  43. ← N (N ∪ T)* Grammar Implementors Framework Implementors 43

  44. ← N (N ∪ T)* indent, command plus, dedent Grammar Implementors Framework Implementors 44

  45. ← N (N ∪ T)* indent, command plus, dedent Grammar Implementors [ :context | [ :contex t | | column referenceIndentation stack | | column indentation stack | (context isBeginOfLine) ifFalse: [ (context isBeginOfLine) ifFalse: [ ^ Failure ^ Failure ]. ]. context consumeLeadingWhitespace. context consumeLeadingWhitespace. , command plus, column := context stream column. " Save the current column " " Restore previous column from the " column := context stream column. " stack and compare with current " stack := (context propertyAt: #indent). stack := (context propertyAt: #indent). indentation := stack top. stack pop. referenceIndentation := stack top. (column > indentation) ifTrue: [ stack push: column. (column == referenceIndentation) ifTrue: [ ^ #indent ^ #dedent ]. ]. ^ Failure ^ Failure ] ] Framework Implementors 45

  46. Context-Sensitive Turing Machine Parsing Parsing Contexts Contexts Parsing Parsing Contexts Contexts Context-Free Regular Regular Expression Complex 46 Simple

  47. Context-Sensitive Turing Machine Parsing Parsing Contexts Contexts Context-Free Regular Regular Expression Complex 47 Simple

  48. Context-Sensitive Turing Machine Parsing Parsing Contexts Contexts Context-Free Regular Regular Expression Complex 48 Simple

  49. Parsing Parsing Contexts Contexts 49

  50. Parsing Parsing Contexts Contexts indentation 50

  51. Parsing Parsing Contexts Contexts indentation tolerant parsing 51

  52. Parsing Parsing Contexts Contexts indentation tolerant parsing debugging 52

  53. http://smalltalkhub.com/#!/~JanKurs/PetitParser 53

  54. Turing Machine Parsing Parsing Contexts Contexts Regular Expression 54

  55. Parsing String Turing Context Machine Parsing Parsing Contexts Contexts Regular Expression Parsing String Context 55

  56. Parsing String Turing Context Machine Parsing Parsing Contexts Contexts Regular Expression Parsing String Context ← ∪ N (N T)* Grammar Framework Implementor Implementors 56

  57. Parsing String Turing Context Machine Parsing Parsing Contexts Contexts Regular Expression Parsing String Context ← ∪ N (N T)* Parsing Parsing Contexts Contexts indentation tolerant parsing Grammar Framework debugging Implementor Implementors 57

Recommend

CSC 4181 Compiler Construction Parsing 1 1 Outline Top-down v.s. Bottom-up Top-down parsing

CSC 4181 Compiler Construction Parsing 1 1 Outline Top-down v.s. Bottom-up Top-down parsing

CSC 4181 Compiler Construction Parsing 1 1 Outline Top-down v.s. Bottom-up Top-down parsing Bottom-up parsing Recursive-descent Shift-reduce parsers parsing LR(0) parsing LL(1) parsing LR(0) items LL(1) parsing

429 views • 29 slides
Introduction to Bottom-Up Parsing Shift-reduce parsing The LR parsing algorithm

Introduction to Bottom-Up Parsing Shift-reduce parsing The LR parsing algorithm

Outline Review LL parsing Introduction to Bottom-Up Parsing Shift-reduce parsing The LR parsing algorithm Constructing LR parsing tables 2 Top-Down Parsing: Review Top-Down Parsing: Review Top-down parsing expands a

470 views • 13 slides
How Contexts Matter Understanding in Dialogues Y UN -N UNG (V IVIAN ) C HEN Word-Level Contexts

How Contexts Matter Understanding in Dialogues Y UN -N UNG (V IVIAN ) C HEN Word-Level Contexts

How Contexts Matter Understanding in Dialogues Y UN -N UNG (V IVIAN ) C HEN Word-Level Contexts in Sentences Learning from Prior Knowledge K nowledge-Guided S tructural A ttention N etworks (K-SAN) [Chen et al., 16] Learning from

790 views • 26 slides
Top-Down Parsing Slides modified from Louden Book and Dr. Scherger Top Down Parsing A

Top-Down Parsing Slides modified from Louden Book and Dr. Scherger Top Down Parsing A

Compiler Design and Construction Top-Down Parsing Slides modified from Louden Book and Dr. Scherger Top Down Parsing A top-down parsing algorithm parses an input string of tokens by tracing out the steps in a leftmost derivation. Such an

1.12k views • 96 slides
Statistical Dependency Parsing in Korean: From Corpus Generation To Automatic Parsing Workshop on

Statistical Dependency Parsing in Korean: From Corpus Generation To Automatic Parsing Workshop on

Statistical Dependency Parsing in Korean: From Corpus Generation To Automatic Parsing Workshop on Statistical Parsing of Morphologically-Rich Languages 12th International Conference on Parsing Technologies Jinho D. Choi & Martha Palmer

199 views • 19 slides
Basic Parsing Algorithms Chart Parsing Seminar Recent Advances in Parsing Technology WS

Basic Parsing Algorithms Chart Parsing Seminar Recent Advances in Parsing Technology WS

Basic Parsing Algorithms Chart Parsing Seminar Recent Advances in Parsing Technology WS 2011/2012 Anna Schmidt Talk Outline Chart Parsing Basics Chart Parsing Algorithms Earley Algorithm CKY Algorithm Basics

1.12k views • 73 slides
Overview Introduction Lexicalized TAG, Advantages of parsing with LTAG Parsing LTAGs

Overview Introduction Lexicalized TAG, Advantages of parsing with LTAG Parsing LTAGs

Parsing with Lexicalized TAG (1) Extracting and comparing LTAG (2) Presentation by Philip John Gorinski Seminar Recent Advances in Parsing Technology Saarland University, Winter Term 2011/12 (1) Yves Shabes, Aravind K. Joshi, 1990 (2)

514 views • 36 slides
* 07/16/96 Plan for Today Shift-reduce parsing The problem with predictive top down parsing

* 07/16/96 Plan for Today Shift-reduce parsing The problem with predictive top down parsing

* 07/16/96 Plan for Today Shift-reduce parsing The problem with predictive top down parsing CS453 : Shift Reduce Parsing LR parsing: bottom up Unambiguous Grammars LR(0) and SLR Parse Tables Performs a right-most derivation in

316 views • 6 slides
1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G ,

1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G ,

1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G , to decide if w L ( G ), and if so, to construct a derivation or a parse tree for w . Parsing is studied in courses in compilers. To be efficient

519 views • 12 slides
Bottom-up parsing LR parsing Construct parse tree for input from leaves up LR( k ) parsing

Bottom-up parsing LR parsing Construct parse tree for input from leaves up LR( k ) parsing

Bottom-up parsing LR parsing Construct parse tree for input from leaves up LR( k ) parsing reducing a string of tokens to single start symbol L eft-to-right scan of input, R ightmost derivation (inverse of deriving a string of tokens from

413 views • 5 slides
Bottom Up Parsing Also known as Shift-Reduce parsing More powerful than top down Dont

Bottom Up Parsing Also known as Shift-Reduce parsing More powerful than top down Dont

9/26/2012 Bottom Up Parsing Also known as Shift-Reduce parsing More powerful than top down Dont need left factored grammars Can handle left recursion Bottom Up Parsing Attempt to construct parse tree from an input string

654 views • 6 slides
Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP

Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP

Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP February 2, 2015 Roadmap Dependency parsing Graph-based dependency parsing Maximum spanning tree CLE Algorithm Learning

899 views • 87 slides
Statistical Parsing Parsing context-free languages ar ltekin University of Tbingen

Statistical Parsing Parsing context-free languages ar ltekin University of Tbingen

Statistical Parsing Parsing context-free languages ar ltekin University of Tbingen Seminar fr Sprachwissenschaft November 8, 2016 Recap Parsing basics CKY Earley Closing Ingredients of a (natural language) parser .

1.17k views • 51 slides
Generalised Parsing and Combinator Parsing A Happy Marriage? L. Thomas van Binsbergen

Generalised Parsing and Combinator Parsing A Happy Marriage? L. Thomas van Binsbergen

Conventional Parser Combinators Aspects of Generalised Parsing Combinators for Generalised Parsing Generalised Parsing and Combinator Parsing A Happy Marriage? L. Thomas van Binsbergen ltvanbinsbergen@acm.org Royal Holloway, University of

452 views • 30 slides
Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs

Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs

Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs Semantic Parsing Dependency parsing models the syntactic structure between words in a sentence. Dependency Parsing vs Semantic Parsing

637 views • 31 slides
Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr

Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr

Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr Sprachwissenschaft November 2016 Recap/background Dependency grammar Dependency parsing Evaluation Summary Ingredients of a parser

1.44k views • 89 slides
Carolyn Peterken Part O rt One ne Debunking the myths Contexts change But there have been

Carolyn Peterken Part O rt One ne Debunking the myths Contexts change But there have been

Carolyn Peterken Part O rt One ne Debunking the myths Contexts change But there have been leaders in diverse contexts over thousands of years So many models so much complexity But is there one right model and how central are

428 views • 26 slides
Plan for Today Predictive parsing as a specific subclass of recursive descent parsing

Plan for Today Predictive parsing as a specific subclass of recursive descent parsing

Plan for Today Predictive parsing as a specific subclass of recursive descent parsing complexity comparisons with general parsing might have to left-factor necessary to remove left-recursion (MiniSVG modification as example) Working

346 views • 5 slides
Lectures 7 and 8. Parsing (syntax analysis) Wei Le 2015.9 Bottom Up Parsing Recognize many

Lectures 7 and 8. Parsing (syntax analysis) Wei Le 2015.9 Bottom Up Parsing Recognize many

Lectures 7 and 8. Parsing (syntax analysis) Wei Le 2015.9 Bottom Up Parsing Recognize many programming languages, used in practice More general, more powerful parser: less restrictions on grammars, e.g., can handle left-recursive grammar

680 views • 64 slides
LL1 Parsing a b $ S 1 A 4 B 3 2 a b b b 1. S --> a B

LL1 Parsing a b $ S 1 A 4 B 3 2 a b b b 1. S --> a B

LL1 Parsing a b $ S 1 A 4 B 3 2 a b b b 1. S --> a B Input 2. B --> 3. B --> A S 4. A --> b B Parse Stack Compare Input and Stacktop 1 LL1 Parsing a b $ S 1 A

694 views • 17 slides
Programming Languages: Parsing Onur Tolga S ehito glu Computer Engineering,METU 27 May

Programming Languages: Parsing Onur Tolga S ehito glu Computer Engineering,METU 27 May

Programming Languages:Parsing Programming Languages: Parsing Onur Tolga S ehito glu Computer Engineering,METU 27 May 2009 Programming Languages:Parsing Outline 1 Parsing Top-down Parsing Recursive Descent Parser LL Parsers

215 views • 8 slides
Models of Human Parsing Experimental Data 2 Informatics 2A: Lecture 22 Eye-tracking Reading

Models of Human Parsing Experimental Data 2 Informatics 2A: Lecture 22 Eye-tracking Reading

Human Parsing Human Parsing Experimental Data Experimental Data Bottom-Up Parser Bottom-Up Parser Left Corner Parser Left Corner Parser Human Parsing 1 Cognitive Constraints Garden Paths Dimensions of Parsing Models of Human Parsing

537 views • 9 slides
Top-Down Parsing Top-Down Parsing #1 Extra Credit Question Given this grammar G: E

Top-Down Parsing Top-Down Parsing #1 Extra Credit Question Given this grammar G: E

Top-Down Parsing Top-Down Parsing #1 Extra Credit Question Given this grammar G: E E + T E T T T * int T int T ( E ) Is the string int * (int + int) in L(G)? Give a derivation or prove that it

1.08k views • 61 slides
CKY Parsing Ling 571 Deep Processing Techniques for NLP January 12, 2011 Roadmap

CKY Parsing Ling 571 Deep Processing Techniques for NLP January 12, 2011 Roadmap

CKY Parsing Ling 571 Deep Processing Techniques for NLP January 12, 2011 Roadmap Motivation: Parsing (In) efficiency Dynamic Programming Cocke-Kasami-Younger Parsing Algorithm Chomsky Normal Form Conversion

689 views • 42 slides

More recommend