Parsing @ IDE V. Zaytsev @ Parsing @ SLE @ SPLASH Grammars in a - PowerPoint PPT Presentation

Sep 25, 2023 •312 likes •506 views

Parsing @ IDE V. Zaytsev @ Parsing @ SLE @ SPLASH Grammars in a broad sense Grammars in a narrow sense Which one? recognise programs in a language parse and interpret parse and translate parse and compile semiparse and

Parsing @ IDE V. Zaytsev @ Parsing @ SLE @ SPLASH
Grammars in a broad sense
Grammars in a narrow sense Which one? recognise programs in a language ● parse and interpret ● ● parse and translate ● parse and compile semiparse and analyse ● document ● ● domain model ● verify & validate
Grammars for IDE support ● idea from attending PLDI ● What is specific to grammars used in IDEs? What IDE features need grammar support? ● ● How to provide it better? ● OK to be “in a broad sense”
Main principles fast ● ● partial not limited by parsing in a narrow sense ●
Syntax highlighting ● colour-code tokens ● commonly implemented with regexes ● easy if the tokeniser is precise ● good luck with C++, PL/I, etc ● embryos of common interfaces ● novel solutions possible
Code completion ● complete keywords ● suggest matching tokens guide indentation ● name suggestions ● ● drop down members
Word selection ● select a word, highlight “the same thing” ● “cheap” visualisation ● liked by devs ● not researched at all
Code folding ● blocks in composite statements ○ NOT a solution! ● hierarchical entities handful of top constructs? ●
Visual editing ● let graphs be graphs ● let tables be tables ● let window panels be window panels ● don’t let your dreams be dreams! ● projectional?
Debugging ● only relevant for executable programs ● step over ● step into ● breakpoint ● watch ● cross language boundaries
Testing ● discover tests ● running tests live ○ ● analysing tests ○ green and red ○ coverage ● incrementality why not advanced features? ● ○ model-based ○ test generation ○ fuzzing ○ mutants
Coding conventions ● formatting ● pretty-printing ● naming calling ● ● templates ● deprecating language constructs ● satisfying global constraints smell detection ●
Refactoring ● recommender systems ● ReSharper, CodeMaid, etc ● grammars are originally rewriting systems seldom used for rewriting ● ● can be insanely complex ● hard to do right ● hard to v&v
Navigating the codebase ● go to definition ● find references ● analyse dependencies ● analyse co-changes
Configuring a build ● compiling ● deploying ● delivering ● versioning ● building in the right order
Helping ● tooltips ● hover infoboxes ● API guidance ● explaining errors recommending fixes ○
Conclusion ● IDEs are built ad hoc IDEs are built with a framework bias ● ● there is [or can be] a class of IDE-specific grammars ● mostly greenfield research ● way beyond [single] grammars [in a narrow sense] vastly different user stories ● ○ JS: live to the extreme C++: many changes, always incomplete info ○ ○ C#: style and paradigm switching PL/I: cache to the extreme (yesterday’s trees are good enough) ○ ● Please do it (willing to collab)

Recommend

CSC 4181 Compiler Construction Parsing 1 1 Outline Top-down v.s. Bottom-up Top-down parsing

CSC 4181 Compiler Construction Parsing 1 1 Outline Top-down v.s. Bottom-up Top-down parsing Bottom-up parsing Recursive-descent Shift-reduce parsers parsing LR(0) parsing LL(1) parsing LR(0) items LL(1) parsing

429 views • 29 slides

Introduction to Bottom-Up Parsing Shift-reduce parsing The LR parsing algorithm

Outline Review LL parsing Introduction to Bottom-Up Parsing Shift-reduce parsing The LR parsing algorithm Constructing LR parsing tables 2 Top-Down Parsing: Review Top-Down Parsing: Review Top-down parsing expands a

471 views • 13 slides

Top-Down Parsing Slides modified from Louden Book and Dr. Scherger Top Down Parsing A

Compiler Design and Construction Top-Down Parsing Slides modified from Louden Book and Dr. Scherger Top Down Parsing A top-down parsing algorithm parses an input string of tokens by tracing out the steps in a leftmost derivation. Such an

1.12k views • 96 slides

Statistical Dependency Parsing in Korean: From Corpus Generation To Automatic Parsing Workshop on

Statistical Dependency Parsing in Korean: From Corpus Generation To Automatic Parsing Workshop on Statistical Parsing of Morphologically-Rich Languages 12th International Conference on Parsing Technologies Jinho D. Choi & Martha Palmer

199 views • 19 slides

Basic Parsing Algorithms Chart Parsing Seminar Recent Advances in Parsing Technology WS

Basic Parsing Algorithms Chart Parsing Seminar Recent Advances in Parsing Technology WS 2011/2012 Anna Schmidt Talk Outline Chart Parsing Basics Chart Parsing Algorithms Earley Algorithm CKY Algorithm Basics

1.12k views • 73 slides

Overview Introduction Lexicalized TAG, Advantages of parsing with LTAG Parsing LTAGs

Parsing with Lexicalized TAG (1) Extracting and comparing LTAG (2) Presentation by Philip John Gorinski Seminar Recent Advances in Parsing Technology Saarland University, Winter Term 2011/12 (1) Yves Shabes, Aravind K. Joshi, 1990 (2)

514 views • 36 slides

* 07/16/96 Plan for Today Shift-reduce parsing The problem with predictive top down parsing

* 07/16/96 Plan for Today Shift-reduce parsing The problem with predictive top down parsing CS453 : Shift Reduce Parsing LR parsing: bottom up Unambiguous Grammars LR(0) and SLR Parse Tables Performs a right-most derivation in

316 views • 6 slides

1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G ,

1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G , to decide if w L ( G ), and if so, to construct a derivation or a parse tree for w . Parsing is studied in courses in compilers. To be efficient

520 views • 12 slides

Bottom-up parsing LR parsing Construct parse tree for input from leaves up LR( k ) parsing

Bottom-up parsing LR parsing Construct parse tree for input from leaves up LR( k ) parsing reducing a string of tokens to single start symbol L eft-to-right scan of input, R ightmost derivation (inverse of deriving a string of tokens from

413 views • 5 slides

Bottom Up Parsing Also known as Shift-Reduce parsing More powerful than top down Dont

9/26/2012 Bottom Up Parsing Also known as Shift-Reduce parsing More powerful than top down Dont need left factored grammars Can handle left recursion Bottom Up Parsing Attempt to construct parse tree from an input string

655 views • 6 slides

Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP

Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP February 2, 2015 Roadmap Dependency parsing Graph-based dependency parsing Maximum spanning tree CLE Algorithm Learning

899 views • 87 slides

Statistical Parsing Parsing context-free languages ar ltekin University of Tbingen

Statistical Parsing Parsing context-free languages ar ltekin University of Tbingen Seminar fr Sprachwissenschaft November 8, 2016 Recap Parsing basics CKY Earley Closing Ingredients of a (natural language) parser .

1.17k views • 51 slides

Generalised Parsing and Combinator Parsing A Happy Marriage? L. Thomas van Binsbergen

Conventional Parser Combinators Aspects of Generalised Parsing Combinators for Generalised Parsing Generalised Parsing and Combinator Parsing A Happy Marriage? L. Thomas van Binsbergen ltvanbinsbergen@acm.org Royal Holloway, University of

452 views • 30 slides

Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs

Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs Semantic Parsing Dependency parsing models the syntactic structure between words in a sentence. Dependency Parsing vs Semantic Parsing

637 views • 31 slides

Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr

Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr Sprachwissenschaft November 2016 Recap/background Dependency grammar Dependency parsing Evaluation Summary Ingredients of a parser

1.44k views • 89 slides

Plan for Today Predictive parsing as a specific subclass of recursive descent parsing

Plan for Today Predictive parsing as a specific subclass of recursive descent parsing complexity comparisons with general parsing might have to left-factor necessary to remove left-recursion (MiniSVG modification as example) Working

346 views • 5 slides

Lectures 7 and 8. Parsing (syntax analysis) Wei Le 2015.9 Bottom Up Parsing Recognize many

Lectures 7 and 8. Parsing (syntax analysis) Wei Le 2015.9 Bottom Up Parsing Recognize many programming languages, used in practice More general, more powerful parser: less restrictions on grammars, e.g., can handle left-recursive grammar

680 views • 64 slides

LL1 Parsing a b $ S 1 A 4 B 3 2 a b b b 1. S --> a B

LL1 Parsing a b $ S 1 A 4 B 3 2 a b b b 1. S --> a B Input 2. B --> 3. B --> A S 4. A --> b B Parse Stack Compare Input and Stacktop 1 LL1 Parsing a b $ S 1 A

694 views • 17 slides

Programming Languages: Parsing Onur Tolga S ehito glu Computer Engineering,METU 27 May

Programming Languages:Parsing Programming Languages: Parsing Onur Tolga S ehito glu Computer Engineering,METU 27 May 2009 Programming Languages:Parsing Outline 1 Parsing Top-down Parsing Recursive Descent Parser LL Parsers

215 views • 8 slides

Models of Human Parsing Experimental Data 2 Informatics 2A: Lecture 22 Eye-tracking Reading

Human Parsing Human Parsing Experimental Data Experimental Data Bottom-Up Parser Bottom-Up Parser Left Corner Parser Left Corner Parser Human Parsing 1 Cognitive Constraints Garden Paths Dimensions of Parsing Models of Human Parsing

537 views • 9 slides

Top-Down Parsing Top-Down Parsing #1 Extra Credit Question Given this grammar G: E

Top-Down Parsing Top-Down Parsing #1 Extra Credit Question Given this grammar G: E E + T E T T T * int T int T ( E ) Is the string int * (int + int) in L(G)? Give a derivation or prove that it

1.08k views • 61 slides

CKY Parsing Ling 571 Deep Processing Techniques for NLP January 12, 2011 Roadmap

CKY Parsing Ling 571 Deep Processing Techniques for NLP January 12, 2011 Roadmap Motivation: Parsing (In) efficiency Dynamic Programming Cocke-Kasami-Younger Parsing Algorithm Chomsky Normal Form Conversion

689 views • 42 slides

Parsing Parsing involves: determining if a string belongs to a language, and

Parsing Parsing involves: determining if a string belongs to a language, and constructing structure of string if it belongs to language. Two approaches to constructing parsers: 1. Top down parsing: Our focus is on table-driven

428 views • 18 slides

Compiler Construction Lecture 7: Bottom-up parsing 2020-01-28 Michael Engel Includes material

Compiler Construction Lecture 7: Bottom-up parsing 2020-01-28 Michael Engel Includes material by Jan Christian Meyer and Rich Maclin (UNM) Overview Top-down parsing revisited Bottom-up parsing Comparison to top-down parsing

539 views • 20 slides