Syntactic list of tokens analysis Syntactic analyzer grammar: - PowerPoint PPT Presentation

Syntactic list of tokens analysis Syntactic analyzer ● grammar: context free ● format: BNF ● (implementation: pushdown (stack) automaton ) parse tree (symbol table?) 90

● Check that the program respects the syntax Description ● Generate parse tree and (and maybe of the start symbol table) for code generation syntax ● Context free grammar – Parsing is represented by rules – Most commonly using BNF (or EBNF) – Rules do not depend on the context in which they appear. 91

● Notation to describe syntax ● Widely used in many areas of computing BNF (Backus-Naur ● Describe syntax by giving syntax rules Form) on how things can be composed from other things ● Terminal symbol : basic tokens coming from lexical analysis (not composed of other symbols) ● Non-terminal symbol : symbol created by a syntax rule from other symbols (both terminal and non-terminal) 92

Simple <variable definition> ::= examples of identifier identifier '=' <expression> ';' syntactic (type) (name) rules <iteration statement> ::= 'while' '(' <expression> ')' <statement> <assignment> ::= identifier '=' <expression> ';' <statement> ::= <iteration statement> <statement> ::= <assignment> <statement> ::= <compound statement> Statements, <compound statement> ::= '{' <statement seq> '}' Code block <statement seq> ::= <statement> <statement seq> ::= <statement> <statement seq> 93

BNF rules: <expr> ::= <expr> + <term> Example of <expr> ::= <expr> - <term> a grammar <expr> ::= <term> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> ) Non-terminals expr, term, factor, number terminals number , ( , ) , + , - , * , / expr Beginning symbol 94

2 * ( 12 + 3 ) Deriving exp. <expr> ::= <expr> + <term> 2 * ( 12 + <expr> ::= <expr> - <term> <expr> ::= <term> 3 ) <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> ) 95

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) Deriving exp. <expr> ::= <expr> + <term> 2 * ( 12 + <expr> ::= <expr> - <term> <expr> ::= <term> 3 ) <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> ) 96

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) Deriving exp. <expr> ::= <expr> + <term> 2 * ( 12 + <expr> ::= <expr> - <term> <expr> ::= <term> 3 ) <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> ) 97

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) Deriving = number * ( <term> + <term> ) exp. <expr> ::= <expr> + <term> 2 * ( 12 + <expr> ::= <expr> - <term> <expr> ::= <term> 3 ) <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> ) 98

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) Deriving = number * ( <term> + <term> ) = number * ( <expr> + <term> ) exp. <expr> ::= <expr> + <term> 2 * ( 12 + <expr> ::= <expr> - <term> <expr> ::= <term> 3 ) <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> ) 99

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) Deriving = number * ( <term> + <term> ) = number * ( <expr> + <term> ) exp. = number * ( <expr> ) <expr> ::= <expr> + <term> 2 * ( 12 + <expr> ::= <expr> - <term> <expr> ::= <term> 3 ) <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> ) 100

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) Deriving = number * ( <term> + <term> ) = number * ( <expr> + <term> ) exp. = number * ( <expr> ) <expr> ::= <expr> + <term> = number * <factor> 2 * ( 12 + <expr> ::= <expr> - <term> <expr> ::= <term> 3 ) <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> ) 101

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) Deriving = number * ( <term> + <term> ) = number * ( <expr> + <term> ) exp. = number * ( <expr> ) <expr> ::= <expr> + <term> = number * <factor> 2 * ( 12 + <expr> ::= <expr> - <term> = <factor> * <factor> <expr> ::= <term> 3 ) <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> ) 102

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) Deriving = number * ( <term> + <term> ) = number * ( <expr> + <term> ) exp. = number * ( <expr> ) <expr> ::= <expr> + <term> = number * <factor> 2 * ( 12 + <expr> ::= <expr> - <term> = <factor> * <factor> <expr> ::= <term> 3 ) = <term> * <factor> <term> ::= <term> * <factor> <term> ::= <term> / <factor> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> ) 103

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) Deriving = number * ( <term> + <term> ) = number * ( <expr> + <term> ) exp. = number * ( <expr> ) <expr> ::= <expr> + <term> = number * <factor> 2 * ( 12 + <expr> ::= <expr> - <term> = <factor> * <factor> <expr> ::= <term> 3 ) = <term> * <factor> <term> ::= <term> * <factor> <term> ::= <term> / <factor> = <term> <term> ::= <factor> <factor> ::= number <factor> ::= ( <expr> ) 104

2 * ( 12 + 3 ) (lexer) = number * ( number + number ) = number * ( <factor> + <factor> ) Deriving = number * ( <term> + <term> ) = number * ( <expr> + <term> ) exp. = number * ( <expr> ) <expr> ::= <expr> + <term> = number * <factor> 2 * ( 12 + <expr> ::= <expr> - <term> = <factor> * <factor> <expr> ::= <term> 3 ) = <term> * <factor> <term> ::= <term> * <factor> <term> ::= <term> / <factor> = <term> <term> ::= <factor> = <expr> (Success!) <factor> ::= number <factor> ::= ( <expr> ) 105

Expression 2 * ( 12 + 3 ) E = expression E as a parse tree Tm = term Fc = factor Tm Tm Fc Fc E E Tm Tm Fc Fc num(2) * ( num(12) + num(3) ) 106

Ambiguous grammar <expr> ::= <expr> + <expr> <expr> ::= <expr> - <expr> <expr> ::= <expr> * <expr> <expr> ::= <expr> / <expr> <expr> ::= ( <expr> ) <expr> ::= number 107

"Dangling else" Ambiguity if E1 then if E1 then example if E2 then if E2 then S1 S1 Pascal: else else (also C/C++) S2 S2 Grammar: <stmt> ::= <if_stmt> | ... <if_stmt> ::= if <expr> then <stmt> <if_stmt> ::= if <expr> then <stmt> else <stmt> 108

<if-stmt> <if-stmt> ”Dangling else” in if <expr> then <stmt> else <stmt> if <expr> then <stmt> parse trees <if-stmt> <if-stmt> if <expr> then <stmt> if <expr> then <stmt> else <stmt> <stmt> ::= <if_stmt> | ... <if_stmt> ::= if <expr> then <stmt> <if_stmt> ::= if <expr> then <stmt> else <stmt> 109