IT:AD:Irony

Summary

Irony is a language parser that is getting traction.

Converts Text into an Abstract Syntax Tree (AST) in two steps (Scanner, Parser) Irony extends this by optionally injecting Token Filters between the two modules:

Text Source -> SCANNER -> tokens -> (TOKEN FILTER)* ->tokens -> PARSER -> AST

The Scanner/Lexer/Tokenizer:

* converts text stream into meaninful Tokens (Terminal/Non-Terminal).
* exclude whitespace and comments.
* pass the Tokens to the any `Token Filters` first, then the `Parser`, in order to assemble into an AST.

The optional Token Filters can be used to inject extra tokens before or after received tokens, or inject new tokens.

The Parser is a LALR(1), or Shift/Reduce parser.

  • Starts in Initial State
  • Consumes input tokens
  • Retrives appropripriate Action from the Action Table for the current State, and Executes it.
  • Arrives at Final State.

The result is a tree of Tokens (Shifted from the input) or replaced/Reduced Abstract Tokens.