What does it do?

Transformers take in one AST and turn it into a different one. In this case we will use a Transformer to take the AST we got from the Parser and turn it into one that we can use in the Code Generator. Since we have control of all parts of the compiler, we had the option of making the AST we needed directly from the parser. We could jump directly from the Parser to the Code Generator by being a thinking ahead when building the first AST, however this is a step that I want to learn about and it helps us visualize the steps of the compiler as different parts of a final process that can be isolated and can work with other steps as necessary.

In reality, there are many different parsers that different tools use that may fit their needs. ESLint uses the 'espree' parser, which produces an AST that fits their needs. While TypeScript uses its own parser that once again, fits their needs. However, the espree parser is an isolated product that could very well be used by other projects to produce a familiar AST.

The Goal

The AST that we will be transforming into will be very similar to the original, with some exceptions:

  1. Program:
    1. Every line in the body property is an ExpressionStatement node instead of a CallExpression node
      1. Expression Nodes represent a line
  2. CallExpression
    1. New 'callee' property contains an Identifier node, which just identifies/gives a name to a variable
      1. This replaces the 'name' property
    2. 'arguments' property replaces the 'params' array, serves the same purpose
      1. 'arguments' is the correct nomenclature since this is data we pass into the function
  3. NumberLiteral
    1. Unchanged
  4. StringLiteral
    1. Unchanged
  5. ExpressionStatement (new node)
    1. Represent a line of our program.
    2. A Program can have many Expressions.
    3. An Expression can only have one direct child CallExpression

The new AST is defined in TypeScript like so:

https://gist.github.com/SammyIsra/cf5cf8130bc418229ff9342bc7a5c677

You can see a comparison of two trees that represent the exact same code but in the different formats here, using the same code we have been compiling in the previous sections:

https://gist.github.com/SammyIsra/0f7d350f1b50f71b3fa710df0e5d463f

https://gist.github.com/SammyIsra/3bc08becf86c7d06d819dd2dddd61b6e

Remember, we go SimpleAST ⇒ TransformedAST.

Also, quick note that AST are generally large by nature. Printing them like this and seeing all of it usually is not very viable. You can imagine what an actual AST with hundreds of lines of code and with nodes that have a lot more data would look like.

This step is very different from the Transformer that was defined in the original the-super-tiny-compiler

This is the one step of the compiler where I decided to scrap the existing logic and write my own transformer based on what the expected input of the Code Generator is, and the output of the Parser is.

Because there exist two versions of the transformer, I will explain both in some detail. However, I only implemented one transformer, so the original Transformer will be explained in concept and not with a demonstration.