Search for a command to run...
I'm Manuel and I work at NearForm as a Full Remote Software Developer from 🇮🇹 Italy. I'm one of the Fastify maintainers since 2019. Contributing to Open Source Software teaches me something new every day. You should join this extraordinary world.
No comments yet. Be the first to comment.
How to stream React components to the browser using Fastify and reply.hijack()
Did you see the "Verified" badge on some GitHub commits and wonder how to get it for your own commits? Well,GitHub shows the ‘Verified’ badge when a commit has a valid signature from a trusted key (GPG, SSH, S/MIME, or GitHub’s own). In this article...
Part 0: From Instinct to System
We share smart strategies for ensuring a slow client doesn’t crash your server
I was looking at the Fastify plugin list and found myself asking, "What is this @fastify/request-context plugin?"So, I checked it out, and here I am, writing a post about it to share what it is and how it can be useful for you! What is AsyncLocalStor...
You may have seen the term "AST" in multiple contexts without digging into what it actually means. It is a common term in compiler theory, and it is also used in the context of static analysis. In this post, we will see what an AST is, and we will focus on how it is a key part of the JavaScript language and ecosystem.
The AST acronym stands for Abstract Syntax Tree. It is a tree data structure representing any structured text file, so every standardized syntax can be represented through an AST.
Since the AST is "abstract", it does not have a standard representation, because every language may have its own specific dictionary.
However, the common concept shared across all the ASTs, is the tree representation where the first node describes the document/program's entry point.
Generally, the first step to run a piece of software is to parse the source code and build an AST. This operation is called parsing and is performed by the parser component.
📄 Source ➞ ⚙️ Parser ➞ 🌲 AST
If the parser is unable to parse the source code, it will throw an error. The source code is invalid and the AST cannot be generated.
The AST is generated to process source code files, but it can be generated from any text file such as Markdown, JSON, or even a GraphQL Document.
When the parser builds a valid AST, the following step is the transform:
📄 Source ➞ ⚙️ Parser ➞ 🌲 AST ➞ ⚙️ Transform ➞ 🌲 AST (Transformed)
The AST is manipulated and transformed during this phase to generate a new AST.
Some examples of transformations are:
the babel tool that transforms the AST to generate a new AST that is compatible with the target environment
source code minification, where the AST is transformed to remove unnecessary characters and to reduce the file size
code formatting tools, where the AST is transformed to add/remove spaces and new lines to improve the code readability
Finally, the new AST is then passed to the compiler:
📄 Source ➞ ⚙️ Parser ➞ 🌲 AST ➞ ⚙️ Transform ➞ 🌲 AST (Transformed) ➞ ⚙️ Compiler ➞ 📄 Output
The compiler generates output such as:
bytecode to be executed
a new source code file derived from the original source code
some console output such as a warning or an error or suggestions
All the tools we use in our daily work are based on the AST and are used to improve developer experience.
The code completion, refactoring, linting and formatting are all powered by the source code tree representation! This is how our IDE and powerful editors implement extraordinary features!
Let's focus on the Node.js runtime and see how the AST is built.
JavaScript is a dynamic language, but the first step for every high-level programming language is to parse the source code and build an AST. Without completing this step, our code will never be executed.
In fact, if you try to run a JS file that contains invalid JavaScript code, the runtime will throw a Syntax error even if the function is never called.
function foo() {
~ this is not javascript code! ~
}
Running node index.js will throw the error:
~ this is not javascript code! ~
^^
SyntaxError: Unexpected identifier
Under the hood, Node.js relies on the Google V8 engine to parse the source code and build the AST. The tree representation is then passed to the Ignition interpreter that builds the final bytecode.
The AST generated by V8 is not accessible from the JavaScript code, but it is possible to generate an AST from JavaScript using one of the many available libraries:
acorn: the core module of many parsers
espree: the eslint's parser
@typescript-eslint/typescript-estree: used by prettier
...and a lot more
For example, the following code:
function hello(name) {
console.log('Hello ' + name)
}
hello('foo')
The following AST can represent it:
{
"type": "Program",
"start": 0,
"end": 70,
"body": [
{
"type": "FunctionDeclaration",
"start": 0,
"end": 56,
"id": {
"type": "Identifier",
"start": 9,
"end": 14,
"name": "hello"
},
"expression": false,
"generator": false,
"async": false,
"params": [
{
"type": "Identifier",
"start": 16,
"end": 20,
"name": "name"
}
],
"body": {
"type": "BlockStatement",
"start": 22,
"end": 56,
"body": [
{
"type": "ExpressionStatement",
"start": 26,
"end": 54,
"expression": {
"type": "CallExpression",
"start": 26,
"end": 54,
"callee": {
"type": "MemberExpression",
"start": 26,
"end": 37,
"object": {
"type": "Identifier",
"start": 26,
"end": 33,
"name": "console"
},
"property": {
"type": "Identifier",
"start": 34,
"end": 37,
"name": "log"
},
"computed": false,
"optional": false
},
"arguments": [
{
"type": "BinaryExpression",
"start": 38,
"end": 53,
"left": {
"type": "Literal",
"start": 38,
"end": 46,
"value": "Hello ",
"raw": "'Hello '"
},
"operator": "+",
"right": {
"type": "Identifier",
"start": 49,
"end": 53,
"name": "name"
}
}
],
"optional": false
}
}
]
}
},
{
"type": "ExpressionStatement",
"start": 58,
"end": 70,
"expression": {
"type": "CallExpression",
"start": 58,
"end": 70,
"callee": {
"type": "Identifier",
"start": 58,
"end": 63,
"name": "hello"
},
"arguments": [
{
"type": "Literal",
"start": 64,
"end": 69,
"value": "foo",
"raw": "'foo'"
}
],
"optional": false
}
}
],
"sourceType": "module"
}
In the initial section, we said that the AST is not standardized.
Moreover, the AST generated by v8 is a tuned and optimized AST designed for its engine, so it is a tailor-made AST.
As mentioned before, many libraries can generate an AST from a JavaScript source code. At this point, the ESTree community organization defines an (unofficial) standard for JavaScript ASTs.
However, there is a nice story behind the JavaScript AST baseline structure:
Once upon a time, an unsuspecting Mozilla engineer created an API in Firefox that exposed the SpiderMonkey engine's JavaScript parser as a JavaScript API. Said engineer documented the format it produced, and this format caught on as a lingua franca for tools that manipulate JavaScript source code.
The specification maintained by the ESTree community follows the ECMAScript standard and its naming conventions.
A handy tool to explore the AST is AST Explorer.
You can copy-paste the source code and see the AST generated by the selected parser.
ℹ️ Navigate the AST
I find it very useful to explore the AST generated by a GraphQL schema when I need to implement amercuriusplugin!
Of course, an awesome list 🕶 of AST-related projects exists!
I hope this AST post was helpful to understand how the AST is used in the JavaScript ecosystem.
Now you should see your IDE and the eslint tool with different eyes! There was no magic, just a tree data structure!
If you are willing to build an eslint or graphql plugin, the information in this post will help you to start your journey!
If you enjoyed this article, comment, share, and follow me on Twitter @ManuEomm!