Webpack ~ Build Your Own - rohit120582sharma/Documentation GitHub Wiki

The inner workings of a bundler

Bundler implementation consists of three main parts. At first, the entry file is parsed and its dependencies are extracted. Next, a dependency graph is built recursively as depicted in the example below. Finally, everything is packaged into a single file that can run properly in the browser.

At the first stage the input is parsed and an Abstract Syntax Tree (AST) is generated. The AST is then traversed in order to extract the dependencies, such as those denoted by import statements.

Then uses a queue-based implementation to create the dependency graph. An efficient queue is used instead of recursion. In this process every such value child is also pushed to the queue, leading to the processing of all child assets as well, and eventually mapping the dependencies of the entire input.

References

Build your own Webpack - Ronen Amiel

Useful tools for parsing, AST traversing, transpiling and more

Babylon JS parser

Babylon is JavaScript Parser which is used in Babel too. Babylon parses JavaScript and creates an AST out of it. JavaScript Parser is a tool that can read and understand JavaScript and It can generate a higher level model called AST (Abstract Syntax Tree)

AST explorer

AST explorer is a utility that displays ASTs such as the ones generated by the Babylon JS parser.

Babel-traverse

Babel-traverse is used for traversing the AST generated by the Babylon JS parser. Babel-traverse maintains the overall tree state, and is responsible for replacing, removing, and adding nodes.

Path

This native Node.js module handles file paths. In this case, it is used to retrieve the absolute path of each input file based on its relative path.

Babel

Babel is a JavaScript transpiler that converts edge JavaScript into plain old ES5 JavaScript that can run in any browser (even the old ones). It makes available all the syntactical sugar that was added to JavaScript with the new ES6 specification, including classes, fat arrows and multiline strings.

The Babel transpiler transforms the code, which in this case is formatted as an AST, to make it compatible with any browser.

Babel-preset-env

In Babel, a preset is a set of plugins used to support particular language features. Babel-preset-env is a new preset that lets you specify an environment and automatically enables the necessary plugins.

Getting Start

Let’s install following dependencies:

npm install --save [email protected] [email protected] [email protected] [email protected]

Run the application

node bundler.js

Coding

bundler.js

const fs = require('fs');
const path = require('path');
const babylon = require('babylon');
const babelTraverse = require('babel-traverse').default;
const babel = require('babel-core');

let ID = 0;

/**
 * 
 * @param {string} filename 
 * Read the content of the file using `fs` module
 * Parse it using `babylon` module
 * Traverse and extract "ImportDeclaration" using `babel-traverse` module
 * Transform using `babel-core` module
 */
function createAsset(filename) {
    const content = fs.readFileSync(filename, 'utf-8');
    const ast = babylon.parse(content, {
        sourceType: 'module'
    });
    const dependencies = [];
    babelTraverse(ast, {
        ImportDeclaration: ({node}) => {
            dependencies.push(node.source.value);
        }
    });
    const id = ID++;
    const {code} = babel.transformFromAst(ast, null, {
        presets: ['env']
    });

    return {
        id,
        filename,
        dependencies,
        code,
    };
}

function createGraph(filename) {
    const mainAsset = createAsset(filename);
    const queue = [mainAsset];

    for(let asset of queue) {
        const dirname = path.dirname(asset.filename);
        asset.mapping = {};
        asset.dependencies.forEach(relativePath => {
            const absolutePath = path.join(dirname, relativePath);
            const childAsset = createAsset(absolutePath);
            asset.mapping[relativePath] = childAsset.id;
            queue.push(childAsset);
        });
    }
    return queue;
}

function bundle(graph) {
    let modules = ``;
    graph.forEach(mod => {
        modules += `${mod.id}: [
            function(require, module, exports) { ${mod.code} },
            ${JSON.stringify(mod.mapping)},
        ],`;
    });
    return `(function(modules){
        function require(id){
            const localModule = {
                exports: {}
            };
            const [fn, mapping] = modules[id];
            function localRequire(relativePath) {
                return require(mapping[relativePath]);
            }
            fn(localRequire, localModule, localModule.exports);
            return localModule.exports;
        }
        require(0);
    })({${modules}})`;
}

const graph = createGraph('./src/index.js');
const result = bundle(graph);

console.log(result);

src/index.js

import message from './message.js';

console.log(message);

src/message.js

import name from './name.js';

const message = `Hello ${name}`;
export default message;

src/name.js

const name = 'Wix Engineering!';

export default name;