[Webpack source code analysis] How to compile and package Webpack

[Webpack source code analysis] How to compile and package Webpack

Webpack source code analysis

This article will be longer, so it is recommended to collect it.


Essentially, webpack is a static module packaging tool for modern Javascript applications. When webpack processing application, it will be constructed inside of a dependency graph ( dependency Graph ), each module required by this project dependency graph maps, and generates more than one or more of the bundle .

Starting from v4.0.0, webpack can be used out of the box without using any configuration files. However, webpack will assume that the entry point of the project is src/index , and then output the result in dist/main.js , and enable compression and optimization in the production environment.

Usually, your project needs to continue to expand this capability, for this you can create a webpack.config.js file in the project root directory , and webpack will automatically use it.

Core idea

Compiler and Compilation

Compiler and Compilation classes are the core modules of webpack and both inherit from Tapable .

Compiler is responsible for controlling the entire life cycle of webpack packaging from a macro level, while Compilation is responsible for reading file content at a micro level, translating with loader, analyzing dependencies with AST syntax tree , recursively compiling dependent files, etc. work.

Overall packaging process

  1. Read webpack configuration and command line parameters to generate the final configuration;
  2. Start webpack , create Compiler or MultiCompiler instances according to the configuration , instantiate all internal plug-ins and plug-ins configured by yourself, and mount them to different hooks on the compiler through the instance method pluginInstance.apply(compiler) , and package the project;
  3. In the packaging phase ( compiler.run() ), there will be some preparations at the beginning, and then call the compiler 's compile method to enter the compilation preparation stage.
  4. The compilation preparation link will be triggered first
    Hook, then trigger
    Hook, and then create a new Compilation instance.
  5. During the creation of the compilation , it will trigger
    Hook, register the factory instances of the different types of modules created before to the dependencyFactories of compilation for
    Phase use.
  6. compiler.hooks.make
    The stage is the compilation stage. The most time-consuming link will trigger different hooks on the compilation . 1. it will start parsing from the entry file ( entry ), and acorn will generate the AST syntax tree , and replace the import, require and other syntax with webpack custom module loading methods, such as
    , And analyze the dependent files, generate a dependent list, repeat the previous operations, and compile recursively. After all modules are loaded, the make phase ends.
  7. After the make phase, compilation calls
    Method to enter the compilation
    Stage, will trigger
    Waiting for hooks, as can be seen from the hook name, Tree Shaking , Code Spliting , code compression, etc. are all completed at this stage. Then enter
  8. compiler.hooks.emit
    In the stage, the neo-async library is used to write files in parallel.


Read configuration

When it starts to execute

npx webpack
When, first check whether it is installed
(webpack-command is a simplified version of webpack-cli, currently deprecated). Then instantiate
, Call the run method.
Use commander to encapsulate commands and execute build command by default .

//webpack-cli/lib/webpack -cli.js 1454 rows const loadedConfig = the await loadConfig (foundDefaultConfigFile.path); const evaluatedConfig = the await evaluateConfig (loadedConfig, options.argv || {}); duplicated code

By default , the configuration information is read from webpack.config , .webpack/webpack.config , .webpack/webpackfile , and evaluateConfig is called to merge the configuration file information and command line parameters.

Instantiate Compiler

//webpack-cli/lib/webpack-cli.js line 1847 compiler = this .webpack( config.options, callback ? ( error, stats ) => { if (error && this .isValidationError(error)) { this .logger.error(error.message); process.exit( 2 ); } callback(error, stats); } : callback, ); Copy code

If config.options is an array, then instantiate

, Otherwise instantiate

= Options new new WebpackOptionsDefaulter () Process (Options);. Copy Code

The remaining unspecified default configuration will be added to the options .

new NodeEnvironmentPlugin({ infrastructureLogging : options.infrastructureLogging }).apply(compiler); Copy code

With the help of graceful-fs module, for
Provide the ability to read and write files.

if (options.plugins && Array .isArray(options.plugins)) { for ( const plugin of options.plugins) { if ( typeof plugin === "function" ) { plugin.call(compiler, compiler); } else { plugin.apply(compiler); } } } Copy code

Next, traverse all the configured

Method, subscribe to different
Life cycle. For example, what we commonly use
Will subscribe
Hook, the plug-in will replace the qualified code when compiling.

//webpack/lib/webpack.js 57 rows compiler.options = new new WebpackOptionsApply () Process (Options, Compiler);. Copy Code

WebpackOptionsApply based webpack configuration DevTools , target and other fields, again adding to the built-in plug-in configuration, and subscribe to the corresponding life cycle. The most important of these is the EntryOptionPlugin plug-in, without which webpack cannot find the entry file:

//webpack/lib/WebpackOptionsApply.js Line 290 new EntryOptionPlugin().apply(compiler); compiler.hooks.entryOption.call(options.context, options.entry); Copy code

EntryOptionPlugin will subscribe

Hook, and then trigger the hook in the next line. At this time, depending on whether it is a single entry or multiple entry, you can choose to instantiate SingleEntryPlugin or MultiEntryPlugin (when entry is a function, it will instantiate DynamicEntryPlugin ), and the plug-in will subscribe
. When the compiler triggers later
When hooking, it will execute
, From the entry file to perform dependency analysis, compilation and other operations.

Start (compiler.run())

After the above operations are completed, the compiler calls the run method to start packaging.

//webpack/lib/Compiler.js line 312 this .hooks.beforeRun.callAsync( this , err => { if (err) return finalCallback(err); this .hooks.run.callAsync( this , err => { if (err) return finalCallback(err); this .readRecords( err => { if (err) return finalCallback(err); this .compile(onCompiled); }); }); }); Copy code

Trigger in turn

Hook, and then call compile to enter the compilation phase.

Compilation preparation phase (compiler.compile(onCompiled))

//webpack/lib/Compiler.js line 661 const params = this.newCompilationParams(); //webpack/lib/Compiler.js line 651 newCompilationParams() { const params = { normalModuleFactory: this.createNormalModuleFactory(), contextModuleFactory: this.createContextModuleFactory(), compilationDependencies: new Set() }; return params; } Copy code

When the compiler 's compile method is called , a new one will be created for instantiation

The parameter object, which contains two module factories: normalModuleFactory and contextModuleFactory .

NormalModule is easy to understand. It is a normal module determined at compile time. However, you may be a little confused when you see the contextModule . What is this module?

require ( 'template/' + name + '.js' ); copy the code

When the above code appears, webpack will parse the require() call and extract useful information from it.

Directory: ./template Regular expression:/^.*\.js$/ Copy code

Then a contextModule will be generated.

If the following is a contextMoudle with an id of 2 , it contains a map containing references to all modules in the template directory:

{ 'a.js' : 10 , 'b.js' : 11 } Copy code


the require ( 'Template/' + name + '.js' ); //runtime name is A //compilation process will convert __webpack_require __ ( 21 is ) (name + '.js' ) copying the code

! Note: In order to meet the requirements of dynamic require, all eligible modules will be packaged into the bundle.

Next, it will trigger

Hook, and then create a new Compilation instance. During the creation process, it will trigger
Hook to register the factory instances of the different types of modules created before to the dependencyFactories of compilation of for
Phase use.

Compile stage compiler.hooks.make

Instantiate Compilation , enter

Stage, trigger SingleEntryPlugin the callback function of subscription, execute
, Starting from the entry file, use loader-runner to execute the qualified loader , then use acorn to generate the AST syntax tree, get the dependencies, and then recursively execute the previous operations until all dependencies are loaded.

//SingleEntryPlugin.js 40 lines compiler.hooks.make.tapAsync( "SingleEntryPlugin" , ( compilation, callback ) => { const {entry, name, context} = this ; const dep = SingleEntryPlugin.createDependency(entry, name); //dep is the entry module compilation.addEntry(context, dep, name, callback); //context is the result of process.cwd() } ); Copy code

Continue to call later
, And after a series of operations, it will execute
, This is the entry module to execute the compilation.

//NormalModule.js line 287 build ( options, compilation, resolver, fs, callback ) { //... return this .doBuild(options, compilation, resolver, fs, err => { //... }); } //... doBuild ( options, compilation, resolver, fs, callback ) { //Create a context shared by all loaders of the current module const loaderContext = this .createLoaderContext( resolver, options, compilation, fs ); // runLoaders({ resource : this .resource, //module path loaders : this .loaders, //loaders context : loaderContext, //context readResource : fs.readFile.bind(fs) //ability to read file content }) } Copy code

Will use readResouce to read the contents of the file, and execute the loader in the order from right to left . Then call
( Acorn ) Generate the AST syntax tree, read the dependencies and store them in the dependencies of the module , and then call the addModuleDependencies method of compilation to process each dependency asynchronously using the neo-async library .

Icon of the make phase:

Code optimization (compilation.seal stage)

I intercepted part of the source code to see what was done in the seal phase?

//Compilation.js line 1186 seal(callback) { this.hooks.seal.call(); while ( this.hooks.optimizeDependenciesBasic.call(this.modules) || this.hooks.optimizeDependencies.call(this.modules) || this.hooks.optimizeDependenciesAdvanced.call(this.modules) ) { /* empty */ } this.hooks.afterOptimizeDependencies.call(this.modules); this.hooks.beforeChunks.call(); //... } Copy code

It can be seen that at this stage, the optimization configuration of webpack.config.js is in effect, and code optimization work such as Tree Shaking and Code Spliting is completed at this stage.

Output files to the specified output directory (compiler.hooks.emit stage)

The code is optimized, it will be executed almost

The onCompiled callback function in, let s take a look at what is inside?

const onCompiled = ( err, compilation ) => { //... this .emitAssets(compilation, err => { if (err) return finalCallback(err); //... }); //... } Copy code

We see that it is called again

Method, the meaning of the output file. The specific source code is not shown, it is used inside
Asynchronous output file, file
The operation is through


Webpack will just talk about it here, there are still some template replacements, how to implement Code Spliting, and so on. If you are interested, you can take a look at the source code for yourself.

Here is a simple webpack packaging logic for a webpack demo I wrote (the loader-runner and the final output logic is not finished, you can also refer to it).

Reference :