[Webpack source code analysis] How to compile and package Webpack

[Webpack source code analysis] How to compile and package Webpack

Webpack source code analysis

This article will be longer, so it is recommended to collect it.

concept

Essentially, webpack is a static module packaging tool for modern Javascript applications. When webpack processing application, it will be constructed inside of a dependency graph ( dependency Graph ), each module required by this project dependency graph maps, and generates more than one or more of the bundle .

Starting from v4.0.0, webpack can be used out of the box without using any configuration files. However, webpack will assume that the entry point of the project is src/index , and then output the result in dist/main.js , and enable compression and optimization in the production environment.

Usually, your project needs to continue to expand this capability, for this you can create a webpack.config.js file in the project root directory , and webpack will automatically use it.

Core idea

Compiler and Compilation

Compiler and Compilation classes are the core modules of webpack and both inherit from Tapable .

Compiler is responsible for controlling the entire life cycle of webpack packaging from a macro level, while Compilation is responsible for reading file content at a micro level, translating with loader, analyzing dependencies with AST syntax tree , recursively compiling dependent files, etc. work.

Overall packaging process

  1. Read webpack configuration and command line parameters to generate the final configuration;
  2. Start webpack , create Compiler or MultiCompiler instances according to the configuration , instantiate all internal plug-ins and plug-ins configured by yourself, and mount them to different hooks on the compiler through the instance method pluginInstance.apply(compiler) , and package the project;
  3. In the packaging phase ( compiler.run() ), there will be some preparations at the beginning, and then call the compiler 's compile method to enter the compilation preparation stage.
  4. The compilation preparation link will be triggered first
    compiler.hooks.beforeCompile
    Hook, then trigger
    compiler.hooks.compile
    Hook, and then create a new Compilation instance.
  5. During the creation of the compilation , it will trigger
    compiler.hooks.compilation
    Hook, register the factory instances of the different types of modules created before to the dependencyFactories of compilation for
    compiler.hooks.make
    Phase use.
  6. compiler.hooks.make
    The stage is the compilation stage. The most time-consuming link will trigger different hooks on the compilation . 1. it will start parsing from the entry file ( entry ), and acorn will generate the AST syntax tree , and replace the import, require and other syntax with webpack custom module loading methods, such as
    __webpack_require__
    , And analyze the dependent files, generate a dependent list, repeat the previous operations, and compile recursively. After all modules are loaded, the make phase ends.
  7. After the make phase, compilation calls
    seal
    Method to enter the compilation
    seal
    Stage, will trigger
    compilation.hooks.seal
    ,
    compilation.hooks.optimize
    ,
    compilation.hooks.optimizeTree
    Waiting for hooks, as can be seen from the hook name, Tree Shaking , Code Spliting , code compression, etc. are all completed at this stage. Then enter
    emit
    stage.
  8. compiler.hooks.emit
    In the stage, the neo-async library is used to write files in parallel.

Icon:

Read configuration

When it starts to execute

npx webpack
When, first check whether it is installed
webpack-cli
or
webpack-command
(webpack-command is a simplified version of webpack-cli, currently deprecated). Then instantiate
WebpackCli
, Call the run method.
webpack-cli
Use commander to encapsulate commands and execute build command by default .

//webpack-cli/lib/webpack -cli.js 1454 rows const loadedConfig = the await loadConfig (foundDefaultConfigFile.path); const evaluatedConfig = the await evaluateConfig (loadedConfig, options.argv || {}); duplicated code

By default , the configuration information is read from webpack.config , .webpack/webpack.config , .webpack/webpackfile , and evaluateConfig is called to merge the configuration file information and command line parameters.

Instantiate Compiler

//webpack-cli/lib/webpack-cli.js line 1847 compiler = this .webpack( config.options, callback ? ( error, stats ) => { if (error && this .isValidationError(error)) { this .logger.error(error.message); process.exit( 2 ); } callback(error, stats); } : callback, ); Copy code

If config.options is an array, then instantiate

MultiCompiler
, Otherwise instantiate
Compiler
.

= Options new new WebpackOptionsDefaulter () Process (Options);. Copy Code

WebpackOptionsDefaulter
The remaining unspecified default configuration will be added to the options .

new NodeEnvironmentPlugin({ infrastructureLogging : options.infrastructureLogging }).apply(compiler); Copy code

NodeEnvironmentPlugin
With the help of graceful-fs module, for
compiler
Provide the ability to read and write files.

if (options.plugins && Array .isArray(options.plugins)) { for ( const plugin of options.plugins) { if ( typeof plugin === "function" ) { plugin.call(compiler, compiler); } else { plugin.apply(compiler); } } } Copy code

Next, traverse all the configured

plugin
,transfer
apply
Method, subscribe to different
compiler
Life cycle. For example, what we commonly use
webpack.DefinePlugin
Will subscribe
compiler.hooks.compilation
Hook, the plug-in will replace the qualified code when compiling.

//webpack/lib/webpack.js 57 rows compiler.options = new new WebpackOptionsApply () Process (Options, Compiler);. Copy Code

WebpackOptionsApply based webpack configuration DevTools , target and other fields, again adding to the built-in plug-in configuration, and subscribe to the corresponding life cycle. The most important of these is the EntryOptionPlugin plug-in, without which webpack cannot find the entry file:

//webpack/lib/WebpackOptionsApply.js Line 290 new EntryOptionPlugin().apply(compiler); compiler.hooks.entryOption.call(options.context, options.entry); Copy code

EntryOptionPlugin will subscribe

compiler.hooks.entryOption
Hook, and then trigger the hook in the next line. At this time, depending on whether it is a single entry or multiple entry, you can choose to instantiate SingleEntryPlugin or MultiEntryPlugin (when entry is a function, it will instantiate DynamicEntryPlugin ), and the plug-in will subscribe
compiler.hooks.comilation
with
compiler.hooks.make
. When the compiler triggers later
make
When hooking, it will execute
compilation.addEntry...
, From the entry file to perform dependency analysis, compilation and other operations.

Start (compiler.run())

After the above operations are completed, the compiler calls the run method to start packaging.

//webpack/lib/Compiler.js line 312 this .hooks.beforeRun.callAsync( this , err => { if (err) return finalCallback(err); this .hooks.run.callAsync( this , err => { if (err) return finalCallback(err); this .readRecords( err => { if (err) return finalCallback(err); this .compile(onCompiled); }); }); }); Copy code

Trigger in turn

beforeRun
,
run
Hook, and then call compile to enter the compilation phase.

Compilation preparation phase (compiler.compile(onCompiled))

//webpack/lib/Compiler.js line 661 const params = this.newCompilationParams(); //webpack/lib/Compiler.js line 651 newCompilationParams() { const params = { normalModuleFactory: this.createNormalModuleFactory(), contextModuleFactory: this.createContextModuleFactory(), compilationDependencies: new Set() }; return params; } Copy code

When the compiler 's compile method is called , a new one will be created for instantiation

Compilation
The parameter object, which contains two module factories: normalModuleFactory and contextModuleFactory .


NormalModule is easy to understand. It is a normal module determined at compile time. However, you may be a little confused when you see the contextModule . What is this module?

require ( 'template/' + name + '.js' ); copy the code

When the above code appears, webpack will parse the require() call and extract useful information from it.

Directory: ./template Regular expression:/^.*\.js$/ Copy code

Then a contextModule will be generated.

If the following is a contextMoudle with an id of 2 , it contains a map containing references to all modules in the template directory:

{ 'a.js' : 10 , 'b.js' : 11 } Copy code

then

the require ( 'Template/' + name + '.js' ); //runtime name is A //compilation process will convert __webpack_require __ ( 21 is ) (name + '.js' ) copying the code

! Note: In order to meet the requirements of dynamic require, all eligible modules will be packaged into the bundle.


Next, it will trigger

beforeCompile
,
compile
Hook, and then create a new Compilation instance. During the creation process, it will trigger
compiler.hooks.compilation
Hook to register the factory instances of the different types of modules created before to the dependencyFactories of compilation of for
compiler.hooks.make
Phase use.

Compile stage compiler.hooks.make

Instantiate Compilation , enter

compiler.hooks.make
Stage, trigger SingleEntryPlugin the callback function of subscription, execute
compilation.addEntry()
, Starting from the entry file, use loader-runner to execute the qualified loader , then use acorn to generate the AST syntax tree, get the dependencies, and then recursively execute the previous operations until all dependencies are loaded.

//SingleEntryPlugin.js 40 lines compiler.hooks.make.tapAsync( "SingleEntryPlugin" , ( compilation, callback ) => { const {entry, name, context} = this ; const dep = SingleEntryPlugin.createDependency(entry, name); //dep is the entry module compilation.addEntry(context, dep, name, callback); //context is the result of process.cwd() } ); Copy code

addEntry()
Continue to call later
this._addModuleChain(....)
, And after a series of operations, it will execute
module.build(...)
, This is the entry module to execute the compilation.

//NormalModule.js line 287 build ( options, compilation, resolver, fs, callback ) { //... return this .doBuild(options, compilation, resolver, fs, err => { //... }); } //... doBuild ( options, compilation, resolver, fs, callback ) { //Create a context shared by all loaders of the current module const loaderContext = this .createLoaderContext( resolver, options, compilation, fs ); // runLoaders({ resource : this .resource, //module path loaders : this .loaders, //loaders context : loaderContext, //context readResource : fs.readFile.bind(fs) //ability to read file content }) } Copy code

runLoaders
Will use readResouce to read the contents of the file, and execute the loader in the order from right to left . Then call
this.parser.parse(code)
( Acorn ) Generate the AST syntax tree, read the dependencies and store them in the dependencies of the module , and then call the addModuleDependencies method of compilation to process each dependency asynchronously using the neo-async library .

Icon of the make phase:

Code optimization (compilation.seal stage)

I intercepted part of the source code to see what was done in the seal phase?

//Compilation.js line 1186 seal(callback) { this.hooks.seal.call(); while ( this.hooks.optimizeDependenciesBasic.call(this.modules) || this.hooks.optimizeDependencies.call(this.modules) || this.hooks.optimizeDependenciesAdvanced.call(this.modules) ) { /* empty */ } this.hooks.afterOptimizeDependencies.call(this.modules); this.hooks.beforeChunks.call(); //... } Copy code

It can be seen that at this stage, the optimization configuration of webpack.config.js is in effect, and code optimization work such as Tree Shaking and Code Spliting is completed at this stage.

Output files to the specified output directory (compiler.hooks.emit stage)

The code is optimized, it will be executed almost

compiler.compile(onCompiled)
The onCompiled callback function in, let s take a look at what is inside?

const onCompiled = ( err, compilation ) => { //... this .emitAssets(compilation, err => { if (err) return finalCallback(err); //... }); //... } Copy code

We see that it is called again

this.emitAssets()
Method, the meaning of the output file. The specific source code is not shown, it is used inside
aeo-async
Asynchronous output file, file
io
The operation is through
compiler.outputFileSystem
Achieved.

summary

Webpack will just talk about it here, there are still some template replacements, how to implement Code Spliting, and so on. If you are interested, you can take a look at the source code for yourself.

Here is a simple webpack packaging logic for a webpack demo I wrote (the loader-runner and the final output logic is not finished, you can also refer to it).

Reference :