cache-loader implements build caching

Basic Principles of cache-loader

cache-loader is a specialized loader in the Webpack ecosystem designed to cache intermediate build results. Its working principle is quite straightforward: during the build process, it saves the module content processed by loaders to the local file system. If the source file remains unchanged during the next build, it directly reads the result from the cache, skipping the time-consuming loader processing.

This loader is typically placed at the beginning of the loader array so it can intercept subsequent loader processing requests. Its typical workflow is as follows:

1. Calculate the cache identifier for the current module (usually using the file path and modification time)
2. Check if a valid result exists in the cache
3. If a valid cache exists, return the cached result directly
4. If no cache exists or the cache is invalid, pass the request to subsequent loaders
5. Write the final processing result to the cache

module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        use: [
          'cache-loader',
          'babel-loader'
        ]
      }
    ]
  }
}

Installation and Basic Configuration

To use cache-loader, first install it via npm or yarn:

npm install --save-dev cache-loader
# or
yarn add --dev cache-loader

The basic configuration is very simple—just add cache-loader before the loader you want to cache. Here’s a typical configuration example:

module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        use: [
          {
            loader: 'cache-loader',
            options: {
              cacheDirectory: path.resolve('.cache'),
              cacheIdentifier: 'v1'
            }
          },
          'babel-loader'
        ],
        include: path.resolve('src')
      }
    ]
  }
}

In this configuration, we specify the cache directory as the .cache folder in the project root and set the cache identifier to v1. When modifying the Babel configuration or upgrading the Babel version, you can change this identifier to force cache invalidation.

Advanced Configuration Options

cache-loader provides several configuration options to meet the needs of different scenarios:

cacheDirectory

Specifies the storage location for cache files. The default is path.resolve('.cache-loader'). It’s recommended to set it to a path within the project directory:

{
  loader: 'cache-loader',
  options: {
    cacheDirectory: path.resolve(__dirname, '.cache/loader')
  }
}

cacheIdentifier

A string identifier. When this identifier changes, all caches are invalidated. This is particularly useful when upgrading loaders or modifying configurations:

{
  loader: 'cache-loader',
  options: {
    cacheIdentifier: `babel-config-${JSON.stringify(babelConfig)}-${babelVersion}`
  }
}

cacheKey

A custom function to generate cache keys, allowing decisions based on more factors:

{
  loader: 'cache-loader',
  options: {
    cacheKey: (options, request) => {
      return `${options.cacheIdentifier}|${request.path}|${fs.statSync(request.path).mtimeMs}`
    }
  }
}

read/write

Custom functions for cache read/write logic:

{
  loader: 'cache-loader',
  options: {
    read: (cacheKey, callback) => {
      customCacheStore.get(cacheKey, callback)
    },
    write: (cacheKey, data, callback) => {
      customCacheStore.set(cacheKey, data, callback)
    }
  }
}

Performance Optimization Practices

Selective Caching

Not all files are suitable for caching. Typically, we only cache files that take a long time to process:

module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        use: [
          {
            loader: 'cache-loader',
            options: {
              cacheDirectory: path.resolve('.cache/js')
            }
          },
          'babel-loader'
        ],
        include: path.resolve('src')
      },
      {
        test: /\.scss$/,
        use: [
          'style-loader',
          {
            loader: 'cache-loader',
            options: {
              cacheDirectory: path.resolve('.cache/css')
            }
          },
          'css-loader',
          'sass-loader'
        ]
      }
    ]
  }
}

Multi-Level Caching

For large projects, you can combine memory-fs to implement memory + disk multi-level caching:

const MemoryFS = require('memory-fs')
const mfs = new MemoryFS()

module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        use: [
          {
            loader: 'cache-loader',
            options: {
              read: (key, callback) => {
                if (mfs.existsSync(key)) {
                  return callback(null, mfs.readFileSync(key))
                }
                fs.readFile(key, callback)
              },
              write: (key, data, callback) => {
                mfs.writeFileSync(key, data)
                fs.writeFile(key, data, callback)
              }
            }
          },
          'babel-loader'
        ]
      }
    ]
  }
}

Parallel Processing

Combine with thread-loader for parallel processing:

module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        use: [
          'cache-loader',
          {
            loader: 'thread-loader',
            options: {
              workers: 4
            }
          },
          'babel-loader'
        ]
      }
    ]
  }
}

Common Issues and Solutions

Cache Invalidation Issues

When encountering cache update problems, troubleshoot from the following aspects:

1. Check if the file modification time has been updated
2. Confirm whether the cacheIdentifier correctly reflects configuration changes
3. Verify the cache directory permissions

// Debug cache key generation
{
  loader: 'cache-loader',
  options: {
    cacheKey: (options, request) => {
      const key = customCacheKey(options, request)
      console.log('Cache key:', key)
      return key
    }
  }
}

Cache Pollution

Issues may arise when multiple projects use the same cache directory. Solutions:

1. Set up independent cache directories for each project
2. Include project-specific information in the cacheIdentifier

{
  loader: 'cache-loader',
  options: {
    cacheDirectory: path.resolve(`.cache/${process.env.NODE_ENV || 'development'}`),
    cacheIdentifier: `${packageJson.name}-${packageJson.version}`
  }
}

Inconsistent Build Results

Sometimes caching may lead to inconsistent build results. Resolve this by:

// Disable caching in CI environments
{
  loader: process.env.CI ? 'babel-loader' : 'cache-loader'
}

// Or force cache clearance
if (process.env.CLEAR_CACHE) {
  require('rimraf').sync(path.resolve('.cache'))
}

Integration with Other Tools

Working with HardSourceWebpackPlugin

HardSourceWebpackPlugin provides module-level caching and can complement cache-loader:

const HardSourceWebpackPlugin = require('hard-source-webpack-plugin')

module.exports = {
  plugins: [
    new HardSourceWebpackPlugin({
      cacheDirectory: path.resolve('.cache/hard-source/[confighash]')
    })
  ],
  module: {
    rules: [
      {
        test: /\.js$/,
        use: [
          'cache-loader',
          'babel-loader'
        ]
      }
    ]
  }
}

Combining with DllPlugin

For stable third-party libraries, using DllPlugin yields better results:

// webpack.dll.config.js
module.exports = {
  entry: {
    vendor: ['react', 'react-dom', 'lodash']
  },
  output: {
    path: path.resolve('.cache/dll'),
    filename: '[name].dll.js',
    library: '[name]_[hash]'
  },
  plugins: [
    new webpack.DllPlugin({
      path: path.join('.cache/dll', '[name]-manifest.json'),
      name: '[name]_[hash]'
    })
  ]
}

// webpack.config.js
module.exports = {
  plugins: [
    new webpack.DllReferencePlugin({
      manifest: require('./.cache/dll/vendor-manifest.json')
    })
  ],
  module: {
    rules: [
      {
        test: /\.js$/,
        use: [
          'cache-loader',
          'babel-loader'
        ],
        exclude: /node_modules/
      }
    ]
  }
}

Advanced Caching Strategies

Content-Based Caching

By default, cache-loader determines cache validity based on file modification time. You can switch to content-based caching:

const crypto = require('crypto')

{
  loader: 'cache-loader',
  options: {
    cacheKey: (options, request) => {
      const fileContent = fs.readFileSync(request.path)
      const contentHash = crypto.createHash('md5').update(fileContent).digest('hex')
      return `${options.cacheIdentifier}|${request.path}|${contentHash}`
    }
  }
}

Distributed Caching

For team development, you can share caches:

const AWS = require('aws-sdk')
const s3 = new AWS.S3()

{
  loader: 'cache-loader',
  options: {
    read: (key, callback) => {
      s3.getObject({ Bucket: 'our-cache-bucket', Key: key }, (err, data) => {
        if (err && err.code === 'NoSuchKey') {
          return fs.readFile(key, callback)
        }
        callback(err, err ? null : data.Body)
      })
    },
    write: (key, data, callback) => {
      fs.writeFile(key, data, (err) => {
        if (err) return callback(err)
        s3.putObject({ Bucket: 'our-cache-bucket', Key: key, Body: data }, callback)
      })
    }
  }
}

Monitoring and Optimization

Cache Hit Rate Statistics

Track cache hit rates using custom read/write functions:

let hits = 0
let misses = 0

{
  loader: 'cache-loader',
  options: {
    read: (key, callback) => {
      fs.readFile(key, (err, data) => {
        if (err) {
          misses++
          return callback(err)
        }
        hits++
        callback(null, data)
      })
    },
    write: (key, data, callback) => {
      fs.writeFile(key, data, callback)
    }
  }
}

// Print statistics after compilation
compiler.hooks.done.tap('CacheStatsPlugin', () => {
  console.log(`Cache hit rate: ${hits / (hits + misses) * 100}%`)
})

Cache Size Control

Implement automatic cleanup of old caches:

const MAX_CACHE_SIZE = 100 * 1024 * 1024 // 100MB

{
  loader: 'cache-loader',
  options: {
    write: (key, data, callback) => {
      fs.writeFile(key, data, (err) => {
        if (err) return callback(err)
        
        // Check cache directory size
        getDirectorySize('.cache', (size) => {
          if (size > MAX_CACHE_SIZE) {
            cleanupOldCacheFiles('.cache', MAX_CACHE_SIZE / 2)
          }
          callback()
        })
      })
    }
  }
}

function getDirectorySize(dir, callback) {
  // Implement logic to get directory size
}

function cleanupOldCacheFiles(dir, targetSize) {
  // Implement logic to clean up old files
}