Response compression and performance optimization

Response Compression and Performance Optimization

As a lightweight Node.js framework, Koa2 often faces the issue of excessively large response bodies when handling HTTP requests. Response compression can effectively reduce the amount of data transmitted and improve application performance. Gzip and deflate are two common compression algorithms, which can be easily implemented in Koa2 using middleware.

Basic Usage of Compression Middleware

koa-compress is the most commonly used compression middleware in Koa2, built on Node.js's native zlib module. Installation is straightforward:

npm install koa-compress

Basic configuration example:

const compress = require('koa-compress');
const Koa = require('koa');

const app = new Koa();

app.use(compress({
  threshold: 2048, // Only compress responses larger than this byte count
  gzip: {
    flush: require('zlib').constants.Z_SYNC_FLUSH
  },
  deflate: {
    flush: require('zlib').constants.Z_SYNC_FLUSH
  },
  br: false // Disable Brotli compression
}));

Compression Threshold and Content Type

Setting a reasonable compression threshold avoids unnecessary compression for small resources:

app.use(compress({
  threshold: 1024, // 1KB
  filter(contentType) {
    // Only compress responses of specific types
    return /text|javascript|json|font/i.test(contentType);
  }
}));

Advanced Configuration for Brotli Compression

Brotli is a more efficient compression algorithm than gzip but requires Node.js 11.7.0+:

app.use(compress({
  br: {
    params: {
      [zlib.constants.BROTLI_PARAM_QUALITY]: 11 // Highest compression level
    }
  }
}));

Performance Optimization Practices

Static Resource Handling

Combine with koa-static to compress static resources:

const static = require('koa-static');

app.use(compress());
app.use(static('public', {
  setHeaders(res) {
    res.setHeader('Cache-Control', 'public, max-age=31536000');
  }
}));

API Response Optimization

Example of JSON response compression:

router.get('/api/data', async (ctx) => {
  const bigData = await fetchDataFromDB(); // Fetch large amounts of data
  ctx.body = bigData; // Automatically compressed
});

Streaming Response Handling

Use streaming compression for large files:

const fs = require('fs');
const path = require('path');

router.get('/large-file', async (ctx) => {
  ctx.type = 'text/plain';
  ctx.body = fs.createReadStream(path.join(__dirname, 'large.txt'));
});

Caching Strategies and Compression

Proper caching reduces repeated compression:

app.use(async (ctx, next) => {
  ctx.set('Cache-Control', 'no-cache');
  await next();
  if (ctx.fresh) {
    ctx.status = 304;
    ctx.body = null;
  }
});

Compression and HTTPS Performance

Be mindful of CPU overhead in HTTPS environments:

const https = require('https');
const fs = require('fs');

const options = {
  key: fs.readFileSync('server.key'),
  cert: fs.readFileSync('server.crt'),
  ciphers: 'ECDHE-RSA-AES128-GCM-SHA256' // Optimize cipher suites
};

https.createServer(options, app.callback()).listen(443);

Monitoring and Tuning

Use performance monitoring to adjust compression parameters:

const { performance } = require('perf_hooks');

app.use(async (ctx, next) => {
  const start = performance.now();
  await next();
  const duration = performance.now() - start;
  ctx.set('X-Response-Time', `${duration.toFixed(2)}ms`);
});

Multi-Level Compression Strategies

Apply different compression strategies for different routes:

const apiRouter = new Router();
apiRouter.use(compress({ threshold: 512 }));

const staticRouter = new Router();
staticRouter.use(compress({ threshold: 2048 }));

app.use(apiRouter.routes());
app.use(staticRouter.routes());

Client Negotiation Mechanism

Properly handle compression formats supported by the client:

app.use(async (ctx, next) => {
  await next();
  const acceptEncoding = ctx.headers['accept-encoding'] || '';
  ctx.vary('Accept-Encoding'); // Inform via Vary header
});

Memory Management

Memory control for large responses:

app.use(compress({
  flush: zlib.constants.Z_PARTIAL_FLUSH, // Partial flushing to reduce memory usage
  chunkSize: 16 * 1024 // 16KB chunk size
}));

Error Handling

Error handling during compression:

app.on('error', (err, ctx) => {
  if (err.code === 'Z_DATA_ERROR') {
    console.error('Compression data error:', err);
    ctx.status = 500;
    ctx.body = 'Compression processing failed';
  }
});

Testing and Benchmarking

Use autocannon for stress testing:

const autocannon = require('autocannon');

autocannon({
  url: 'http://localhost:3000',
  connections: 100,
  duration: 10
}, console.log);

Real-World Case: E-commerce API Optimization

Comparison before and after optimizing a product list API:

// Before optimization
router.get('/products', async (ctx) => {
  const products = await Product.find().lean();
  ctx.body = products;
});

// After optimization
router.get('/products', async (ctx) => {
  const products = await Product.find()
    .select('name price image') // Only fetch necessary fields
    .lean()
    .cache('1 hour'); // Add caching
  ctx.body = products;
});