Response compression and transmission optimization

Response Compression and Transmission Optimization

As a lightweight Node.js framework, Koa2 often faces HTTP response transmission efficiency issues. Response compression can significantly reduce network data transfer volume and improve application performance. Through proper middleware configuration and optimization strategies, a performance leap at the transmission layer can be achieved without altering business logic.

Compression Algorithm Selection and Comparison

Common compression algorithms include gzip, deflate, and br (Brotli), each with distinct characteristics:

• gzip: Best compatibility, moderate compression ratio (~70%)
• deflate: Faster than gzip but with slightly lower compression ratio
• br: Modern algorithm, highest compression ratio (up to 85%), but with higher CPU consumption

const compression = require('koa-compress')

// Support multiple compression algorithms simultaneously
app.use(compression({
  threshold: 2048, // Compress only when exceeding 2KB
  gzip: { level: 6 },
  br: { params: { [zlib.constants.BROTLI_PARAM_QUALITY]: 4 } }
}))

Threshold and Content-Type Control

Not all responses are suitable for compression; reasonable thresholds should be set:

1. Files smaller than 1KB may increase in size after compression
2. Binary files like images/videos are already compressed and don't need reprocessing
3. Dynamic content shows the most noticeable compression effects

app.use(compression({
  filter: (contentType) => {
    return /text|javascript|json|xml/i.test(contentType)
  },
  threshold: 1024
}))

Cache Strategy Combined with Compression

Combining compression with caching maximizes performance:

app.use(async (ctx, next) => {
  ctx.set('Cache-Control', 'public, max-age=3600')
  ctx.set('Vary', 'Accept-Encoding') // Distinguish between compressed versions
  await next()
})

Stream Compression for Large Files

For large file responses, stream processing prevents memory overflow:

const fs = require('fs')
const zlib = require('zlib')

app.use(async (ctx) => {
  ctx.type = 'text/plain'
  ctx.body = fs.createReadStream('./large.log')
    .pipe(zlib.createGzip())
})

Performance Monitoring and Tuning

Verify compression effectiveness through monitoring tools:

# Check compression ratio
curl -I -H "Accept-Encoding: gzip" http://localhost:3000

Typical optimization metrics:

• Compressed size reduced by 60%-80%
• CPU load increase not exceeding 15%
• TTFB time increase controlled within 20ms

Client Negotiation Mechanism

Properly handle client capability negotiation:

app.use(async (ctx, next) => {
  const acceptEncoding = ctx.headers['accept-encoding'] || ''
  ctx.compress = acceptEncoding.includes('gzip')
  await next()
})

Special Handling for Binary Data

Optimization strategies for pre-compressed formats:

app.use(async (ctx, next) => {
  if (ctx.path.endsWith('.jpg')) {
    ctx.set('Content-Encoding', 'identity') // Disable secondary compression
  }
  await next()
})

HTTP/2 Header Compression

Additional configuration required in HTTP/2 environments:

const http2 = require('http2')
const server = http2.createSecureServer({}, app.callback())

// Enable HTTP/2 header compression
server.on('stream', (stream, headers) => {
  stream.respond({
    ':status': 200,
    'content-encoding': 'gzip'
  })
})

Compression and Security Considerations

Note security-related details:

1. CRIME attack prevention: Avoid compressing sensitive cookies
2. Compression bomb protection: Limit maximum decompression size

const safeCompress = require('koa-safe-compress')
app.use(safeCompress({
  maxSize: '10mb' // Limit maximum decompressed size to 10MB
}))

Actual Performance Test Data

Test environment comparison (1MB JSON response):

Dynamic Content Compression Strategy

Special handling for API responses:

router.get('/api/data', async (ctx) => {
  const data = await fetchData()
  ctx.body = data
  
  // Determine compression level based on data type
  if (data.type === 'highPriority') {
    ctx.response.compress = { level: 9 }
  }
})

Multi-Level Cache Integration

Combining with Redis multi-level caching:

const redis = require('redis')
const client = redis.createClient()

app.use(async (ctx, next) => {
  const cacheKey = `${ctx.url}:gzip`
  const cached = await client.get(cacheKey)
  
  if (cached) {
    ctx.body = cached
    ctx.set('X-Cache', 'HIT')
  } else {
    await next()
    if (ctx.status === 200) {
      client.setex(cacheKey, 3600, ctx.body)
    }
  }
})

Mobile-Specific Optimization

Adjustments for mobile network characteristics:

app.use(async (ctx, next) => {
  const isMobile = /Mobile/i.test(ctx.headers['user-agent'])
  if (isMobile) {
    ctx.compressOptions = { level: 4 } // Lower compression level to save battery
  }
  await next()
})

Error Handling and Fallback Mechanism

Ensure compression failures don't affect normal responses:

app.use(async (ctx, next) => {
  try {
    await next()
  } catch (err) {
    if (err.code === 'Z_DATA_ERROR') {
      ctx.remove('Content-Encoding')
      ctx.app.emit('error', err, ctx)
    }
  }
})

Content Encoding Negotiation Implementation

Complete content negotiation example:

app.use(async (ctx) => {
  const encodings = ctx.acceptsEncodings()
  const body = generateResponseBody()
  
  if (encodings.includes('br')) {
    ctx.set('Content-Encoding', 'br')
    ctx.body = brotliCompress(body)
  } else if (encodings.includes('gzip')) {
    ctx.set('Content-Encoding', 'gzip')
    ctx.body = gzipSync(body)
  } else {
    ctx.body = body
  }
})

Modern Browser Feature Utilization

Alternative solution using Compression Streams API:

// Browser-side decompression example
fetch('/compressed-data')
  .then(res => res.body)
  .then(stream => {
    const ds = new DecompressionStream('gzip')
    return stream.pipeThrough(ds)
  })

Server-Side Rendering Optimization

Special handling for SSR scenarios:

app.use(async (ctx) => {
  const html = await renderToString(ctx)
  if (ctx.compress) {
    ctx.type = 'text/html'
    ctx.body = compressHTML(html) // Custom HTML compression
  } else {
    ctx.body = html
  }
})

Balancing Performance and Quality

Find optimal configuration based on business requirements:

// Quality-first configuration
const qualityFirst = {
  level: zlib.constants.BROTLI_MAX_QUALITY,
  [zlib.constants.BROTLI_PARAM_SIZE_HINT]: fs.statSync(file).size
}

// Speed-first configuration
const speedFirst = {
  level: zlib.constants.BROTLI_MIN_QUALITY,
  [zlib.constants.BROTLI_PARAM_MODE]: zlib.constants.BROTLI_MODE_TEXT
}

Continuous Optimization and A/B Testing

Dynamically adjust compression strategies:

app.use(abTest({
  variants: [
    { name: 'HighCompression', options: { level: 9 } },
    { name: 'FastCompression', options: { level: 3 } }
  ],
  metric: 'pageLoadTime' 
}))