Memory leak troubleshooting

Basic Concepts of Memory Leaks

A memory leak refers to allocated memory in a program that is not properly released, leading to a gradual reduction in available memory. In Node.js, memory leaks can cause application performance degradation or even crashes. Common memory leak scenarios include misuse of global variables, unreleased closures, and unremoved event listeners.

Common Memory Leak Scenarios

Misuse of Global Variables

// Bad example
function leak() {
  leakedArray = new Array(1000000).fill('*'); // Not declared with var/let/const
}

// Correct approach
function noLeak() {
  const localArray = new Array(1000000).fill('*');
}

Global variables persist in memory until the process ends. In Node.js, module-level variables are effectively "semi-global," so their lifecycle must be carefully managed.

Uncleared Timers

// Leak example
setInterval(() => {
  const data = fetchData();
  processData(data);
}, 1000);

// Correct approach
const intervalId = setInterval(/* ... */);
// Clear when no longer needed
clearInterval(intervalId);

Uncleared timers prevent variables in related closures from being garbage-collected.

Unremoved Event Listeners

const EventEmitter = require('events');
const emitter = new EventEmitter();

function createListener() {
  const heavyObject = new Array(1000000).fill('*');
  emitter.on('event', () => {
    console.log(heavyObject.length);
  });
}

// Correct approach
function createProperListener() {
  const heavyObject = new Array(1000000).fill('*');
  const listener = () => {
    console.log(heavyObject.length);
  };
  emitter.on('event', listener);
  // Remove when no longer needed
  return () => emitter.off('event', listener);
}

Memory Leak Detection Tools

Built-in Node.js Tools

1. Start with the --inspect flag for debugging
2. Chrome DevTools memory profiler
3. process.memoryUsage() API

setInterval(() => {
  const memoryUsage = process.memoryUsage();
  console.log(`RSS: ${memoryUsage.rss / 1024 / 1024} MB`);
  console.log(`HeapTotal: ${memoryUsage.heapTotal / 1024 / 1024} MB`);
  console.log(`HeapUsed: ${memoryUsage.heapUsed / 1024 / 1024} MB`);
}, 1000);

Third-Party Tools

1. heapdump: Generate heap snapshots

   npm install heapdump
   

   const heapdump = require('heapdump');
   
   // Manually generate a snapshot
   heapdump.writeSnapshot('/path/to/snapshot.heapsnapshot');
   

2. v8-profiler: Detailed CPU and memory profiling

   const profiler = require('v8-profiler-next');
   const snapshot = profiler.takeSnapshot();
   snapshot.export((error, result) => {
     fs.writeFileSync('snapshot.heapsnapshot', result);
     snapshot.delete();
   });
   

Analyzing Heap Snapshots

1. Load .heapsnapshot files in Chrome DevTools
2. Focus on:
   • Retained Size: Total memory occupied by the object and its reference chain
   • Shallow Size: Memory occupied by the object itself
   • Distance: Distance from GC roots

Common suspicious objects:

• Large arrays
• Large objects in closures
• Uncleared caches

Practical Case Studies

Case 1: Uncleared Cache

const cache = {};

function processRequest(req) {
  if (!cache[req.url]) {
    cache[req.url] = generateResponse(req);
  }
  return cache[req.url];
}

Improved solution:

const LRU = require('lru-cache');
const cache = new LRU({
  max: 100,  // Maximum cache items
  maxAge: 1000 * 60 * 5 // 5 minutes
});

function processRequest(req) {
  if (!cache.has(req.url)) {
    cache.set(req.url, generateResponse(req));
  }
  return cache.get(req.url);
}

Case 2: Unhandled Promises

function asyncOperation() {
  return new Promise((resolve) => {
    // Forgot to call resolve
    // resolve(data);
  });
}

// Accumulated unresolved Promises consume memory
setInterval(asyncOperation, 100);

Solution:

function asyncOperation() {
  return new Promise((resolve) => {
    resolve();
  }).catch(err => {
    console.error(err);
  });
}

Best Practices for Preventing Memory Leaks

1. Use strict mode

   'use strict';
   

2. Monitor memory usage

   const memwatch = require('memwatch-next');
   memwatch.on('leak', (info) => {
     console.error('Memory leak detected:', info);
   });
   

3. Regular stress testing

   autocannon -c 100 -d 60 http://localhost:3000
   

4. Code review focus areas:

   • Global state management
   • Event listener lifecycles
   • Large object caching strategies
   • Asynchronous operation completion

Advanced Debugging Techniques

Core Dump Analysis

1. Generate core dump

   ulimit -c unlimited
   node --abort-on-uncaught-exception app.js
   

2. Analyze with llnode

   llnode node -c core.1234
   > v8 findjsobjects
   > v8 findjsinstances Array
   

Memory Growth Comparative Analysis

1. Take initial heap snapshot
2. Perform suspicious operations
3. Take subsequent heap snapshots
4. Compare object allocation differences between snapshots

const { writeHeapSnapshot } = require('v8');
const fs = require('fs');

// First snapshot
writeHeapSnapshot('snapshot1.heapsnapshot');

// After performing operations
setTimeout(() => {
  writeHeapSnapshot('snapshot2.heapsnapshot');
}, 10000);