Inter-process communication

Basic Concepts of Inter-Process Communication (IPC)

Inter-Process Communication (IPC) is a mechanism provided by the operating system that allows different processes to exchange data and information. In Node.js, IPC is particularly important because Node.js adopts a single-threaded event loop model and relies on child processes to handle CPU-intensive tasks. IPC mechanisms enable the main process and child processes to collaborate and share data and state.

Node.js offers various IPC methods, including but not limited to: pipes, message queues, shared memory, semaphores, and sockets. Each method has its applicable scenarios, advantages, and disadvantages, and developers need to choose the most suitable communication method based on specific requirements.

The Child Process Module in Node.js

The built-in child_process module in Node.js is the core for handling inter-process communication. This module provides several methods to create child processes:

const { spawn, exec, execFile, fork } = require('child_process');

Among these, the fork() method is specifically designed to create Node.js child processes and automatically establishes an IPC channel:

// parent.js
const { fork } = require('child_process');
const child = fork('child.js');

child.on('message', (msg) => {
  console.log('Message from child process:', msg);
});

child.send({ hello: 'world' });

// child.js
process.on('message', (msg) => {
  console.log('Message from parent process:', msg);
  process.send({ foo: 'bar' });
});

Common Patterns of Inter-Process Communication

Message Passing Pattern

This is the most commonly used IPC method in Node.js, enabling bidirectional communication through process.send() and process.on('message'). Messages are serialized in JSON format for transmission, allowing complex objects to be passed:

// Parent process sends a complex object
child.send({
  type: 'config',
  data: {
    port: 3000,
    env: 'production',
    features: ['logging', 'monitoring']
  }
});

// Child process receives and processes
process.on('message', (msg) => {
  if (msg.type === 'config') {
    console.log('Received configuration:', msg.data);
  }
});

Stream-Based Communication

For large data transfers, standard input/output streams (stdio) can be used for communication:

// Use spawn to create a child process and communicate via streams
const { spawn } = require('child_process');
const child = spawn('node', ['child.js'], {
  stdio: ['pipe', 'pipe', 'pipe', 'ipc'] // Explicitly enable IPC
});

child.stdout.on('data', (data) => {
  console.log(`Child process output: ${data}`);
});

child.stdin.write('Parent process sends data via stdin\n');

Advanced IPC Techniques

Shared Memory

Although Node.js does not directly support shared memory, it can be implemented using third-party modules like shared-memory:

const SharedMemory = require('shared-memory');
const memory = new SharedMemory('my-shared-memory', 1024);

// Process A writes data
memory.write(0, Buffer.from('Hello from Process A'));

// Process B reads data
const data = memory.read(0, 20);
console.log(data.toString()); // Output: Hello from Process A

Using Message Brokers

For distributed systems, message brokers like Redis or RabbitMQ can be used for cross-machine process communication:

// Using Redis pub/sub
const redis = require('redis');
const subscriber = redis.createClient();
const publisher = redis.createClient();

subscriber.on('message', (channel, message) => {
  console.log(`Received message: ${message} from channel: ${channel}`);
});

subscriber.subscribe('my-channel');

// Another process
publisher.publish('my-channel', 'This is a cross-process message');

Error Handling in Inter-Process Communication

Reliable IPC requires robust error handling mechanisms:

child.on('error', (err) => {
  console.error('Child process error:', err);
});

child.on('exit', (code, signal) => {
  if (code !== 0) {
    console.warn(`Child process exited abnormally, code: ${code}, signal: ${signal}`);
  }
});

// Timeout handling
const timeout = setTimeout(() => {
  child.kill('SIGTERM');
}, 5000);

child.on('exit', () => clearTimeout(timeout));

Performance Optimization and Best Practices

Message Batching

Frequent small messages can degrade performance; consider batching them:

// Not recommended
for (let i = 0; i < 1000; i++) {
  child.send({ index: i });
}

// Recommended
const batch = [];
for (let i = 0; i < 1000; i++) {
  batch.push({ index: i });
}
child.send({ type: 'batch', data: batch });

Serialization Optimization

Serializing/deserializing large objects consumes CPU resources:

// Use more efficient serialization methods
const msgpack = require('msgpack-lite');
child.send(msgpack.encode(largeObject));

process.on('message', (msg) => {
  const data = msgpack.decode(msg);
});

Practical Application Scenarios

IPC in Microservices Architecture

In a microservices architecture, different services can communicate via IPC:

// service-a.js
const { fork } = require('child_process');
const serviceB = fork('service-b.js');

serviceB.send({ 
  action: 'getUser',
  params: { id: 123 }
});

// service-b.js
process.on('message', async ({ action, params }) => {
  if (action === 'getUser') {
    const user = await db.getUser(params.id);
    process.send({ 
      status: 'success',
      data: user 
    });
  }
});

Worker Process Pool

Create a process pool to handle high CPU-load tasks:

const { Worker, isMainThread, parentPort } = require('worker_threads');

if (isMainThread) {
  // Main thread code
  const workerPool = Array(4).fill().map(() => new Worker(__filename));
  
  workerPool.forEach(worker => {
    worker.on('message', result => {
      console.log('Received result:', result);
    });
  });
  
  // Distribute tasks
  workerPool[0].postMessage({ task: 'heavyCalculation', data: 1000 });
} else {
  // Worker thread code
  parentPort.on('message', ({ task, data }) => {
    if (task === 'heavyCalculation') {
      const result = performHeavyCalculation(data);
      parentPort.postMessage(result);
    }
  });
  
  function performHeavyCalculation(n) {
    // Simulate CPU-intensive computation
    let result = 0;
    for (let i = 0; i < n; i++) {
      for (let j = 0; j < n; j++) {
        result += i * j;
      }
    }
    return result;
  }
}

Security Considerations

Inter-process communication must account for security issues:

// Validate message source
process.on('message', (msg, handle) => {
  if (typeof msg !== 'object' || !msg.type) {
    return; // Ignore invalid messages
  }
  
  // Check if sender is trusted
  if (handle && !isTrustedSender(handle)) {
    console.warn('Received message from untrusted source');
    return;
  }
  
  // Process message...
});

// Limit message size
const MAX_MESSAGE_SIZE = 1024 * 1024; // 1MB
child.on('message', (msg) => {
  if (JSON.stringify(msg).length > MAX_MESSAGE_SIZE) {
    child.kill('SIGTERM');
    throw new Error('Message too large');
  }
});

Debugging and Monitoring

Debugging IPC communication requires specialized tools and techniques:

// Log all IPC messages
const debug = require('debug')('ipc');
child.on('message', (msg) => {
  debug('Received message %O', msg);
});

// Monitor IPC performance
const start = process.hrtime();
child.send('ping');
child.once('message', () => {
  const diff = process.hrtime(start);
  console.log(`IPC round-trip time: ${diff[0] * 1e3 + diff[1] / 1e6}ms`);
});

// Use --inspect-brk to debug child processes
const child = fork('child.js', [], {
  execArgv: ['--inspect-brk=9229']
});