Integration and configuration of the logging system

A logging system is an indispensable component in backend development, especially in the Express framework. Proper logging configuration helps developers quickly identify issues, analyze request behavior, and monitor system status. From basic console output to file storage and integration with third-party services, the flexibility and extensibility of a logging system directly impact development efficiency.

Choosing and Configuring Logging Middleware

The Express ecosystem offers various logging middleware options, with the most commonly used being the combination of morgan and winston. morgan focuses on HTTP request logging, while winston provides more general logging functionality. Install the basic dependencies:

npm install morgan winston

Basic configuration example:

const express = require('express');
const morgan = require('morgan');
const winston = require('winston');

const app = express();

// Morgan configuration
app.use(morgan('combined'));  // Use predefined format

// Winston basic configuration
const logger = winston.createLogger({
  level: 'info',
  format: winston.format.json(),
  transports: [
    new winston.transports.Console(),
    new winston.transports.File({ filename: 'error.log', level: 'error' })
  ]
});

app.get('/', (req, res) => {
  logger.info('Homepage accessed');
  res.send('Hello World');
});

Customizing Log Formats

Real-world projects often require custom log formats. morgan supports custom tokens and format strings:

// Custom morgan token
morgan.token('request-id', req => req.headers['x-request-id'] || 'none');

app.use(morgan(':request-id :method :url :status :response-time ms'));

For winston, complex formats can be achieved by combining multiple formats:

const { combine, timestamp, label, printf } = winston.format;

const myFormat = printf(({ level, message, label, timestamp }) => {
  return `${timestamp} [${label}] ${level}: ${message}`;
});

logger.add(new winston.transports.Console({
  format: combine(
    label({ label: 'API' }),
    timestamp(),
    myFormat
  )
}));

Multi-Environment Logging Strategies

Development and production environments typically require different logging strategies. Switch configurations using environment variables:

const isProduction = process.env.NODE_ENV === 'production';

if (isProduction) {
  logger.add(new winston.transports.File({ 
    filename: 'combined.log',
    format: winston.format.combine(
      winston.format.timestamp(),
      winston.format.json()
    )
  }));
} else {
  logger.add(new winston.transports.Console({
    format: winston.format.combine(
      winston.format.colorize(),
      winston.format.simple()
    )
  }));
}

Special Handling for Error Logs

Errors in Express require special capturing and logging. Create an error-handling middleware:

app.use((err, req, res, next) => {
  logger.error({
    message: err.message,
    stack: err.stack,
    path: req.path,
    method: req.method
  });
  
  res.status(500).json({ error: 'Internal Server Error' });
});

For uncaught exceptions and Promise rejections, global handling is needed:

process.on('uncaughtException', (error) => {
  logger.error('Uncaught Exception:', error);
  process.exit(1);
});

process.on('unhandledRejection', (reason, promise) => {
  logger.error('Unhandled Rejection at:', promise, 'reason:', reason);
});

Log File Rotation and Compression

In production environments, log files need automatic rotation. Use winston-daily-rotate-file:

const DailyRotateFile = require('winston-daily-rotate-file');

logger.add(new DailyRotateFile({
  filename: 'application-%DATE%.log',
  datePattern: 'YYYY-MM-DD',
  zippedArchive: true,
  maxSize: '20m',
  maxFiles: '14d'
}));

Structured Logging and Cloud Service Integration

Modern logging systems often require structured data for analysis. Configure JSON format and integrate with systems like ELK:

const { ElasticsearchTransport } = require('winston-elasticsearch');

const esTransport = new ElasticsearchTransport({
  level: 'info',
  clientOpts: { node: 'http://localhost:9200' }
});

logger.add(esTransport);

// Example of structured logging
logger.info('User login', {
  userId: 12345,
  ip: '192.168.1.1',
  userAgent: req.headers['user-agent']
});

Performance Optimization and Log Sampling

In high-traffic scenarios, control log volume with sampling strategies:

app.use(morgan('combined', {
  skip: (req, res) => res.statusCode < 400,  // Only log error requests
  stream: {
    write: (message) => {
      if (Math.random() < 0.1) {  // 10% sampling rate
        logger.info(message.trim());
      }
    }
  }
}));

Sensitive Information Filtering

Filter sensitive information like passwords in logs. Create a morgan skip function:

const sensitiveFields = ['password', 'creditCard'];

app.use(morgan((tokens, req, res) => {
  let logData = {
    method: tokens.method(req, res),
    url: maskSensitiveData(tokens.url(req, res)),
    status: tokens.status(req, res)
  };
  return JSON.stringify(logData);
}));

function maskSensitiveData(url) {
  const urlObj = new URL(url, 'http://dummy.com');
  sensitiveFields.forEach(field => {
    if (urlObj.searchParams.has(field)) {
      urlObj.searchParams.set(field, '******');
    }
  });
  return urlObj.pathname + urlObj.search;
}

Detailed Request/Response Logging

Sometimes, full request and response bodies need to be logged, but be mindful of memory usage:

const expressWinston = require('express-winston');

app.use(expressWinston.logger({
  winstonInstance: logger,
  requestWhitelist: ['url', 'method', 'headers', 'body'],
  responseWhitelist: ['body', 'statusCode'],
  bodyBlacklist: ['password']
}));

Logging and Monitoring System Integration

Integrate logs with APM systems like New Relic:

const newrelic = require('newrelic');

logger.add(new winston.transports.Console({
  format: winston.format.printf((info) => {
    newrelic.recordCustomEvent('NodeLog', {
      level: info.level,
      message: info.message
    });
    return info.message;
  })
}));

Distributed Tracing in Microservices

In microservice architectures, logs from multiple services need to be correlated. Integrate OpenTelemetry:

const { DiagConsoleLogger, DiagLogLevel, diag } = require('@opentelemetry/api');
const { WinstonInstrumentation } = require('@opentelemetry/instrumentation-winston');

diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.INFO);

const winstonInstrumentation = new WinstonInstrumentation({
  enabled: true,
  logHook: (span, record) => {
    record['traceId'] = span.spanContext().traceId;
  }
});