Audit Log

The Concept of Audit Log

Audit logs are a core mechanism for recording system operations, used to track data changes, user activities, and system events. In MongoDB, audit logs capture database operations (such as CRUD), authentication events, and administrative commands, providing critical evidence for security compliance and troubleshooting. Unlike regular logs, audit logs emphasize immutability and integrity, often requiring separate storage and retention for specific periods.

Configuring Audit Logs in MongoDB

To enable the audit feature, add the following parameters to the mongod.conf configuration file:

auditLog:
  destination: file
  format: JSON
  path: /var/log/mongodb/audit.json
  filter: '{ atype: { $in: ["authenticate", "createCollection"] } }'

Three output destinations are supported:

• File: Directly written to disk (supports JSON/BSON formats)
• Syslog: Sent to the system log service
• Console: Output to the standard output stream

The filter example above only records authentication and collection creation events. In production environments, more complex filtering conditions may be required:

// Dynamically set audit filters
db.adminCommand({
  setParameter: 1,
  auditAuthorizationSuccess: true,
  auditFilter: {
    $or: [
      { "param.command": { $in: ["dropDatabase", "shutdown"] } },
      { "param.ns": /^protectedDB\./ }
    ]
  }
})

Classification and Structure of Audit Events

MongoDB categorizes audit events into six major types:

1. Authentication Events:

   • authenticate: Successful login
   • authCheck: Permission verification

   {
     "atype": "authenticate",
     "ts": { "$date": "2023-08-20T03:45:12.483Z" },
     "local": { "ip": "192.168.1.15", "port": 27017 },
     "remote": { "ip": "10.2.3.4", "port": 54132 },
     "users": [{ "user": "admin", "db": "admin" }],
     "result": 0
   }
   

2. CRUD Operations:

   • Details of insert/update/delete operations

   {
     "atype": "delete",
     "ts": { "$date": "2023-08-20T03:47:22.156Z" },
     "param": {
       "ns": "medical.records",
       "query": { "patientId": "P10023" }
     }
   }
   

3. Administrative Operations:

   • DDL operations like createCollection/dropDatabase
   • Records of user role changes

Storage Optimization Strategies for Audit Logs

For large-scale deployments, special attention is required:

Sharded Cluster Configuration:

# Each shard has its own audit path
sharding:
  clusterRole: "shardsvr"
auditLog:
  destination: file
  path: "/data/shard01/audit.log"

Log Rotation Solution:

# Configure using the logrotate tool
/var/log/mongodb/audit.json {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    sharedscripts
    postrotate
        killall -SIGUSR1 mongod
    endscript
}

Storage Separation Example:

// Write audit logs to a dedicated collection
use admin
db.createCollection("system.audit", {
  capped: true,
  size: 1024 * 1024 * 1024, // 1GB capacity
  storageEngine: { wiredTiger: { configString: "block_compressor=zstd" } }
})

Practical Analysis of Audit Logs

Using Aggregation Pipeline for log analysis:

db.system.audit.aggregate([
  {
    $match: {
      "atype": "update",
      "ts": { "$gt": ISODate("2023-08-01") }
    }
  },
  {
    $group: {
      _id: "$param.ns",
      count: { $sum: 1 },
      users: { $addToSet: "$users.user" }
    }
  },
  { $sort: { count: -1 } }
])

Example of security alert rules:

// Detect abnormal delete operations
const alerts = db.system.audit.find({
  "atype": "delete",
  "param.ns": /^finance\./,
  "ts": { "$gt": new Date(Date.now() - 3600000) },
  "$expr": { "$gt": [{ "$size": "$param.query" }, 5] }
})

Compliance Requirements Implementation

Example configuration for GDPR compliance:

auditLog:
  destination: file
  path: /var/log/mongodb/gdpr_audit.log
  filter: '{
    $or: [
      { "param.ns": /\.personal_data$/ },
      { "atype": { $in: ["createUser", "dropUser"] } }
    ]
  }'

HIPAA medical data audit solution:

// Use change streams for real-time monitoring
const pipeline = [
  { $match: {
    operationType: { $in: ["insert", "update", "delete"] },
    "ns.db": "medical",
    "fullDocument.ssn": { $exists: true }
  }}
];

db.watch(pipeline).on("change", change => {
  db.hipaa_audit.insertOne({
    timestamp: new Date(),
    operator: change.operationType,
    documentKey: change.documentKey,
    clientInfo: change.clientInfo
  });
});

Performance Impact and Tuning

The primary performance impacts of audit logs come from:

1. I/O Pressure Test Data:
   • No audit: Average throughput of 12,000 ops/sec
   • File audit: 9,200 ops/sec (23% decrease)
   • Syslog audit: 7,800 ops/sec (35% decrease)

Optimization solutions:

storage:
  engine: wiredTiger
  wiredTiger:
    engineConfig:
      # Dedicated cache for audit logs
      auditLogCacheSizeGB: 2

Asynchronous write configuration:

db.adminCommand({
  setParameter: 1,
  auditLogFlushIntervalMillis: 1000  // Batch write every 1 second
})