Replica set configuration and management

Basic Concepts of Replica Sets

A MongoDB replica set is a group of mongod processes that maintain the same dataset, providing data redundancy and high availability. A replica set typically consists of multiple nodes, including one primary node and several secondary nodes. The primary node receives all write operations, while secondary nodes maintain data synchronization by replicating the primary's operation log. When the primary node becomes unavailable, the replica set automatically elects a new primary node to ensure continuous service availability.

A typical replica set configuration requires at least three nodes:

• One primary node (Primary)
• Two secondary nodes (Secondary)
• An optional arbiter node (Arbiter)

// Example of a MongoDB client connecting to a replica set
const { MongoClient } = require('mongodb');
const uri = "mongodb://node1.example.com:27017,node2.example.com:27017,node3.example.com:27017/?replicaSet=myReplicaSet";
const client = new MongoClient(uri);

Replica Set Configuration

Initializing a Replica Set

To configure a replica set, first start the mongod process on each node and specify the replica set name. Below are example commands to start three nodes:

# Node 1
mongod --replSet myReplicaSet --dbpath /data/node1 --port 27017

# Node 2
mongod --replSet myReplicaSet --dbpath /data/node2 --port 27018

# Node 3
mongod --replSet myReplicaSet --dbpath /data/node3 --port 27019

To initialize the replica set, connect to one of the nodes via the mongo shell and execute the initialization command:

// Execute in the mongo shell
rs.initiate({
  _id: "myReplicaSet",
  members: [
    { _id: 0, host: "node1.example.com:27017" },
    { _id: 1, host: "node2.example.com:27018" },
    { _id: 2, host: "node3.example.com:27019" }
  ]
})

Replica Set Member Configuration

Replica set members can be configured with different roles and priorities:

// Configuring priorities and hidden nodes
rs.reconfig({
  _id: "myReplicaSet",
  version: 2,
  members: [
    { _id: 0, host: "node1.example.com:27017", priority: 2 },
    { _id: 1, host: "node2.example.com:27018", priority: 1 },
    { 
      _id: 2, 
      host: "node3.example.com:27019", 
      priority: 0,
      hidden: true,
      slaveDelay: 3600  // Delayed synchronization by 1 hour
    }
  ]
})

Replica Set Management

Monitoring Replica Set Status

Use the rs.status() command to view detailed status information of the replica set:

// Get replica set status
rs.status()

// Check replica set configuration
rs.conf()

// View oplog status
rs.printReplicationInfo()

// View member synchronization status
rs.printSlaveReplicationInfo()

Adding and Removing Nodes

Dynamically add new nodes to the replica set:

// Add a new node
rs.add("node4.example.com:27020")

// Remove a node
rs.remove("node3.example.com:27019")

Failover Testing

Manually trigger a primary node stepdown to test failover:

// Force the primary node to step down and become a secondary
rs.stepDown(300)  // Cannot become primary again for 300 seconds

// Check the current primary node
rs.isMaster()

Advanced Configuration Options

Read/Write Concern Settings

Configure read/write concern levels for the replica set:

// Set write concern to majority acknowledgment
db.products.insertOne(
  { item: "envelope", qty: 100 },
  { writeConcern: { w: "majority", wtimeout: 5000 } }
)

// Set read preference to secondary
const collection = client.db("test").collection("products", {
  readPreference: 'secondary'
})

Data Synchronization Control

Configure oplog size and synchronization behavior:

# Specify oplog size (MB) at startup
mongod --replSet myReplicaSet --oplogSize 2048

Adjust oplog size during runtime:

// Connect to the local database of the primary node
use local

// Resize the oplog
db.adminCommand({replSetResizeOplog: 1, size: 2048})

Security Configuration

Replica Set Authentication

Enable internal member authentication for the replica set:

# mongod.conf configuration file
security:
  keyFile: /path/to/keyfile
replication:
  replSetName: myReplicaSet

Create a keyfile and set permissions:

openssl rand -base64 756 > /path/to/keyfile
chmod 400 /path/to/keyfile

TLS/SSL Encryption

Configure TLS encryption between replica set members:

# mongod.conf configuration file
net:
  tls:
    mode: requireTLS
    certificateKeyFile: /path/to/server.pem
    CAFile: /path/to/ca.pem
replication:
  replSetName: myReplicaSet

Performance Optimization

Read/Write Splitting

Utilize secondary nodes to share read load:

// Set read preference to the nearest node
const collection = client.db("test").collection("products", {
  readPreference: 'nearest'
})

// Set read preference to nodes with specific tags
const collection = client.db("test").collection("products", {
  readPreference: {
    mode: 'secondary',
    tags: [{ region: 'east' }]
  }
})

Index Management

Ensure secondary nodes also have appropriate indexes:

// Create an index on the primary node
db.products.createIndex({ item: 1 })

// Check index status on a secondary node
db.products.getIndexes()

Common Issue Handling

Node Synchronization Delay

Address node synchronization delays:

// Check replication lag
db.printSlaveReplicationInfo()

// Possible solutions:
// 1. Increase oplog size
// 2. Optimize network connections
// 3. Upgrade hardware performance

Election Issues

Diagnose election failures:

// View election metrics
rs.status().electionMetrics

// Common causes:
// 1. Network partition
// 2. Clock desynchronization between nodes
// 3. Configuration errors

Backup and Recovery

Replica Set Backup Strategy

Perform hot backups using mongodump:

# Backup from a secondary node
mongodump --host node2.example.com --port 27018 --out /backup/path

# Backup with oplog
mongodump --oplog --out /backup/path

Point-in-Time Recovery

Use oplog for point-in-time recovery:

# Restore base backup
mongorestore --host node1.example.com --port 27017 /backup/path

# Apply oplog up to a specific timestamp
mongorestore --oplogReplay --oplogLimit "1631059200" /backup/path