Distributed transactions (cross-shard transactions)

The Concept of Distributed Transactions (Cross-Shard Transactions)

Distributed transactions refer to transactional operations that span multiple databases or shards, requiring the guarantee of atomicity, consistency, isolation, and durability (ACID). In MongoDB, cross-shard transactions allow write operations to be executed across multiple shards while ensuring that these operations either all succeed or all fail. This is crucial for applications requiring strong consistency.

MongoDB began supporting multi-document transactions within replica sets in version 4.0, and version 4.2 extended this functionality to support multi-document transactions across shards. This means developers can perform transactional operations involving multiple shards in a sharded cluster without worrying about data inconsistency.

Implementation Principles of MongoDB Cross-Shard Transactions

MongoDB's cross-shard transactions are implemented based on the two-phase commit (2PC) protocol. The core workflow is as follows:

1. Prepare Phase: The transaction coordinator sends a prepare command to all participating shards. Each shard executes the transaction operation but does not commit, storing the result in the transaction log.

2. Commit/Abort Phase: If all shards prepare successfully, the coordinator sends a commit command; if any shard fails to prepare, the coordinator sends an abort command.

3. Recovery Mechanism: If the coordinator fails during the commit process, the shards periodically communicate with the coordinator to determine the final state of the transaction.

// Example: Using Node.js driver to execute cross-shard transactions
const { MongoClient } = require('mongodb');

async function runTransaction() {
  const client = await MongoClient.connect('mongodb://localhost:27017');
  const session = client.startSession();
  
  try {
    session.startTransaction({
      readConcern: { level: 'snapshot' },
      writeConcern: { w: 'majority' }
    });
    
    const db1 = client.db('db1').collection('users');
    const db2 = client.db('db2').collection('orders');
    
    await db1.updateOne(
      { _id: 'user123' },
      { $inc: { balance: -100 } },
      { session }
    );
    
    await db2.insertOne(
      { userId: 'user123', amount: 100, date: new Date() },
      { session }
    );
    
    await session.commitTransaction();
    console.log('Transaction committed');
  } catch (error) {
    await session.abortTransaction();
    console.error('Transaction aborted:', error);
  } finally {
    session.endSession();
    client.close();
  }
}

runTransaction();

Performance Considerations for Cross-Shard Transactions

While cross-shard transactions provide strong consistency guarantees, they come with certain performance overheads:

1. Increased Latency: The two-phase commit requires additional network round trips, increasing transaction completion time.

2. Lock Contention: Locks are held during transactions, potentially blocking other operations.

3. Resource Consumption: Transaction logs require additional storage space and I/O operations.

To optimize performance, consider the following strategies:

• Minimize transaction duration
• Reduce the number of shards involved in transactions
• Design shard keys appropriately to keep related data on the same shard
• Use appropriate read and write concern levels

Transaction Timeout and Retry Mechanisms

MongoDB provides timeout and automatic retry mechanisms for cross-shard transactions:

1. Transaction Timeout: Default is 60 seconds, configurable via transactionLifetimeLimitSeconds.

2. Retryable Writes: The driver can automatically retry certain retryable errors.

// Example of configuring transaction options
const transactionOptions = {
  readConcern: { level: 'snapshot' },
  writeConcern: { w: 'majority', j: true },
  maxCommitTimeMS: 10000, // 10-second timeout
  readPreference: 'primary'
};

Practical Application Scenarios

E-commerce Platform Order Processing:

1. After a user places an order, the following steps are required:
   • Deduct payment from the user account (user shard)
   • Create an order record (order shard)
   • Update inventory (product shard)

// E-commerce transaction example
async function processOrder(userId, productId, quantity) {
  const session = client.startSession();
  try {
    session.startTransaction();
    
    // Deduct user balance
    await usersCollection.updateOne(
      { _id: userId },
      { $inc: { balance: -totalPrice } },
      { session }
    );
    
    // Create order record
    await ordersCollection.insertOne({
      userId,
      productId,
      quantity,
      totalPrice,
      status: 'pending'
    }, { session });
    
    // Update inventory
    await inventoryCollection.updateOne(
      { _id: productId },
      { $inc: { stock: -quantity } },
      { session }
    );
    
    await session.commitTransaction();
  } catch (error) {
    await session.abortTransaction();
    throw error;
  } finally {
    session.endSession();
  }
}

Monitoring and Troubleshooting

Key metrics for monitoring cross-shard transactions:

1. Transaction Statistics:

   db.runCommand({ serverStatus: 1 }).transactions
   

2. Current Active Transactions:

   db.currentOp({ 'lsid': { $exists: true } })
   

3. Transaction Logs: Check transaction-related entries in the mongod logs.

Common troubleshooting methods:

• Transaction timeout: Check if the transaction duration is too long
• Lock waits: Analyze lock contention
• Network partitions: Check connectivity between cluster nodes

Best Practices and Limitations

Best Practices:

1. Design appropriate shard keys to minimize cross-shard operations.
2. Avoid long-running operations within transactions.
3. Set reasonable transaction timeout periods.
4. Implement proper error handling and retry logic.

Limitations:

1. Cannot create or delete collections within transactions.
2. Cannot modify shard key values.
3. Transaction size limits (default 16MB).
4. Certain commands cannot be executed within transactions (e.g., createIndex).

// Error example: Operations not allowed in transactions
async function invalidTransaction() {
  const session = client.startSession();
  try {
    session.startTransaction();
    
    // Error: Cannot create collections within transactions
    await db.createCollection('newCollection', { session });
    
    await session.commitTransaction();
  } catch (error) {
    console.error('This will fail:', error);
    await session.abortTransaction();
  }
}

Comparison with Other Database Transactions

Comparison of MongoDB cross-shard transactions with other database solutions:

1. Comparison with Traditional RDBMS:

   • MongoDB uses optimistic concurrency control instead of locking mechanisms.
   • Default isolation level is snapshot isolation.
   • No table-level locks, supporting higher concurrency.

2. Comparison with NewSQL Databases:

   • MongoDB adopts eventual consistency as the default mode.
   • Cross-shard transactions are optional features.
   • More suitable for flexible data models.

3. Comparison with NoSQL Solutions:

   • Provides stronger guarantees than most NoSQL databases.
   • Supports multi-document ACID transactions.
   • Does not require strict data models.