Usage and limitations of multi-document transactions

Basic Concepts of Multi-Document Transactions

MongoDB introduced support for multi-document transactions in version 4.0, allowing operations across multiple documents to be executed within a single transaction. Transactions provide ACID properties (Atomicity, Consistency, Isolation, and Durability), ensuring that multiple operations either all succeed or all fail and roll back. This is particularly important for complex business scenarios that require data consistency.

The implementation of transactions in MongoDB is similar to relational databases but with some specific limitations. Transactions must be executed within a session, and by default, operations in a transaction wait for locks to be released. MongoDB uses snapshot isolation, ensuring that transactions see a consistent snapshot of the data.

const session = db.getMongo().startSession();
session.startTransaction({
    readConcern: { level: 'snapshot' },
    writeConcern: { w: 'majority' }
});

try {
    const users = session.getDatabase('test').users;
    const orders = session.getDatabase('test').orders;
    
    users.insertOne({ _id: 1, name: 'Alice', balance: 100 });
    orders.insertOne({ _id: 1, userId: 1, amount: 50 });
    
    await session.commitTransaction();
} catch (error) {
    await session.abortTransaction();
    throw error;
} finally {
    session.endSession();
}

Use Cases for Transactions

Multi-document transactions are particularly suitable for operations that require atomicity across multiple collections or documents. Typical scenarios include bank transfers, order processing, inventory management, etc. In these scenarios, multiple operations must be executed as a single unit; otherwise, data inconsistency may occur.

For example, the order creation process in an e-commerce system:

1. Deduct funds from the user's account
2. Create an order record
3. Reduce inventory quantity These three operations must be executed as an atomic unit.

async function createOrder(userId, productId, quantity) {
    const session = client.startSession();
    try {
        session.startTransaction();
        
        // 1. Deduct user balance
        const product = await products.findOne({ _id: productId }, { session });
        await users.updateOne(
            { _id: userId },
            { $inc: { balance: -product.price * quantity } },
            { session }
        );
        
        // 2. Create order
        await orders.insertOne({
            userId,
            productId,
            quantity,
            total: product.price * quantity,
            date: new Date()
        }, { session });
        
        // 3. Reduce inventory
        await products.updateOne(
            { _id: productId },
            { $inc: { stock: -quantity } },
            { session }
        );
        
        await session.commitTransaction();
    } catch (error) {
        await session.abortTransaction();
        throw error;
    } finally {
        session.endSession();
    }
}

Performance Impact of Transactions

Using transactions incurs significant performance overhead, primarily due to the following factors:

1. Lock contention: Transactions acquire locks on the documents being operated on, which may cause other operations to wait.
2. Memory usage: Transactions require maintaining operation logs and snapshots, increasing memory pressure.
3. Network round-trips: Transactions require additional communication to coordinate commits or rollbacks.

On sharded clusters, the performance impact of transactions is even more pronounced because coordinating transaction states across multiple shards is necessary. MongoDB recommends keeping transaction durations under 1 second, as long-running transactions may cause performance issues or even timeouts.

You can optimize transaction performance by:

• Minimizing the number and scope of operations within a transaction.
• Avoiding time-consuming computations or I/O operations within transactions.
• Using appropriate indexes to speed up queries within transactions.
• Considering splitting large documents that are frequently updated into smaller documents.

Limitations and Constraints of Transactions

MongoDB multi-document transactions have some important limitations to note:

1. Collection restrictions: Cannot create or delete collections or create indexes within a transaction.
2. DDL operation restrictions: Database management operations (e.g., creating users, modifying collection options) cannot be executed within a transaction.
3. Shard key modification: Cannot modify a document's shard key value within a transaction.
4. Size limit: The total size of operations in a single transaction cannot exceed 16MB.
5. Time limit: By default, transactions can run for a maximum of 60 seconds, adjustable via transactionLifetimeLimitSeconds.
6. Cursor restrictions: Cursors created within a transaction cannot be used outside the transaction.

// Error example: Attempting to create a collection within a transaction
async function invalidTransaction() {
    const session = client.startSession();
    try {
        session.startTransaction();
        
        // This will throw an error
        await db.createCollection('newCollection', { session });
        
        await session.commitTransaction();
    } catch (error) {
        console.error('Transaction failed:', error);
        await session.abortTransaction();
    } finally {
        session.endSession();
    }
}

Transactions and Replica Sets

When using transactions in a replica set environment, consider the following factors:

1. Write concern: Transactions should use the "majority" write concern to ensure data is written to most nodes.
2. Read concern: Typically use "snapshot" read concern to guarantee a consistent view.
3. Election impact: Primary node failures may cause ongoing transactions to abort.
4. Oplog size: Large transactions may consume significant oplog space, affecting replication.

In production environments, it is recommended to configure a sufficiently large oplog to accommodate expected transaction volumes. Additionally, applications should be prepared to handle transactions aborted due to primary node switches.

// Transaction example in a replica set environment
async function replicaSetTransaction() {
    const session = client.startSession();
    try {
        session.startTransaction({
            readConcern: { level: 'snapshot' },
            writeConcern: { w: 'majority', j: true }
        });
        
        // Transaction operations...
        
        await session.commitTransaction();
    } catch (error) {
        if (error.errorLabels && error.errorLabels.includes('TransientTransactionError')) {
            // Temporary error, can retry
            console.log('Transient error, retrying...');
            return replicaSetTransaction();
        }
        await session.abortTransaction();
        throw error;
    } finally {
        session.endSession();
    }
}

Transactions and Sharded Clusters

Using transactions in a sharded cluster is more complex, with additional considerations:

1. Sharded transactions: All shards participating in the transaction must be configured as replica sets.
2. Performance impact: Cross-shard transactions are much slower than single-shard transactions.
3. Configuration limits: Transactions cannot involve more than 1,000 shards.
4. mongos version: All mongos instances must run versions that support multi-shard transactions.

Cross-shard transactions require two-phase commits, which increases latency. When designing data models, try to keep related data on the same shard.

// Sharded cluster transaction example
async function shardedClusterTransaction() {
    const session = client.startSession();
    try {
        session.startTransaction({
            readConcern: { level: 'snapshot' },
            writeConcern: { w: 'majority' }
        });
        
        // Ensure these operations involve as few shards as possible
        await db.users.updateOne(
            { _id: 'user1' }, 
            { $inc: { balance: -100 } },
            { session }
        );
        
        await db.orders.insertOne(
            { userId: 'user1', amount: 100, date: new Date() },
            { session }
        );
        
        await session.commitTransaction();
    } catch (error) {
        if (error.errorLabels && error.errorLabels.includes('UnknownTransactionCommitResult')) {
            // Commit result unknown, need to check transaction status
            console.log('Unknown commit result, checking status...');
        }
        await session.abortTransaction();
        throw error;
    } finally {
        session.endSession();
    }
}

Error Handling and Retry Logic

Transactions may fail for various reasons, and applications should implement appropriate error handling and retry mechanisms. MongoDB provides special error labels to help identify retryable errors:

1. TransientTransactionError: Temporary error; the entire transaction can be safely retried.
2. UnknownTransactionCommitResult: Commit result unknown; the transaction status needs to be checked.

async function runTransactionWithRetry(txnFunc, maxRetries = 3) {
    let retryCount = 0;
    while (retryCount < maxRetries) {
        try {
            return await txnFunc();
        } catch (error) {
            console.error('Transaction error:', error);
            
            if (error.errorLabels && 
                (error.errorLabels.includes('TransientTransactionError') ||
                 error.errorLabels.includes('UnknownTransactionCommitResult'))) {
                retryCount++;
                console.log(`Retrying transaction (attempt ${retryCount})...`);
                continue;
            }
            
            throw error;
        }
    }
    throw new Error(`Transaction failed after ${maxRetries} attempts`);
}

// Usage example
await runTransactionWithRetry(async () => {
    const session = client.startSession();
    try {
        session.startTransaction();
        // Transaction operations...
        await session.commitTransaction();
    } catch (error) {
        await session.abortTransaction();
        throw error;
    } finally {
        session.endSession();
    }
});

Transaction Monitoring and Diagnostics

Monitoring transaction performance is important for maintaining system health. MongoDB provides various tools and commands to monitor transactions:

1. currentOp: View running transactions.

db.adminCommand({ currentOp: true, $or: [
    { op: 'command', 'command.abortTransaction': { $exists: true } },
    { op: 'command', 'command.commitTransaction': { $exists: true } },
    { op: 'command', 'command.startTransaction': { $exists: true } }
]})

2. serverStatus: View transaction statistics.

db.serverStatus().transactions

3. mongostat: Command-line tool to view transaction metrics.
4. Audit logs: Configure audit logs to record transaction operations.

Key metrics include:

• Number of active transactions
• Transaction duration
• Commit/abort counts
• Lock wait times

Alternatives and Best Practices

In some scenarios, alternatives to multi-document transactions can be considered:

1. Embedded documents: Model related data as nested structures within a single document.
2. Optimistic concurrency control: Use version numbers or timestamps to detect conflicts.
3. Two-phase commit pattern: Implement application-level atomicity guarantees.
4. $merge: Use the $merge operation in aggregation pipelines to implement complex updates.

Best practices include:

• Keep transactions short; avoid long-running transactions.
• Minimize the number of operations within a transaction.
• Design appropriate indexes for transaction operations.
• Implement robust error handling and retry logic.
• Thoroughly test transaction performance before deploying to production.

// Example of using embedded documents as an alternative to multi-document transactions
// Embed orders in user documents to achieve atomic updates
await users.updateOne(
    { _id: userId, 'orders.orderId': { $ne: orderId } },
    { 
        $inc: { balance: -amount },
        $push: { 
            orders: {
                orderId,
                amount,
                date: new Date()
            }
        }
    }
);