Transaction processing and atomic operations

Basic Concepts of Transaction Processing

Transaction processing is a core mechanism in database operations, ensuring that a group of operations either all succeed or none execute. In Mongoose, transaction processing is implemented through sessions, allowing developers to maintain consistency across multiple document operations. Transactions adhere to the ACID properties: Atomicity, Consistency, Isolation, and Durability.

const session = await mongoose.startSession();
session.startTransaction();

try {
  await User.create([{ name: 'Alice' }], { session });
  await Account.create([{ userId: '...', balance: 100 }], { session });
  await session.commitTransaction();
} catch (error) {
  await session.abortTransaction();
  throw error;
} finally {
  session.endSession();
}

Necessity of Atomic Operations

In distributed systems, atomic operations ensure the integrity of data modifications. Mongoose provides various atomic operation methods, such as findOneAndUpdate and updateMany, which complete the read-modify-write process in a single operation. When multiple clients modify the same document simultaneously, atomic operations prevent data races and inconsistencies.

// Atomically increment a counter
await Counter.findOneAndUpdate(
  { _id: 'userCount' },
  { $inc: { value: 1 } },
  { new: true, upsert: true }
);

Transaction Implementation in Mongoose

Mongoose transactions are based on MongoDB's multi-document transactions and require a replica set or sharded cluster environment. Each operation within a transaction must be associated with a session, and transactions have a default timeout of 60 seconds. Key limitations include the inability to create or delete collections or perform administrative operations.

const transferMoney = async (from, to, amount) => {
  const session = await mongoose.startSession();
  try {
    session.startTransaction();
    
    const fromAccount = await Account.findOneAndUpdate(
      { _id: from, balance: { $gte: amount } },
      { $inc: { balance: -amount } },
      { new: true, session }
    ).exec();
    
    if (!fromAccount) throw new Error('Insufficient funds');
    
    await Account.findOneAndUpdate(
      { _id: to },
      { $inc: { balance: amount } },
      { session }
    ).exec();
    
    await session.commitTransaction();
    return true;
  } catch (error) {
    await session.abortTransaction();
    throw error;
  } finally {
    session.endSession();
  }
};

Detailed Explanation of Atomic Operators

MongoDB provides a rich set of atomic update operators, all of which can be used in Mongoose:

• $set: Set field values
• $unset: Remove fields
• $inc: Increment or decrement numerical values
• $push/$addToSet: Array operations
• $pull: Remove elements from an array
• $bit: Bitwise operations

// Example of a complex atomic operation
await Product.findOneAndUpdate(
  { _id: productId, stock: { $gte: quantity } },
  {
    $inc: { stock: -quantity, sold: quantity },
    $push: { buyers: userId },
    $set: { lastSoldAt: new Date() }
  },
  { new: true }
);

Transaction Isolation Levels

MongoDB offers two isolation levels: read uncommitted and snapshot isolation. By default, transactions see a snapshot of the data as it was at the start of the transaction. Write operations block other transactions from modifying the same documents until the transaction is committed or rolled back.

// Isolation level example
const session1 = await mongoose.startSession();
session1.startTransaction();

const session2 = await mongoose.startSession();
session2.startTransaction();

// Session 1 reads data
const user1 = await User.findById('123', null, { session: session1 });

// Session 2 modifies the same document
await User.findByIdAndUpdate('123', { name: 'Bob' }, { session: session2 });

// Session 1 reads again, result remains unchanged (snapshot isolation)
const user1Again = await User.findById('123', null, { session: session1 });
console.log(user1.name === user1Again.name); // true

Performance Optimization Strategies

While transactions and atomic operations are powerful, they can impact performance. Optimization methods include:

1. Minimizing transaction duration
2. Avoiding time-consuming operations within transactions
3. Using appropriate read and write concern levels
4. Implementing optimistic concurrency control for frequently updated fields

// Optimistic concurrency control example
const product = await Product.findById(productId);
const updated = await Product.findOneAndUpdate(
  {
    _id: productId,
    version: product.version // Check if the version has changed
  },
  {
    $set: { price: newPrice },
    $inc: { version: 1 } // Increment the version number
  }
);

if (!updated) {
  throw new Error('Concurrent modification conflict, please retry');
}

Error Handling Patterns

Robust transaction processing requires comprehensive error handling. Common error types include:

• Concurrent modification conflicts
• Deadlocks
• Timeouts
• Network interruptions
• Validation failures

const withRetry = async (fn, maxRetries = 3) => {
  let attempts = 0;
  while (attempts < maxRetries) {
    try {
      return await fn();
    } catch (error) {
      if (error.message.includes('WriteConflict') && attempts < maxRetries - 1) {
        attempts++;
        await new Promise(resolve => setTimeout(resolve, 100 * attempts));
        continue;
      }
      throw error;
    }
  }
};

await withRetry(async () => {
  const session = await mongoose.startSession();
  try {
    session.startTransaction();
    // Transaction operations...
    await session.commitTransaction();
  } catch (error) {
    await session.abortTransaction();
    throw error;
  } finally {
    session.endSession();
  }
});

Practical Application Scenarios

Typical applications in e-commerce systems include:

1. Order creation and inventory deduction
2. Payment processing and account balance updates
3. Coupon redemption and usage
4. User point adjustments

// Order processing example
const processOrder = async (orderData) => {
  const session = await mongoose.startSession();
  try {
    session.startTransaction();
    
    // 1. Create order
    const order = await Order.create([orderData], { session });
    
    // 2. Deduct inventory
    for (const item of orderData.items) {
      const updated = await Product.findOneAndUpdate(
        { _id: item.product, stock: { $gte: item.quantity } },
        { $inc: { stock: -item.quantity } },
        { session }
      );
      if (!updated) throw new Error(`Product ${item.product} is out of stock`);
    }
    
    // 3. Process payment
    if (orderData.paymentMethod === 'wallet') {
      await User.findOneAndUpdate(
        { _id: orderData.user, balance: { $gte: orderData.total } },
        { $inc: { balance: -orderData.total } },
        { session }
      );
    }
    
    await session.commitTransaction();
    return order;
  } catch (error) {
    await session.abortTransaction();
    throw error;
  } finally {
    session.endSession();
  }
};

Advanced Transaction Patterns

For complex business scenarios, consider the following patterns:

1. Saga pattern: Break long transactions into multiple local transactions
2. Two-phase commit: Ensure consistency across multiple data sources
3. Compensating transactions: Execute reverse operations upon failure

// Saga pattern example
const placeOrderSaga = async (orderData) => {
  try {
    // Step 1: Create order (reserved status)
    const order = await Order.create({
      ...orderData,
      status: 'reserved'
    });
    
    try {
      // Step 2: Deduct inventory
      await reduceInventory(order.items);
      
      // Step 3: Process payment
      await processPayment(order.user, order.total);
      
      // All steps succeeded, confirm the order
      await Order.findByIdAndUpdate(order._id, { status: 'confirmed' });
      return order;
    } catch (error) {
      // Any step failed, cancel the order
      await Order.findByIdAndUpdate(order._id, { status: 'cancelled' });
      // Execute compensating operations...
      throw error;
    }
  } catch (error) {
    throw error;
  }
};