Common pitfalls in data modeling

Common Pitfalls in Data Modeling

Data modeling is the core of database design, but in document databases like MongoDB, developers often fall into specific traps due to habitual thinking. These mistakes can lead to poor query performance, scalability issues, or loss of data consistency.

Excessive Nesting Causes Query Complexity Explosion

Document databases allow unlimited nesting levels, but abusing this feature can cause serious problems. For example, in an e-commerce system's product category design:

// Bad example: Overly deep nesting
{
  "category": {
    "level1": "Electronics",
    "level2": {
      "name": "Phones",
      "level3": {
        "name": "Smartphones",
        "level4": {
          "brands": ["Apple", "Samsung"]
        }
      }
    }
  }
}

This design leads to:

1. Needing full paths to query specific brands: db.products.find({"category.level2.level3.level4.brands": "Apple"})
2. Update operations requiring all parent fields
3. Inability to index brand fields separately

The improved solution should use a flattened structure:

{
  "category": "Electronics/Phones/Smartphones",
  "brands": ["Apple", "Samsung"]
}

Blindly Applying Relational Paradigms

Directly migrating foreign key relationships to MongoDB is a classic anti-pattern. For example, in an order system design:

// Bad example: Relational thinking
// orders collection
{
  "_id": "order123",
  "user_id": "user456",
  "items": ["item789", "item012"]
}

// Correct approach: Appropriate embedding
{
  "_id": "order123",
  "user": {
    "_id": "user456",
    "name": "John Doe"
  },
  "items": [
    {
      "_id": "item789",
      "name": "Wireless Earbuds",
      "price": 299
    }
  ]
}

Key considerations:

• Embedded documents should not exceed the 16MB limit
• Frequently updated subdocuments should be in separate collections
• Reference relationships require application-level transactions

Ignoring Read/Write Ratio Impact on Design

Different read/write scenarios require different modeling approaches. For example, in a news comment system:

// Poor design for high-write scenarios
{
  "_id": "news123",
  "title": "Breaking News",
  "comments": [
    { "user": "A", "text": "..." },
    { "user": "B", "text": "..." }
    // Continuously growing array
  ]
}

// Optimized solution: Bucketing strategy
{
  "_id": "news123_bucket1",
  "news_id": "news123",
  "comments": [
    // Store 50 comments per bucket
  ]
}

Key considerations:

• Write-intensive data should avoid single-document bloat
• Read-intensive data can tolerate some redundancy
• Bucket size should balance query frequency and document size

Mismatched Index Strategies and Query Patterns

Inefficient indexes are more dangerous than no indexes. User query scenario:

// User collection
{
  "_id": "user1",
  "name": "Jane Doe",
  "age": 30,
  "address": {
    "city": "Beijing",
    "district": "Haidian"
  }
}

// Bad index: Single-field index
db.users.createIndex({ "name": 1 })

// Actual query: Multi-condition combination
db.users.find({
  "name": /^Zhang/,
  "age": { "$gt": 25 },
  "address.city": "Beijing"
})

// Should create compound index
db.users.createIndex({
  "name": 1,
  "age": 1,
  "address.city": 1
})

Special notes:

• ESR rule (Equality, Sort, Range) determines index field order
• Index field selectivity affects actual performance
• Covered queries can avoid collection scans

Poor Time-Series Data Modeling

Typical problems in IoT device data storage:

// Original design: Separate document per reading
{
  "device_id": "sensor01",
  "timestamp": ISODate("2023-01-01T00:00:00Z"),
  "value": 23.5
}
// Causes document explosion

// Optimized solution: Time bucketing
{
  "device_id": "sensor01",
  "start_time": ISODate("2023-01-01T00:00:00Z"),
  "end_time": ISODate("2023-01-01T01:00:00Z"),
  "readings": [
    { "time": ISODate("2023-01-01T00:00:00Z"), "value": 23.5 },
    // Aggregate hourly data
  ]
}

Advanced techniques:

• Use MongoDB 5.0+ time-series collections
• Implement hot/cold data tiering
• Pre-aggregate key metrics

Transaction Abuse Causes Performance Bottlenecks

While MongoDB supports multi-document transactions, misuse can cripple systems:

// Unreasonable cross-document transaction
try {
  session.startTransaction();
  await orders.insertOne({...}, { session });
  await inventory.updateOne({...}, { session });
  await payment.createOne({...}, { session });
  session.commitTransaction();
} catch (e) {
  session.abortTransaction();
}

// Better solution: Redesign the model
{
  "_id": "order123",
  "items": [
    { "product_id": "p1", "qty": 2 }
  ],
  "inventory_locked": true  // Use status flags
}

Important notes:

• Transactions default to 60-second timeout
• Sharded cluster transactions are more expensive
• Consider compensation transaction patterns

Lack of Schema Evolution Planning

Version control neglect causing migration disasters:

// Original user model
{
  "_id": "user1",
  "login": "user1@example.com"
}

// New requirement: Support multiple emails
// Wrong approach: Direct structure modification
{
  "_id": "user1",
  "emails": ["user1@example.com"]
}

// Correct solution: Versioned handling
{
  "_id": "user1",
  "schema_version": 2,
  "emails": {
    "primary": "user1@example.com",
    "secondary": []
  }
}

Migration strategies:

• Support both formats during transition
• Use $jsonSchema validation
• Incremental migration to avoid downtime

Neglecting Shard Key Selection Impact

Sharded cluster design mistakes:

// Bad shard key: Low-cardinality field
sh.shardCollection("test.orders", { "status": 1 })

// Queries cause broadcast operations
db.orders.find({ "customer_id": "cust123" })

// Ideal shard key: Compound field
sh.shardCollection("test.orders", 
  { "customer_id": 1, "order_date": -1 })

Selection principles:

• Ensure sufficient cardinality
• Match primary query patterns
• Avoid write hotspots
• Consider shard key immutability