Operations on nested documents and subdocuments

Basic Concepts of Nested Documents and Subdocuments

In Mongoose, nested documents refer to a structure where one document is directly embedded within another. This design pattern allows related data to be organized together without the need for additional collections or references. Subdocuments are another manifestation of nested documents, typically existing as a property of a parent document.

const childSchema = new mongoose.Schema({
  name: String,
  age: Number
});

const parentSchema = new mongoose.Schema({
  children: [childSchema],
  address: String
});

In this example, childSchema is nested within parentSchema as an array of subdocuments. Mongoose automatically creates an _id field for subdocuments unless explicitly disabled.

Creating Nested Documents

There are several ways to create documents containing nested documents. The most straightforward method is to specify the subdocument content while creating the parent document:

const Parent = mongoose.model('Parent', parentSchema);

const newParent = new Parent({
  address: '123 Main St',
  children: [
    { name: 'Alice', age: 8 },
    { name: 'Bob', age: 5 }
  ]
});

You can also create subdocument instances first and then add them to the parent document:

const child1 = { name: 'Charlie', age: 10 };
const child2 = { name: 'Diana', age: 7 };

const anotherParent = new Parent({
  address: '456 Oak Ave',
  children: [child1, child2]
});

Querying Nested Documents

When querying nested documents, Mongoose provides various operators and methods. The most basic query can access nested fields using dot notation:

Parent.find({ 'children.age': { $gt: 6 } }, (err, parents) => {
  console.log(parents);
});

Use $elemMatch for more complex queries:

Parent.find({
  children: {
    $elemMatch: {
      age: { $gt: 5 },
      name: /^A/
    }
  }
});

Updating Nested Documents

There are multiple methods for updating nested documents. You can use the traditional update operation:

Parent.updateOne(
  { _id: parentId, 'children._id': childId },
  { $set: { 'children.$.age': 9 } },
  (err, result) => {
    // Handle result
  }
);

Mongoose also provides the more convenient findOneAndUpdate:

Parent.findOneAndUpdate(
  { _id: parentId, 'children._id': childId },
  { $inc: { 'children.$.age': 1 } },
  { new: true }
).then(updatedParent => {
  console.log(updatedParent);
});

Deleting Nested Documents

To remove subdocuments from a parent document, use the $pull operator:

Parent.updateOne(
  { _id: parentId },
  { $pull: { children: { _id: childId } } },
  (err, result) => {
    // Handle result
  }
);

To delete all subdocuments that meet certain conditions:

Parent.updateMany(
  {},
  { $pull: { children: { age: { $lt: 3 } } },
  { multi: true }
);

Validation of Nested Documents

Mongoose automatically applies validation rules from the subdocument schema. For example, if the subdocument schema requires the name field:

const strictChildSchema = new mongoose.Schema({
  name: { type: String, required: true },
  age: Number
});

const strictParentSchema = new mongoose.Schema({
  children: [strictChildSchema]
});

Attempting to save a subdocument without a name will trigger a validation error:

const invalidParent = new Parent({
  children: [{ age: 4 }]  // Missing required name field
});

invalidParent.save(err => {
  console.log(err);  // Validation error
});

Middleware with Nested Documents

Middleware can be used on nested documents. For example, preprocessing before saving:

childSchema.pre('save', function(next) {
  if (this.age < 0) {
    this.age = 0;
  }
  next();
});

This middleware executes before each subdocument is saved, ensuring the age is not negative.

Operations on Nested Document Arrays

Mongoose provides special methods for manipulating nested document arrays. The push method can add new subdocuments:

Parent.findById(parentId, (err, parent) => {
  parent.children.push({ name: 'Eva', age: 6 });
  parent.save();
});

Use the id method to quickly find a specific subdocument:

const parent = await Parent.findById(parentId);
const child = parent.children.id(childId);
console.log(child);

Differences Between Nested Documents and Population

While both nested documents and population establish relationships between documents, they work differently. Population uses references and additional queries:

const refChildSchema = new mongoose.Schema({
  name: String,
  age: Number
});

const refParentSchema = new mongoose.Schema({
  children: [{ type: mongoose.Schema.Types.ObjectId, ref: 'Child' }]
});

In contrast, nested documents store all data within the same document, eliminating the need for additional operations during queries.

Performance Considerations for Nested Documents

Nested documents are suitable for the following scenarios:

• The number of subdocuments is limited and will not grow indefinitely
• Subdocuments are primarily accessed together with the parent document
• Atomic updates of both parent and subdocuments are required

They are not suitable for:

• Situations where the number of subdocuments could become very large
• Cases where subdocuments need to be queried or updated independently
• Scenarios where multiple parent documents share the same subdocuments

Depth Limitations of Nested Documents

Mongoose does not limit nesting depth by default, but you can implement custom validation:

const deepSchema = new mongoose.Schema({
  level1: {
    level2: {
      level3: {
        value: String
      }
    }
  }
});

function validateDepth(doc, maxDepth) {
  // Implement depth validation logic
}

deepSchema.pre('save', function(next) {
  if (validateDepth(this, 3)) {
    next(new Error('Maximum nesting depth exceeded'));
  } else {
    next();
  }
});

Indexing Nested Documents

Create indexes on nested fields to improve query performance:

parentSchema.index({ 'children.name': 1 });
parentSchema.index({ 'children.age': -1 });

Compound indexes are also applicable:

parentSchema.index({ 'children.name': 1, 'children.age': -1 });

Atomic Operations with Nested Documents

Mongoose ensures that updates to a single document (including its nested documents) are atomic. For example, you can atomically update both the parent and subdocument:

Parent.findOneAndUpdate(
  { _id: parentId },
  {
    $set: { address: 'New Address' },
    $inc: { 'children.$.age': 1 }
  },
  { new: true }
);

Schema Design Tips for Nested Documents

When designing nested document schemas, consider the following factors:

1. Access patterns: Which fields are frequently queried together
2. Update frequency: Which fields are often updated independently
3. Size limitations: MongoDB documents have a maximum size of 16MB

// Good design: Frequently accessed data grouped together
const userSchema = new mongoose.Schema({
  profile: {
    name: String,
    avatar: String,
    bio: String
  },
  preferences: {
    theme: String,
    notifications: Boolean
  }
});

Migration Strategies for Nested Documents

When modifying nested document structures, consider these migration approaches:

1. Bulk update all documents:

Parent.updateMany(
  {},
  { $rename: { 'children.grade': 'children.level' } }
);

2. Use migration scripts for complex cases:

async function migrateChildren() {
  const parents = await Parent.find({});
  
  for (const parent of parents) {
    parent.children.forEach(child => {
      if (child.grade) {
        child.level = convertGradeToLevel(child.grade);
        delete child.grade;
      }
    });
    await parent.save();
  }
}

Transactions with Nested Documents

When operating on nested documents within MongoDB transactions, the entire document is treated as an atomic unit:

const session = await mongoose.startSession();
session.startTransaction();

try {
  const parent = await Parent.findById(parentId).session(session);
  parent.children.push(newChild);
  await parent.save();
  
  await session.commitTransaction();
} catch (error) {
  await session.abortTransaction();
  throw error;
} finally {
  session.endSession();
}

Virtual Properties for Nested Documents

Add virtual properties to nested documents:

childSchema.virtual('ageGroup').get(function() {
  if (this.age < 5) return 'Toddler';
  if (this.age < 12) return 'Child';
  return 'Teen';
});

const parent = await Parent.findById(parentId);
console.log(parent.children[0].ageGroup);  // Returns age group based on age

JSON Serialization of Nested Documents

Control JSON output of nested documents:

childSchema.set('toJSON', {
  transform: (doc, ret) => {
    ret.id = ret._id;
    delete ret._id;
    delete ret.__v;
    return ret;
  }
});

parentSchema.set('toJSON', {
  virtuals: true
});

Default Values for Nested Documents

Set default values for nested documents:

const settingsSchema = new mongoose.Schema({
  notifications: { type: Boolean, default: true },
  theme: { type: String, default: 'light' }
});

const userSchema = new mongoose.Schema({
  settings: { type: settingsSchema, default: () => ({}) }
});

Conditional Validation for Nested Documents

Validate subdocuments based on parent document state:

parentSchema.path('children').validate(function(children) {
  if (this.type === 'large_family' && children.length < 3) {
    return false;
  }
  return true;
}, 'Large families must have at least 3 children');

Query Performance Optimization for Nested Documents

Methods to optimize nested document queries:

1. Use projection to return only needed fields:

Parent.find({}, 'address children.name');

2. Create indexes for common query conditions:

parentSchema.index({ 'children.age': 1, address: 1 });

3. Use $slice to limit returned array elements:

Parent.findById(parentId, { children: { $slice: 5 } });

Nested Documents with GraphQL

Handling nested documents in GraphQL:

const ParentType = new GraphQLObjectType({
  name: 'Parent',
  fields: () => ({
    address: { type: GraphQLString },
    children: {
      type: new GraphQLList(ChildType),
      resolve(parent) {
        return parent.children;
      }
    }
  })
});

const ChildType = new GraphQLObjectType({
  name: 'Child',
  fields: {
    name: { type: GraphQLString },
    age: { type: GraphQLInt }
  }
});

Version Control for Nested Documents

Implement version control for nested documents:

const versionedChildSchema = new mongoose.Schema({
  name: String,
  age: Number,
  version: { type: Number, default: 0 }
});

childSchema.pre('save', function(next) {
  if (this.isModified()) {
    this.version += 1;
  }
  next();
});

Parallel Processing of Nested Documents

Safely process nested documents in parallel:

async function updateChildAge(parentId, childId, newAge) {
  const parent = await Parent.findById(parentId);
  const child = parent.children.id(childId);
  
  if (!child) throw new Error('Child not found');
  
  child.age = newAge;
  await parent.save();
}

Bulk Insert for Nested Documents

Efficiently insert multiple nested documents:

async function addMultipleChildren(parentId, newChildren) {
  await Parent.findByIdAndUpdate(
    parentId,
    { $push: { children: { $each: newChildren } } },
    { new: true }
  );
}

Reference Counting in Nested Documents

Implement reference counting in nested documents:

const tagSchema = new mongoose.Schema({
  name: String,
  count: { type: Number, default: 0 }
});

const postSchema = new mongoose.Schema({
  tags: [tagSchema]
});

postSchema.pre('save', function(next) {
  this.tags.forEach(tag => {
    tag.count = this.tags.filter(t => t.name === tag.name).length;
  });
  next();
});

Type Conversion in Nested Documents

Handle type conversion in nested documents:

childSchema.path('age').set(function(v) {
  return parseInt(v, 10) || 0;
});

const parent = new Parent({
  children: [{ age: '10' }]  // Automatically converted to number 10
});

Encryption for Nested Documents

Encrypt sensitive nested fields:

const crypto = require('crypto');
const secret = 'my-secret-key';

childSchema.pre('save', function(next) {
  if (this.isModified('ssn')) {
    const cipher = crypto.createCipher('aes-256-cbc', secret);
    this.ssn = cipher.update(this.ssn, 'utf8', 'hex') + cipher.final('hex');
  }
  next();
});

childSchema.methods.decryptSsn = function() {
  const decipher = crypto.createDecipher('aes-256-cbc', secret);
  return decipher.update(this.ssn, 'hex', 'utf8') + decipher.final('utf8');
};

Internationalization for Nested Documents

Implement multilingual support for nested documents:

const i18nStringSchema = new mongoose.Schema({
  en: String,
  es: String,
  fr: String
});

const productSchema = new mongoose.Schema({
  name: i18nStringSchema,
  description: i18nStringSchema
});

productSchema.methods.getName = function(lang) {
  return this.name[lang] || this.name.en;
};

Audit Logs for Nested Documents

Track change history for nested documents:

const auditLogSchema = new mongoose.Schema({
  timestamp: { type: Date, default: Date.now },
  operation: String,
  data: mongoose.Schema.Types.Mixed
});

const auditableSchema = new mongoose.Schema({
  content: String,
  logs: [auditLogSchema]
});

auditableSchema.pre('save', function(next) {
  if (this.isModified()) {
    this.logs.push({
      operation: this.isNew ? 'create' : 'update',
      data: this.getChanges()
    });
  }
  next();
});

Caching Strategies for Nested Documents

Optimize caching for nested documents:

const cache = new Map();

parentSchema.statics.findByIdWithCache = async function(id) {
  if (cache.has(id)) {
    return cache.get(id);
  }
  
  const parent = await this.findById(id).lean();
  cache.set(id, parent);
  return parent;
};

parentSchema.post('save', function(doc) {
  cache.set(doc._id.toString(), doc.toObject());
});

Testing Strategies for Nested Documents

Example tests for nested documents:

describe('Parent Model', () => {
  it('should validate child age', async () => {
    const parent = new Parent({
      children: [{ name: 'Test', age: -1 }]
    });
    
    await expect(parent.save()).rejects.toThrow();
  });

  it('should update nested child', async () => {
    const parent = await Parent.create({ /* ... */ });
    await Parent.updateOne(
      { _id: parent._id, 'children._id': parent.children[0]._id },
      { $set: { 'children.$.name': 'Updated' } }
    );
    
    const updated = await Parent.findById(parent._id);
    expect(updated.children[0].name).toBe('Updated');
  });
});

Migration from Nested to Independent Collections

Strategy for migrating nested documents to independent collections:

async function migrateChildrenToCollection() {
  const parents = await Parent.find({});
  const Child = mongoose.model('Child', childSchema);
  
  for (const parent of parents) {
    const childDocs = parent.children.map(child => ({
      ...child.toObject(),
      parent: parent._id
    }));
    
    await Child.insertMany(childDocs);
    await Parent.updateOne(
      { _id: parent._id },
      { $set: { children: [] } }
    );
  }
}

Aggregation Operations with Nested Documents

Handling nested documents in aggregation pipelines:

Parent.aggregate([
  { $unwind: '$children' },
  { $match: { 'children.age': { $gte: 5 } } },
  { $group: {
    _id: '$_id',
    address: { $first: '$address' },
    childrenCount: { $sum: 1 }
  }}
]);

Geospatial Queries with Nested Documents

Using geospatial queries with nested documents:

const locationSchema = new mongoose.Schema({
  type: { type: String, default: 'Point' },
  coordinates: { type: [Number], index: '2dsphere' }
});

const placeSchema = new mongoose.Schema({
  name: String,
  locations: [locationSchema]
});

placeSchema.index({ 'locations': '2dsphere' });

Place.find({
  locations: {
    $near: {
      $geometry: {
        type: 'Point',
        coordinates: [longitude, latitude]
      },
      $maxDistance: 1000
    }
  }
});

Full-Text Search with Nested Documents

Implementing full-text search for nested documents:

parentSchema.index({
  'children.name': 'text',
  'children.description': 'text'
});

Parent.find(
  { $text: { $search: 'keyword' } },
  { score: { $meta: 'textScore' } }
).sort({ score: { $meta: 'textScore' } });

Access Control for Nested Documents

Role-based access control for nested documents:

parentSchema.methods.filterChildrenForRole = function(role) {
  if (role === 'admin') {
    return this.children;
  } else {
    return this.children.map(child => ({
      name: child.name,
      age: child.age
      // Hide sensitive fields