Array processing ($unwind, $filter, $slice, etc.)

Array Processing ($unwind, $filter, $slice, etc.)

MongoDB provides a rich set of array operators for processing and manipulating array fields in documents. These operators can be used in aggregation pipelines, as well as in query and update operations. Array processing is a very common operation in MongoDB, and mastering these operators can greatly enhance the flexibility and efficiency of data processing.

$unwind Operator

The $unwind operator is used to split each element in an array field into separate documents. This is useful for scenarios where each element in the array needs to be processed individually.

// Sample data
db.orders.insertMany([
  { _id: 1, items: ["apple", "banana", "orange"] },
  { _id: 2, items: ["pear", "grape"] }
])

// Using $unwind to expand the array
db.orders.aggregate([
  { $unwind: "$items" }
])

// Result
[
  { _id: 1, items: "apple" },
  { _id: 1, items: "banana" },
  { _id: 1, items: "orange" },
  { _id: 2, items: "pear" },
  { _id: 2, items: "grape" }
]

$unwind also supports some optional parameters:

• preserveNullAndEmptyArrays: Defaults to false. If set to true, documents with null, empty, or non-existent array fields will be retained.

db.orders.aggregate([
  { $unwind: { path: "$items", preserveNullAndEmptyArrays: true } }
])

$filter Operator

The $filter operator is used to filter array elements based on specified conditions, returning a new array.

// Sample data
db.products.insertMany([
  { _id: 1, name: "Laptop", prices: [1200, 1100, 1000] },
  { _id: 2, name: "Phone", prices: [800, 750, 700] }
])

// Using $filter to select elements with prices greater than 1000
db.products.aggregate([
  {
    $project: {
      name: 1,
      highPrices: {
        $filter: {
          input: "$prices",
          as: "price",
          cond: { $gt: ["$$price", 1000] }
        }
      }
    }
  }
])

// Result
[
  { _id: 1, name: "Laptop", highPrices: [1200, 1100] },
  { _id: 2, name: "Phone", highPrices: [] }
]

$slice Operator

The $slice operator is used to return a subset of an array, specifying the starting position and the number of elements.

// Sample data
db.blogposts.insertMany([
  { _id: 1, title: "Post 1", comments: ["Great!", "Nice", "Awesome"] },
  { _id: 2, title: "Post 2", comments: ["First", "Second", "Third", "Fourth"] }
])

// Get the first 2 comments
db.blogposts.aggregate([
  {
    $project: {
      title: 1,
      firstTwoComments: { $slice: ["$comments", 2] }
    }
  }
])

// Result
[
  { _id: 1, title: "Post 1", firstTwoComments: ["Great!", "Nice"] },
  { _id: 2, title: "Post 2", firstTwoComments: ["First", "Second"] }
]

// Get 2 comments starting from the 2nd element
db.blogposts.aggregate([
  {
    $project: {
      title: 1,
      sliceComments: { $slice: ["$comments", 1, 2] }
    }
  }
])

$map Operator

The $map operator applies an expression to each element in an array and returns the resulting array.

// Sample data
db.products.insertMany([
  { _id: 1, name: "Laptop", prices: [1200, 1100, 1000] },
  { _id: 2, name: "Phone", prices: [800, 750, 700] }
])

// Apply a discount to each element in the prices array
db.products.aggregate([
  {
    $project: {
      name: 1,
      discountedPrices: {
        $map: {
          input: "$prices",
          as: "price",
          in: { $multiply: ["$$price", 0.9] } // 10% discount
        }
      }
    }
  }
])

// Result
[
  { _id: 1, name: "Laptop", discountedPrices: [1080, 990, 900] },
  { _id: 2, name: "Phone", discountedPrices: [720, 675, 630] }
]

$reduce Operator

The $reduce operator combines array elements into a single value using an expression.

// Sample data
db.sales.insertMany([
  { _id: 1, items: [10, 20, 30] },
  { _id: 2, items: [5, 15, 25] }
])

// Calculate the sum of array elements
db.sales.aggregate([
  {
    $project: {
      total: {
        $reduce: {
          input: "$items",
          initialValue: 0,
          in: { $add: ["$$value", "$$this"] }
        }
      }
    }
  }
])

// Result
[
  { _id: 1, total: 60 },
  { _id: 2, total: 45 }
]

$size Operator

The $size operator returns the length of an array.

// Sample data
db.blogposts.insertMany([
  { _id: 1, title: "Post 1", comments: ["Great!", "Nice", "Awesome"] },
  { _id: 2, title: "Post 2", comments: ["First", "Second"] }
])

// Get the number of comments
db.blogposts.aggregate([
  {
    $project: {
      title: 1,
      commentCount: { $size: "$comments" }
    }
  }
])

// Result
[
  { _id: 1, title: "Post 1", commentCount: 3 },
  { _id: 2, title: "Post 2", commentCount: 2 }
]

$concatArrays Operator

The $concatArrays operator concatenates multiple arrays.

// Sample data
db.students.insertMany([
  { _id: 1, name: "Alice", grades1: [80, 85], grades2: [90, 95] },
  { _id: 2, name: "Bob", grades1: [70, 75], grades2: [60, 65] }
])

// Merge two grade arrays
db.students.aggregate([
  {
    $project: {
      name: 1,
      allGrades: { $concatArrays: ["$grades1", "$grades2"] }
    }
  }
])

// Result
[
  { _id: 1, name: "Alice", allGrades: [80, 85, 90, 95] },
  { _id: 2, name: "Bob", allGrades: [70, 75, 60, 65] }
]

$arrayElemAt Operator

The $arrayElemAt operator returns the element at a specified position in an array.

// Sample data
db.products.insertMany([
  { _id: 1, name: "Laptop", prices: [1200, 1100, 1000] },
  { _id: 2, name: "Phone", prices: [800, 750, 700] }
])

// Get the second price
db.products.aggregate([
  {
    $project: {
      name: 1,
      secondPrice: { $arrayElemAt: ["$prices", 1] }
    }
  }
])

// Result
[
  { _id: 1, name: "Laptop", secondPrice: 1100 },
  { _id: 2, name: "Phone", secondPrice: 750 }
]

$in Operator

The $in operator checks if a value exists in an array.

// Sample data
db.users.insertMany([
  { _id: 1, name: "Alice", roles: ["admin", "editor"] },
  { _id: 2, name: "Bob", roles: ["editor"] },
  { _id: 3, name: "Charlie", roles: ["viewer"] }
])

// Find users with the "admin" role
db.users.find({
  roles: { $in: ["admin"] }
})

// Result
[
  { _id: 1, name: "Alice", roles: ["admin", "editor"] }
]

$all Operator

The $all operator checks if an array contains all specified elements.

// Sample data
db.courses.insertMany([
  { _id: 1, title: "Math", tags: ["algebra", "geometry", "calculus"] },
  { _id: 2, title: "Physics", tags: ["mechanics", "optics"] },
  { _id: 3, title: "Chemistry", tags: ["organic", "inorganic", "physical"] }
])

// Find courses with both "algebra" and "geometry" tags
db.courses.find({
  tags: { $all: ["algebra", "geometry"] }
})

// Result
[
  { _id: 1, title: "Math", tags: ["algebra", "geometry", "calculus"] }
]

Array Update Operators

MongoDB provides a series of array update operators for modifying array fields:

$push Operator

The $push operator adds an element to an array.

// Add one element to the array
db.students.updateOne(
  { _id: 1 },
  { $push: { scores: 85 } }
)

// Add multiple elements to the array
db.students.updateOne(
  { _id: 1 },
  { $push: { scores: { $each: [90, 92] } } }
)

$addToSet Operator

The $addToSet operator adds an element to an array only if it does not already exist.

db.students.updateOne(
  { _id: 1 },
  { $addToSet: { scores: 85 } } // Only adds 85 if it doesn't exist
)

$pop Operator

The $pop operator removes the first or last element from an array.

// Remove the last element
db.students.updateOne(
  { _id: 1 },
  { $pop: { scores: 1 } }
)

// Remove the first element
db.students.updateOne(
  { _id: 1 },
  { $pop: { scores: -1 } }
)

$pull Operator

The $pull operator removes all elements from an array that match the specified condition.

db.students.updateOne(
  { _id: 1 },
  { $pull: { scores: { $gte: 90 } } } // Remove all scores greater than or equal to 90
)

Array Query Operators

MongoDB provides some operators specifically for querying arrays:

$elemMatch Operator

The $elemMatch operator queries documents where at least one array element satisfies all specified conditions.

// Sample data
db.students.insertMany([
  { _id: 1, name: "Alice", scores: [ { math: 80 }, { math: 90 } ] },
  { _id: 2, name: "Bob", scores: [ { math: 70 }, { math: 85 } ] }
])

// Find students with a math score greater than 85
db.students.find({
  scores: { $elemMatch: { math: { $gt: 85 } } }
})

// Result
[
  { _id: 1, name: "Alice", scores: [ { math: 80 }, { math: 90 } ] }
]

Array Index Query

You can directly query using array indices:

// Find students whose first math score is greater than 85
db.students.find({
  "scores.0.math": { $gt: 85 }
})

Array Aggregation Operators

In aggregation pipelines, there are other useful array operators:

$first and $last Operators

The $first and $last operators return the first or last element of an array.

db.students.aggregate([
  {
    $project: {
      name: 1,
      firstScore: { $first: "$scores" },
      lastScore: { $last: "$scores" }
    }
  }
])

$isArray Operator

The $isArray operator checks if a value is an array.

db.students.aggregate([
  {
    $project: {
      name: 1,
      isScoresArray: { $isArray: "$scores" }
    }
  }
])

Combining Array Operators

In practical applications, multiple array operators are often combined to achieve complex data processing.

// Example: Calculate the average of each student's highest score
db.students.aggregate([
  {
    $project: {
      name: 1,
      maxScore: {
        $reduce: {
          input: "$scores",
          initialValue: 0,
          in: { $max: ["$$value", "$$this.math"] }
        }
      }
    }
  },
  {
    $group: {
      _id: null,
      avgMaxScore: { $avg: "$maxScore" }
    }
  }
])

Performance Considerations

When working with large arrays, consider the following performance issues:

1. Index usage: You can create indexes for array fields, but be aware of the limitations of multikey indexes.
2. Array size: Large arrays can impact query performance.
3. Operation complexity: Some array operations have higher time complexity.

// Create an index for an array field
db.students.createIndex({ "scores.math": 1 })