Third-party monitoring tools (Prometheus, Grafana, Ops Manager)

Third-party Monitoring Tools (Prometheus, Grafana, Ops Manager)

MongoDB performance monitoring and operations management rely on the support of third-party tools. Prometheus, Grafana, and Ops Manager are three common solutions, each with distinct features and applicable scenarios.

Monitoring MongoDB with Prometheus

Prometheus is an open-source monitoring system particularly suited for monitoring time-series data. It retrieves metric data from target services via a pull mechanism and stores it in a local time-series database.

Configuring MongoDB Exporter

To enable Prometheus to monitor MongoDB, you need to deploy the MongoDB Exporter first. This is a specialized tool for exposing MongoDB metrics to Prometheus.

# Download and run MongoDB Exporter
wget https://github.com/percona/mongodb_exporter/releases/download/v0.30.0/mongodb_exporter-0.30.0.linux-amd64.tar.gz
tar -xzf mongodb_exporter-0.30.0.linux-amd64.tar.gz
./mongodb_exporter --mongodb.uri=mongodb://username:password@localhost:27017

Prometheus Configuration Example

Add the MongoDB Exporter as a scrape target in Prometheus's configuration file:

scrape_configs:
  - job_name: 'mongodb'
    static_configs:
      - targets: ['localhost:9216']  # Default port for MongoDB Exporter
    metrics_path: '/metrics'

Key Monitoring Metrics

Prometheus can monitor various key MongoDB metrics:

• Operation counters: inserts, queries, updates, deletes, etc.
• Connection counts: current connections, available connections
• Memory usage: resident memory, virtual memory
• Replica set status: node roles, replication lag

Visualizing MongoDB Metrics with Grafana

Grafana is an open-source visualization tool that integrates seamlessly with Prometheus to create rich monitoring dashboards.

Importing MongoDB Dashboards

The Grafana community provides several ready-made MongoDB dashboard templates:

1. Log in to Grafana
2. Click "+" > Import
3. Enter the dashboard ID (e.g., 2583)
4. Select the Prometheus data source

Custom Dashboard Example

Create a dashboard displaying MongoDB operation statistics:

// Query example
{
  "targets": [
    {
      "expr": "rate(mongodb_opcounters_insert_total[5m])",
      "legendFormat": "Inserts",
      "refId": "A"
    },
    {
      "expr": "rate(mongodb_opcounters_query_total[5m])",
      "legendFormat": "Queries",
      "refId": "B"
    }
  ],
  "title": "Operations Rate",
  "type": "timeseries"
}

Alert Configuration

Grafana can set alerts based on Prometheus metrics:

1. Create an alert rule
2. Set conditions (e.g., connection count exceeds a threshold)
3. Configure notification channels (Email, Slack, etc.)

MongoDB Ops Manager

Ops Manager is MongoDB's official enterprise-grade management platform, offering comprehensive monitoring, backup, and automation features.

Key Features

1. Performance Monitoring: Real-time monitoring of cluster status and performance metrics
2. Automation: Automated deployment, upgrades, and scaling of MongoDB
3. Backup and Recovery: Continuous backup and point-in-time recovery
4. Security: Centralized management of user permissions and audit logs

Deployment Architecture

A typical Ops Manager deployment includes the following components:

• Ops Manager application server
• MongoDB database (stores configuration and metadata)
• Backup Daemon (for backup operations)
• Monitoring Agent (installed on each monitored node)

Monitoring Agent Configuration

Install the monitoring agent on each MongoDB node:

# Download and install the agent
curl -OL https://downloads.mongodb.com/on-prem-mms/tar/monitoring-agent/latest/linux/mongodb-mms-monitoring-agent-latest.x86_64.tar.gz
tar -xzf mongodb-mms-monitoring-agent-latest.x86_64.tar.gz
cd mongodb-mms-monitoring-agent-*

# Configure the agent
echo "mmsGroupId=<your-group-id>" >> monitoring-agent.config
echo "mmsApiKey=<your-api-key>" >> monitoring-agent.config
echo "mmsBaseUrl=<Ops-Manager-URL>" >> monitoring-agent.config

# Start the agent
./mongodb-mms-monitoring-agent -conf=monitoring-agent.config

Comparison with Prometheus/Grafana

Advanced Monitoring Scenarios

Sharded Cluster Monitoring

For sharded clusters, pay special attention to the following metrics:

• Data distribution balance between shards
• Query router (mongos) performance
• Config server status

Prometheus query example:

# View data size differences between shards
topk(3, mongodb_shard_stats_dataSize_bytes)

# Monitor mongos connection count
sum(mongodb_network_numRequests) by (instance)

Slow Query Analysis

Combine MongoDB's profiler with monitoring tools to analyze slow queries:

1. Enable the profiler:

db.setProfilingLevel(1, { slowms: 100 })

2. Create a slow query dashboard in Grafana:

-- Use $group and $sort to aggregate slow queries
db.system.profile.aggregate([
  { $match: { millis: { $gt: 100 } } },
  { $group: { _id: "$command", count: { $sum: 1 }, avgMillis: { $avg: "$millis" } } },
  { $sort: { avgMillis: -1 } },
  { $limit: 10 }
])

Capacity Planning

Use historical monitoring data for capacity planning:

1. Collect key metric trends:

   • Data growth rate
   • Memory usage patterns
   • IOPS requirements

2. Create a prediction dashboard in Grafana:

# Predict disk space requirements
predict_linear(mongodb_db_stats_storageSize_bytes[7d], 86400 * 30)

Integration and Automation

CI/CD Pipeline Integration

Integrate monitoring with deployment processes:

# Jenkins pipeline example
pipeline {
  stages {
    stage('Deploy') {
      steps {
        sh 'ansible-playbook deploy-mongodb.yml'
      }
    }
    stage('Monitor') {
      steps {
        sh '''
          curl -X POST -H "Content-Type: application/json" \
          -d '{"name":"MongoDB Deployment","text":"New version deployed"}' \
          http://grafana/api/annotations
        '''
      }
    }
  }
}

Automated Remediation Scripts

Automated responses based on monitoring metrics:

# Automatically increase connection pool size when connection count is too high
import requests
from prometheus_api_client import PrometheusConnect

prom = PrometheusConnect(url="http://prometheus:9090")
current_conn = prom.get_current_metric_value('mongodb_connections_current')[0]['value'][1]

if float(current_conn) > 1000:
    requests.post(
        "http://opsmanager/api/public/v1.0/groups/{groupId}/automationConfig",
        json={"setParameter": {"maxIncomingConnections": 1500}}
    )

Security Monitoring Considerations

Audit Log Monitoring

Monitor critical security events:

# Prometheus rule example
groups:
- name: security.rules
  rules:
  - alert: UnauthorizedAccessAttempt
    expr: increase(mongodb_audit_events_total{type:"authenticate", success="false"}[1m]) > 5
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "Multiple failed authentication attempts detected"

Encryption Monitoring

Ensure TLS connections are functioning properly:

# Monitor encrypted connection ratio
sum(mongodb_network_bytesIn{encrypted="true"}) / sum(mongodb_network_bytesIn) * 100

Performance Optimization Reference

Index Usage Monitoring

Identify underutilized indexes:

// MongoDB shell query
db.collection.aggregate([
  { $indexStats: {} },
  { $project: { name: 1, accesses: { $sum: "$accesses.ops" } } },
  { $sort: { accesses: 1 } }
])

Memory Pressure Metrics

Identify memory pressure issues:

# Monitor page fault rate
rate(mongodb_extra_info_page_faults_total[5m])