Comparison between synthetic surveillance and real surveillance

Definitions of Synthetic Monitoring and Real User Monitoring

Synthetic Monitoring is an active monitoring method triggered by simulating user behavior or predefined scripts in a specific environment. For example, using automation tools to periodically access key pages and record performance metrics:

// Puppeteer example: Simulating page load and collecting metrics
const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  
  // Start performance tracing
  await page.tracing.start({path: 'trace.json'});
  await page.goto('https://example.com');
  await page.tracing.stop();
  
  const metrics = await page.evaluate(() => {
    return {
      FCP: performance.getEntriesByName('first-contentful-paint')[0].startTime,
      LCP: performance.getEntriesByName('largest-contentful-paint')[0].renderTime
    };
  });
  
  console.log(metrics);
  await browser.close();
})();

Real User Monitoring (RUM) collects performance data from actual user visits. Typically implemented by injecting scripts via browser APIs:

// Using Performance API to collect real user metrics
window.addEventListener('load', () => {
  const perfData = {
    navigationTiming: performance.timing,
    paintMetrics: {
      fp: performance.getEntriesByName('first-paint')[0].startTime,
      fcp: performance.getEntriesByName('first-contentful-paint')[0].startTime
    },
    resourceTiming: performance.getEntriesByType('resource')
  };
  
  // Send data to analytics server
  navigator.sendBeacon('/analytics', JSON.stringify(perfData));
});

Differences in Data Collection Methods

Synthetic monitoring is typically triggered in the following scenarios:

• Scheduled tasks (e.g., checking homepage load every 15 minutes)
• Automated tests in deployment pipelines
• Geographically distributed probe nodes

Real user monitoring data characteristics include:

1. Device fragmentation data (different CPU/GPU capabilities)
2. Network condition fluctuations (4G weak signal/WiFi jitter)
3. User interaction timing variations (scroll/click timing)

Typical data collection code structure comparison:

// Complete test scenario for synthetic monitoring
describe('Checkout Flow', () => {
  it('should load under 2s', async () => {
    const start = Date.now();
    await page.goto('/checkout');
    expect(Date.now() - start).toBeLessThan(2000);
  });
});

// Error capture for real user monitoring
window.addEventListener('error', (event) => {
  trackError({
    message: event.message,
    stack: event.error.stack,
    userAgent: navigator.userAgent
  });
});

Performance Metric Comparison Dimensions

Loading Performance Metric Differences

Synthetic monitoring typically reports:

• DNS lookup time (stable in controlled environments)
• TCP connection time (optimized lab network)
• Full load time (no background process interference)

Real user data shows:

• 90th percentile LCP may be 3-5 times higher than synthetic data
• Median FCP on mobile devices is usually worse than desktop
• Long tasks caused by third-party scripts are more common

// Long task monitoring for real users
const observer = new PerformanceObserver((list) => {
  list.getEntries().forEach(entry => {
    if (entry.duration > 50) {
      console.warn('Long task detected:', entry);
    }
  });
});
observer.observe({entryTypes: ['longtask']});

Interaction Metric Comparison

Synthetic monitoring can precisely measure:

• Response delay after programmatic clicks
• Virtual scroll frame rate
• Form submission API latency

Real monitoring captures:

• Input delay due to user device lag
• Animation frame drops on low-end phones
• Delayed JS execution in background tabs

// Example of interaction test for synthetic monitoring
await page.click('#submit-btn');
const apiLatency = await page.evaluate(() => {
  return window.performance.getEntriesByType('resource')
    .find(r => r.name.includes('/api/submit')).duration;
});

// Input delay monitoring for real users
const input = document.querySelector('input');
input.addEventListener('input', (e) => {
  const lag = Date.now() - e.timeStamp;
  if (lag > 100) reportInputLag(lag);
});

Complementary Application Scenarios

Advantages of Synthetic Monitoring

1. Benchmark testing before new feature releases

// Pre-release environment performance check
benchmark('Search Results', async () => {
  await page.type('#search', 'test');
  await page.waitForSelector('.result-item');
  return getPaintMetrics();
});

2. Infrastructure change validation (e.g., CDN switch)
3. Geographic performance benchmarks (via global probe nodes)

Irreplaceable Scenarios for Real Monitoring

1. Discovering specific device/browser issue combinations

// Collecting user environment fingerprints
const hardwareConcurrency = navigator.hardwareConcurrency || 'unknown';
const deviceMemory = navigator.deviceMemory || 'unknown';

2. Detecting third-party service performance degradation
3. Identifying actual bottleneck steps in user flows

Challenges in Data Accuracy

Limitations of Synthetic Monitoring

• Cannot simulate real CPU throttling (even with --cpu-throttling parameter)
• Cache behavior differs from production environments
• Does not trigger real user browser extension interference

// Lab-simulated extension interference case
window.chrome.runtime.sendMessage = () => {
  // Some ad blockers rewrite this API
  return new Promise(resolve => setTimeout(resolve, 200));
};

Data Noise in Real Monitoring

• Need to handle extreme outliers (data collected when users minimize browsers)
• Distinguish meaningful performance degradation from user behavior variations
• Sampling strategy affects data representativeness

// Example of filtering invalid data
function isValidMeasurement(timing) {
  return timing.loadEventEnd > 0 && 
    timing.navigationStart < timing.fetchStart &&
    timing.domComplete > timing.domLoading;
}

Implementation Cost Comparison

Synthetic Monitoring Infrastructure

1. Requires maintaining test script version control

# Example of versioned monitoring script deployment
version: 2023-08
scripts:
  - path: /checkout
    thresholds:
      LCP: 2500ms
      CLS: 0.1

2. Costs for global probe nodes (AWS CloudWatch Synthetics ~$0.0012/run)
3. Test data storage and analysis pipelines

Ongoing Costs of Real Monitoring

• Data cleaning ETL processes (handling TB-scale logs)
• User privacy compliance costs (GDPR data anonymization)
• False positive handling in real-time alert systems

// Example of data anonymization
function anonymize(data) {
  return {
    ...data,
    ip: data.ip.substring(0, data.ip.lastIndexOf('.') + 1) + '0',
    userId: hash(data.userId + 'salt')
  };
}

In-Depth Technical Implementation Comparison

Advanced Techniques for Synthetic Monitoring

1. Using WebPageTest private instances for deep analysis

// WebPageTest script example
[settings]
timeout = 60
firstViewOnly = true

[script]
navigate https://example.com
execAndWait document.querySelector('#login').click()

2. Combining Lighthouse CI for regression detection
3. Multi-step transaction monitoring (e.g., checkout flow)

Fine Control in Real Monitoring

1. Sampling strategies based on user groups

// Sampling by user group
const sampleRate = user.isPremium ? 1.0 : 0.1;
if (Math.random() < sampleRate) {
  sendPerformanceData();
}

2. Correlation with key business metrics (e.g., conversion rate vs. TTI)
3. Association analysis between frontend errors and performance data

Toolchain Ecosystem

Main synthetic monitoring solutions:

• Commercial: Catchpoint, Dynatrace Synthetic
• OSS: Sitespeed.io, Artillery
• Cloud: AWS Synthetics, Google Cloud Monitoring

Typical real monitoring solutions:

• Full-stack: New Relic, Datadog RUM
• Specialized: SpeedCurve, LogRocket
• Self-hosted: Elastic APM, Sentry Performance

Example hybrid deployment architecture:

User Browser --[Beacon]--> RUM Collector --[Kafka]--> 
                           ↓
Real-time Alert System     Data Warehouse
                           ↑
Synthetic Nodes --[API]--> Unified Analysis Layer

Different Focuses in Data Visualization

Synthetic monitoring dashboards typically display:

• Historical trend comparisons (before/after version releases)
• Geographic heatmaps
• SLA compliance rates (e.g., "98% of tests <2s load")

Real monitoring analysis requires:

• Percentile distribution histograms (P75/P90/P99)
• Device/browser matrix views
• User journey waterfall flows (identifying critical path blockages)

// Example of percentile calculation
function calculatePercentiles(data, percentiles) {
  data.sort((a,b) => a - b);
  return percentiles.map(p => {
    const index = Math.ceil(data.length * p) - 1;
    return data[Math.max(0, index)];
  });
}

Differences in Anomaly Detection Mechanisms

Synthetic monitoring alert strategies:

• Fixed thresholds (e.g., API response >500ms)
• Standard deviation detection based on historical data
• Multi-location consistency checks (3+ node anomalies)

Real monitoring requires:

• Dynamic baselines (weekday/weekend pattern recognition)
• User group comparative analysis (new vs. returning users)
• Detection of slow-growing degradations

// Pseudocode for dynamic baseline algorithm
function detectAnomaly(current, historical) {
  const hourOfDay = new Date().getHours();
  const similarPeriods = historical.filter(d => 
    d.hour === hourOfDay && d.dayType === 'weekday');
  const avg = calculateAverage(similarPeriods);
  return current > avg * 1.5;
}

Performance Optimization Feedback Loop

Synthetic monitoring-driven optimizations:

1. Identifying blocking points in resource loading chains

# WebPageTest before/after optimization comparison
Before: document.write blocking parsing 1200ms
After: Async script loading → 300ms reduction in blocking

2. Validating CDN configuration effects through repeated tests
3. A/B testing of preload strategies

Real monitoring-guided improvements:

• Progressive enhancement deployment by user group

<!-- Fallback for low-end devices -->
<script>
if (navigator.deviceMemory < 2) {
  document.write('<link rel="stylesheet" href="lite.css">');
}
</script>

• Retry mechanism optimization in real user paths
• Resource adjustment based on actual network conditions

Long-Term Value of Monitoring Data

Synthetic monitoring data is used for:

• Infrastructure selection decisions (comparing CDN vendors)
• Risk assessment for framework upgrades (e.g., React 16→18 migration)
• Capacity planning (stress test benchmarks)

Real monitoring data helps:

• Feature prioritization (optimizing high-frequency paths)
• Technical debt quantification (usage rates of poorly performing legacy code)
• Business impact analysis (speed vs. conversion rate correlation models)

// Example of business impact analysis
function analyzeImpact(perfData, revenueData) {
  return perfData.map(user => {
    const rev = revenueData.find(u => u.id === user.id);
    return {
      loadTime: user.LCP,
      conversion: rev ? 1 : 0
    };
  });
}

Impact of Emerging Technologies

New developments in synthetic monitoring:

• Using Playwright for more realistic interaction simulation

// Playwright multi-context testing
const { chromium } = require('playwright');
const browser = await chromium.launch();
const context = await browser.newContext({
  userAgent: 'Mozilla/5.0 (iPhone; CPU iPhone OS 13_2 like Mac OS X)'
});

• Computer vision-based content change detection
• Containerized global probe node deployment

Frontier directions in real monitoring:

• Using Web Vitals API's INP metric
• WebAssembly-based high-performance data preprocessing
• Deep integration with OpenTelemetry standards

// OpenTelemetry frontend integration example
const { WebTracerProvider } = require('@opentelemetry/web');
const provider = new WebTracerProvider();
provider.register();

const tracer = provider.getTracer('rum');
tracer.startSpan('checkout-flow').end();