Anomaly detection visualization

Core Requirements for Anomaly Detection Visualization

Anomaly detection visualization needs to intuitively display outliers, abnormal patterns, or sudden trends in data. ECharts, as a powerful visualization library, can clearly present data anomalies through its rich chart types and interactive features. The key lies in selecting appropriate visual encoding methods to create a sharp contrast between abnormal and normal data.

Basic Anomaly Detection Chart Types

Line Charts and Area Charts

Line charts are suitable for showing abnormal fluctuations in time-series data. By setting threshold lines or marking abnormal points, data exceeding normal ranges can be quickly identified.

option = {
  xAxis: {
    type: 'category',
    data: ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun']
  },
  yAxis: {
    type: 'value'
  },
  series: [{
    data: [820, 932, 901, 934, 1290, 1330, 1320],
    type: 'line',
    markPoint: {
      data: [
        { type: 'max', name: 'Max' },
        { type: 'min', name: 'Min' }
      ]
    },
    markArea: {
      data: [[
        { yAxis: 800 },
        { yAxis: 950 }
      ]]
    }
  }]
};

Scatter Plots and Bubble Charts

Scatter plots effectively display outliers in multidimensional data by using different colors or sizes to distinguish abnormal values.

option = {
  xAxis: {},
  yAxis: {},
  series: [{
    symbolSize: 20,
    data: [
      [10.0, 8.04],
      [8.0, 6.95],
      [13.0, 7.58],
      [9.0, 8.81],
      [11.0, 8.33],
      [14.0, 9.96],
      [6.0, 7.24],
      [4.0, 4.26],
      [12.0, 10.84],
      [7.0, 4.82],
      [5.0, 5.68],
      [30.0, 15.0]  // Outlier
    ],
    type: 'scatter',
    itemStyle: {
      color: function(params) {
        return params.data[0] > 20 ? '#ff0000' : '#5470c6';
      }
    }
  }]
};

Advanced Anomaly Detection Visualization Techniques

Box Plots and Outlier Display

Box plots are a standard method for displaying data distribution and outliers, implemented in ECharts through the boxplot series.

option = {
  dataset: [{
    source: [
      [850, 740, 900, 1070, 930, 850, 950],
      [960, 940, 960, 940, 880, 800, 850],
      [880, 800, 840, 750, 750, 700, 720],
      [1150, 990, 1000, 1020, 930, 980, 990],
      [890, 810, 810, 820, 800, 770, 750],
      [890, 840, 780, 810, 760, 810, 790],
      [1000, 950, 960, 1000, 1100, 1050, 1000],
      [920, 910, 905, 925, 955, 960, 950]
    ]
  }, {
    transform: {
      type: 'boxplot',
      config: { itemNameFormatter: 'expr {value}' }
    }
  }],
  xAxis: { type: 'category' },
  yAxis: { type: 'value' },
  series: [{
    name: 'boxplot',
    type: 'boxplot',
    datasetIndex: 1,
    itemStyle: {
      color: '#b8c5f2',
      borderColor: '#5470c6'
    },
    emphasis: {
      itemStyle: {
        color: '#5470c6',
        borderColor: '#333'
      }
    }
  }]
};

Heatmaps and Matrix Plots

Heatmaps are suitable for displaying abnormal patterns in high-dimensional data, highlighting abnormal areas through color variations.

option = {
  tooltip: {
    position: 'top'
  },
  grid: {
    height: '50%',
    top: '10%'
  },
  xAxis: {
    type: 'category',
    data: ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun'],
    splitArea: {
      show: true
    }
  },
  yAxis: {
    type: 'category',
    data: ['Morning', 'Afternoon', 'Evening', 'Night'],
    splitArea: {
      show: true
    }
  },
  visualMap: {
    min: 0,
    max: 10,
    calculable: true,
    orient: 'horizontal',
    left: 'center',
    bottom: '15%'
  },
  series: [{
    name: 'Anomaly Frequency',
    type: 'heatmap',
    data: [
      [0, 0, 5], [1, 0, 7], [2, 0, 3], [3, 0, 5], [4, 0, 2], [5, 0, 1], [6, 0, 0],
      [0, 1, 1], [1, 1, 2], [2, 1, 4], [3, 1, 8], [4, 1, 1], [5, 1, 0], [6, 1, 0],
      [0, 2, 0], [1, 2, 1], [2, 2, 2], [3, 2, 9], [4, 2, 1], [5, 2, 0], [6, 2, 0],
      [0, 3, 0], [1, 3, 1], [2, 3, 1], [3, 3, 10], [4, 3, 2], [5, 3, 0], [6, 3, 0]
    ],
    label: {
      show: true
    },
    emphasis: {
      itemStyle: {
        shadowBlur: 10,
        shadowColor: 'rgba(0, 0, 0, 0.5)'
      }
    }
  }]
};

Interactive Anomaly Detection Features

Data Brushing and Linking

Use the brush component to enable interactive selection of abnormal data, linking with other charts for analysis.

option = {
  brush: {
    toolbox: ['rect', 'polygon', 'keep', 'clear'],
    xAxisIndex: 0
  },
  toolbox: {
    feature: {
      brush: {
        type: ['rect', 'polygon', 'clear']
      }
    }
  },
  xAxis: {
    data: ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun']
  },
  yAxis: {},
  series: [{
    type: 'bar',
    data: [120, 200, 150, 80, 70, 110, 130],
    itemStyle: {
      color: function(params) {
        var colorList = ['#c23531','#2f4554','#61a0a8','#d48265','#91c7ae','#749f83','#ca8622'];
        return params.data > 150 ? '#c23531' : colorList[params.dataIndex];
      }
    }
  }]
};

Dynamic Thresholds and Adaptive Detection

Combine ECharts' data processing capabilities to implement dynamic threshold calculation and visualization.

function calculateThreshold(data) {
  const mean = data.reduce((a, b) => a + b) / data.length;
  const std = Math.sqrt(data.reduce((sq, n) => sq + Math.pow(n - mean, 2), 0) / data.length);
  return [mean - 2*std, mean + 2*std];
}

const data = [23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42];
const [lower, upper] = calculateThreshold(data);

option = {
  xAxis: {
    type: 'category',
    data: Array.from({length: data.length}, (_, i) => `Day ${i+1}`)
  },
  yAxis: {
    type: 'value'
  },
  series: [{
    data: data,
    type: 'line',
    markArea: {
      silent: true,
      data: [[
        { yAxis: lower },
        { yAxis: upper }
      ]],
      itemStyle: {
        color: 'rgba(135, 206, 235, 0.2)'
      }
    },
    markPoint: {
      data: data.map((value, index) => {
        return (value < lower || value > upper) ? {
          coord: [`Day ${index+1}`, value],
          symbol: 'circle',
          symbolSize: 10,
          itemStyle: {
            color: '#ff0000'
          }
        } : null;
      }).filter(Boolean)
    }
  }]
};

Multidimensional Anomaly Detection Visualization

Parallel Coordinates

Parallel coordinates are suitable for displaying abnormal patterns in high-dimensional data, highlighting abnormal records through line colors.

option = {
  parallelAxis: [
    {dim: 0, name: 'CPU Usage'},
    {dim: 1, name: 'Memory Usage'},
    {dim: 2, name: 'Disk IO'},
    {dim: 3, name: 'Network Traffic'},
    {dim: 4, name: 'Response Time'}
  ],
  series: {
    type: 'parallel',
    lineStyle: {
      width: 1,
      opacity: 0.5
    },
    data: [
      [80, 70, 65, 50, 120],
      [85, 75, 70, 55, 130],
      [90, 80, 75, 60, 140],
      [95, 85, 80, 65, 150],
      [30, 20, 15, 10, 200],  // Abnormal record
      [75, 65, 60, 45, 110],
      [82, 72, 67, 52, 125],
      [88, 78, 73, 58, 135],
      [92, 82, 77, 62, 145],
      [25, 15, 10, 5, 210]   // Abnormal record
    ],
    itemStyle: function(params) {
      const data = params.data;
      return {
        color: data[4] > 180 ? '#ff0000' : '#5470c6'
      };
    }
  }
};

Radar Chart for Anomaly Pattern Recognition

Radar charts are suitable for displaying periodic data or abnormal patterns across multiple metrics.

option = {
  radar: {
    indicator: [
      { name: 'CPU', max: 100 },
      { name: 'Memory', max: 100 },
      { name: 'Disk', max: 100 },
      { name: 'Network', max: 100 },
      { name: 'Response', max: 200 }
    ],
    radius: '65%'
  },
  series: [{
    type: 'radar',
    data: [
      {
        value: [80, 70, 65, 50, 120],
        name: 'Normal Pattern',
        areaStyle: {
          color: 'rgba(84, 112, 198, 0.5)'
        }
      },
      {
        value: [95, 85, 80, 65, 180],
        name: 'Abnormal Pattern',
        areaStyle: {
          color: 'rgba(255, 0, 0, 0.5)'
        }
      }
    ]
  }]
};

Real-Time Anomaly Detection Visualization

Dynamic Data Updates

For real-time data streams, charts need to be dynamically updated to reflect the latest anomalies.

let data = [];
let now = new Date();

for (let i = 0; i < 100; i++) {
  data.push(randomData());
}

function randomData() {
  now = new Date(+now + 1000);
  const value = Math.random() * 100;
  return {
    name: now.toString(),
    value: [
      now.toLocaleTimeString(),
      value > 90 ? value + 50 : value  // Simulate outlier
    ]
  };
}

option = {
  xAxis: {
    type: 'category',
    boundaryGap: false
  },
  yAxis: {
    type: 'value',
    boundaryGap: [0, '10%']
  },
  series: [{
    type: 'line',
    showSymbol: false,
    data: data,
    markPoint: {
      data: [],
      symbol: 'pin',
      symbolSize: 50,
      label: {
        formatter: function(params) {
          return params.value > 140 ? 'Anomaly' : '';
        }
      }
    }
  }]
};

setInterval(function() {
  data.shift();
  const newData = randomData();
  data.push(newData);
  
  if (newData.value[1] > 140) {
    option.series[0].markPoint.data.push({
      coord: [newData.value[0], newData.value[1]],
      itemStyle: { color: '#ff0000' }
    });
  }
  
  myChart.setOption({
    series: [{
      data: data
    }]
  });
}, 1000);

Anomaly Event Marking and Tooltips

Mark anomaly events in time series and provide detailed tooltips.

option = {
  xAxis: {
    type: 'category',
    data: ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun']
  },
  yAxis: {
    type: 'value'
  },
  series: [{
    data: [820, 932, 901, 934, 1290, 1330, 1320],
    type: 'line',
    markPoint: {
      data: [
        {
          coord: ['Fri', 1290],
          value: 'Abnormal Peak',
          symbol: 'pin',
          symbolSize: 50,
          itemStyle: {
            color: '#ff0000'
          },
          label: {
            formatter: 'Anomaly Event',
            position: 'top'
          }
        },
        {
          coord: ['Sat', 1330],
          value: 'Highest Value',
          symbol: 'pin',
          symbolSize: 50,
          itemStyle: {
            color: '#ff0000'
          },
          label: {
            formatter: 'Severe Anomaly',
            position: 'top'
          }
        }
      ]
    },
    markLine: {
      data: [
        {
          type: 'average',
          name: 'Average'
        },
        {
          yAxis: 1100,
          name: 'Threshold Line'
        }
      ]
    }
  }],
  tooltip: {
    trigger: 'axis',
    formatter: function(params) {
      const data = params[0].data;
      let tip = `${params[0].axisValue}<br/>`;
      tip += `${params[0].seriesName}: ${data}`;
      
      if (data > 1100) {
        tip += '<br/><span style="color:#ff0000">Warning: Threshold Exceeded</span>';
      }
      
      return tip;
    }
  }
};

Performance Optimization for Anomaly Detection Visualization

Rendering Strategies for Large Data Volumes

When dealing with large-scale datasets, downsampling or incremental rendering techniques are required.

// Downsampling function
function downsample(data, factor) {
  const result = [];
  for (let i = 0; i < data.length; i += factor) {
    const chunk = data.slice(i, i + factor);
    const avg = chunk.reduce((sum, val) => sum + val, 0) / chunk.length;
    result.push(avg);
  }
  return result;
}

// Raw data
const rawData = Array.from({length: 100000}, () => Math.random() * 100);

// Add outliers
for (let i = 0; i < 10; i++) {
  const index = Math.floor(Math.random() * rawData.length);
  rawData[index] = 150 + Math.random() * 50;
}

// Downsampling
const downsampledData = downsample(rawData, 100);

option = {
  xAxis: {
    type: 'category',
    data: Array.from({length: downsampledData.length}, (_, i) => i)
  },
  yAxis: {
    type: 'value'
  },
  series: [{
    data: downsampledData,
    type: 'line',
    symbol: 'none',
    lineStyle: {
      width: 1
    },
    markPoint: {
      data: downsampledData
        .map((value, index) => (value > 140) ? {coord: [index, value]} : null)
        .filter(Boolean),
      symbol: 'circle',
      symbolSize: 5,
      itemStyle: {
        color: '#ff0000'
      }
    }
  }]
};

WebGL Accelerated Rendering

For extremely large datasets, ECharts' WebGL extension can be used for GPU acceleration.

// Requires echarts-gl
const data = new Float32Array(1000000);
for (let i = 0; i < data.length; i++) {
  data[i] = Math.random() * 100;
  if (Math.random() < 0.001) {
    data[i] = 150 + Math.random() * 50; // Add outliers
  }
}

option = {
  xAxis3D: {
    type: 'value'
  },
  yAxis3D: {
    type: 'value'
  },
  zAxis3D