Catch the Anomaly
Monitoring is not the hard part. Deciding what counts as wrong is. Here is a live stream with three detectors reading it at once — push a spike into the feed, drag the threshold, and watch the same data turn into three different sets of alerts.
What you are looking at
Every point is scored three times
The z-score, the EWMA band and the isolation forest all run on every point as it lands. Switching method does not re-run anything it shows you the answer that detector had already reached. Hover any point to see all three at once.
Sensitivity is a dial, not a setting
Drag the threshold down and you catch more of what was injected, and start flagging ordinary noise with it. Drag it up and the false positives stop, along with the quieter real anomalies. The counters move together, which is the whole problem in miniature.
Nothing is running off screen
The stream, all three detectors and the scoring run on this page, in JavaScript. The isolation forest really does rebuild its 200 trees every 28 points. Reset replays the same series from the same seed, so you can put two methods against identical data.
Why they disagree
Each of the three is good at something the other two are bad at. The pattern break button is the quickest way to see it: inject one, then switch between the methods without resetting.
Rolling z-score
Mean and σ of the last 40 points; the new point is measured against them.
- Catches
- Sharp spikes and dips, immediately.
- Misses
- A sustained level shift it flags hard, then the window fills with the new level and it goes quiet. And it cannot see a flatline at all: a stuck sensor reads dead on the mean.
Moving average + σ band
An EWMA centre line with a σ band drawn around it.
- Catches
- Fast, cheap, and it re-centres on a genuine change in level within a few points.
- Misses
- That same speed is the weakness. It is dragged towards whatever it is being shown, so a slow drift walks it along and a long anomaly gets absorbed into normal.
Isolation forest
200 small random trees over three contextual features how far the point is from the local level, how far it moved, and how much the series has been moving. Anomalies need fewer splits to isolate.
- Catches
- Shape rather than size. It catches the flatline and the oscillation the other two argue about, because roughness near zero is as unusual as roughness far too high.
- Misses
- It is the noisiest of the three on the easy cases. At the default it flags roughly one ordinary point in twenty, against one in thirty for the z-score, and it is worse on the smooth sensor feed than on the noisy ones. Its score also has no natural units: 0.6 means nothing until you have watched a few hundred points go by.
A demonstration of the method, not a product. The same approach scoring a live stream against what it has just been doing, and firing only when it is worth someone’s attention is what we build into monitoring for payments, plant telemetry and infrastructure.
Talk to us about your own problem →