Precursor scenarios
Cerebro saves the state before each outage and alerts you when that scenario repeats, with the cause from the previous time.
Many outages give warning: memory climbs, errors show up in the logs, a dependency starts failing. Cerebro saves that scenario from each real outage and, when it sees it repeating, alerts you before the component goes down again and tells you what the cause was the previous time.
What it saves from each outage
For each incident that was a real outage of a component with Cerebro turned on, Cerebro saves:
The footprint: how its metrics, trends, log errors, notifications and previous incidents looked in the hours before the outage (2 hours by default; you can change it in Settings). Only what was out of the ordinary for that component counts.
The incident and its probable cause.
Linked items in that window: its dependencies from the inventory and their incidents, the Cerebro alerts on them, and the security detections on the server and its neighbors.
How many times it repeated: each time in the history the state looked similar, and whether or not it ended in an outage.
It builds this from all the history ARGUNIS has for the component (up to 13 months).
When it alerts
Every time it scores (every 15 minutes), Cerebro compares the current state of each component with its scenarios:
Similarity: from 0 to 100%. From 80% it can alert.
Weight: goes up with each repetition that ended in an outage and down with those that didn't, and with your verdicts on the alerts: Useful and I acted and prevented it add, False alarm subtracts.
If the scenario repeats with enough weight, the component's outage probability goes up and a predictive alert opens. Its first signal says, for example: “87% similar to the scenario before the outage of srv-pagos on 12-10 (INC-00321); ended in an outage 3 of 4 times. The probable cause that time: …”.
A scenario does not alert if there were no clear signals before that outage, or if it repeats very often without ending in an outage (more than the week's alert budget).
Like other predictive alerts, they don't open incidents: they are for acting ahead of time.
Where to see them
Cerebro → Scenarios: all the scenarios in your company. At the top, Similar today: those more than 50% similar to the current state. Below, All scenarios.
Component page in Cerebro: the Precursor scenarios section with that component's scenarios.
Incident page: Scenario before the outage, if Cerebro saved one.
Each scenario shows the component, the outage date and its incident, the probable cause, how many times it ended in an outage, its weight, whether it is similar today and, when you open Linked items, the dependencies, incidents, alerts and detections from that window.