A risk score is an argument you are not allowed to hear
Why Aegis shows you a rule, its evidence and its query instead of a number between 0 and 100 — and the two places where we admit a score would be more convenient.
Almost every security product you can buy will hand you a number. Risk: 87. Severity: Critical (9.1). Confidence: High. The number is doing a job — it is compressing a chain of reasoning into something sortable — and I understand entirely why it exists.
We do not ship one. Here is the argument, and then the two cases where I think we are wrong.
What the number is hiding
A risk score is the output of a function. Somewhere there is a weighting: this signal is worth 30, that asset tag multiplies by 1.4, this identity is privileged so add 20, the model’s anomaly likelihood contributes up to 25. The function is real, it is knowable, and it is almost never shown to you.
That matters because you cannot argue with a number. When your platform team tells you the 87 is wrong — that the workload is a load-test harness, that the identity is a break-glass account that is supposed to do that once a quarter — you have no surface to correct. You can suppress the alert. You cannot fix the reasoning. So the suppression list grows, and the suppression list is where the reasoning actually lives, undocumented, in a UI.
What we show instead
A situation in Aegis has four things and no score:
The pattern. A named correlation pattern with its conditions listed, which of them matched, and which did not. It is a YAML file, it lives in your Git repository, and the console links to the exact line.
The evidence. The normalised events that matched, in full, not summarised. If the pattern matched on “source ASN not seen in 90 days”, you get the ASN, the 90-day history that justifies “not seen”, and the count of observations backing it.
The query. The literal query that produced the evidence, in a box, copyable. Run it yourself, change the window, change the threshold, see what else it catches. This is the one people use most and the one that took us longest to build, because it required the detection engine and the ad-hoc query engine to be the same engine.
The neighbours. What else this identity, workload or account did in the surrounding window — not filtered by relevance, because relevance is the thing under dispute.
Sorting without a score
The obvious objection: with 26 open situations, how do I know which to open first?
We sort by three orthogonal facts rather than one blended number:
- Reach — how many distinct workloads, identities and data stores the involved principal can touch. This is computed from your actual IAM graph, and it is a count, not a judgement.
- Novelty — how many times this pattern has matched for you before, and how it was resolved each time. A pattern you have closed as benign eleven times sorts down.
- Motion — whether the underlying activity is still happening. A situation where events are still arriving sorts above one that stopped four hours ago.
Three numbers you can each interrogate beat one number you cannot. Reach is the one customers find most useful and the one no scoring model I have seen exposes separately, because it is expensive to compute and boring to demo.
Where I think we are wrong
Executive reporting. A CISO presenting to a board genuinely needs a compressed number, and “we had 26 situations, sorted by reach, novelty and motion” is not it. Our customers build this number themselves, in their BI tool, from our export. That is a worse experience than us providing it, and the reason we do not is closer to squeamishness than principle.
The first week. A new customer with no baselines and no resolution history has weak novelty signal and no tuning. In week one, a rough severity heuristic really would help them triage, and our answer — run in observe-only for a fortnight — asks for patience that a team in the middle of an incident does not have.
We have not solved either. If we ship a score, it will be one you can open, edit and put in your own repository, or it will not be worth having.
The short version
Show the argument. If the argument is too long to show, the argument is probably not very good. And if a vendor cannot show you the query behind an alert, ask whether that is because it would be embarrassing.