A rule that catches 99% of attacks while erring on only 1% of normal events is routinely presented as a strong control during architecture reviews. The claim is rarely questioned because the numbers appear solid. Yet the arithmetic of rare events shows that almost every alert the rule generates will still be false—not because the rule is poorly written, but because genuine attacks are statistically rare.
The Base-Rate Problem in Numbers
Consider an organization that processes 200,000 logins per day, two of which represent real compromises. A rule advertised with 99% sensitivity and 1% false-positive rate produces:
- True positives: 2 × 0.99 = 1.98
- False positives: 199,998 × 0.01 = 2,000
- Total alerts: 2,002
- Precision: 0.10%
An analyst therefore faces roughly one thousand alerts for every genuine finding. The imbalance arises because the 1% error applied to the large population of normal events overwhelms the 99% hit rate applied to the tiny population of attacks.
Bayes' Theorem Formalizes the Outcome
Precision is expressed as:
Precision = (P × Se) / (P × Se + (1 − P) × FPR)
where P is the base rate of attacks, Se is sensitivity, and FPR is the false-positive rate on benign events. Because the numerator cannot exceed P, even perfect sensitivity cannot overcome a large denominator term driven by (1 − P) × FPR when attacks are rare.
What Actually Improves Precision
Raising sensitivity from 99% to 100% adds only 0.02 true positives while leaving 2,000 false positives untouched; precision remains 0.10%. Reducing the false-positive rate, however, produces immediate results:
- 1.00% FPR → 2,002 alerts, 0.10% precision
- 0.10% FPR → 202 alerts, 0.98% precision
- 0.01% FPR → 22 alerts, 9.01% precision
Lowering the false-positive rate by two orders of magnitude reduces daily workload from thousands of events to a manageable two dozen while preserving nearly all true detections.
Population Scoping and Cascaded Rules
Applying the same rule only to the 4,000 administrative and service-account logins (one of the two compromises) raises precision to 2.42% and cuts alerts to 41. The rule itself is unchanged; only its scope has been narrowed. A two-stage cascade achieves similar gains: a cheap broad filter followed by an expensive enrichment step on the remaining candidates can drop 98% of false positives while losing only 5% of true positives, yielding 94% overall sensitivity at 4.5% precision.
Operational Consequences and Recommended Metrics
Rules with 0.1% precision train analysts to close alerts without examination. Over time the rule is moved to low priority, then to reporting only, and finally disabled—coverage metrics still look perfect while actual detection has vanished. The article therefore advocates tracking two figures together: the absolute number of confirmed detections and the percentage of alerts that prove true. Reviewing quarterly tickets against these paired metrics quickly identifies rules that consume analyst attention without delivering value.
