← All projects Detection engineering

An MFA fatigue detection, and the test data I guessed wrong

Sigma · Elastic Security · Okta Verify · GitHub Actions

What this demonstrates: Detection engineering end to end: Sigma logic, an Elastic threshold rule, test cases that run in CI, and checking test data against real telemetry before turning a rule on.

Homelab project: a fake company, real systems, my own test traffic.

In short: I wrote a rule for MFA push fatigue as code, with test cases that CI runs on every change. Before turning it on, I checked it against one real push. The event name for "push sent" was right. But a real denial is logged under a different event than the one my test data used, so I fixed the test data. The rule is in the repo and passes its tests. It isn't turned on yet.

Problem

An attacker who already has a password can send push prompt after push prompt, hoping the user taps Approve to make them stop. This is called MFA fatigue or MFA bombing (MITRE ATT&CK T1621). A single push is a normal sign-in. A burst of pushes to one person is the signal.

Design

A Sigma rule for one event: Okta sent an Okta Verify push (system.push.send_factor_verify_push). Sigma keeps the logic portable across SIEMs.
The count lives in the Elastic rule: 5 or more pushes for one user.name in one run. It runs every 5 minutes and looks back 6 minutes. My failed sign-in rule has the same shape, because the Sigma converters can't produce a Kibana threshold rule.
Counted by ingest time, not event time, so events that arrive late still count in the next run.
Triage notes in the rule: who was targeted, from where, whether pushes were denied, and above all whether a push was approved or a session started afterwards. If so, treat the password as known and reset it. A person approves that action.

Test cases

Each rule has made-up events with the expected result. A script runs every rule's cases on every pull request, alongside a Terraform scan.

Case	Expected
Five pushes in four minutes, four denied, the fifth approved (fatigue that worked)	fires
Four pushes in two minutes for one user	quiet
Two pushes, then approved (slow phone)	quiet
Five pushes to five users in two minutes (busy morning)	quiet
Five pushes to one user over 40 minutes (slow drip)	quiet: a known gap

The slow-drip case is there on purpose. It records what the rule misses, so a later change that claims to catch it has a test to pass.

What I got wrong

I wrote the rule and its test data from Okta's event type catalog, without checking them against my lab's data. So I marked the rule as a draft and shipped it turned off.

Before turning it on, I checked it against real data. The "push sent" event was right. My lab already held 19 real ones from my own sign-ins, all with a user.name to count by. Then I signed in and denied a push on my phone.

My test data and my triage notes expected the denial as user.mfa.okta_verify.deny_push. Okta logged nothing by that name. A denied push shows up as a failed MFA check with a reason code:

Before: the denial in my test datasynthetic event, guessed from the catalog

event.action:   user.mfa.okta_verify.deny_push
event.outcome:  failure
user.name:      user-a@example-company
okta.outcome.reason: Push rejected

After: the shape of a real denialsynthetic event, copied from the real one's fields

event.action:   user.authentication.auth_via_mfa
event.outcome:  failure
user.name:      user-a@example-company
okta.debug_context.debug_data.factor: OKTA_VERIFY_PUSH
okta.outcome.reason: INVALID_CREDENTIALS
# followed by a failed user.session.start with the same reason

The values are made up. The field names and codes match what Okta logged for my real denied push.

The rule's query only counts pushes sent, so detection wasn't affected. The risk was in triage: notes that point to the wrong denial event could lead an analyst to find no denials and mistake an attack for a slow phone. I fixed the triage notes and the test data, and all 16 cases across my rules still pass.

I'd already learned this once with an earlier check: test data has to copy real data, not my idea of it. Tests that pass on a guessed format only prove the guess.

How I tested it

The rule test runner: 5 cases for this rule (1 fires, 4 stay quiet), 16 across all rules, run in CI on every pull request.
The runner's own self-tests. I changed one that broke every time a rule was added, because it pinned the total number of cases. It still checks that a skipped case isn't counted as a pass.
Against real data: the 19 earlier push events matched the query's fields, and one live denial confirmed what a denial looks like.

Limits

The rule is turned off. The 5-push threshold is a starting value and hasn't fired on real data.
A slow drip, one push every 10 minutes, never reaches 5 in one window. A companion rule on denials would catch it.
I tested a denial that wasn't reported as suspicious. A reported one may log a different event.
My admin account uses number matching, where a push only works if I match the number shown on the sign-in screen. That defeats blind approval. The rule still matters for accounts or policies without it.