From identity attack to investigated alert

Password guessing against a fake employee

One sign-in with a made-up outside address, then five wrong passwords for user-a, all from the same browser and address within 71 seconds. Okta logs each one as a failed user.session.start.

12:58:12  user.session.start  FAILURE  someone@outside.example
12:59     user.session.start  FAILURE  user-a@example-company   VERIFICATION_ERROR
  ...     (5 for user-a in about six seconds, the last at 12:59:23)

Limit: the target account was never activated, so this is guessing at an account that can't sign in. No account takeover was simulated.

The events reach the SIEM

Elastic Agent reads Okta's audit log every minute, signing in as its own read-only OAuth app. All six failures were in Elastic, with one source address between them. An hourly check matches every Okta event ID against Elastic, so a lost event would alert on its own.

logs-okta.system-*   user.session.start   failure   6
  user-a: 5     distinct source addresses: 1

The detection fires

The rule counts failed sign-ins per user: 5 or more in one run fires. It runs every 5 minutes, looks back 6 minutes, and counts by ingest time so late events still count. It fired once for user-a.

ALERT  Okta: repeated failed sign-ins for one user
       user.name: user-a@example-company   count: 5

The AI agent investigates

A timer checks for new alerts every 5 minutes and claims each alert once. The agent can call only read-only tools, at most 15 per run, with a token cap. Here it looked up the alert, the user's sign-ins, everything else from that address, the account and its groups.

tick: 1 open alerts in 24h, 1 not yet run, running 1
totals: 6 of 15 tool calls, 1 model request,
        9,515 of 150,000 tokens, 0 strings that looked like injection
posted to Slack (verdict)
exit=0 verdict

The "0 strings that looked like injection" is a pattern count for this one run's tool output. It shows nothing suspicious came back this time; it doesn't prove the agent resists injection. That's covered by tests that assume the model obeys injected text and check that code rejects what it asks for.

The risk score is computed in code, not by the model. The model's answer has to pass checks before it's posted: every piece of evidence must cite events the tools returned.

Personal data replaced before the model sees it

The lab's sign-ins come from my own devices, and anyone can type anything into a sign-in page. So addresses, places and outside emails leave the tools as stand-ins. The fake employee's address stays, because the verdict has to say who the alert is about.

source address    203.0.113.x          →  [masked-ip-1]
location          (city, country)      →  withheld
outside username  someone@outside...   →  [email-1]
fake employee     user-a@example-company  (kept)

Afterwards I scanned the run's 18 audit records: no outside address in any of them. Those records keep tool arguments, not tool output, so the post itself is where the stand-in shows.

The verdict in Slack

Bionic Footwear triage: suspicious, risk 40/100
Needs a person: yes
User: user-a
Summary: Five failed sign-ins for user-a came from [masked-ip-1]
within about six seconds. The same address also tried one other
account, [email-1], once. The account has never been activated
and none of the sign-ins succeeded. Worth watching; nothing shows
a compromise.
Recommended (advice only, nothing was done):
- watch_user
- ask_user: should this account see sign-in attempts at all?
Not checked: reputation of [masked-ip-1], country (withheld)

It also flagged that the alert's time window (1:00–1:03) didn't match the events (12:59). That's expected: the rule counts by ingest time, and the window is the run's. I'd rather it raise a small mismatch than smooth it over.

A person decides

Today the agent has no tools that change anything. It recommends from a fixed list, and I act by hand. For this alert there was nothing to do: it was my own test, against an account that can't sign in.

The design for approvals, written and reviewed but not built:

suspend_user or revoke_sessions recommended
  → Slack post gets Approve / Deny, for me only
  → the button carries a one-time ID, not the target
  → expires after 15 minutes; at most 3 actions an hour
  → protected accounts (admins, service accounts) get no buttons
  → a separate process with a narrow Okta role acts
  → it checks the change is visible, then records it
unsuspend stays manual