API key leak / repo compromise — methodology
an api key leak is rarely one bad commit. it is a credential in git history, a force-push that hides it from main, secret scanning that fires too late, and a cloud abuse window measured in minutes. your job is to recover the introducing commit from reflog, correlate vcs access to cloudtrail, map iam escalation paths, and prove the same key material across git, aws, and k8s exports — before the attacker rotates their own keys and stops logging.
what evidence exists and how fast it dies
| artifact | volatility | time to loss |
|---|---|---|
| git bundle / .git with reflog | persistent if captured | reflog expires on gc — days to weeks on active repos |
| github enterprise audit log export | rolling | 180 days default — export immediately |
| github secret scanning alert | persistent in ui | alert fires after push — attacker may clone before you see it |
| aws cloudtrail management events | rolling | 90 days default — StopLogging ends the trail |
| iam policy document for leaked principal | persistent until changed | attacker may attach broader policy within minutes |
| k8s secret export (base64) | point-in-time | secrets rotate — stale export still proves prior material |
the first 10 minutes
- disable or deactivate the leaked access key — do not delete until cloudtrail is exported.
- clone or bundle the repo with full .git — preserve reflog before gc runs.
- export github enterprise audit log for the repo org — 30 days minimum.
- export cloudtrail management events for the account — all regions, 14 days back.
- pull the iam policy attached to the leaked principal — before attacker attaches a broader one.
- check for CreateAccessKey, AttachUserPolicy, AssumeRole, GetSecretValue, StopLogging in cloudtrail.
- export k8s secrets from namespaces the ci bot could reach — compare decoded material.
- block attacker source ip at edge if identified in github audit or cloudtrail.
- notify secret scanning / security team — scan all repos for the same key pattern.
- begin the path below.
the path
1. git repository forensic analyzer
repo bundle or .git archive. recovers deleted commits via reflog, scans blob history for AKIA/ghp patterns, and pins the introducing commit.why first: force-push hides the leak from main — reflog is where the credential still lives.
2. github audit log parser
enterprise audit json/jsonl export. structured rows for clone, push, oauth, secret_scanning_alert, and member events.why second: you need a clean event inventory before correlating vcs activity to cloud abuse timestamps.
3. github audit log analyzer
same audit export — timeline view with force-push, secret scanning, and suspicious access heuristics across org repos.why third: cross-check parser output — clone from attacker ip often precedes aws api calls by minutes.
4. aws cloudtrail forensic deep analyzer
cloudtrail json records. surfaces privilege escalation paths, credential theft, bulk GetSecretValue, and lateral movement patterns.why fourth: the leaked key's first aws calls tell you scope before the attacker pivots roles.
5. aws cloudtrail log forensic analyzer
same cloudtrail export — operation breakdown, attacker ip clustering, StopLogging and CreateAccessKey flags.why fifth: second parser catches timeline gaps — CreateAccessKey + AttachUserPolicy often pair in escalation chains.
6. aws iam policy analyzer
iam policy json for the leaked principal (ci-deploy-bot). expands wildcards, flags PassRole + ec2/lambda surface, plain-english risk summary.why sixth: the key alone is not the blast radius — the attached policy defines what abuse was already possible.
7. iam escalation graph
same policy document. builds escalation chains across 15 patterns — PassRole, sts:AssumeRole, iam:* combos.why seventh: graph view shows multi-hop paths that flat policy review misses.
8. kubernetes secrets decoder
cluster secret yaml export. decodes base64 aws + github tokens — compare material to git leak and cloudtrail caller identity.why last: k8s exports often hold the same key the attacker already pulled from secrets manager — corroborates scope.
common false leads
- main branch is clean so no leak — force-push removes commits from main, not from reflog.
- secret scanning cleared it — scanning fires on push; attacker may have cloned before the alert.
- rotating the key fixes everything — cloudtrail shows what already happened with the old key.
- the github token is unrelated — same commit often leaks aws + github credentials together.
- iam policy looked fine at triage — attacker AttachUserPolicy may have broadened it after compromise.
what we can tell you, what we can't
we can tell you:
- which commit introduced akia/ghp patterns and whether it was reachable from main
- github audit timeline — clone, force-push, secret_scanning_alert, oauth, member changes
- cloudtrail privilege escalation, bulk secret reads, and attacker ip clustering
- iam policy wildcard expansion and escalation graph paths for the leaked principal
- decoded k8s secret material matching git leak and cloudtrail caller identity
we can't tell you:
- live aws account state — offline analysis of exports you provide
- whether the key was exploited before you detected it — requires complete cloudtrail retention
- attribution to a named threat actor — intel and law enforcement territory
- automatic key revocation — you must disable keys in iam/console
handing it off
- platform / devsecops: introducing commit hash, leaked key id (redacted), iam escalation paths, affected repos, k8s namespaces with matching secret material.
- cloud / iam team: cloudtrail timeline, CreateAccessKey events, StopLogging proof, policy diff before/after, rotation status for all principals in the escalation graph.
- outside counsel / breach coach: preservation log — git bundle hash, audit export timestamps, cloudtrail export scope, notification timeline for secret scanning.
further reading
reference investigation
synthetic fixture novak-api-key-leak — NovaPay ci-deploy-bot key committed to novak-payments-api, force-pushed but recoverable from reflog. attacker cloned from 198.51.100.77, triggered secret scanning, then iam escalation + secrets manager reads matching k8s export. seed novak-api-key-leak:v1. compare output via npm run check:flagship.
fixture download: evidence zip · proof page: /forensics/proof/novak-api-key-leak · case playbook: case type tools