AML Alert Prioritization — Exploratory Data Analysis
1. Approach
Each alert (signal_id) carries several months of prior transaction history. Our hypothesis going in:
escalation is driven less by absolute transaction volume and more by behavioral deviation — a change
in an account's own pattern shortly before the alert, relative to its own longer-run baseline.
We built aggregate features across multiple trailing windows (1d / 3d / 7d / 30d / full history, plus 7–14d and 30–60d comparison windows), derived deviation and growth ratios between recent and historical windows, and added burstiness (inter-transaction gap statistics) and diversity (transaction-type entropy, night/weekend activity share) features. A 5-fold LightGBM classifier was trained and validated out-of-fold, then folds were averaged for the test-set submission.
2. Dataset overview
Escalation is a minority outcome: roughly 17% of alerts are escalated, 83% dismissed. Class imbalance was handled via stratified k-fold validation rather than resampling, since ROC-AUC is threshold-free.
Median history depth is ~460 transactions per alert (min 1, max 2,279) — enough to make trailing-window baseline comparisons statistically meaningful for the vast majority of alerts, addressing the main concern raised before feature work began.
3. Key EDA observations
Escalated alerts show a marginally different inflow/outflow mix than dismissed ones, though the effect is subtle rather than a clean separator on its own.
Transaction-type composition (card / bank transfer / cash / international) differs slightly
between outcomes — international transfers (xalqaro) are rare (<0.5% of all transactions) but
carry outsized importance in the model.
Transaction density in the days immediately preceding the alert vs. dismissed and escalated outcomes — both groups show broadly similar timing distributions, which foreshadows the modest predictive power we ultimately observed.
The standardized transaction-size index (miqdor_indeksi) distribution is nearly
identical between escalated and dismissed alerts — this is the clearest single indication that transaction size
alone is not a strong escalation driver in this dataset.
4. What the model actually learned
The strongest features are the extremes of the full-history transaction-size distribution
(amt_max_full, amt_min_full), a short-window night-activity deviation, and
kirim/naqd ratio shifts across windows — consistent with the "deviation from baseline" hypothesis, though each
individual feature's marginal signal is weak.
Out-of-fold predicted probabilities do separate the two classes, but with substantial overlap — the model is meaningfully better than random, not a clean discriminator.
5. Honest conclusion
Out-of-fold ROC-AUC: ≈0.59. We validated this isn't a feature-engineering gap: point-biserial correlations between the single strongest features and the target all stay under 0.06 in magnitude, and we found no seasonality signal (escalation rate is flat across months and weekdays of the alert date). Expanding from ~60 aggregate features to ~120 (adding short windows, burstiness, entropy, growth ratios) moved OOF AUC by less than 0.01. Our reading: this synthetic dataset's transaction history carries a genuine but limited amount of escalation-relevant signal, and 0.59 is close to what's recoverable from these tables alone — not a modeling shortfall we ran out of time to fix.
The features that do carry signal point toward a believable story: extreme single-transaction values, international-transfer share, and short-window deviations in cash/night activity relative to an account's own history. A production system would likely need account-level context beyond transaction logs (KYC risk tier, prior investigation history, counterparty network features) to meaningfully improve on this.