Before / After Comparison Workbench
Replay Lab
Capture Rate (Recall)+11.50pp
86.8%vs 75.3% baseline
347 of 400 synthetic fraud txns caught
Rule Precision+15.57pp
33.0%vs 17.4% baseline
F1 score: 47.8% · 347 TP / 705 FP
False Positive Rate (FPR)-3.68pp
3.60%vs 7.28% baseline
705 legit txns flagged (440 declined)
Approval Rate+3.38pp
94.74%vs 91.36% baseline
Decline rate: 3.75% · Review: 1.51%
Review Queue Size-1,034
302vs 1,336 baseline
37 fraud / 265 legit in manual queue
$ Fraud Caught vs $ Good Declined+$22,336
$170,637caught
$50,210 good $ declined (+$27,255 vs base)
BEFORE · BASELINE
35.2msStarter Baseline (3 Rules)
True Positive (Fraud Caught)TP
301
188 decline · 113 review$148,300
False Negative (Missed Fraud)FN
99
approved fraud$31,886
False Positive (Legit Flagged)FP
1,427
204 decline · 1223 review$22,956 decl.
True Negative (Legit Approved)TN
18,173
clean pass-through91.36% appr.
AFTER · CANDIDATE
15.0msTuned Candidate v2 (5 Rules)
True Positive (Fraud Caught)TP
347
310 decline · 37 review$170,637
False Negative (Missed Fraud)FN
53
approved fraud$9,550
False Positive (Legit Flagged)FP
705
440 decline · 265 review$50,210 decl.
True Negative (Legit Approved)TN
18,895
clean pass-through94.74% appr.
Threshold Sensitivity Sweep
New Account High-Ticket + Young Device
Vary one numeric cutoff while holding all other candidate rules fixed to locate the precision/recall elbow.
Capture Rate (Recall)Approval RateFalse Positive Rate (right axis)
14 thresholds swept in 174.3msamount >
$350current
Capture (Recall)
86.8%
FPR
3.60%
Approval
94.74%
Fraud Caught
$170,637
Good Declined
$50,210
Per-Rule Contribution (Tuned Candidate v2 (5 Rules))
First-match triggers determine decision ownership by ascending priority order.
| Pri | Rule | Action | 1st Match | Total Hits | Fraud | FP | Prec. | $ Caught |
|---|---|---|---|---|---|---|---|---|
| #1 | Velocity Burst + Composite Score | decline | 653 | 653 | 213 | 440 | 32.6% | $77,793 |
| #2 | New Account High-Ticket + Young Device | decline | 90 | 127 | 90 | 0 | 100.0% | $72,743 |
| #3 | CVV & AVS Hard Mismatch | decline | 7 | 20 | 7 | 0 | 100.0% | $3,557 |
| #4 | Cross-Border Risky BIN / High Distance | review | 66 | 234 | 27 | 39 | 40.9% | $12,758 |
| #5 | Late-Night Digital Goods Review | review | 236 | 367 | 10 | 226 | 4.2% | $3,787 |
Rule Overlap
Transactions matching multiple candidate rules simultaneously.
Velocity Burst + Composite Score111 txns
∩ Cross-Border Risky BIN / High Distance17.0% of A
Cross-Border Risky BIN / High Distance79 txns
∩ Late-Night Digital Goods Review33.8% of A
New Account High-Ticket + Young Device74 txns
∩ Cross-Border Risky BIN / High Distance58.3% of A
New Account High-Ticket + Young Device69 txns
∩ Late-Night Digital Goods Review54.3% of A
Velocity Burst + Composite Score64 txns
∩ Late-Night Digital Goods Review9.8% of A
Velocity Burst + Composite Score37 txns
∩ New Account High-Ticket + Young Device5.7% of A
Highest-dollar legitimate synthetic transactions flagged by the candidate rule set. Inspecting FPs reveals where guardrails (device age, AVS/CVV, higher thresholds) save good revenue.
| Txn ID | Amount | Decision / Rule | Acct / Dev Age | Geo / Dist | 1h Card | AVS / CVV | MCC / Hr | Synthetic Profile |
|---|---|---|---|---|---|---|---|---|
txn_002081 BIN 411111 | $734.50 | reviewLate-Night Digital Goods Review | 190d / 12d | IN→IN (45km) | 1 | U / M | 6051 · 04h | taylor.tanner.5449@example.com vel 33 · clean |
txn_000521 BIN 424242 | $713.70 | reviewLate-Night Digital Goods Review | 875d / 271d | IN→IN (20km) | 1 | U / M | 6051 · 02h | skyler.dalton.4322@example.com vel 34 · clean |
txn_002929 BIN 510510 | $627.60 | reviewLate-Night Digital Goods Review | 62d / 278d | US→US (55km) | 1 | Y / M | 6051 · 03h | sawyer.foley.0811@example.com vel 25 · clean |
txn_000155 BIN 510510 | $546.71 | reviewLate-Night Digital Goods Review | 75d / 97d | US→US (2km) | 1 | Y / M | 6051 · 01h | hayden.marlow.1497@example.com vel 22 · clean |
txn_002100 BIN 424242 | $528.70 | reviewLate-Night Digital Goods Review | 69d / 138d | US→US (22km) | 1 | Y / M | 6051 · 03h | skyler.nolan.5520@example.com vel 23 · clean |
txn_001138 BIN 378282 | $527.96 | reviewLate-Night Digital Goods Review | 135d / 900d | US→US (21km) | 2 | U / M | 6051 · 04h | avery.marlow.8727@example.com vel 60 · clean |
txn_001726 BIN 510510 | $469.41 | reviewLate-Night Digital Goods Review | 1400d / 399d | IN→IN (14km) | 1 | Y / M | 6051 · 01h | quinn.keller.2396@example.com vel 37 · clean |
txn_001975 BIN 510510 | $434.07 | reviewLate-Night Digital Goods Review | 47d / 31d | US→US (15km) | 1 | Y / M | 6051 · 00h | skyler.corbin.3972@example.com vel 28 · clean |
txn_001645 BIN 555555 | $353.60 | declineVelocity Burst + Composite Score | 93d / 100d | US→US (9km) | 4 | Y / M | 6051 · 04h | hayden.foley.1729@example.com vel 100 · clean |
txn_000628 BIN 601111 | $352.68 | declineVelocity Burst + Composite Score | 41d / 387d | US→US (99km) | 4 | U / M | 6051 · 08h | jamie.marlow.5052@example.com vel 100 · clean |
txn_002688 BIN 601111 | $350.77 | reviewLate-Night Digital Goods Review | 764d / 76d | IN→IN (11km) | 1 | Y / U | 6051 · 00h | alex.nolan.9030@example.com vel 25 · clean |
txn_003249 BIN 601111 | $347.89 | declineVelocity Burst + Composite Score | 16d / 11d | US→US (7km) | 4 | U / M | 5411 · 02h | jamie.sutton.3345@example.com vel 100 · clean |
txn_000642 BIN 424242 | $346.17 | reviewLate-Night Digital Goods Review | 100d / 75d | IN→IN (305km) | 1 | Y / M | 6051 · 02h | hayden.harlan.4609@example.com vel 30 · clean |
txn_002244 BIN 411111 | $341.76 | reviewLate-Night Digital Goods Review | 41d / 20d | US→US (8km) | 1 | Y / M | 6051 · 02h | parker.sutton.5785@example.com vel 30 · clean |
txn_002861 BIN 510510 | $328.63 | reviewLate-Night Digital Goods Review | 492d / 226d | US→US (46km) | 1 | U / M | 6051 · 02h | reese.foley.0220@example.com vel 30 · clean |
txn_000380 BIN 378282 | $316.41 | reviewLate-Night Digital Goods Review | 377d / 24d | IN→IN (9km) | 1 | Y / M | 6051 · 05h | quinn.caleb.3360@example.com vel 35 · clean |
txn_002442 BIN 555555 | $315.39 | reviewLate-Night Digital Goods Review | 1072d / 12d | IN→IN (42km) | 1 | Y / M | 6051 · 03h | sawyer.caleb.7948@example.com vel 35 · clean |
txn_002507 BIN 424242 | $315.01 | reviewLate-Night Digital Goods Review | 159d / 206d | IN→IN (6km) | 1 | U / M | 6051 · 04h | jamie.bauer.7843@example.com vel 24 · clean |
txn_001175 BIN 400005 | $298.39 | reviewLate-Night Digital Goods Review | 487d / 24d | IN→IN (3km) | 1 | Y / M | 6051 · 03h | jamie.foley.8564@example.com vel 24 · clean |
txn_002634 BIN 411111 | $294.07 | reviewLate-Night Digital Goods Review | 978d / 122d | IN→IN (12km) | 1 | Y / M | 6051 · 01h | rowan.vance.9058@example.com vel 36 · clean |