Krish Swamy

Learning model development is easy, but building your first fraud model is where reality hits. I’ve seen experienced data scientists miss key evaluation metrics when working on onboarding fraud, especially when fraud rates are below 1% and you’re flying blind with no behavioural history. Accuracy and AUC might look impressive, but they tell you nothing about operational effectiveness. You need to think in terms of fraud capture at top X%, fraud positive rate, lift, and threshold tuning. These aren’t just for post-model dashboards. They should guide how you tune hyperparameters, set thresholds, and communicate model performance to the business. If your model boasts 99% accuracy but hasn’t caught a single fraudster, it’s not a fraud model. It’s a compliance checkbox. Use metrics that reflect real-world impact. Because in onboarding fraud, the difference between a good model and a great one isn’t how it scores in a notebook. It’s how well it protects your front door. Read my post to understand the evaluation metrics that matter in fraud. #FraudDetection #OnboardingFraud #MachineLearning #ModelEvaluation #DataScience #AIinBanking #RiskAnalytics #PrecisionMatters #ModelMetrics #FinancialCrime #MLforFraud #HyperparameterTuning #BusinessImpact #RealWorldML #DataDrivenDecisions https://lnkd.in/gyMV_Z_Y

48

2 Comments

Enterprise AI governance fails at a predictable moment: the first time a regulator asks which model version executed a specific underwriting, pricing, or claims decision—and the organization cannot answer. This is not a policy problem. Policy governs intent. Architecture governs execution. Financial institutions scaling probabilistic systems inherit a structural mismatch. Pilots tolerate ambiguity because errors remain contained. Enterprise operations cannot. Once model outputs shape material customer decisions across millions of transactions, the operational substrate built for deterministic, rule-based logic offers no native way to trace, bind, or audit probabilistic decision behavior. Closing that gap requires four explicit architectural controls: ↳ Cryptographic model lineage: CI/CD pipelines must enforce signed artifacts across the full promotion path, from training through inference. Without this, no audit trail can establish which model version executed a contested decision. ↳ Execution boundaries: Probabilistic outputs must operate inside deterministic constraints. Hard business rules, approval thresholds, and scoped permissions must bind system behavior independently of model confidence. The model proposes; the architecture decides. ↳ Automated circuit breakers: Production systems need circuit breakers that monitor execution telemetry. When drift, instability, or policy violations exceed thresholds, these mechanisms must automatically trigger isolation or graceful degradation to deterministic fallbacks. At scale, monitoring becomes part of the enforcement architecture. ↳ Compliance as a pipeline stage: Engineering teams must embed validation and release controls inside the deployment pipeline. When an unverified model reaches production, it creates governance debt that compounds with every decision it influences. The test is forensic: given a disputed decision, can the system reconstruct the exact model version, the decision input context, the deterministic controls applied, and the signed release record? Organizations that cannot pass this test are not governing probabilistic systems. They are merely operating them. Architectural discipline is now the primary differentiator. As model capability commoditizes, the ability to deploy it accountably remains scarce. #EnterpriseArchitecture #SystemsArchitecture #AIGovernance #FinancialServices #MLOps

6

1 Comment

After 10 years in data science, it’s rare to come across a shift this significant—AI agents are no longer just an idea, they’re being deployed. Unlike traditional models that predict, these agents act. They interpret context, connect across systems, and execute tasks—autonomously. From underwriting and reporting to customer support and operations, businesses are using them to drive real outcomes. This isn’t about replacing humans—it’s about freeing us from the repetitive, so we can focus on strategy, insight, and impact. As someone who’s spent a decade building data solutions, I see this as the next chapter: moving from dashboards to decision-making, from insight to intelligent action. We’re not just building models anymore. We’re building systems that think. #AI #DataScience #AIagents #Automation #Innovation #FutureOfWork

18

6 Comments

A significant proportion of AI implementations at major financial institutions require rework after initial regulatory review. I've seen the same pattern across multiple Chief of Staff roles in financial services. AI teams build strong models. Risk teams understand the regulations. But the translation layer between them is often missing. That gap is where delivery slows — and where value gets lost. At a Fortune 500 financial institution, I reduced AI audit preparation time by 40% by introducing structured translation protocols: • AI teams document model decisions in regulatory-aligned language from day one • Risk teams review technical architecture during development, not after deployment • Cross-functional standups built around shared vocabulary and success metrics • Pre-submission validation using actual regulatory frameworks The breakthrough wasn't better AI or tighter compliance controls. It was better coordination — designed into the process, not added at the end. Most organisations still treat this as a handoff problem: build, then pass to Risk. But regulatory readiness has to be embedded in development itself. This is where the Chief of Staff is uniquely effective. Not just coordinating delivery — but translating between technical possibility and regulatory reality, across teams with different incentives and languages. When AI teams understand compliance constraints early, they build differently. When Risk teams engage during development, decisions move faster. Are you treating AI governance as a gate — or designing it as a bridge? #AIGovernance #ChiefOfStaff #ModelRiskManagement #FinancialServices #ResponsibleAI

1

Every institution has blind spots. The challenge is knowing where they are before they become losses. That goal shaped the work of my team in developing Experian's new Fraud Peer Benchmarking Report. The report highlights: • Where your exposure is increasing fastest • How your defenses compare to organizations with similar risk profiles • Where targeted investment could deliver immediate impact For lenders and fraud executives, this external context is no longer optional. Threats evolve by industry segment, channel, and portfolio mix — often faster than internal data can reveal. This new report is an important step toward more proactive risk management. If you're interested in learning more, please reach out. https://lnkd.in/gyiVaPsE

11

Major microstructure changes and rough dates that you should know when backtesting with CME market data: Jan 2, 2013 - Messaging Efficiency Program becomes effective. Jun 3, 2013 - Self-match prevention functionality launched. Mar 9, 2014 - New iLink and MSGW architecture (Phase 1) launched for TAS markets. Sep 28, 2014 - CME introduces MDP 3.0 based on SBE. Sep 21, 2015 - BTIC launched. Jan 31, 2016 - New iLink and MSGW architecture pre-launch for remaining markets (Phase 2). Apr 10, 2016 - New iLink and MSGW architecture in production for remaining markets (Phase 2). Jul 10, 2016 - Major performance enhancements to CME Globex. (?) Jul 17, 2016 - Production rollout of iLink nanosecond timestamp enhancement and tag 5979. May 21, 2017 - Production rollout of MBO for final market segment. Jul 17, 2017 - Russell index futures and options return to CME. Sep 24, 2017 - Implied functionality for FX futures and SD calendar spreads. Dec 18, 2017 - CME launches crypto futures with Bitcoin futures. Mar 25, 2018 - FX Link launched. May 13, 2018 - TACO launched. Jul 29, 2018 - iLink performance enhancements. Jan 13, 2019 - 9-digit price precision extension. Feb 3, 2019 - Launch of consolidated iLink fill messages. May 6, 2019 - Micro E-mini futures launched. Jun 17, 2020 - Production rollout of binary iLink 3.0 for final market segment. Jun 27, 2021 - Major performance enhancements to CME Globex. Jun 27, 2021 - 3:15 - 3:30 PM CT pause eliminated for CME equity index futures and options. Sep 17, 2021 - Standard-sized S&P 500 index futures and options delisted. Apr 14, 2023 - Eurodollar futures and options converted to SOFR contracts. Jun 26, 2023 - Eurodollar futures and options delisted with expiration of June contracts. Jan 7, 2024 - Major performance enhancements to CME Globex. Jan 7, 2024 - Mass quote messaging capacity increased. Mar 25, 2024 - Micro Treasury futures launched. Need a microstructure-aware vendor and market depth going back to 2009? You get both with #Databento.

164

8 Comments

Ever feel like today’s AI models are getting smarter but somehow less useful? You’re not imagining it. LLM benchmarks have become the obsession. Model makers are locked in a cycle of chasing leaderboard points on MMLU, GSM8K, and other tests no one actually cares about outside of research labs. The result? Models that ace exams but fall apart in real-world use. They misinterpret vague instructions. They forget what you said three messages ago. They can’t think beyond a single step. But sure, the score went up by 0.3 percent. Benchmarks were supposed to be a guide. Instead, they’ve become a crutch. A way to show “progress” without improving anything that matters. The questions we should be asking are simple: Can the model follow messy, human instructions? Can it reason through problems, not just regurgitate answers? Can it actually help people get real work done? If you’ve worked with these models, you’ve felt the gap. You’ve seen the hype, and then you’ve seen it fall short when it counts. Are we building for people, or are we building for points? I’d love to hear what others are seeing in the wild. #AI #LLMs #Benchmarks #MachineLearning #RealWorldAI

27

3 Comments

I am reading yesterday's news on #JPMorgan implementing #fees for data that's been freely accessible to #fintechs for years¹. These charges could reach hundreds of millions of dollars, with some potentially exceeding 1,000% of transaction revenue for affected companies². Seems banks are finally recognizing their data as a strategic asset rather than a regulatory obligation. JPMorgan has distributed pricing sheets to data aggregators, with premium charges for payment-focused services³. We observed similar dynamics in Central Europe under #PSD2, where banks explored monetizing mandatory data sharing requirements⁴. The key difference: European regulation mandated open access, while JPMorgan operates from a position of choice—creating substantially more pricing leverage. I am really keen on whether other major banks will follow this precedent or leverage free data access as a competitive differentiator. The answer will determine whether we see accelerated consolidation or increased innovation in alternative data strategies. --- Sources: 1. Bloomberg: "JPMorgan Tells Fintechs They Have to Pay Up for Customer Data" - https://lnkd.in/db45bYDd 2. Bloomberg: "JPMorgan Hands Fintech Firms the Bill: Evening Briefing Americas" - https://lnkd.in/d_Fb_4Y6 3. American Banker: "JPMorganChase to charge data aggregators for customer data" - https://lnkd.in/dU6CEJtX 4. EOS Global Collection: "With PSD2, the EU is revolutionizing online payments" - https://lnkd.in/dZF8PtZk #Fintech #BankingInnovation #DataMonetization #FinancialServices #DigitalTransformation

8

A recent Amazon paper on Insight Agents is an example of the agentic AI patterns and agentic analytics systems I’ve been writing about recently. The system follows an orchestrator–worker pattern: a manager agent handles intent detection, routing, and control; specialized agents perform data retrieval and insight generation; and LLMs are used primarily for reasoning and synthesis, not execution. Analytics is treated as a plan-and-execute workflow, with governance built directly into the architecture through out-of-domain detection, API-based data access, and explicit control flow. Evaluated via human review, the system achieves ~90% accuracy with interactive latency (~14s P90)—showing that agentic analytics can be both reliable and operational at scale when orchestration and constraints are first-class design decisions. Paper: https://lnkd.in/eE_tDkGc #AgenticAI #AgenticAnalytics #AIArchitecture

11

1 Comment