Gopichand Talluri

Breaking the Black Box: Stress-Testing, Governance, and Resilience of LLM-Driven Intelligent Systems in High-Stakes Domains

Abstract:

The promise of Large Language Models in production is enormous. The reality is messier. Across financial services, healthcare, and enterprise computing, organizations are discovering that deploying an LLM is the easy part: keeping it reliable, accountable, and secure under real-world conditions is where the hard work begins.

This keynote is built around that gap. Drawing from hands-on experience designing and operating LLM-based systems within a financial institution, the talk examines what happens when these models are pushed beyond their comfort zone: targeted input manipulation that flips fraud detection outcomes, poisoned retrieval pipelines that corrupt the logic models rely on, and performance degradation under the kind of load spikes that financial systems routinely see. These aren't theoretical vulnerabilities; they are failure modes that standard benchmarks simply don't surface, and that only reveal themselves once a system is running in production against real, motivated actors.

Finance turns out to be a uniquely honest testbed for intelligent systems. The stakes are high, the regulatory requirements are unforgiving, and the pressure from bad actors is constant. Lessons learned there translate directly to any domain where a model's outputs carry real consequences  whether that's a fraud alert, a compliance flag, or an automated decision in a vision-based pipeline.

The second half of the talk shifts from breaking things to building them right. It covers the governance and observability layer that separates a research prototype from a production-ready system: how to monitor a model that drifts silently over time, how metadata-driven pipeline architecture reduces brittleness, and how Retrieval-Augmented Generation can serve as a practical grounding mechanism that keeps models current and auditable without the cost of full retraining. Explainability is treated not as a regulatory checkbox but as an engineering necessity, the thing that makes it possible to catch problems before they become incidents.

The goal of this session is not to make the audience skeptical of LLMs, but to give researchers and practitioners a concrete framework for thinking about resilience as a design criterion from day one, not something bolted on after the fact.

Keywords: Large Language Models, Stress-Testing, AI Resilience, Fraud Detection, Drift Detection, AI Governance, Retrieval-Augmented Generation, Explainable AI, Intelligent Systems, Financial AI, MLOps

Profile:

Gopichand Talluri is a Senior Software Engineer at Annslo Tech Inc, USA, where he works on building and scaling LLM-based intelligent systems for fraud detection, financial compliance, and data engineering. Over the years, his work has gravitated toward a question that doesn't get enough attention in the research community: what does it actually take to make these systems work reliably in the real world, not just on benchmarks?

That question has shaped both his engineering work and his research. He has published five journal papers on topics ranging from bias and data poisoning in fraud detection pipelines and explainable AI in financial decision systems, to LLM lifecycle management and metadata-driven self-optimizing ETL frameworks. He has six more papers currently under review at international conferences, with a particular focus on stress-testing LLM pipelines under high-load and attack conditions, and hybrid vector-aware retrieval architectures. On the applied side, he has filed patent applications in four countries  India, the United States, Germany, and the United Kingdom  covering innovations in intelligent pipeline automation, LLM governance, real-time fraud analytics, and enterprise data interfaces.

He believes the most interesting problems in AI right now live at the boundary between what models can do in a lab and what they need to do in the field and that financial services, with its combination of regulatory pressure and attack-surface complexity, is one of the best places to work through those problems.