Mr. Vivek Venkatesan

Designing Cost-Effective Data Platforms: Engineering for Scale Without Breaking the Bank

Abstract:

In the era of massive data volumes and complex cloud ecosystems, cost can become the hidden enemy of innovation. This talk offers a practical guide to designing cost-efficient data architectures that don’t compromise on performance or scalability. Drawing from real-world experience at Fortune 500 organizations, I’ll share techniques to optimize data processing, storage, and AI integration costs across platforms like AWS Glue, Snowflake, and Redshift.

We’ll explore strategies like decoupling compute from storage, intelligent partitioning, minimizing data scans, and managing model inference costs using serverless AI patterns. This session is ideal for engineers, architects, and leaders looking to strike the right balance between innovation and infrastructure cost.

Profile:

Vivek Venkatesan is a Lead Data Engineer at a Fortune 500 global investment firm—one of the world’s largest investment management companies. With over 15 years of experience in data engineering, AI integration, and scalable cloud architectures, he has led high-impact digital transformation initiatives across healthcare, insurance, and finance.

Vivek’s work during the COVID-19 pandemic included designing a real-time exposure alerting system for a major healthcare organization—preventing workplace infections using intelligent data flows. At Vanguard, his innovations have resulted in over $250,000 in annual infrastructure savings while advancing real-time decision intelligence. He actively mentors engineers, contributes to AI-driven frameworks, and actively serves as a manuscript peer reviewer for reputed international journals and conferences.