Kuladeep Sandra

Row-Based vs Columnar Execution in Spark: Practical Tradeoffs from Large-Scale Data Engineering

Abstract:

This talk explores the practical differences between row-based and columnar execution in Apache Spark, with a focus on performance, memory efficiency, and real-world tradeoffs in large-scale data processing environments. Based on hands-on experience in enterprise data engineering, the session will highlight how execution format affects query performance, resource utilization, and overall system design. It will also discuss when columnar execution provides measurable benefits, where row-based processing remains relevant, and how these choices influence modern lakehouse and analytics architectures. The session is intended to provide practical lessons for data engineers, architects, and practitioners working with Spark at scale.

Profile:

Kuladeep Sandra is a Senior Manager at Dell Technologies with extensive experience in data engineering, cloud infrastructure, and enterprise data platform architecture. He has led large-scale initiatives in modern data platforms, lakehouse architecture, and governed analytics environments, with a focus on building scalable, secure, and practical data solutions. His work spans open data architectures, query engines, Kubernetes-based platforms, and enterprise AI governance. Kuladeep is passionate about translating complex engineering concepts into real-world architectural outcomes and sharing insights that help organizations improve performance, reliability, and data platform maturity.