In this talk, we will explore the core components of Apache Spark and examine what makes it a powerful and efficient engine for large-scale data processing. We'll discuss how Spark achieves performance through in-memory computing, DAG execution, and parallelism, while also highlighting common challenges related to scalability and resource optimization in big data environments. Finally, we'll introduce how AI techniques—such as intelligent workload management and adaptive query optimization—can be leveraged to enhance performance and efficiency in Spark-based architectures.
Co-sponsored by: Fairleigh Dickinson University
Speaker(s): Hina Gandahi
Agenda:
Fairleigh Dickinson University
1000 River Road, Building: Muscarelle Center, Room Number: 105
Teaneck, New Jersey, United States 07666
For additional information about the venue and parking, please contact
Dr. Hong Zhao
zhao@fdu.edu
Virtual: https://events.vtools.ieee.org/m/486209
