Explain the differences between RDDs, DataFrames, and Datasets in Apache Spark. When would you use each?

Sample answer preview

Spark offers three main data abstractions, each building on the previous one with additional capabilities. Understanding when to use each is important for writing efficient Spark applications. RDDs, or Resilient Distributed Datasets, are the foundational data structure in Spark.