How does Apache Spark differ from Hadoop MapReduce? Why has Spark become the preferred processing framework?

Sample answer preview

Apache Spark and Hadoop MapReduce are both distributed processing frameworks, but Spark has largely replaced MapReduce for most use cases due to fundamental architectural advantages. The most significant difference is in-memory processing.