Challenge: Analyzing Sales Data with Spark SQL

You are given a flights dataset as a list of rows. Load it into a DataFrame, register it as a temporary view, and answer the following using spark.sql(). Store results in the specified variables:

1. Find the top 3 routes (unique AirportFrom + AirportTo pairs) by average Length – store as a list of tuples [(origin, destination, avg_length), ...] in top_routes_by_length;
2. For each airline, find the flight with the longest Length using a window function with row_number() – store as a DataFrame in longest_flight_per_airline with columns Airline, Flight, Length;
3. Count how many delayed flights (Delay == 1) per DayOfWeek – store as a list of tuples [(day_of_week, count), ...] sorted by DayOfWeek ascending in delays_by_dow.

Print all results.