Challenge: Predicting Flight Delays

You are given a flights dataset as a list of rows. Load it into a DataFrame using createDataFrame and train a binary classification model to predict whether a flight is delayed (Delay == 1). Complete all steps and store results in the specified variables:

1. Fill nulls in Delay and Length with 0;
2. Add a LABEL column – 1.0 if Delay == 1, otherwise 0.0;
3. Add IS_WEEKEND – 1 if DayOfWeek >= 6, otherwise 0;
4. Split into train (80%) and test (20%) with seed=42;
5. Build a Pipeline with StringIndexer on Airline, VectorAssembler on ["Length", "Time", "IS_WEEKEND", "AIRLINE_IDX"], and RandomForestClassifier with numTrees=10, maxDepth=3, seed=42;
6. Fit the pipeline and generate predictions on the test set – store in predictions;
7. Compute AUC-ROC – store in auc_roc (rounded to 4 decimal places);
8. Compute accuracy – store in accuracy (rounded to 4 decimal places).

Print both metrics.