Review Questions

1. C ✅

2. B ✅

3. B ❌ C

   1. C. For imbalanced datasets, AUC PR is a way to minimize false positives compared to AUC ROC.
   • If the goal is to minimize false positives while detecting fraud broadly, C (maximize AUC PR) is generally the best objective.
   • If you have a specific recall target (like 50%) and want to maximize precision exactly there, then B is appropriate.
   • In practice, C is preferred for robustness and overall better balance.

4. C ❌ B

   1. B. Since the model is performing well with training data, it is a case of data leakage. Cross‐validation is one of the strategies to overcome data leakage. We covered this in Chapter 2.
   • B is a better choice because it addresses the root cause (data leakage) more systematically by improving validation methodology.
   • C might help in some cases but is a limited fix and might degrade model performance by removing useful information unnecessarily.
   • For time-series data, proper validation (like nested CV with temporal splits) is crucial to avoid leakage and get realistic performance estimates.

5. A, B ✅

6. D ❌ A

   1. A. Use a tf.data.Dataset.prefetch transformation.

   Create a tf.data.Dataset.prefetch transformation.

   • prefetch() allows the input pipeline to fetch data asynchronously, overlapping data preprocessing and model execution.
   • Helps reduce input latency by preparing the next batch while the current batch is being processed.
   • Crucial for pipelines where data does not fit in memory and comes from various sources.
   • This is a best practice for efficient, low-latency input pipelines.

7. C ✅

8. A ✅

9. D ❌ A

   1. A. TensorFlow Transform is the most scalable way to transform your training and testing data for production workloads.
   • tf.Transform (TensorFlow Transform) is a library that allows you to define data preprocessing and feature engineering pipelines that can be shared between training and serving.
   • More importantly, tf.Transform supports specifying and tracking metrics on specific subsets of data (called slicing).
   • This means you can:
     • Train on all available data for best model performance.
     • Define performance metrics on different slices/subsets (e.g., by location, product category, time period).
     • Monitor these metrics continuously to detect drifts or degradation in specific segments before pushing models to production.
   • This approach is streamlined (integrated into the pipeline), scalable (works with big data), and reliable (metrics are consistent and comparable).

   b. Why not other options?

   • B: Using the entire dataset and a single overall metric like AUC ROC ignores performance variation on different data subsets. This can hide poor performance on specific, important segments.
   • C: Using the last relevant week as validation only assesses recent performance but doesn't provide a scalable way to monitor multiple subsets or different slices of data continuously.
   • D: K-fold cross-validation is often not suitable for time-series data (which this is) because it can mix past and future data, causing leakage and unrealistic performance estimates. Also, it doesn’t inherently track performance on specific data subsets

10. A ❌ D

    1. D. Since the model is underperforming with production data, there is a training‐serving skew. Using a tf.Transform pipeline helps prevent this skew by creating transformations separately for training and testing.
    • Option A is vague and incomplete — just applying transformations during training is not sufficient.
    • Option D explicitly solves the problem of applying consistent, leak-free transformations during both training and serving, which is why it's the recommended solution.