%%html
<!-- The customized css for the slides -->
<link rel="stylesheet" type="text/css" href="../styles/python-programming-introduction.css"/>
<link rel="stylesheet" type="text/css" href="../styles/basic.css"/>
<link rel="stylesheet" type="text/css" href="../../assets/styles/basic.css" />

43.19. Model Selection#

43.19.1. Introduction#

  • Model Selection is the process of choosing the best model among all the potential candidate models for a given problem.

  • The aim of the model selection process is to select a machine learning algorithm that evaluates to perform well against all the different parameters.

43.19.2. Outline#

  • Over-fitting and under-fitting

  • Bias variance tradeoff

  • L1 and L2 Regularization

  • Early stopping

  • Dropout

  • Tuning the hyper-parameters of an estimator

43.19.3. Over-fitting and under-fitting#

  • Regression

43.19.4. Over-fitting and under-fitting#

  • Regression

    • Training data points

43.19.5. Over-fitting and under-fitting#

  • Regression

    • Over-fitting model fits very well on training data

43.19.6. Over-fitting and under-fitting#

  • Regression

    • Over-fitting model fits poorly on test data

43.19.7. Over-fitting and under-fitting#

  • Regression

    • Under-fitting model fits poorly on training data

43.19.8. Over-fitting and under-fitting#

  • Regression

    • Under-fitting model fits poorly on test data

43.19.9. Over-fitting and under-fitting#

  • Classification

43.19.10. Bias variance tradeoff#

  • Graphical illustration of variance and bias

43.19.11. Bias variance tradeoff#

  • Model complexity v.s. error

43.19.12. L1 and L2 regularization#

\[L2\ Loss = Loss + {\lambda}\sum_{i} w_i^2\]
\[L1\ Loss = Loss + {\lambda}\sum_{i} \lvert w \rvert\]

43.19.13. L1 and L2 Regularization#

43.19.14. L1 and L2 Regularization#

43.19.15. L1 and L2 Regularization#

43.19.16. L1 and L2 Regularization#

43.19.17. L1 and L2 Regularization#

  • The impact of the value of \(\lambda\)

43.19.18. Early stopping#

43.19.19. Dropout#

43.19.20. Prediction after dropout#

43.19.21. Conclusions#

  • Training size matters

43.19.22. Conclusions#

  • How to choose a good model

43.19.23. Conclusions#