// model evaluation & validation · application
R is used to put both Model Evaluation and Model Validation techniques into practice, using packages such as Metrics and caret for computing evaluation metrics and for running cross-validation.
For Model Evaluation, the Metrics package is installed and loaded to evaluate regression models, using functions such as sse to calculate error on the test dataset. For evaluating classification models, the caret package is imported, which allows computation of the confusion matrix along with a whole host of other metrics, while the ROCR package is used to plot the ROC curve and calculate the AUC score.
For Model Validation, the caret package's trainControl function is used to configure k-fold cross-validation, repeated cross-validation, leave-one-out cross-validation (LOOCV) and bootstrap resampling, which are then passed into the train function to validate models such as Logistic Regression and Decision Trees on the Titanic dataset.
TM