// pillar 04 · validation
A model is not finished when it is built; it has to be shown to work. This section covers the measures that judge how well a model predicts and the validation techniques that test it against unseen data.
Once the model is built, it becomes important to evaluate and validate the model.
There are multiple measures that can be used to find how good a classifier is classifying the data or how good a regression model is predicting. Model Evaluation methods can be successfully applied to the models that run in a supervised learning environment and thus the measures of evaluation are discussed for various Regression and Classification Models. The various Model Evaluation methods don't address the problem of overfitting and it is very important to find how well the model will perform with unseen data. This brings us to Data Validation where we mimic this scenario where our model is applied to a test data which acts as an unseen data for us to understand how good our model is at generalising.
This section, like all the other sections, is also divided into Theory and Application where in Theory, various evaluation measures and validation techniques are discussed. Here the need for such measures along with the limitations and advantages of each of these measures is explored.
In the Application section, the codes pertaining to Model Evaluation and Validation are provided in Python and R language.
The various model evaluation methods are explored. The way they work, the advantages and limitations in each of these measures are explored along with the comparison of these measures of evaluation and the different insights each of these measures provide to us. The back-end working of different model validation techniques are discussed to give you an idea how they solve the problem of overfitting and provide us with a better idea of how our model will perform in the real world.
The application of model evaluation and validation is relatively easy than understanding all the theory behind it. Simple one line codes are provided in Python and R language. We explore the difference in the accuracy when applying different validation techniques on the same dataset. This will help us in understanding the advantages that some of these measures have over others.
TM