Overview

Teaching: 40 min
Exercises: 0 min

Questions

• How do we measure the accuracy of ML model

Objectives

• Learn different metrics with caret

4 Evaluation Metrics

• Evaluation Metric is an essential part in any Machine Learning project.
• It measures how good or bad is your Machine Learning model
• Different Evaluation Metrics are used for Regression model (Continuous output) or Classification model (Categorical output).

4.1 Regression model Evaluation Metrics

4.1.1 Correlation Coefficient (R) or Coefficient of Determination (R2):

cor(prediction,testing)
cor.test(prediction,testing)

4.1.2 Root Mean Square Error (RMSE) or Mean Square Error (MSE)

The postResample function gives RMSE, R2 and MAE at the same time:

postResample(prediction,testing$Ozone)

4.2. Classification model Evaluation Metrics

4.2.1 Confusion Matrix

• A confusion matrix is a technique for summarizing the performance of a classification algorithm.
• You can learn more about Confusion Matrix here

For binary output (classification problem with only 2 output type, also most popular):

 confusionMatrix(predict,testing)

Key Points

• Caret