Training Machine Learning model using Model based Prediction

Overview

Teaching: 20 min
Exercises: 0 min

Questions

• What is model based prediction algorithm in ML?

Objectives

• Learn to use different Model based prediction for Machine Learning training

8.1 Naive Bayes

• Assuming data follow a probabilistic model
• Assuming all predictors are independent (Naïve assumption)
• Use Bayes’s theorem to identify optimal classifiers
• More information on Aplication of Bayes’s Theorem in ML can be found here

8.1.1 Implementation Naive Bayes

Split data

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

iris = load_iris()
X = iris.data
y = iris.target
X_train, X_test, y_train, y_test = train_test_split(X,y,train_size=0.6, random_state = 123)

Train data using Naive Bayes

from sklearn.naive_bayes import GaussianNB
model_NB = GaussianNB().fit(X_train,y_train)
model_NB.score(X_train,y_train)
model_NB.score(X_test,y_test)

In addition to GaussianNB, sklearn also includes: MultinomialNB, ComplementNB, BernoulliNB, CategoricalNB. More information on Naive Bayes using sklearn can be found here

8.2 Linear Discriminent Analysis

• LDA is a supervised learning model that is similar to logistic regression in that the outcome variable is categorical and can therefore be used for classification.
• LDA is useful with two or more class of objects

8.2.1 Implementation LDA

Using the same iris data set, the LDA model is built:

from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
model_LDA = LinearDiscriminantAnalysis().fit(X_train,y_train)
model_LDA.score(X_train,y_train)
model_LDA.score(X_test,y_test)

8.2.2 Ensemble approach (Bagging) with LDA

from sklearn.ensemble import BaggingClassifier
model_LDAbag = BaggingClassifier(base_estimator = model_LDA,n_estimators=100,
                                 bootstrap=True, n_jobs=-1,
                                 random_state=123)
model_LDAbag.fit(X_train,y_train)
model_LDAbag.score(X_train,y_train)
model_LDAbag.score(X_test,y_test)

Key Points

• Naive Bayes, Linear Discriminent Analyst