Naïve Bayes Algorithm

Introduction:

Naïve Bayes is a popular machine learning algorithm used for classification tasks, Particularly in text and natural processing application, such as spam email detection and sentimate analysis. The algorithm based on Bayes’s  theorm and is known for its simplicity and efficiency.

Naïve Bayes Classifier Algorithm:

Ø Naïve Bayes algorithm is a supervised learning algorithm, which is based on Bayes theorem used for solving classification problems.

Ø It is mainly used in text classification that includes a high-dimensional training dataset.

Ø Naïve Bayes Classifier is one of the simple and most effective Classification algorithms which helps in building the fast machine learning models that can make quick prediction.

Bayes' Theorem:

Ø Bayes' theorem is also known as Bayes' Rule or Bayes' law, which is used to determine the probability of a hypothesis with prior knowledge. It depends on the conditional probability.

Ø Naïve Bayes is also called Maximizing a Posterior(MAP).

Ø The formula for Bayes' theorem is given as:

Where,

P(A|B) is Posterior probability: Probability of hypothesis A on the observed event B.

P(B|A) is Likelihood probability: Probability of the evidence given that the probability of a hypothesis is true.

P(A) is Prior Probability: Probability of hypothesis before observing the evidence.

P(B) is Marginal Probability: Probability of Evidence.

P(A | B) =

P(A | B)=Posterior

P(B | A) = Likelyhood

P(A)=Prior

P(B)= Normalizing constant or Evidence

Condition for Independence:

P(A |B)=P(B)

P(A∩B)=P(A)*P(B)

Naïve Assumptions:

Ø  All the input feature or Independent feature are independent with respect to other.

Ø  All the inputs features(Columns) are equally importance.

Types of Naive Bayes Classifiers:

1.     Multinomial Naive Bayes: Used for text classification, where features are discrete and represent the frequency of terms in a document. It's often used for tasks like document classification and spam detection.

2.     Gaussian Naive Bayes: Assumes that features follow a Gaussian (normal) distribution. It's suitable for continuous-valued features.

3.     Bernoulli Naive Bayes: Applicable when features are binary (0/1), representing the presence or absence of specific attributes. It's often used in document classification, especially for binary feature data.

Advantages of Naive Bayes:

• Simple and easy to implement.
• Efficient and computationally inexpensive.
• Works well with high-dimensional data.
• Can handle large datasets.

Limitations of Naive Bayes:

• Assumes feature independence, which may not hold in all cases.
• Can be sensitive to irrelevant features.
• Requires a relatively large amount of training data for accurate results.
• May not perform well with imbalanced datasets.

How Exactly Naïve Bayes Work ?

1.     We will find Probability of every class given features.

P(Ci | x1,x2,x3……..xn)

2.     We will assign a class to a record which has the highest probability

Output= argmax P(Ci | x1,x2,x3……..xn)

Argmax—>which class got the highest probability