Day - 2 Probability in Machine Learning

Certainly! Probability is a fundamental concept in machine learning that helps us understand and model uncertainty. It provides a mathematical framework to quantify the likelihood of events or outcomes occurring.

Here are some key concepts in probability that are relevant to machine learning :

What is Probability?

Probability can be defined as the ratio of the number of favorable outcomes to the total number of outcomes of an event.

For an experiment having 'n' number of outcomes, the number of favorable outcomes can be denoted by x. The formula to calculate the probability of an event is as follows.

Probability(Event) = Favorable Outcomes/Total Outcomes = x/n

Probability is used to predict the outcomes for the tossing of coins, rolling of dice, or drawing a card from a pack of playing cards.

Terminology of Probability Theory

The following terms in probability theorey help in a better understanding of the concepts of probability.

• Experiment : A trial or an operation conducted to produce an outcome is called an experiment.
• Sample Space : All the possible outcomes of an experiment together constitute a sample space.

For example, the sample space of tossing a coin is {head, tail}.

• Favorable Outcome : An event that has produced the desired result or expected event is called a favorable outcome.

For example, when we roll two dice, the possible/favorable outcomes of getting the sum of numbers on the two dice as 4 are (1,3), (2,2), and (3,1).

• Trial : A trial denotes doing a random experiment.
• Random Experiment : An experiment that has a well-defined set of outcomes is called a random experiment.

For example, when we toss a coin, we know that we would get ahead or tail, but we are not sure which one will appear.

• Event : The total number of outcomes of a random experiment is called an event.
• Equally Likely Events : Events that have the same chances or probability of occurring are called equally likely events. The outcome of one event is independent of the other.

For example, when we toss a coin, there are equal chances of getting a head or a tail.

• Exhaustive Events : When the set of all outcomes of an event is equal to the sample space, we call it an exhaustive event.
• Mutually Exclusive Events : Events that cannot happen simultaneously are called mutually exclusive events.

For example, the climate can be either hot or cold. We cannot experience the same weather simultaneously.

• Independent Events : Events that are not affected by other events are independent events.
• Dependent Events : Events that are affected by other events are known as dependent events.

Events in Probability

In probability theory, an event is a set of outcomes of an experiment or a subset of the sample space. If P(E) represents the probability of an event E, then, we have,

• P(E) = 0 if and only if E is an impossible event
• P(E) = 1 if and only if E is a certain event
• 0 ≤ P(E) ≤ 1

Suppose, we are given two events, "A" and "B", then the probability of event A, P(A) > P(B) if and only if event "A" is more likely to occur than the event "B". Sample space(S) is the set of all of the possible outcomes of an experiment and n(S) represents the number of outcomes in the sample space.

P(E) = n(E)/n(S)

P(E’) = 1 - (n(E)/n(S))

E’ represents that the event will not occur.

Therefore, now we can also conclude that, P(E) + P(E’) = 1

Probability Formula

The probability equation defines the likelihood of the happening of an event. It is the ratio of favorable outcomes to the total favorable outcomes. The probability formula can be expressed as,

i.e., P(A) = n(A)/n(S)

where,

• P(A) is the probability of an event 'B'.
• n(A) is the number of favorable outcomes of an event 'B'.
• n(S) is the total number of events occurring in a sample space.

Probability Theorems

The following theorems of probability are helpful to understand the applications of probability and also perform the numerous calculations involving probability.

Theorem 1 : The sum of the probability of happening of an event and not happening of an event is equal to 1. P(A) + P(A') = 1.

Theorem 2 : The probability of an impossible event or the probability of an event not happening is always equal to 0. P(ϕ) = 0.

Theorem 3 : The probability of a sure event is always equal to 1. P(A) = 1

Theorem 4 : The probability of happening of any event always lies between 0 and 1. 0 < P(A) < 1

Theorem 5 : If there are two events A and B, we can apply the formula of the union of two sets and we can derive the formula for the probability of happening of event A or event B as follows.

P(A∪B) = P(A) + P(B) - P(A∩B)

Also for two mutually exclusive events A and B, we have P( A U B) = P(A) + P(B).

Different Probability Formulas

Probability formula with addition rule : Whenever an event is the union of two other events, say A and B, then

• P(A or B) = P(A) + P(B) - P(A∩B)
• P(A ∪ B) = P(A) + P(B) - P(A∩B)

Probability formula with the complementary rule : Whenever an event is the complement of another event, specifically, if A is an event, then P(not A) = 1

• P(A) or P(A') = 1 - P(A)
• P(A) + P(A′) = 1

Probability formula with the conditional rule : When event A is already known to have occurred, the probability of event B is known as conditional probability and is given by:

• P(A∣B) = P(A∩B)/P(B)

Probability formula with the multiplication rule : Whenever an event is the intersection of two other events, that is, events A and B need to occur simultaneously. Then

• P(A ∩ B) = P(A)⋅P(B) (in case of independent events)
• P(A∩B) = P(A)⋅P(B∣A) (in case of dependent events)