Day - 5 Probability in Machine Learning

Today we are going to discuss about 2 very important Distribution in Probability and Statistics :

1. Binomial Distribution
2. Bernoulli Distribution

Binomial Distribution

In statistics and probability theory, the Binomial Distribution is the probability distribution that is discrete and applicable to events having only two possible results in an experiment, either success or failure. (the prefix “bi” means two, or twice).

What Is Binomial Distribution?

The binomial distribution is the probability distribution of a binomial random variable. A random variable is a real-valued function whose domain is the sample space of a random experiment.

Let us consider an example to understand this better :

Tossing a fair coin twice. This is a binomial experiment. There are 4 possible outcomes of this experiment. {HH, HT, TH, TT}.

Consider getting one head as the success. Count the number of successes in each possible outcome.

Here n(getting heads) is the success in n repeated trials of a binomial experiment.. n(X) = 0, 1, or 2 is the binomial random variable. The distribution of probability is of a binomial random variable, and this is known as a binomial distribution.

No. of heads(n(X)) : 0, 1, 2

Probability of getting a head :

P(x = 0) = 1/4 = 0.251

P(x = 1) = P(HT) = 1/4 + 1/4 = 0.502

P(x = 2) = P(HH) = 1/4 = 0.25

The binomial distribution represents the probability for 'x' successes of an experiment in 'n' trials, given a success probability 'p' for each trial at the experiment. Two parameters n and p are used here in the binomial distribution. The variable ‘n’ represents the number of trials and the variable ‘p’ states the probability of any one(each) outcome.

Binomial Distribution Formula

The binomial distribution formula is for any random variable X, given by :

P(x,n,p) = nCx p^x (1-p)^n-x  or

P(x,n,p) = nCx p^x (q)^n-x

where p is the probability of success, q is the probability of failure, and n = number of trials.

Binomial Distribution Mean, Variance and Standard Distribution

For a binomial distribution, the mean, variance and standard deviation for the given number of success are represented using the formulas :

• Mean, μ = np
• Variance, σ2 = npq
• Standard Deviation σ= √(npq)

where p is the probability of success q is the probability of failure, where q = 1-p

Properties of Binomial Distribution

The properties of the binomial distribution are :

• There are only two distinct possible outcomes: true/false, success/failure, yes/no.
• There is a fixed number of 'n' times repeated trials in a given experiment.
• The probability of success or failure remains constant for each attempt/trial.
• Only the successful attempts are calculated out of 'n' independent trials.
• Every trial is an independent trial on its own, this means that the outcome of one trial has no effect on the outcome of another trial.

Bernoulli Distribution

Bernoulli Distribution is a type of discrete probability distribution where every experiment conducted asks a question that can be answered only in yes or no. In other words, the random variable can be 1 with a probability p or it can be 0 with a probability (1 - p). Such an experiment is called a Bernoulli trial.

Note : If we have a Binomial Distribution where n = 1 then it becomes a Bernoulli Distribution.

What is Bernoulli Distribution?

A random experiment that can only have an outcome of either 1 or 0 is known as a Bernoulli trial. Such an experiment is used in a Bernoulli distribution.

If in a Bernoulli trial the random variable takes on the value of 1, it means that this is a success. The probability of success is given by p. Similarly, if the value of the random variable is 0, it indicates failure. The probability of failure is q or 1 - p.

Bernoulli Distribution Formula

The Bernoulli distribution is characterized by a single parameter, usually denoted as p, which represents the probability of success in a single trial. The probability mass function (PMF) of the Bernoulli distribution is given by :

P(X = k) = p^k * (1 - p)^(1 - k)

where X is a random variable representing the outcome of a trial, k is the value of the random variable (either 0 or 1), and p is the probability of success.

Bernoulli Distribution Mean, Variance and Standard Distribution

The mean or expected value of a Bernoulli random variable is given by E(X) = p

The variance is given by Var(X) = p * (1 - p)

The standard deviation is given by SD = √(p * (1 - p))

where p is the probability of success in a single trial.

Important Notes on Bernoulli Distribution

• Bernoulli distribution is a discrete probability distribution where the Bernoulli random variable can have only 0 or 1 as the outcome.
• p is the probability of success and 1 - p is the probability of failure.
• The mean of a Bernoulli distribution is E[X] = p and the variance, Var[X] = p(1-p).
• Bernoulli distribution is a special case of binomial distribution when only 1 trial is conducted.