Bayes Theorem Formula

8 February, 2024 Tạ Ngọc Châu Math Formulas 0

5/5 - (1 vote)

Bayes’ Theorem

A mathematical formula used to determine the conditional probability of events

What is the Bayes’ Theorem?

In statistics and probability theory, the Bayes’ theorem (also known as the Bayes’ rule) is a mathematical formula used to determine the conditional probability of events. Essentially, the Bayes’ theorem describes the probability of an event based on prior knowledge of the conditions that might be relevant to the event.

The theorem is named after English statistician, Thomas Bayes, who discovered the formula in 1763. It is considered the foundation of the special statistical inference approach called the Bayes’ inference.

Besides statistics, the Bayes’ theorem is also used in various disciplines, with medicine and pharmacology as the most notable examples. In addition, the theorem is commonly employed in different fields of finance. Some of the applications include but are not limited to, modeling the risk of lending money to borrowers or forecasting the probability of the success of an investment.

Formula for Bayes’ Theorem

The Bayes’ theorem is expressed in the following formula:

Where:

  • P(A|B) – the probability of event A occurring, given event B has occurred
  • P(B|A) – the probability of event B occurring, given event A has occurred
  • P(A) – the probability of event A
  • P(B) – the probability of event B

Note that events A and B are independent events (i.e., the probability of the outcome of event A does not depend on the probability of the outcome of event B).

A special case of the Bayes’ theorem is when event A is a binary variable. In such a case, the theorem is expressed in the following way:

Where:

  • P(B|A–) – the probability of event B occurring given that event A– has occurred
  • P(B|A+) – the probability of event B occurring given that event A+ has occurred

In the special case above, events A– and A+ are mutually exclusive outcomes of event A.

Bayes Theorem Statement

Let E1, E2,…, En be a set of events associated with a sample space S, where all the events E1, E2,…, En have nonzero probability of occurrence and they form a partition of S. Let A be any event associated with S, then according to Bayes theorem,

for any k = 1, 2, 3, …., n

Bayes Theorem Proof

According to the conditional probability formula,

Using the multiplication rule of probability,

Using total probability theorem,

Putting the values from equations (2) and (3) in equation 1, we get

Note:

The following terminologies are also used when the Bayes theorem is applied:

Hypotheses: The events E1, E2,… En is called the hypotheses

Priori Probability: The probability P(Ei) is considered as the priori probability of hypothesis Ei

Posteriori Probability: The probability P(Ei|A) is considered as the posteriori probability of hypothesis Ei
Bayes’ theorem is also called the formula for the probability of “causes”. Since the Ei‘s are a partition of the sample space S, one and only one of the events Ei occurs (i.e. one of the events Ei must occur and the only one can occur). Hence, the above formula gives us the probability of a particular Ei (i.e. a “Cause”), given that the event A has occurred.

Bayes Theorem Derivation

Bayes Theorem can be derived for events and random variables separately using the definition of conditional probability and density.

From the definition of conditional probability, Bayes theorem can be derived for events as given below:

Here, the joint probability P(A ⋂ B) of both events A and B being true such that,

Similarly, from the definition of conditional density, Bayes theorem can be derived for two continuous random variables namely X and Y as given below:

Example of Bayes’ Theorem

Imagine you are a financial analyst at an investment bank. According to your research of publicly-traded companies, 60% of the companies that increased their share price by more than 5% in the last three years replaced their CEOs during the period.

At the same time, only 35% of the companies that did not increase their share price by more than 5% in the same period replaced their CEOs. Knowing that the probability that the stock prices grow by more than 5% is 4%, find the probability that the shares of a company that fires its CEO will increase by more than 5%.

Before finding the probabilities, you must first define the notation of the probabilities.

  • P(A) – the probability that the stock price increases by 5%
  • P(B) – the probability that the CEO is replaced
  • P(A|B) – the probability of the stock price increases by 5% given that the CEO has been replaced
  • P(B|A) – the probability of the CEO replacement given the stock price has increased by 5%.

Using the Bayes’ theorem, we can find the required probability:

Thus, the probability that the shares of a company that replaces its CEO will grow by more than 5% is 6.67%.

Examples and Solutions

Some illustrations will improve the understanding of the concept.

Example 1:

A bag I contains 4 white and 6 black balls while another Bag II contains 4 white and 3 black balls. One ball is drawn at random from one of the bags, and it is found to be black. Find the probability that it was drawn from Bag I.

Solution:

Let E1 be the event of choosing bag I, E2 the event of choosing bag II, and A be the event of drawing a black ball.

Then,

Also, P(A|E1) = P(drawing a black ball from Bag I) = 6/10 = 3/5

P(A|E2) = P(drawing a black ball from Bag II) = 3/7

By using Bayes’ theorem, the probability of drawing a black ball from bag I out of two bags,

Example 2:

A man is known to speak the truth 2 out of 3 times. He throws a die and reports that the number obtained is a four. Find the probability that the number obtained is actually a four.

Solution:

Let A be the event that the man reports that number four is obtained.

Let E1 be the event that four is obtained and E2 be its complementary event.

Then, P(E1) = Probability that four occurs = 1/6.

P(E2) = Probability that four does not occur = 1- P(E1) = 1 – (1/6) = 5/6.

Also, P(A|E1)= Probability that man reports four and it is actually a four = 2/3

P(A|E2) = Probability that man reports four and it is not a four = 1/3.

By using Bayes’ theorem, probability that number obtained is actually a four, P(E1|A)

Bayes Theorem Applications

One of the many applications of Bayes’ theorem is Bayesian inference, a particular approach to statistical inference. Bayesian inference has found application in various activities, including medicine, science, philosophy, engineering, sports, law, etc. For example, we can use Bayes’ theorem to define the accuracy of medical test results by considering how likely any given person is to have a disease and the test’s overall accuracy. Bayes’ theorem relies on consolidating prior probability distributions to generate posterior probabilities. In Bayesian statistical inference, prior probability is the probability of an event before new data is collected.

Practice Problems

Solve the following problems using Bayes Theorem.

  1. A bag contains 5 red and 5 black balls. A ball is drawn at random, its colour is noted, and again the ball is returned to the bag. Also, 2 additional balls of the colour drawn are put in the bag. After that, the ball is drawn at random from the bag. What is the probability that the second ball drawn from the bag is red?
  2. Of the students in the college, 60% of the students reside in the hostel and 40% of the students are day scholars. Previous year results report that 30% of all students who stay in the hostel scored A Grade and 20% of day scholars scored A grade. At the end of the year, one student is chosen at random and found that he/she has an A grade. What is the probability that the student is a hosteler? 
  3. From the pack of 52 cards, one card is lost. From the remaining cards of a pack, two cards are drawn and both are found to be diamond cards. What is the probability that the lost card is a diamond?

Frequently Asked Questions on Bayes Theorem

Q1

What is meant by Bayes theorem in probability?

In Probability, Bayes theorem is a mathematical formula, which is used to determine the conditional probability of the given event. Conditional probability is defined as the likelihood that an event will occur, based on the occurrence of a previous outcome.

Q2

How is Bayes theorem different from conditional probability?

As we know, Bayes theorem defines the probability of an event based on the prior knowledge of the conditions related to the event. In case, if we know the conditional probability, we can easily find the reverse probabilities using the Bayes theorem.

Q3

When can we use Bayes theorem?

Bayes theorem is used to find the reverse probabilities if we know the conditional probability of an event.

Q4

What is the formula for Bayes theorem?

The formula for Bayes theorem is:
P(A|B)= [P(B|A). P(A)]/P(B)
Where P(A) and P(B) are the probabilities of events A and B.
P(A|B) is the probability of event A given B
P(B|A) is the probability of event B given A.

Q5

Where can we use Bayes theorem?

Bayes rule can be used in the condition while answering the probabilistic queries conditioned on the piece of evidence.

What Is the History of Bayes’ Theorem?

The theorem was discovered among the papers of the English Presbyterian minister and mathematician Thomas Bayes and published posthumously by being read to the Royal Society in 1764.1 Long ignored in favor of Boolean calculations, advances in calculation capacity have led to an increase in applications using Bayes’ theorem. It is now applied to a wide variety of probability calculations, including financial calculations, genetics, drug use, and disease control.

What Does Bayes’ Theorem State?

Bayes’ Theorem states that the conditional probability of an event, based on the occurrence of another event, is equal to the likelihood of the second event given the first event multiplied by the probability of the first event.

What Is Calculated in Bayes’ Theorem?

Bayes’ Theorem calculates the conditional probability of an event, based on the values of specific related known probabilities.

What Is a Bayes’ Theorem Calculator?

A Bayes’ Theorem Calculator figures the probability of Event A conditional on Event B, given the prior probabilities of A and B, and the probability of B conditional on A. It calculates conditional probabilities based on known probabilities.

How Is Bayes’ Theorem Used in Machine Learning?

Bayes Theorem provides a useful method for thinking about the relationship between a data set and a probability. In other words, the theorem says that the probability of a given hypothesis being true based on specific observed data can be stated as finding the probability of observing the data given the hypothesis multiplied by the probability of the hypothesis being true regardless of the data, divided by the probability of observing the data regardless of the hypothesis.