Probability

In this section students will:

  1. Identify the sample space of a probability of event
  2. Calculate basic probabilities
  3. Use probability notation

Terminology

Probability is a measure that is associated with how certain we are of outcomes of a particular experiment or activity.

An experiment is a planned operation carried out under controlled conditions. If the result is not predetermined, then the experiment is said to be a chance experiment.

Flipping one fair coin twice is an example of an experiment.

A result of an experiment is called an outcome. The sample space of an experiment is the set of all possible outcomes.

The uppercase letter \(S\) is used to denote the sample space.

For example, if you flip one fair coin, \(S={H,T}\) where \(H=\) heads and \(T=\) tails are the outcomes.
An event is any combination of outcomes. Upper case letters like \(A\) and \(B\) represent events.

For example, if the experiment is to flip one fair coin, event \(A\) might be getting at most one head.

The probability of an event \(A\) is written \(P(A)\).

The probability of any outcome is the **long-term relative frequency* of that outcome.

Probabilities are between zero and one, inclusive (that is, zero and one and all numbers between these values).

\(P(A)=0\) means the event \(A\) can never happen.
\(P(A)=1\) means the event \(A\) always happens.
\(P(A)=0.5\) means the event \(A\) is equally likely to occur or not to occur.

Equally likely means that each outcome of an experiment occurs with equal probability.

If you toss a fair coin, a Head (H) and a Tail (T) are equally likely to occur.

To calculate the probability of an event \(A\) when all outcomes in the sample space are equally likely, count the number of outcomes for event \(A\) and divide by the total number of outcomes in the sample space. \[P(A)=\frac{number~of~outcomes}{n}=\frac Xn\]

For example, if you toss a fair dime and a fair nickel, the sample space is {HH, TH, HT, TT} where T = tails and H = heads.

The sample space has four outcomes. Let \(A\) = getting one head.
There are two outcomes that meet this condition {HT, TH}, so \(P(A)=2/4=0.5\)

Law of Large Numbers

An important characteristic of probability experiments is known as the law of large numbers which states that as the number of repetitions of an experiment is increased, the relative frequency obtained in the experiment tends to become closer and closer to the theoretical probability.

Even though the outcomes do not happen according to any set pattern or order, overall, the long-term observed relative frequency will approach the theoretical probability. (The word empirical is often used instead of the word observed.)

LLN

Building a function to generate a sequence of successes and failures. The function will return with a vector of 0s and 1s (0=failure, 1=success) based on input of a sample size and probability of success. The idea is to use the function to simulate samples of size 5, then size 6, size 7, and so on until maybe 500. For each block, the mean is calculated and look at the variability in the means decreasing.

Simulation of LLN

 [1] 1 0 1 1 1 1 1 0 1 1

Plot of simulation

LLN

You can see in the graph, as we take more and more samples, the variation is reducing and the probability will converge to a single number (the theoretical probability).

In practice, an event probability might never actually be known but rather estimated from many trials.

Unions (“Or” events) and Intersections (“And” events)

“OR” Event:
An outcome is in the event A OR B if the outcome is in A or is in B or is in both A and B.

For example, let \(A = {1,2,3,4,5}\) and \(B={4,5,6,7,8}\). \(A~OR~B={1,2,3,4,5,6,7,8}\). Notice that 4 and 5 are NOT listed twice.

“AND” Event:
An outcome is in the event A AND B if the outcome is in both A and B at the same time.

For example, let \(A={1,2,3,4,5}\) and \(B={4,5,6,7,8}\). Then \(A~AND~B={4,5}\).

Complement

(Not compliment, but thanks anyways…)

The complement of event \(A\) is denoted \(A'\) (read “A prime”).

\(A'\) consists of all outcomes that are NOT in \(A\).

Notice that \(P(A)+P(A')=1\)

For example, let \(S={1,2,3,4,5,6}\) and let \(A={1,2,3,4}\). Then, \(A'={5,6}\).

Example of flipping two fair coins

When you flip a fair coin (one that is not weighted so that one side is more likely to come up than the other), the chance that it is a head is 50% and the chance that it is a tail is 50%; either side is equally likely.

List out all possible combinations

2 heads: (H,H); 1 head: (H,T), (T,H); 0 heads: (T,T)
\(P(H)=0.5,P(T)=0.5\): flipping coins are independent events

2 heads: \(P(H,H)=P(H1^{st} \cap H2^{nd})=P(H1)P(H2)=\left(\frac12\right)\left(\frac12\right)=\frac14\)

1 head: \(P(H,T)=P[(H1^{st} \cap T2^{nd})\cup (T1^{st} \cap H2^{nd})]\) \(=P(H1)P(T2)+P(T1)P(H2)=\left(\frac12\right)^2+\left(\frac12\right)^2\) \(=\left(\frac12\right)\)

0 heads: \(P(T,T)=P(T1^{st} \cap T2^{nd})=P(T1)P(T2)=\left(\frac12\right)^2=\frac14\)

Probability distribution of number of heads

  Probability table coins

Find the probabilities of the following defined events:

  1. Let \(A=\) at most one tail
  2. Let \(B=\) all tails
  3. Let \(D=\) more than one tail
  4. Let \(E=\) at least one tail in two flips

\[P(A)=P(0~or~1~tail)=P(0~tails)+P(1~tail)=(0.5)^2+(0.5)=0.75\]
\[P(B)=P(all~tails)=P(T~and~T)=P(T)P(T)=(0.5)^2=0.25\]
\[P(D)=P(more~than~one~tail)=P(2~tails)=(0.5)^2=0.25\]
\[P(E)=P(at~least~one~tail)=1-P(no~tails)=1-(0.5)^2=1-0.25=0.75\]

Independent and Mutually Exclusive Events

In this section students will:

  1. Determine if two events are mutually exclusive
  2. Determine if two events are independent
  3. Calculate conditional probability rules to calculate probabilities

Independent Events

Two events A and B are independent if the knowledge that one occurred does not affect the chance the other occurs.

For example, the outcomes of two rolls of a fair die are independent events. The outcome of the first roll does not change the probability for the outcome of the second roll.

Two events are independent if the following are true: \[P(A~AND~B)=P(A)P(B)\]
To show two events are independent, you must show if it meets the above condition or not

If two events are NOT independent, then we say that they are dependent. If it is not known whether A and B are independent or dependent, assume they are dependent until you can show otherwise.

Independence example

Let event \(A=\) learning Spanish. Let event \(B=\) learning German. Then \(A~AND~B\) = learning Spanish and German. Suppose \(P(A)=0.4\), \(P(B)=0.2\) and \(P(A~AND~B)=0.08\). Are events \(A\) and \(B\) independent? Use \(P(A~and~B)=P(A)P(B)\) to determine independence

\(0.08?=?(0.4)(0.2) \Rightarrow 0.08=0.08 \therefore\) (therefore) \(A\) and \(B\) are independent

Sampling with replacement or without replacement

Sampling may be done with replacement (SWR) or without replacement (SWOR).

With replacement: If each member of a population is replaced after it is picked, then that member has the possibility of being chosen more than once.

When sampling is done with replacement, then events are considered to be independent, meaning the result of the first pick will not change the probabilities for the second pick.

Without replacement: When sampling is done without replacement, each member of a population may be chosen only once. In this case, the probabilities for the second pick are affected by the result of the first pick.

When sampling is done without replacement, then events are considered to be dependent or not independent.

SWR or SWOR?

You have a fair, well-shuffled deck of 52 cards. It consists of four suits. The suits are clubs, diamonds, hearts and spades. There are 13 cards in each suit consisting of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, J (jack), Q (queen), K (king) of that suit. Three cards are picked at random.

  1. Suppose you know that the picked cards are Q of spades, K of hearts and Q of spades. Can you decide if the sampling was with or without replacement?
  2. Suppose you know that the picked cards are Q of spades, K of hearts, and J of spades. Can you decide if the sampling was with or without replacement?

Mutually Exclusive Events

A and B are mutually exclusive events if they cannot occur at the same time.

This means that A and B do not share any outcomes and \(P(A~AND~B)=\varnothing\) (\(P(A~AND~B)=0\); it does not exist).

If it is not known whether A and B are mutually exclusive, assume they are not until you can show otherwise.

ME example

A student goes to the library. Let events B = the student checks out a book and D = the student checks out a DVD. Suppose that \(P(B) = 0.40\), \(P(D) = 0.30\) and \(P(B~AND~D)=0.20\).
Are B and D independent? Are B and D mutually exclusive?

Basic Rules of Probability

In this section students will:

  1. Use the multiplication rule for calculate probabilities
  2. Use the addition rules to calculate probabilities

Multiplication Rule

Multiplication rule ( THIS RULE IS FOR INDEPENDENT EVENTS ONLY!!!)

If two events, \(A\) and \(B\) are independent, the intersection of \(A\) and \(B\) can be calculated as:

\[P(A\cap B)=P(A)P(B)\]
This equation can also be used to prove or disprove independence

Again, it cannot be assumed, you have to either be told they are independent or prove it mathematically with the above formula.

Addition Rule

Addition rule
The probability of the union of \(A\) and \(B\) is:
\[P(A \cup B)=P(A)+P(B)-P(A\cap B)\]
This formula can be modified if the union of one or both complements is required; if the union is given, then the intersection can be solved for. If two events are mutually exclusive, then the intersection is an empty set (it does not exist), the intersection is the sum of the probabilities of the two events. \(P(A \cup B)=P(A)+P(B)\)

Last Major Concepts

For a probability to be valid, probabilities must be between 0 and 1 (0% and 100%): \(0\leq P(A)\leq1\)

The other concept/rule is that the sum of the probabilities for an experiment must sum to 1 (the sample space): \(\sum P(A_i)=1=S\)

Rules recap

  1. Valid probability rule: \(0\leq P(A)\leq1\)
  2. Total probability rule: \(\sum P(A_i)=1=S\)
  3. Complement rule: \(P(A')=1-P(A)\)
  4. Addition rule: \(P(A \cup B)=P(A)+P(B)-P(A\cap B)\)
  5. Multiplication rule: \(P(A\cap B)=P(A)P(B)\) (independent events only)

Pay attention to the formulas. They can be modified for finding complements, as well as solving for unknown values with a bit of algebra

Confusion Matrix

One way to help calculating probabilities of intersections, many time for use in other calculations

  1. Create a 2X2 table, including row and column headers and totals
  2. Label the rows with \(P(A),P(A')\) and the columns with \(P(B),P(B')\)
  3. The row totals are the values of \(P(A),P(A')\) and the columns totals are \(P(B),P(B')\)
  4. The grand total (bottom right corner cell) is always 1 (\(\sum p(x_i)=1\))
  5. The upper left cell of the matrix (table) is \(P(A\cap B)\); it has to be given or calculated
  6. Solve for the other 3 cells of the matrix by way of row and column totals

Matrix setup

  Confusion matrix ## Matrix example

Suppose that \(P(A)=0.5\), \(P(B)=0.3\), \(P(A \cap B)=0.2\)

Confusion matrix example
Confusion matrix example

\(P(A \cap B')=0.5-0.2=0.3\)
\(P(A'\cap B)=0.3-0.2=0.1\)
\(P(A'\cap B')=0.5-0.1=0.4\) or \(=0.7-0.3=0.4\)

More matrix

Are \(A\) and \(B\) independent? Use \(P(A\cap B)=P(A)P(B)\) to prove it.

\[P(A\cap B)?=?P(A)P(B)\Rightarrow 0.2\neq(0.5)(0.5)\]

Since the statement is not true, events \(A\) and \(B\) are not independent (they are dependent).

Contingency Tables

In this section students will:

  1. Calculate basic probabilities using contingency tables

A contingency table provides a way of portraying data that can facilitate calculating probabilities.

The table helps in determining conditional probabilities quite easily.

The table displays sample values in relation to two different variables that may be dependent or contingent on one another.

Table displays sample values in relation to two different variables that may be dependent (contingent) on one another. You can display with counts or relative frequencies (probabilities)

Cell counts or probabilities are intersections, row and column totals are the individual counts or probabilities

Refer to the confusion matrix, it is the same thing. The inside cells of the table are intersections between rows and columns, with the row and column totals being the single events (\(A\), \(B\), etc.)

A basic probability, where all events have an equal chance of happening (equal probability of occurrence), is

\[P(X)=\frac{events~of~interest}{n}=\frac{X}{n}\]

Table of counts

The following table shows a random sample of 100 hikers and the areas of hiking they prefer.

 

Hiking survey results
Hiking survey results

Counts table example

\[P(male)=\frac{55}{100}=0.55 \text{ , }P(female)=\frac{45}{100}=0.45\] \[P(lakes')=\frac{34+25}{100}=0.59\]
\[P(male \cap coastine)=\frac{16}{100}=0.16\]

Venn diagram I

A Venn diagram is a graph (picture) that represents the events of an experiment. It usually is a box that represents the sample space \(S\) and has circles that represent the events (\(A\), \(B\), etc.)

Beethoven
Beethoven

Venn diagram II

This one shows two events, \(A\) and \(B\) with an intersection.

\[P(A)=0.5,~P(B)=0.5,~P(A\cap B)=0.2\]

Venn diagram 1

Venn diagram III

This one shows two events, \(A\) and \(B\) for disjoint/mutually exclusive events.

\[P(A)=0.45,~P(B)=0.35,~P(A\cap B)=\varnothing\]

Venn diagram 2