PSM1_001 Introduction and Probability

From (Walpole et al., 2017):

Probability

Sample Space

In the study of statistics, we are concerned basically with presentation and interpretation of chance outcomes that occur in a planned study or scientific investigation. The statistician is often dealing with either numerical data, representing counts or measurements, or categorical data, which can be classified according to some criterion.

Definition:

The set of all possible outcomes of a statistical experiment is called the sample space and is represented by symbol .

Each outcome in a sample space is called an element or a member of the sample space, or simply a sample point. If the sample space has a finite number of elements, we may list the members separated by commas and enclosed in braces. Thus, the sample space , of all possible outcomes when a coin is flipped, may be written

where and correspond to heads and tails, respectively.

Example:

Consider the experiment of tossing a die. If we are interested in the number that shows on the top face, the sample face is

If we are interested only in whether the number is even or odd, the sample space is simply

Events

For any given experiment, we may be interested in the occurrence of certain events rather than in the occurrence of a specific element in the sample space. For instance, we may be interested in the event that the outcome when a die is tossed is divisible by . This will occur if the outcome is an element of the subset of the sample space is the example above.
To each event we assign a collection of sample points, which constitute a subset of the sample space. That subset represents all of the elements for which the event is true.

Definition:

An event is subset of a sample space.

It is conceivable that an event may be a subset that includes the entire sample space or a subset of called the null set and denoted by the symbol , which contains no elements at all. For example, if

then must be the null set, since the only possible factors of are the odd numbers and .

Various Set Operations

Definition:

The complement of an event with respect to is the subset of all elements of that are not in . We denote the complement of by the symbol .

Example: be the event that a red card is selected from an ordinary deck of playing cards, and let be the entire deck. Then is the event that the card selected from the deck is not a red card but a black card.

Let

Definition:

The Intersection of the two events and , denoted by the symbol , is the event containing all elements that are common to and .

Definition:

Two events and are mutually exclusive, or disjoint, if , that is, if and have no elements in common.

Definition:

The union of two events and , denoted by the symbol , is the event containing all the elements that belong to or or both.

The relationship between events and the corresponding sample space can be illustrated graphically by means of Venn diagrams. In a Venn diagram we let the sample space a rectangle and represent events by circles drawn inside the rectangle.

Events represented by various regions. (Walpole et al., 2017).

Thus, in the figure above, we see that:

Some useful rules:
1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

Counting Sample Points

One of the problems that the statistician must consider and attempt to evaluate is the element of chance associated with the occurrence of certain events when an experiment is performed. These problems belong in the field of probability. In many cases, we shall be able to solve a probability problem by counting the number of points in the sample space without actually listing each element. The fundamental principle of counting, often referred to as the multiplication rule, is the following:

Theorem:

If an operation can be performed in ways, and if for each of these ways a second operation can be performed in ways, then the two operations can be performed together in ways.

This rule can be extended to cover any number of operations. Supposed for instance, that a customer wishes to buy a new cell phone and can choose from brands, sets of capability, and colors. These three classifications result in different ways for a customer to order one of these phones. The generalized multiplication rule covering operations is stated in the following.

Theorem:

If an operation can be performed in ways, and if for each of these a second operation can be performed in ways, and for each of the first two a third operation can be performed in ways, and so forth, then the sequence of operations can be performed in ways.

Example:

How many even four-digit numbers can be formed from the digits and if each digit can be used only once?
Solution:
Since the number must be even, we have only choices for the units position. However, for a four-digit number the thousands position cannot be . Hence, we consider the units position in two parts, or not . If the units position is (i.e., ), we have choices for the thousands position, for the hundreds position, and for the tens position. Therefore, in this case we have a total of

even four-digit numbers. On the other hand, if the units position is not (i.e., ), we have choices for the thousands position, for the hundreds position, for the tens position. Therefore, in this case we have a total of

even four-digit numbers.
Since the above two cases are mutually exclusive, the total number of even four-digit numbers can be calculated as

Permutation

Definition:

A permutation is an arrangement of all or part of a set of objects.

Consider the three letters and . The possible permutations are and . Thus, we see that there are distinct arrangements. There are choices for the first position. No matter which letter is chosen, there are always choices for the second position. No matter which two letters are chosen for the first two positions, there is only choice for the last position, giving a total of:

Or simply:

Theorem:

The number of permutations of objects is .

Now consider the number of permutations that are possible by taking two letters at a time from four. These would be and . Using the rule above, we have two positions to fill, with choices for the first and then choices for the second, for a total of

In general, distinct objects taken at a time can be arranged in

ways. We represent this product by the symbol

As a result, we have the following theorem:

Theorem:

The number of permutations of distinct object taken at a time is

Example:

In one year, three awards (research, teaching, and service) will be given to a class of graduate students in a statistics department. If each student can receive at most one award, how many possible selection are there?

Solution:
Since the awards are distinguishable, it is a permutation problem. The total number of sample points is

Permutations that occur by arranging object in a circle are called circular permutations. Two circular permutations are not considered different unless corresponding objects in the two arrangements are preceded or followed by a different object as we proceed in a clockwise direction. For example, if people are playing bridge, we do not have a new permutation if there all move one position in a clockwise direction. By considering one person in a fixed position and arranging the other three in ways, we find there are distinct arrangements for the bridge game.

Theorem:

The number of permutations of objects arranged in a circle is

So far we have considered permutations of distinct objects. That is, all the objects were completely different or distinguishable. Obviously, if the letters and are both equal to , then the permutations of the letters , , and are both equal to , then the permutations of the letters , , and become , , , , and , of which only are distinct. Therefore, with letters, being the same, we have distinct permutations. With different letters , , and , we have distinct permutations. If we let and , we just list only the following distinct permutations: , , , , and . Thus we have distinct permutations.

Theorem:

The number of distinct permutations of things of which are of one kind, of a second kind, , are of a -th kind is

Combination

In many problems, we are interested in the number of ways of selecting objects from without regard to order. These selections are called combinations.

Theorem:

The number of combinations of distinct objects taken at a time is

Note:

Probability of an Event

Definition:

The probability of an event is the sum of the weights of all sample points in . Therefore,

Furthermore, if is a sequence of mutually exclusive events, then

Example:

A coin is tossed twice. What is the probability that at least head occurs?
Solution:
The space for this experiment is

If the coin is balanced, each of these outcomes is equally likely to occur. Therefore, we assign a probability of to each sample point. Then , or . If represents the event of at least head occurring, then

so

Some important rules:
1.

2.

Additive Rules

Often it is easiest to calculate the probability of some event from known probabilities of other events. This may well be true if the event in question can be represented as the union of two other events or as the complement of some event. Several important laws that frequently simplify the computation of probabilities follow. The first, called the additive rule, applies to union of events.

Theorem:

If and are two events, then

Additive rule of probability. (Walpole et al., 2017).

Theorem:

If and are disjoint, then

Which we can generalize:

Theorem:

If are mutually exclusive (disjoint), then

Tip:

When in doubt, consider a Venn diagram.

Example:

How do employees commute to work? (defined as “use at least one a week”), given:

What are the probabilities for “not car”, “train or car”, “bike and not car”?
Solution:
Using the notation

we get:

For the third case, we know that:

So we can simple rearrange:

Conditional Probability, Independence, and the Product Rule

Conditional Probability

The probability of an event occurring when it is known that some event has occurred is called a conditional probability and is denoted by . The symbol is usually read “the probability that occurs given that occurs”, or simply “the probability of , given ”.

Consider the event of getting a perfect square when a die is tossed. The die is constructed so that the even numbers are twice as likely to occur as the odd numbers. Based on sample space , with probabilities of and assigned, respectively, to the odd and even numbers, the probability of occurring is . Now supposed that it is known that the toss of the die resulted in a number greater than . We are now dealing with a reduced sample space , which is a subset of . To find the probability that occurs, relative to the space , we must first assign new probabilities to the elements of proportional to their original probabilities such that their sum is . Assigning a probability of to the odd numbers in and a probability of to the two even numbers, we have , or . Relative to the space , we find that contains the single element . Denoting this event by the symbol , we write , and hence

This example illustrates that events may have different probabilities when considered relative to different sample spaces.
We can also write

Definition:

The conditional probability of , given , denoted by , is defined by:

Another rule:
1.

Independent Events

In the die tossing experiment we note that whereas . That is, , indicating that depends on . Now consider an experiment in which cards are drawn in succession from an ordinary deck, with replacement. The events are defined as

• - the first card is an ace,
• - the second card is a spade.

Since the first card is replaced, our sample space for both the first and the second draw consists of cards, containing aces and spades. Hence:

That is, . When this is true, the events and are said to be independent.
Although conditional probability allows for an alteration of the probability of an event in the light additional material, it also enables us to understand better the very important concept of independence or, in the present context, independent events.

Definition:

Two events and are independent if and only if

Assuming the existences of the conditional probabilities. Otherwise, and are dependant.

Note:

The condition implies that , and conversely.

The Multiplicative Rule

Multiplying the definition of the conditional probability above by , we obtain the following important multiplicative rule (or product rule), which enables us to calculate the probability that two events will both occur.

Theorem:

If in an experiment the events and can both occur, then

Therefore, we can also write:

Example:

Suppose that we have a fuse box containing fuses, of which are defective. If fuses are selected at random and removed from the box in succession without replacing the first, what is the probability that both fuses are defective?
Solution:
We shall let be the event that the first fuse is defective and the event that second fuse is defective; then we interpret as the event that occurs and then occurs after has occurred. The probability of first removing a defective fuse is ; then the probability of removing a second defective fuse from the remaining is . Hence,

Bayesian Statistics

Bayesian statistics is a collection of tools that is used in a special form of statistical inference which applies in the analysis of experimental data in many practical situations in science and engineering. Bayes’ rule is one of the most important rules in probability theory.

Total Probability

Supposed that our sample space is the population of adults in a small town who have completed the requirements for a college degree. We shall categorize them according to gender and employment status. The data are given in the following figure:

Categorization of the Adults in a Small Town.

One of these individuals is to be selected at random for a tour throughout the country to publicize the advantages of establishing new industries in the town. We shall be concerned with the following events:

• - a man is chosen,
• - the one chosen is employed.

Suppose that we are now given the additional information that of those employed and of those unemployed are members of the Rotary Club. We wish to find the probability of the event that the individual selected is a member of the Rotary Club.

Venn diagram for the events and . (Walpole et al., 2017).

Referring to the figure, we can write as the union of the two mutually exclusive events and . Hence, , and using the rules discussed earlier we can write:

From the table we can compute that:

and:

We can also display these probabilities by means of a tree diagram:

Tree diagram for the data above.

It follows that:

A generalization of the foregoing illustration to the case where the sample space is partitioned into subsets is covered by the following theorem, sometimes called the theorem of total probability.

Theorem:

If the events constitute a partition of the sample space such that for , then for any event of ,

Partitioning the sample space . (Walpole et al., 2017).

Example:

In a certain assembly plant, three machines, and , make , and , respectively, of the products. It is known from past experience that and of the products made by each machine, respectively, are defective. Now, suppose that a finished product is randomly selected. What is the probability that it is defective?

Solutions:
Consider the following events:

• - the product is defective
• - the product is made by machine ,
• - the product is made by machine ,
• - the product is made by machine

Applying the rule of elimination, we can write

Tree diagram for the example above.

Referring to the tree diagram of the figure above, we find that the three branches give the probabilities

and hence

Bayes’ Rule

Instead of asking for in the example above, by the rule of eliminations, suppose that we now consider the problem of finding the conditional probability . In other words, supposed that a product was randomly selected and it is defective. What is the probability that this product was made by machine ? Questions of this type can be answered by using the following theorem, called Bayes’ rule:

Theorem:

If the events constitute a partition of the sample space such that for , then for any event in such that ,

Example:

A medical test has a sensitivity (i.e., detects the disease in of people who have it) and specificity (i.e., does not detect the disease in of people who do not have it).

If of the population have this disease (a prevalence), what is the probability that a person who tests positive has the disease (this is called the “positive predictive value”)?

Solution:
Let

• - positive test result
• - disease

Therefore, we know that:

So that:

We want to know what’s the probability of knowing . Using Bayes’ rule:

Exercises

Question 1

and are 2 disjoined events. The following probabilities are given:

Calculate:

Solution:

1. Using the Various Set Operations: Where because and are disjointed.
2. Using the Various Set Operations: Where because and are disjointed.

Question 2

The chance of a student to solve:

• question 1 correctly is .
• question 3 correctly is .
• both questions correctly is .

Part a

What is the chance of a student to solve at least one question correctly?

Solution:
Let’s denote:

• solving question 1 correctly.
• solving question 3 correctly.

We know that:

Solving at least one question correctly corresponds to ” or ”, which is . Therefore:

Part b

What is the chance of a student to solve none of the questions correctly?

Solution:
Solving at none of the questions correctly corresponds to “not and not ”, which is . Therefore:

Part c

What is the chance of a student to solve only one question correctly?

Solution:
Solving only one question correctly corresponds to ” xor ”, which is . Therefore:

We know that . Therefore:

Substituting known values we can see that:

Doing the same for , we can solve the problem. Daniel didn’t want to do it.

Question 3

In how many ways can we choose two people with different nationalities, given out of a group of French, 10 Italians , and 5 English?

Solution:
If there were only French and Italians, then there would simply be ways of performing the given operation. If there were only the Italians and English, it would be different ways. With only the French and the English, there are different ways. Now, simply summing it all up, there are ways for the given operation to be performed.

Question 4

Out of tenants in a common building, a committee contains representatives. How many options of representatives are there:

• With no limitation.
• If tenants cannot be together due to a conflict.

Solution:
In the case where there is no limitation, we can simple calculate using a theorem:

In the second case, the problem is a bit more complicated. We can separate it to three different situations:

1. None of special tenants are in the committee.
2. The first special tenant is in the committee.
3. The second special tenant is in the committee.

For the first situation, we simply use the same theorem again, but now we have people to choose from instead of :

For the second situation, we now already know one of tenants in the committee, so we have more to choose. We also have people to choose from, because one was already chosen, and another we can’t choose. Therefore:

The third situation is exactly the same as the second one, so we have another .
Summing all the different options, we get different options for representatives.

Question 5

Consider a bag containing red balls, 8 white and black. How many ways of arranging the balls are there, in a row, while all the black balls are together?

Solution:
The basic formula is (i.e., there are permutations). So we could say we have balls, and therefore there are permutations. Now we need to subtract from that all the cases where the black balls aren’t together. That’s a bit problematic.
Another thing we can try is to treat all the black balls as one group - just as a single ball. Doing that, we get permutations.
These permutations contain repeated elements ( reds and whites), so the number of ways to arrange these items is:

Question 6

Drinking cups are produced in a -step process: shaping and painting. Experience shows that of the cups are properly shaped.
The probability that a deformed cup is properly painted is . A properly shaped cup will be properly painted with probability.

Part a

What is the probability that a random cup produced is properly shaped but improperly painted?

Solution:
Let’s denote

• : cup is properly shaped.
• : cup is properly painted.

We are given that:

We need to find .

Probability tree for the given problem.

From the tree above, and from the the multiplication rule, we see that:

Part b

What is the probability that a cup is properly painted?

Solution:
Using the total probability rule:

Part c

If a cup is improperly painted, what is the probability it is properly shaped?

Solution:
We need to find . Using bayes’ rule: