Set Theory - Three Axioms - Permutations and Combinations
Probability has as its ultimate goal the ability to predict, with some known degree of certainty, the frequency of occurence of an event. Implicit in this idea is the notion that there is some uncertainty associated with the generation of the event or there is incomplete information with which to determine the exact outcome of an event. Signals that have this property are commonly refered to as stochastic signals. If this is not the case, an event or a "certain" signal is said to be deterministic. As an example, we typically think of a unit step signal as being deterministic and noise on a signal as being probabilistic (or stochastic).
Probability is usually first encountered using a Set Theory formulation. In this context the set of all possible outcomes of a defined experiment (or test or measurement) are refered to as the sample space S. Events (such as A or B) are subsets of the sample space and the probability of the occurence of an event, P(A) or P(B) for example, defines the likelihood or relative frequency of that event in a series of experimental trials.
Essentially, each outcome, xn, in the sample space, S, has an associated non-negative number P(xn) such that 0 < P(xn) < 1 and SUM{P(xn)} = P(S) = 1. The probability of occurence of an event A then is expressed P(A) and is the sum of the probabilities of each of the distinct events, xn, that satisfy the conditions that define A.
Some Recall Questions
What is meant by the term "certain" event? What is the probability of the "certain" event?
What is meant by the term "impossible" event? What is the probability of the "impossible" event?
What do we mean when we say that two events are mutually exclusive or disjoint?
What do we mean when we say that a group of events is collectively exhaustive?
How does the probability of the event (A+B), that is P(A+B), change if events A and B are either dependent or independent?
Probability density functions and cumulative probability distribution functions and their properties are all essentially developed from three fundamental axioms of probability. These axioms are often illustrated and interpreted using set theory.
P(A) > 0 for every event A.
P(S) = 1 for the certain event S.
P(A+B) = P(A) +P(B), if two events A and B are mutually exclusive.
In addition to these axioms, some key auxiliary results prove to be essential.
|
Addition Formula: |
|
|
Conditional Probability |
|
|
Bayes Theorem |
|
Some Recall Questions - Express the previous questions and answers mathematically.
What is meant by the term "certain" event? What is the probability of the "certain" event?
What is meant by the term "impossible" event? What is the probability of the "impossible" event?
What do we mean when we say that two events are mutually exclusive or disjoint?
What do we mean when we say that a group of events is collectively exhaustive?
How is the probability of the event (A+B), that is P(A+B), change if events A and B are either dependent or independent?
Many practical problems deal with experiments or events that occur frequently. In some, indeed many, cases such events have only two outcomes of interests. In such cases, we frequently ask two questions.
If there are n-distinct events or objects, how may different ways are there of selecting r of them (r < n)? These are termed permutations, and the number of possible ways is expressed as,
If there are n-events or objects, how may different ways are there of selecting r of them irrespective of the event or object order (r < n)? These are termed combinations, and the number of possible ways is expressed as,
A Pulse Code Modulated (PCM) word consists of a sequence of binary 1-s and 0-s. If the PCM word is n-bits long, how many distinct words are there? If n=4, what is the probability that a PCM word has two or more consecutive zeros?
A Pulse Amplitude Modulated (PAM) signal consists of a series of pulses with different magnitudes. If a PAM system uses the amplitudes {-1, 0 1}, how many pulses are required in a PAM word to provide at least 100 distinct PAM words?
Return to Random Variables
