|
|
|
|
If X is a random variable with a probability density function px(x), and if g(X) is a function of the random variable, then the expectation, or expected value of the function g(X), denoted as E[g(X)], is defined as,
|
|
The most important property of expectation is linearity; that is the mean of a sum of functions of X is equal to the sum of the means of each function of X. Mathematically, this is expressed as,
|
|
The mean or expected value of a random variable, is simply the expected value of the function g(X) = x. That is,
|
|
This definition is used for discrete, continuous, and mixed random variables.
The expected value is often referred to as the mean value or statistical average of the random variable. Note that it is not simply the usual notion of "average value" since g(X) = x is weighted in the integral above by the probability density function px(x). Rather, it is the value we might "expect" to measure if we sampled the random variable. This has some similarities to more familiar algebraic functions of a variable. That is, if we know x = 3, we can expect g(x) = x = 3. Likewise, if g(x) = 4x + 5, we would expect g(3) = 4(3) + 5 = 17. The difference in the probabilistic case, is that we don't know exactly what x will be, only what is might be.
The variance or mean square value of a random variable is a measure of the "spread" of the random variable about its expected value. In essence, it tells us how much variation there is in the values of the random variable from its mean value. It can be interpreted as an energy measure since it is the weighted sum of the deviations of the random variable from its mean value.
The variance of the random variable X, is determined by calculating the expectation of the function g(X) = (x-m x)2. That is,
|
|
A more generalized expectation that encompasses polynomial functions, is to define the n-th moment of the random variable X about a value x0 as,
|
|
Both the mean and variance of the random variable X are then special cases of this expectation.
The following table summarizes some of the other main results.
|
n |
nth moment about the origin, x0=0 |
nth central moment, x0=m x |
||
|
0 |
E[X0] = 1 |
E[(X-m X)0] = 1 |
||
|
1 |
E[X]= m X = the mean value |
E[(X-m X)1] = m X - m X = 0 |
||
|
2 |
E[X2] = the mean square value |
E[(X-m X)2] = s X2 = the variance = (standard deviation)2
|
||
|
3 |
E[X3] |
E[(X-m X)3] = the skewness
|
||
|
4 |
E[X4] |
E[(X-m X)4] = the kurtosis
|
||
|
Random Variable |
Probability Density Function |
Mean or Expected Value |
Second Moment about the Origin |
Variance |
|
Geometric |
|
1/p |
|
|
|
Binomial |
|
np |
np(np+q) |
npq |
|
Poisson |
|
a |
a 2+a |
a |
|
Uniform |
|
a/2 |
a2/3 |
a2/12 |
|
Gaussian |
|
m X |
s X2 + m X2 |
s X2 |
Unfortunately, this is where the whole business tends to get confusing. In many instances, signals comprise of more than one random variable and the relationships between these multiple random variables become important. Although this does not change the concepts we have discussed already, the mathematics of situation is more complicated since we need to be concerned with conditional probability density and distribution functions. One important class of problems however describes situations where the different random processes are statistically independent. In the context of axiomatic probability this means that the outcome of one event is unaffected by the outcome of some other event. In terms of probability density functions, it means that the pdf of one random variable is unaffected by the pdfs of the other random processes. Some of the relevant results are repeated below, although we will review these in the weeks to come as they become relevant to our current studies.
|
Statistical Independence |
|
|
Expected Value |
|
|
Covariance |
|
|
Correlation Coefficient |
|
|
Sum of independent random variables: Z = X + Y |
|
