**Course leader**: ...... **Course summary**: You should be able to state & use definitions: state, prove and use theorems - hence examples. Main theorems are law of averages and central limit theorem as they make use of most of the precedent definitions and theorems except chapters 2 and 4 - namely Counting and Conditional Probability namely. Cumulative Functions and Moments used for single theory. --- # 1. Uniform Probability Spaces | Definitions | Theorems | Examples | | ----------------------------------------------------------- | ------------------------------------------------------------------------------------ | -------- | | 1.1 [[Uniform Probability Space with Finite Sample Space]]. | | | | 1.2 [[Finite Set]] & [[Cardinality]]. | 1.1 [[Probability of Events in Uniform Probability Space With Finite Sample Space]]. | | | | | | # 2. Counting | Definitions | Theorems | Examples | | ----------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | -------- | | | 2.0 [[Cardinality of Union of Disjoint Sets]] & [[Product Rule for Counting (Fundamental Counting Principle)]]. Proofs not given. | | | 2.1 [[List]]. Set of all $k$-tuples over finite set $A$ with cardinality $n$ is denoted $S_{n,k}(A).$ | 2.1 [[Number of Lists of Length k whose Elements are Taken From a Finite Set]]. | | | 2.2 [[Partial Permutation of n Letters (Ordered Selection)]]. Set of $k$-permutations of finite set $A$ with cardinality $n$ is denoted $O_{n,k}(A).$ | 2.2 [[Number of k-Permutations of n Letters (Injections between Finite Sets)]]. | | | 2.3 [[Combination of n Letters]]. Set of $k$-combinations of finite set $A$ with cardinality $n$ is denoted $C_{n,k}(A).$ | 2.3 [[Number of k-Combinations of n Letters]]. | | | 2.4 [[Permutation of Finite Degree]]. Denoted $S_{n}$ or $S_{n}(A).$ | 2.1 [[Number of Permutations of n Letters]]. | | | 2.5 [[Partition of a Set]]. | 2.4 [[Number of Partitions of n Letters into Subsets of Given Sizes]]. | | # 3. Probability Spaces | Definitions | Theorems | Examples | | ---------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------- | | 3.1 [[Sample Space]]. In ST120, we only consider finite sample spaces. | | | | 3.2 [[Event Space]]. | 3.1.1 [[Event Spaces are Closed Under Finite Unions]]. | | | | 3.1.2 [[Event Spaces are Closed Under Finite Intersections]]. | | | | 3.1.3 [[Event Spaces are Closed under Countably Infinite Intersections]]. | | | 3.2 [[Probability Measure]]. | | | | 3.4 [[Probability Space]]. | 3.2.a [[Probability of Subset of Event]]. | | | | 3.2.b [[Probability of Complement of Event]]. | | | | 3.2.c [[Probability of Empty Set is Zero]]. | | | | 3.3 [[Inclusion-Exclusion Principle for Probability Measure]]. Proof not given. We prove the special case: *[[Probability Measure is Strongly Additive]]. | [[Enumerating surjections between finite sets]]. | | | 3.4 [[Probability of Subset of Event is Less Than or Equal to Probability of Event]]. | | | | 3.5 [[Probability of Union is Less than Sum of Probabilities (Boole's Inequality)]]. | | # 4. Conditional Probability & Independence | Definitions | Theorems | Examples | | ----------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | -------- | | 4.1 [[Conditional Probability]]. | 4.1 [[Conditional Probability Defines Probability Space]]. | | | | 4.2 [[Chain Rule for Probability]]. | | | | 4.3 [[Law of Total Probability]]. | | | | 4.1 [[Bayes' Theorem]]. | | | 4.3 [[Independence of Two Events]]. | 4.3 [[Independent of Event iff Independent of Complement]]; [[Disjoint Events are Independent iff Probability of one is Zero]]; | | | 4.4 [[Pairwise Independent Set of Events]]; [[Mutually Independent Set of Events]]. | | | # 5. Random Variables | Definitions | Theorems | Examples | | --------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 5.1 [[Random Variables]]. | | | | 5.2 [[Probability Distribution of Real-Valued Random Variable]]. | 5.1 [[Probability Distribution of Real-Valued Random Variable is Probability Measure]] | | | 5.3 [[Discrete random variables]]. | | | | 5.4 [[Probability Mass Function of Discrete Real-Valued Random Variable]]. | | | | 5.5 [[Discrete Support of Distribution of Discrete Real-Valued Random Variable]]. | 5.2 [[Probability Distribution of Discrete Real-Valued Random Variable in Terms of Probability Mass Function]] | | | 5.6 *[[Probability Mass Function]]. | 5.3 *[[Probability Mass Function Defines Discrete Real-Valued Random Variable]]. | [[Bernoulli Distribution]]. | | | | [[Geometric Distribution]]; [[Geometric Distribution Probability Mass Function is Probability Mass Function]]. | | | | [[Binomial Distribution]]; [[Binomial Distribution Probability Mass Function is Probability Mass Function]]. | | | | [[Poisson Distribution]]; [[Poisson Distribution Probability Mass Function is Probability Mass Function]]; *[[Binomial Distribution Approximated by Poisson Distribution]]. | # 6. Expectation | Definitions | Theorems | Examples | | ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | 6.1 [[Expectation of Discrete Real-Valued Random Variable]]. | | [[Expectation of Poisson Distribution]]; [[Expectation of Binomial Distribution]]; [[Expectation of Geometric Distribution]]; | | 6.2 [[Integrable Discrete Real-Valued Random Variable]]. | 6.1.1 [[Expectation of Discrete Real-Valued Random Variable is Unitary]]. | 6.2 *[[Event Indicator Function]]. | | | 6.1.2 [[Expectation of Discrete Real-Valued Random Variable is Monotone]]. Proof not given in this chapter. | | | | 6.1.3 [[Expectation of Discrete Real-Valued Random Variable is Linear]]. Proof not given in this chapter. | | | | 6.1 [[Expectation of Real-Valued Function of Discrete Real-Valued Random Variable]]. Proof not given in this chapter. | | | 6.3 [[Square-Integrable Discrete Real-Valued Random Variable]]. | | | | 6.4 [[Variance of Square-Integrable Discrete Real-Valued Random Variable]]. | | [[Variance of Poisson Distribution]]; [[Variance of Geometric Distribution]]; [[Variance of Bernoulli Distribution]]; | | 6.5 [[Standard Deviation of Square-Integrable Discrete Real-Valued Random Variable]]. | [[Variance of Linear Transformation of Square-Integrable Discrete Real-Valued Random Variable]]. Thus $\sigma(aX)=\|a\|\cdot \sigma(x).$ Proof not given. | | # 7. Multivariate Discrete Distributions | Definitions | Theorems | Examples | | --------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | -------- | | 7.1 [[Joint Probability Mass Function of Discrete Real-Valued Random Variables]]. | 7.1 [[Marginal Probability Mass Function of Discrete Real-Valued Random Variable]]. | | | | 7.1 *[[Expectation of Real-Valued Function of Bivariate Discrete Real-Valued Random Variable]]. | | | | 7.1.1 [[Expectation of Discrete Real-Valued Random Variable is Linear]]. Proof given in this chapter. | | | 7.2 [[Independence of Two Discrete Real-Valued Random Variables]]. | 7.1 [[Expectation of Product of Two Independent Discrete Real-Valued Random Variables]]. | | | 7.3 [[Pairwise Independent Set of Discrete Real-Valued Random Variables]]. | 7.2 [[Variance of Sum of Pairwise Independent Square-Integrable Discrete Real-Valued Random Variables]]. | | | 7.4 [[Mutually Independent Set of Discrete Real-Valued Random Variables]]. | | | # 8. The Law of Averages | Definitions | Theorems | Examples | | ----------- | ---------------------------------------------- | -------- | | | 8.1 [[Strong Law of Large Numbers]]. Proof not given here. | | # 9. Covariance | Definitions | Theorems | Examples | | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------- | | 9.1 [[Covariance of Square-Integrable Discrete Real-Valued Random Variables]]. | 9.0.1 [[Covariance of Square Integrable Discrete Real-Valued Random Variables is Symmetric]]. | | | | 9.0.2 [[Covariance of Square Integrable Discrete Real-Valued Random Variable with Itself]]. | | | | 9.1 [[Covariance Square Integrable Discrete Real-Valued Random Variables is Bilinear]]. | | | | 9.2 [[Variance of Sum of Square-Integrable Discrete Real-Valued Random Variables]]. | | | 9.2 [[Uncorrelated Square-Integrable Discrete Real-Valued Random Variables]]. | 9.3 [[Variance of Sum of Uncorrelated Square-Integrable Discrete Real-Valued Random Variables]]. | | | | 9.3.1 *[[Square Root Law]]. | | | | 9.4.1 [[Covariance of Square-Integrable Discrete Real-Valued Random Variables as Expectation of Product minus Product of Expectations]]. | | | | 9.4.2 [[Pairwise Independent Square-Integrable Discrete Real-Valued Random Variables are Uncorrelated]]. Which gives another proof of [[Variance of Sum of Pairwise Independent Square-Integrable Discrete Real-Valued Random Variables]]. | | # 10. Chebyshev's Inequality | Definitions | Theorems | Examples | | ----------- | ----------------------------------------------------------------------------------------------- | -------- | | | 10.1 [[Markov's Inequality for Non-negative Integrable Discrete Real-valued Random Variables]]. | | | | 10.2 [[Chebyshev's Inequality for Square-Integrable Discrete Real-valued Random Variables]]. | | | | 10.3 [[Strong Law of Large Numbers]]. | | # 11. Correlation Coefficient | Definitions | Theorems | Examples | | -------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | | 11.1 [[Correlation Coefficient of Square-Integrable Discrete Real-Valued Random Variables]]. | 11.2 [[Correlation Coefficient of Square-Integrable Discrete Real-Valued Random Variables is Invariant Under Linear Transformations of Variables]]. | | | | 11.2.1 *[[Correlation Coefficient of Square-Integrable Discrete Real-Valued Random Variables is Symmetric]]. | | | | 11.2.2 *[[Correlation Coefficient of Square-Integrable Discrete Real-Valued Random Variables involving Negative]]. | | | | 11.3 [[Absolute Value of Correlation Coefficient of Square-Integrable Discrete Real-Valued Random Variables is Bounded Above by 1]]. | | # 12. Central Limit Theorem | Definitions | Theorems | Examples | | ----------- | --------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | | | 12.1 [[Central Limit Theorem]]. Proof not given in this module. | 12.1 [[ST120 Example 2.1 (Central Limit Theorem Application)]] | # 13. Continuous Random Variables | Definitions | Theorems | Examples | | -------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 13.1 [[Probability Density Function]]. | | | | 13.1 [[Continuous random variables]]. | | 13.2 [[Continuous Uniform Distribution]] | | | | 13.3 [[Normal Distribution]]; [[Standard Normal Random Variable as Transformation of Normal Random Variable]]; *[[Normal Distribution Probability Density Function is Probability Density Function]]; | | | | 13.4 [[Exponential Distribution]]; [[Exponential Distribution is Memoryless]]. | | 13.2 [[Integrable Continuous Real-Valued Random Variable]]. | | 13.1 [[Expectation of Continuous Uniform Distribution]]. | | 13.2 [[Expectation of Integrable Continuous Real-Valued Random Variable]]. | | 13.2 [[Expectation of Exponential Distribution]]. | | | | 13.3 [[Expectation of Standard Normal Distribution]]. | | | | 13.4 [[Continuous Real-Valued Random Variable with Cauchy Distribution is Not Integrable]]. | | 13.3 [[Square-Integrable Continuous Real-Valued Random Variable]]. | 13.1 [[Expectation of Real-Valued Function of Continuous Real-Valued Random Variable]]. | 13.4 [[Comparison of Discrete and Continuous Real-Valued Random Variables]]. | | 13.4 [[Variance of a Square-Integrable Continuous Real-Valued Random Variable]]. | | 13.8 [[Variance of Continuous Uniform Distribution]]. | | | | 13.9 [[Variance of Exponential Distribution]]. | | | | 13.10 [[Variance of Standard Normal Distribution]]. | # 14. A Single Theory For Discrete and Continuous | Definitions | Theorems | Examples | | ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | | 14.1 [[Cumulative Distribution Function of Real-Valued Random Variable]]. | 14.1 [[Cumulative Distribution Determines Probability Distribution for Real-Valued Random Variable]] asserts that any two real-valued random variables with same CDF have the same probability distribution function. Proof not given in this course. | | | 14.2.1 [[Integrable Real-Valued Random Variable]] if 'expectation' is defined and finite. | 14.2 [[Expectation of Integrable Real-Valued Random Variable is Unitary]]. | | | 14.2.2 [[Expectation of Integrable Real-Valued Random Variable]]. Formula not given. | 14.2 [[Expectation of Integrable Real-Valued Random Variable is Linear]]. | | | 14.2.3 [[Square-Integrable Real-Valued Random Variable]]. | 14.2 [[Expectation of Integrable Real-Valued Random Variable is Monotone]]. | | | 14.2.4 [[Variance of Square-Integrable Real-Valued Random Variable]]. | | | | 14.3 [[Covariance of Square-Integrable Real-Valued Random Variables]]. | | | # 15. Joint Distributions and Independence | Definitions | Theorems | Examples | | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------- | | 15.1 [[Joint Probability Density Function of Continuous Real-Valued Random Variables]]. | | | | 15.2 [[Joint Cumulative Distribution Function of Real-Valued Random Variables]]. | | | | 15.3 [[Independence of Two Real-Valued Random Variables]]. | 15.1 [[Joint Cumulative Distribution Function of Two Independent Real-Valued Random Variables]]. | | | | 15.1 [[Expectation of Product of Two Independent Real-Valued Random Variables]]. | | | 15.4 [[Pairwise Independent Set of Real-Valued Random Variables]]. | | | | 15.5 [[Mutually Independent Set of Real-Valued Random Variables]]. | | 15.4 [[Uncorrelated Square-Integrable Real-Valued Random Variables]] | | | | 15.4 [[Variance of Sum of Uncorrelated Square-Integrable Real-Valued Random Variables]]. | | | | 15.4 [[Markov's Inequality for Non-negative Integrable Real-valued Random Variables]]. | | | | 15.5 [[Chebyshev's Inequality for Square-Integrable Real-valued Random Variables]]. | | | | 15.4 [[Strong Law of Large Numbers]]. | | | | 15.4 [[Central Limit Theorem for Real-Valued Random Variables]]. | # 16. Sums of Independent Random Variables | Definitions | Theorems | Examples | | ----------- | ---------------------------------------------------------------------------------------- | -------- | | | 16. [[Probability Distribution of Sum of Two Independent Real-Valued Random Variables]]. | | | | 16.1 *[[Sum of Two Independent Normally Distributed Real-Valued Random Variables]]. | | # 17. Moments and Moment Generating Functions | Definitions | Theorems | Examples | | ------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | | 17. [[Moment generating function of real-valued random variable]]. | | 17.1 [[Moment Generating Function of Geometric Distribution]]. | | | | 17.2 [[Moment Generating Function of Poisson Distribution]]. | | | | 17.3 [[Moment Generating Function of Normal Distribution]]. | | 17.2 [[Raw Moment of Real-Valued Random Variable]]. | 17.1 [[Moment in terms of Moment Generating Function of Real-Valued Random Variable]]. Proof not given in this module. | [[Expectation of Integrable Real-Valued Random Variable is First Raw Moment]]. | | | | | | | | | | | 17.2 [[Moment Generating Function of Linear Transformation of Real-Valued Random Variable]] | | | | 17.3 [[Moment Generating Function of Sum of Two Independent Real-Valued Random Variables]]. | | | | 17.1 [[Moment Generating Function of Real-Valued Random Variable Determines Probability Distribution]]. Proof not given in this module. | [[Sum of Two Independent Normally Distributed Real-Valued Random Variables]] |