Probability Theory: Weak and Strong Law of Large Numbers

Introduction

Probability theory is a branch of mathematics concerned with the analysis of random phenomena. The Weak and Strong Law of Large Numbers are fundamental theorems in probability theory that describe the behavior of averages of random variables.

Weak Law of Large Numbers

The Weak Law of Large Numbers (WLLN) states that the sample average converges in probability towards the expected value as the sample size grows. Formally, if \(X_1, X_2, \ldots, X_n\) are independent and identically distributed (i.i.d.) random variables with expected value \(E[X_i] = \mu\), then for any \(\epsilon > 0\):

\[\lim_{n \to \infty} P\left( \left| \frac{1}{n} \sum_{i=1}^n X_i - \mu \right| < \epsilon \right) = 1\]

Strong Law of Large Numbers

The Strong Law of Large Numbers (SLLN) states that the sample average almost surely converges to the expected value as the sample size grows. Formally, if \(X_1, X_2, \ldots, X_n\) are i.i.d. random variables with expected value \(E[X_i] = \mu\), then:

\[\frac{1}{n} \sum_{i=1}^n X_i \xrightarrow{\text{a.s.}} \mu\]

This means that:

\[P\left( \lim_{n \to \infty} \frac{1}{n} \sum_{i=1}^n X_i = \mu \right) = 1\]

Conclusion

The Weak and Strong Law of Large Numbers are powerful results in probability theory that provide a foundation for statistical inference. They assure us that with a large enough sample size, the sample average will be close to the expected value.