Common Probability Distributions

The Bernoulli distribution models a single trial with a binary outcome: outcome 1 (success) with probability $p$ and outcome 0 (Failure) with probability $(1 - p)$ .
The Binomial distribution is the sum of $n$ independent Bernoulli trials. It models the total number of successes for a fixed $n$ trials.
The Geometric distribution models the number of trials required to get the first success.
- Discrete analog of the Exponential distribution.
The Negative Binomial distribution models the number of trials required to get the r-th success. It can be viewed as the sum of $r$ i.i.d Geometric variables.

	Bernoulli	Binomial	Geometric	Negative Binomial
Notation	$X \sim BERNOULLI (p)$	$X \sim BIN (n, p)$	$X \sim GEO (p)$	$X \sim NB (r, p)$
$f_{X} (x)$	$p^{x} (1 - p)^{1 - x}$ for $x \in {0, 1}$	$(\binom{n}{x}) p^{x} (1 - p)^{n - x}$ for $x = 0, 1, \dots, n$	$p (1 - p)^{x - 1}$ for $x = 1, 2, \dots$	$(\binom{x - 1}{r - 1}) p^{r} (1 - p)^{x - r}$ for $x = r, r + 1, \dots$
$F_{X} (x)$	${\begin{cases} 0 & x < 0 \\ 1 - p & 0 \leq x < 1 \\ 1 & x \geq 1 \end{cases}$	(Sum of PMFs)	$1 - (1 - p)^{x}$	(Sum of PMFs)
$M_{X} (t)$	$(1 - p) + p e^{t}$	$((1 - p) + p e^{t})^{n}$	$\frac{p e^{t}}{1 - (1 - p) e^{t}}$	${(\frac{p e^{t}}{1 - (1 - p) e^{t}})}^{r}$
$E (X)$	$p$	$n p$	$\frac{1}{p}$	$\frac{r}{p}$
$V a r (X)$	$p (1 - p)$	$n p (1 - p)$	$\frac{1 - p}{p^{2}}$	$\frac{r (1 - p)}{p^{2}}$
Params	$0 < p < 1$	$n \in N$ $0 < p < 1$	$0 < p < 1$	$r \in N$ $0 < p < 1$

The Poisson distribution models the number of events occurring in a fixed interval given a constant average rate $λ$ .
- It is derived from the Binomial distribution by letting $n \to \infty$ and $p \to 0$ such that $λ = n p$ remains constant.
The Exponential distribution models the waiting time until the 1st event occurs in a Poisson process, where the average waiting time is $θ = \frac{1}{λ}$ .
- It is the continuous analog of the Geometric distribution.
The Gamma distribution models the waiting time until the $κ$ -th event occurs in a Poisson process. It is the sum of $κ$ i.i.d Exponential variables.
The Weibull distribution generalizes the Exponential distribution to allow for changing failure rates over time.

	Poisson	Exponential	Gamma	Weibull
Notation	$X \sim POI (λ)$	$X \sim EXP (θ)$	$X \sim GAM (θ, κ)$	$X \sim WEI (θ, β)$
$f_{X} (x)$	$e^{- λ} \frac{λ^{x}}{x!}$ for $x = 0, 1, 2, \dots$	$\frac{1}{θ} e^{- x / θ}$ for $x > 0$	$\frac{1}{θ^{κ} Γ (κ)} x^{κ - 1} e^{- x / θ}$ for $x > 0$	$\frac{β}{θ^{β}} x^{β - 1} e^{- (x / θ)^{β}}$ for $x > 0$
$F_{X} (x)$	(Sum of PMFs)	$1 - e^{- x / θ}$ for $x > 0$	(Incomplete)	$1 - e^{- (x / θ)^{β}}$ for $x > 0$
$M_{X} (t)$	$e^{λ (e^{t} - 1)}$	$(1 - t θ)^{- 1}$ for $t < 1 / θ$	$(1 - t θ)^{- κ}$ for $t < 1 / θ$	(Complex)
$E (X)$	$λ$	$θ$	$κ θ$	$θ Γ (1 + \frac{1}{β})$
$V a r (X)$	$λ$	$θ^{2}$	$κ θ^{2}$	$θ^{2} [Γ (1 + \frac{2}{β}) - Γ^{2} (1 + \frac{1}{β})]$
Params	$λ > 0$	$θ > 0$	$θ > 0, κ > 0$	$θ > 0, β > 0$

Memoryless Property: the probability of an event occurring after a duration $s + t$ , given that it has not occurred by time $t$ , is the same as the initial probability that it would not occur after time $s$ .
If $X_{1}, \dots, X_{n}$ are independent exponential random variables with rates $λ_{1}, \dots, λ_{n}$ ,
- Distribution of the Minimum: Then the random variable $Y = min (X_{1}, \dots, X_{n})$ is also exponentially distributed with a rate equal to the sum of the individual rates $Λ = λ_{1} + \dots + λ_{n}$ .
- Competing Exponentials (Race Condition): The probability that a specific variable $X_{k}$ is the first to occur (i.e., is the minimum) is the ratio of its rate to the total rate:

The Normal (Gaussian) distribution models sums of independent random variables (via the Central Limit Theorem). It approximates the Binomial, Poisson, and Gamma distributions when samples are large.
The Chi-Square distribution models the sum of the squares of $ν$ independent standard normal random variables.
- It is mathematically a special case of the Gamma distribution ( $GAM (2, ν / 2)$ ).
The Student's t distribution models the mean of a normal population when the sample size is small and variance is unknown. It is the ratio of a Normal random variable to the square root of a Chi-Square variable. It converges to the Normal distribution as $ν \to \infty$ .

Component	Normal (Gaussian)	Chi-Square	Student's t
Notation	$X \sim N (μ, σ^{2})$	$X \sim χ^{2} (ν)$	$X \sim t (ν)$
$f_{X} (x)$	$\frac{1}{σ \sqrt{2 π}} e^{- \frac{1}{2} (\frac{x - μ}{σ})^{2}}$ for $x \in R$	$\frac{1}{2^{ν / 2} Γ (ν / 2)} x^{ν / 2 - 1} e^{- x / 2}$ for $x > 0$	$\frac{Γ (\frac{ν + 1}{2})}{\sqrt{ν π} Γ (\frac{ν}{2})} (1 + \frac{x^{2}}{ν})^{- \frac{ν + 1}{2}}$
$F_{X} (x)$	$Φ (\frac{x - μ}{σ})$	(Incomplete Gamma)	(Table)
$M_{X} (t)$	$e^{μ t + \frac{1}{2} σ^{2} t^{2}}$	$(1 - 2 t)^{- 1 / 2}$ for $t < 1 / 2$	Does not exist
$E (X)$	$μ$	$ν$	$0$ (if $ν > 1$ )
$V a r (X)$	$σ^{2}$	$2 ν$	$\frac{ν}{ν - 2}$ (if $ν > 2$ )
Params	$μ \in R$ , $σ > 0$	$ν > 0$	$ν > 0$

The Uniform distribution models a random variable where all intervals of the same length within the support $[a, b]$ are equally probable. It represents maximum entropy (minimum information) over an interval.
The Beta distribution models random variables restricted to the interval $[0, 1]$ . It generalizes the Uniform distribution and serves as the conjugate prior for the Bernoulli, Binomial, and Geometric distributions in Bayesian statistics.

TODO:

Maybe do research into other common distributions that are relevant.
Explain how we can sample from these distributions, with the uniform distribution as the base. Could maybe write a separate article on this.