Probability & Statistics Cheatsheet

REFERENCE v1.0

Probability & Statistics Quick Reference

Everything you need day‑to‑day – probability, distributions, and statistical inference.

Probability Basics

Definitions

Experiment – any procedure with uncertain outcome
Sample Space (S) – set of all possible outcomes
Event (E) – subset of sample space
Probability – P(E) = |E| / |S| (equally likely outcomes)

Axioms of Probability

0 ≤ P(E) ≤ 1
P(S) = 1
If E₁, E₂, ... are mutually exclusive: P(∪Eᵢ) = ΣP(Eᵢ)

Key Probability Rules

Complement: P(E') = 1 - P(E)
Union (Addition Rule): P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
Intersection (Multiplication Rule): P(A ∩ B) = P(A) · P(B) if independent
Conditional Probability: P(A|B) = P(A ∩ B) / P(B), P(B) > 0
Bayes' Theorem: P(A|B) = P(B|A) · P(A) / P(B)
Law of Total Probability: P(B) = Σᵢ P(B|Aᵢ) · P(Aᵢ)

Bayes' Theorem (Extended)

P(A|B) = P(B|A) · P(A) / (P(B|A)·P(A) + P(B|A')·P(A'))

Random Variables

Discrete Random Variable

Finite or countable outcomes
PMF: P(X = x)
Σ P(X = x) = 1
E[X] = Σ x · P(X = x)
Var(X) = Σ (x - μ)² · P(X = x)

Continuous Random Variable

Infinite outcomes (interval)
PDF: f(x), f(x) ≥ 0, ∫ f(x) dx = 1
CDF: F(x) = P(X ≤ x)
E[X] = ∫ x · f(x) dx
Var(X) = ∫ (x - μ)² · f(x) dx

Expectation & Variance Properties

E[aX + b] = aE[X] + b
Var(aX + b) = a² Var(X)
E[X + Y] = E[X] + E[Y] (always)
Var(X + Y) = Var(X) + Var(Y) + 2Cov(X,Y)
If independent: Var(X + Y) = Var(X) + Var(Y)
E[XY] = E[X]E[Y] if independent

Common Probability Distributions

Bernoulli

X ~ Bern(p), p ∈ [0,1]
P(X=1) = p, P(X=0) = 1-p
E[X] = p
Var(X) = p(1-p)
Use: Single trial (success/failure)

Binomial

X ~ Bin(n, p)
P(X = k) = C(n,k) pᵏ (1-p)ⁿ⁻ᵏ
E[X] = np
Var(X) = np(1-p)
Use: Number of successes in n trials

Poisson

X ~ Poisson(λ), λ > 0
P(X = k) = (e⁻λ · λᵏ) / k!
E[X] = λ
Var(X) = λ
Use: Count events in interval

Geometric

X ~ Geo(p), p ∈ [0,1]
P(X = k) = (1-p)ᵏ⁻¹ · p
E[X] = 1/p
Var(X) = (1-p) / p²
Use: Number of trials until first success

Uniform (Continuous)

X ~ U(a, b)
f(x) = 1/(b-a), x ∈ [a,b]
E[X] = (a+b)/2
Var(X) = (b-a)²/12
Use: Equal likelihood over interval

Normal (Gaussian)

X ~ N(μ, σ²)
f(x) = 1/(σ√2π) · e⁻(x-μ)²/(2σ²)
E[X] = μ
Var(X) = σ²
68‑95‑99.7 Rule: μ±σ (68%), ±2σ (95%), ±3σ (99.7%)
Z‑score: Z = (X - μ)/σ

Exponential

X ~ Exp(λ), λ > 0
f(x) = λ·e⁻λˣ, x ≥ 0
E[X] = 1/λ
Var(X) = 1/λ²
Memoryless: P(X > s+t | X > s) = P(X > t)
Use: Time between events

Beta

X ~ Beta(α, β)
f(x) = (xᵅ⁻¹ (1-x)ᵝ⁻¹) / B(α, β)
E[X] = α/(α+β)
Var(X) = αβ / ((α+β)²(α+β+1))
Use: Probabilities (0-1)

Normal Table (Z‑Scores)

z	0.00	0.01	0.02	0.03	0.04	0.05	0.06	0.07	0.08	0.09
0.0	0.5000	0.5040	0.5080	0.5120	0.5160	0.5199	0.5239	0.5279	0.5319	0.5359
0.5	0.6915	0.6950	0.6985	0.7019	0.7054	0.7088	0.7123	0.7157	0.7190	0.7224
1.0	0.8413	0.8438	0.8461	0.8485	0.8508	0.8531	0.8554	0.8577	0.8599	0.8621
1.5	0.9332	0.9345	0.9357	0.9370	0.9382	0.9394	0.9406	0.9418	0.9429	0.9441
2.0	0.9772	0.9778	0.9783	0.9788	0.9793	0.9798	0.9803	0.9808	0.9812	0.9817
2.5	0.9938	0.9940	0.9941	0.9943	0.9945	0.9946	0.9948	0.9949	0.9951	0.9952
3.0	0.9987	0.9987	0.9987	0.9988	0.9988	0.9989	0.9989	0.9989	0.9990	0.9990

Values: P(Z ≤ z). For z > 3, use 0.9999. For negative z, use 1 - P(Z ≤ |z|).

Descriptive Statistics

Measures of Central Tendency

Mean (μ): Σxᵢ / n
Median: Middle value when sorted
Mode: Most frequent value
Robust: Median is robust to outliers

Measures of Dispersion

Variance (σ²): Σ(xᵢ - μ)² / n
Sample Variance (s²): Σ(xᵢ - x̄)² / (n-1)
Standard Deviation (σ): √σ²
IQR: Q3 - Q1
Range: Max - Min

Measures of Shape

Skewness: Symmetry of distribution
Positive Skew: Long right tail
Negative Skew: Long left tail
Kurtosis: Tail heaviness

Percentiles & Quartiles

Q1: 25th percentile
Q2: 50th percentile (median)
Q3: 75th percentile
Pₖ: Value below which k% of data falls

Inferential Statistics

Confidence Intervals

// For mean (known σ)
CI = x̄ ± z* · (σ / √n)

// For mean (unknown σ, t‑distribution)
CI = x̄ ± t* · (s / √n)

// For proportion
CI = p̂ ± z* · √(p̂(1-p̂) / n)

// Common z* values
90% CI: z* = 1.645
95% CI: z* = 1.96
99% CI: z* = 2.576

Hypothesis Testing

Steps

State H₀ (null) and H₁ (alternative)
Choose significance level (α = 0.05)
Compute test statistic
Find p‑value or critical value
Reject H₀ if p < α

Common Tests

Z‑test – known σ, large n
t‑test – unknown σ, small n
Chi‑square – categorical data
ANOVA – compare multiple means

Test Statistics

One‑sample Z: z = (x̄ - μ₀) / (σ / √n)
One‑sample t: t = (x̄ - μ₀) / (s / √n), df = n-1
Two‑sample t: t = (x̄₁ - x̄₂) / √(s₁²/n₁ + s₂²/n₂)
Paired t: t = d̄ / (s_d / √n)
Proportion Z: z = (p̂ - p₀) / √(p₀(1-p₀) / n)

p‑value Interpretation

p < 0.01 – strong evidence against H₀
0.01 < p < 0.05 – moderate evidence
0.05 < p < 0.10 – weak evidence
p > 0.10 – insufficient evidence

Correlation & Regression

Correlation Coefficient

r ∈ [-1, 1]
r = 1: perfect positive
r = -1: perfect negative
r = 0: no linear relationship
Formula: r = Σ((xᵢ - x̄)(yᵢ - ȳ)) / √(Σ(xᵢ - x̄)² Σ(yᵢ - ȳ)²)

Simple Linear Regression

y = β₀ + β₁x + ε
β₁ = r · (s_y / s_x)
β₀ = ȳ - β₁x̄
R²: proportion of variance explained
R² = r² (simple regression)

Multiple Linear Regression

y = β₀ + β₁x₁ + β₂x₂ + ... + ε
Adjusted R²: penalises added variables
F‑test: tests overall significance
t‑tests: test individual coefficients
VIF: measures multicollinearity

Assumptions of Linear Regression

Linearity
Independence (no autocorrelation)
Homoscedasticity (constant variance)
Normality of residuals
No multicollinearity

Bayesian Statistics

Prior: P(θ) – belief before data
Likelihood: P(D|θ) – data given parameter
Posterior: P(θ|D) ∝ P(D|θ) · P(θ)
Bayes Factor: ratio of likelihoods
Credible Interval: Bayesian equivalent of confidence interval
Conjugate Priors: prior and posterior from same family

Important Theorems

Law of Large Numbers (LLN)

As n → ∞, sample mean → population mean
Converges in probability

Central Limit Theorem (CLT)

Sample mean distribution → Normal
Mean = μ, variance = σ²/n
Approx normal for n ≥ 30

CLT Conditions

Independent and identically distributed (iid)
Finite variance
n sufficiently large (≥ 30 generally)
Applies to sums, means, proportions

Bayes' Theorem Example

P(A|B) = P(B|A) · P(A) / P(B)

Example: Medical Test
• Disease prevalence: P(D) = 0.01
• Test sensitivity: P(+|D) = 0.95
• Test false positive: P(+|no D) = 0.05

P(D|+) = (0.95 × 0.01) / (0.95×0.01 + 0.05×0.99)
       = 0.0095 / (0.0095 + 0.0495)
       = 0.161

Common Statistical Tests Summary

Test	Purpose	Data Type	Assumptions
Z‑test	Mean vs known value	Continuous	Normal (or large n), known σ
t‑test (1‑sample)	Mean vs known value	Continuous	Normal (or large n), unknown σ
t‑test (2‑sample)	Compare two means	Continuous	Normal, independent
Paired t‑test	Before/after comparison	Continuous	Normal differences
ANOVA	Compare ≥3 means	Continuous	Normal, independent, equal variance
Chi‑Square	Categorical association	Categorical	Expected ≥ 5 per cell
Wilcoxon	Non‑parametric mean	Continuous	No normality assumption
Mann‑Whitney	Compare two means (non‑parametric)	Continuous	No normality assumption

Probability Distributions Quick Reference

Distribution	Parameters	E[X]	Var(X)	Use Case
Bernoulli	p	p	p(1-p)	Single success/failure
Binomial	n, p	np	np(1-p)	# successes in n trials
Poisson	λ	λ	λ	Count over interval
Geometric	p	1/p	(1-p)/p²	Trials until first success
Uniform	a, b	(a+b)/2	(b-a)²/12	Equal likelihood over range
Normal	μ, σ²	μ	σ²	Most common, CLT
Exponential	λ	1/λ	1/λ²	Time between events
Beta	α, β	α/(α+β)	αβ/((α+β)²(α+β+1))	Probabilities (0‑1)

📌 Quick Reference

Bayes: P(A|B) = P(B|A)P(A) / P(B)
CLT: sample mean → Normal for n ≥ 30
Normal: Z = (X-μ)/σ, 68‑95‑99.7 rule
Confidence: μ = x̄ ± z*(σ/√n), z*: 1.645 (90%), 1.96 (95%), 2.576 (99%)
Test types: Z‑test (known σ), t‑test (unknown σ), Chi‑square (categorical)
Correlation: r ∈ [-1,1], r² = variance explained
Regression: y = β₀ + β₁x + ε, β₁ = r·s_y/s_x, β₀ = ȳ - β₁x̄

← Back to All Cheatsheets