Probability distribution Study Guide

📖 Core Concepts Probability distribution – a function that assigns probabilities to all possible outcomes of a random experiment. Sample space (Ω) – the set containing every outcome that can occur. Random variable (X) – a rule that maps each outcome ω ∈ Ω to a numeric value. Discrete vs. absolutely continuous – Discrete RVs take a countable set of values (PMF); continuous RVs take values from an uncountable interval (PDF). PMF (pₓ(x)) – gives \(P(X = x)\) for each admissible value of a discrete RV. PDF (fₓ(x)) – satisfies \(P(a\le X\le b)=\inta^b fₓ(t)\,dt\); for any single point \(P(X=x)=0\). CDF (Fₓ(x)) – cumulative probability \(Fₓ(x)=P(X\le x)=\int{-\infty}^{x} fₓ(t)\,dt\). Kolmogorov axioms – (1) non‑negativity, (2) total probability = 1, (3) countable additivity for disjoint events. 📌 Must Remember Normalization: \(\sumx pₓ(x)=1\) for discrete; \(\int{-\infty}^{\infty} fₓ(x)\,dx=1\) for continuous. PDF–CDF relationship: \(fₓ(x)=\frac{d}{dx}Fₓ(x)\). Uniform [0,1) is the base for most pseudo‑random generators. Inverse‑transform rule: if \(U\sim\text{Uniform}(0,1)\) then \(X=Fₓ^{-1}(U)\) has distribution Fₓ. Key families & parameters: Bernoulli(p), Binomial(n, p), Geometric(p), Negative binomial(r, p), Hypergeometric(N, K, n). Poisson(λ), Exponential(λ), Gamma(k, θ). Normal(μ, σ²), Uniform(a, b), Log‑normal(μ, σ²). Chi‑squared(ν), Student‑t(ν), F(d₁, d₂). 🔄 Key Processes Deriving a CDF from a PDF \[ Fₓ(x)=\int{-\infty}^{x} fₓ(t)\,dt \] Generating a discrete RV from Uniform(0,1) Example Bernoulli(p): generate \(U\sim\text{Uniform}(0,1)\); set \(X=1\) if \(U\le p\), else \(0\). Inverse‑transform sampling (continuous) Compute \(U\sim\text{Uniform}(0,1)\). Set \(X = Fₓ^{-1}(U)\). Example for Exponential(λ): \(X = -\frac{1}{\lambda}\ln(1-U)\). Obtaining marginal distribution For joint PDF \(f{X,Y}(x,y)\): \(fX(x)=\int{-\infty}^{\infty} f{X,Y}(x,y)\,dy\). Constructing a conditional distribution \(f{X|Y}(x|y)=\frac{f{X,Y}(x,y)}{fY(y)}\) (provided \(fY(y)>0\)). 🔍 Key Comparisons Discrete vs. Continuous Values: countable ↔ uncountable. Probability of a point: \(P(X=x)>0\) ↔ \(P(X=x)=0\). PMF vs. PDF Definition: \(pₓ(x)=P(X=x)\) ↔ \(fₓ(x)=\frac{d}{dx}Fₓ(x)\). Sum vs. integral: \(\sum pₓ(x)=1\) ↔ \(\int fₓ(x)dx=1\). Uniform Discrete vs. Uniform Continuous Discrete: equal probability \(1/k\) for each of k outcomes. Continuous: constant density \(1/(b-a)\) over interval \([a,b]\). Binomial vs. Poisson Binomial: fixed number of trials \(n\), success prob p. Poisson: limit of Binomial as \(n\to\infty, p\to0\) with \(λ=np\) fixed. ⚠️ Common Misunderstandings “PDF is a probability” – The PDF itself is not a probability; only its integral over an interval yields a probability. Zero probability ⇒ impossible – For continuous RVs \(P(X=x)=0\) does not mean the value cannot occur; it simply has zero measure. CDF must be differentiable – Not all CDFs have a PDF (e.g., singular Cantor distribution). Uniform pseudo‑random generator gives true randomness – It produces deterministic sequences that appear uniform; not suitable for cryptography without additional safeguards. 🧠 Mental Models / Intuition Mass vs. Density – Think of a PMF as “weight on a set of boxes” (each box gets a finite weight). A PDF is “sand spread over a surface”; the height at a point (density) only tells you how much sand per unit length is there. Inverse‑transform as “inverse map” – The CDF maps a value x to a cumulative probability. Inverse‑transform simply walks backwards: from a random probability back to the corresponding x‑value. Joint → Marginal → Conditional – Visualize a 3‑D landscape (joint). Slicing a vertical plane (fix Y) gives a conditional curve; flattening (integrating) across one axis yields the marginal “shadow” on the other axis. 🚩 Exceptions & Edge Cases Singular continuous distributions (e.g., Cantor) – continuous but no PDF; CDF increases on a set of measure zero. Mixed distributions – Some RVs have both a discrete component (point masses) and a continuous component (density). Boundary of Uniform( a, b ) – Density defined on \([a,b]\) inclusive of a, exclusive of b in the half‑open convention used by many pseudo‑random generators. 📍 When to Use Which Modeling count data → Poisson (rate λ) or Binomial (fixed n, p) if trial number known. Modeling times between events → Exponential (memoryless) or Gamma (sum of k exponentials) for waiting times. Symmetric measurement error → Normal(μ, σ²). Proportions or probabilities → Beta (conjugate prior) for Bayesian updates of Bernoulli/binomial. Heavy‑tailed phenomena (e.g., income) → Log‑normal or Pareto. Generating discrete outcomes → Inverse‑transform via CDF table or simple thresholding of Uniform(0,1). 👀 Patterns to Recognize “Sum of independent normals → normal” – Any linear combination of independent normal variables is normal. “Mean = λ, variance = λ” for Poisson – Identical mean and variance signals a Poisson model. “Memoryless property” → Exponential – Only exponential (continuous) and geometric (discrete) have the lack‑of‑memory property. “Variance > mean” – Over‑dispersion suggests Negative binomial rather than Poisson. 🗂️ Exam Traps Choosing PDF instead of PMF – If a question explicitly mentions a discrete variable, answer with a PMF; using a PDF will be wrong. Confusing \(FX^{-1}(p)\) with quantile – Some textbooks define quantile as the smallest x with \(FX(x) \ge p\); be careful with strict vs. non‑strict inequalities. Assuming all continuous distributions have a density – Singular distributions violate this; such a nuance can appear in “which statement is always true?” items. Boundary values in Uniform generator – If a problem asks for Uniform\([a,b]\) from a generator that returns \([0,1)\), the transformed variable is \(a + (b-a)U\) (never exactly b). Mixing up parameters of Gamma – Some texts use shape‑scale (k, θ), others shape‑rate (k, λ). Verify which convention the exam adopts.