Random variable Study Guide

📖 Core Concepts Random Variable (RV) – a measurable function \(X:\Omega\to\mathbb{R}\) that assigns a real number to each outcome of a random experiment. Distribution (Law) – the push‑forward measure \(PX(B)=P\bigl(X\in B\bigr)\) on the real line; it tells the probabilities of all possible values of \(X\). Cumulative Distribution Function (CDF) – \(FX(x)=P(X\le x)\); uniquely determines the distribution. Probability Mass Function (PMF) – for discrete RVs, \(pX(x)=P(X=x)\) with \(\sumx pX(x)=1\). Probability Density Function (PDF) – for absolutely continuous RVs, \(fX(x)\) with \(\displaystyle\int{-\infty}^{\infty} fX(x)\,dx=1\) and \(P(a\le X\le b)=\inta^b fX(x)\,dx\). Measurability – \(X\) is measurable if \(X^{-1}(B)\in\mathcal{F}\) for every Borel set \(B\subset\mathbb{R}\); equivalently, \(\{\omega:X(\omega)\le x\}\in\mathcal{F}\) for all \(x\). Moments – expectations of powers: \(\mathbb{E}[X]\) (mean), \(\operatorname{Var}(X)=\mathbb{E}[(X-\mathbb{E}[X])^2]\), higher moments \(\mathbb{E}[X^k]\). Moment‑Generating Function (MGF) – \(MX(t)=\mathbb{E}[e^{tX}]\); if it exists, it uniquely characterizes the distribution. --- 📌 Must Remember Discrete vs Continuous: Discrete → countable support, described by PMF. Continuous → uncountable support, described by PDF; \(P(X=x)=0\) for any single point. CDF properties: non‑decreasing, right‑continuous, \(\displaystyle\lim{x\to-\infty}FX(x)=0,\;\lim{x\to\infty}FX(x)=1\). Expectation formulas Discrete: \(\displaystyle\mathbb{E}[X]=\sumx x\,pX(x)\). Continuous: \(\displaystyle\mathbb{E}[X]=\int{-\infty}^{\infty} x\,fX(x)\,dx\). Variance identity: \(\operatorname{Var}(X)=\mathbb{E}[X^2]-(\mathbb{E}[X])^2\). Convolution for sums (independent RVs): \(\displaystyle f{X+Y}(z)=\int{-\infty}^{\infty} fX(x)fY(z-x)\,dx\). Equality in distribution: \(X\stackrel{d}{=}Y\iff FX=FY\). Almost‑sure equality: \(X=Y\) a.s. \(\iff P(X\neq Y)=0\). Convergence hierarchy: a.s. \(\Rightarrow\) in probability \(\Rightarrow\) in distribution (but not vice‑versa). LLN: Sample mean \(\bar Xn\) of i.i.d. variables converges in probability to \(\mathbb{E}[X]\). CLT: \(\displaystyle\frac{\sum{i=1}^n Xi - n\mu}{\sqrt{n}\sigma}\xrightarrow{d} \mathcal N(0,1)\) for i.i.d. \(Xi\) with mean \(\mu\) and variance \(\sigma^2\). --- 🔄 Key Processes Checking Measurability Verify \(\{\omega:X(\omega)\le x\}\in\mathcal{F}\) for every real \(x\). Finding the PDF of a Monotone Transform \(Y=g(X)\) Compute inverse \(x=g^{-1}(y)\). Apply \(fY(y)=fX\!\bigl(g^{-1}(y)\bigr)\,\bigl|\frac{d}{dy}g^{-1}(y)\bigr|\). Finding the PDF of a Non‑Monotone Transform Identify all pre‑images \(\{xi\}\) such that \(g(xi)=y\). Sum contributions: \(fY(y)=\sumi fX(xi)\,\bigl|\frac{dxi}{dy}\bigr|\). Convolution (sum of independent RVs) Write integral (continuous) or sum (discrete) of product of densities/PMFs shifted by the argument. Computing Moments via MGF Differentiate \(MX(t)\): \(\displaystyle\mathbb{E}[X^k]=MX^{(k)}(0)\). --- 🔍 Key Comparisons Discrete vs Continuous RV Support: countable vs uncountable interval. Probability of a point: >0 vs 0. Descriptor: PMF vs PDF. CDF vs PMF/PDF CDF: cumulative probability \(P(X\le x)\). PMF: derivative of CDF at points of mass (discrete jumps). PDF: Radon–Nikodym derivative of CDF where it is absolutely continuous. Equality in Distribution vs Almost‑Sure Equality In distribution: same CDF, may differ on underlying sample space. Almost sure: identical values with probability 1 (stronger). Convergence Types a.s. strongest → implies in probability → implies in distribution. In distribution only cares about limiting CDF, not about probabilities of specific events. --- ⚠️ Common Misunderstandings “PDF gives probability at a point” – false; \(fX(x)\) is a density, not a probability. Only integrals over intervals give probabilities. Confusing convergence in probability with almost‑sure convergence – a.s. convergence requires the entire sequence to settle for almost every outcome, not just that the probability of large deviations shrinks. Treating mixed‑type RVs as purely discrete or continuous – they have both a PMF component (jumps) and a PDF component (continuous part). Assuming independence when only uncorrelated – zero covariance does not guarantee independence unless the joint distribution is known to be jointly normal. --- 🧠 Mental Models / Intuition RV as a “black box”: input = outcome \(\omega\); output = a number. The distribution is what you see outside the box, regardless of the underlying \(\Omega\). PDF as “mass density”: think of a thin sheet of sand; the height at \(x\) tells how much sand you’d collect over a small interval around \(x\). Convolution = “smearing”: adding independent randomness spreads (smears) the distribution; the shape of the sum is the overlap of the two original shapes. Transformations: monotone transforms stretch/compress the density; non‑monotone transforms fold the density and add contributions from each branch. --- 🚩 Exceptions & Edge Cases Mixed‑type RVs: CDF has jumps (discrete mass) and a smooth part; the PDF exists only on intervals where the CDF is absolutely continuous. Non‑invertible transformations with infinite roots: the formula with a sum over pre‑images assumes a finite or countable set of roots; infinite uncountable pre‑images require measure‑theoretic treatment beyond the simple sum. MGF may not exist: heavy‑tailed distributions (e.g., Cauchy) have no finite MGF, yet moments of lower order may still exist. --- 📍 When to Use Which Determine PMF vs PDF: If the support is countable → use PMF. If the support is an interval and \(P(X=x)=0\) for all \(x\) → use PDF. Choose Convolution vs Direct Calculation: Independent sum → convolution integral/sum. Dependent sum → need joint distribution, not convolution. Use MGF: To find moments quickly and when the MGF exists (e.g., normal, exponential). Otherwise compute moments directly from definition. Select Convergence Test: Prove a.s. convergence → use Borel‑Cantelli or almost‑sure criteria. Prove convergence in probability → Chebyshev’s inequality or Slutsky’s theorem. Prove convergence in distribution → show CDFs converge at continuity points or use characteristic functions. --- 👀 Patterns to Recognize Jump in CDF ⇒ discrete mass at that point. Flat segment in PDF ⇒ zero probability density (possible gap in support). Symmetric PDF about 0 → mean = 0 (if it exists). Sum of i.i.d. normals → still normal (closed under convolution). When a transformation is monotone, only one term appears in the density‑change formula. In CLT problems, look for “standardized sum” \((\sum Xi - n\mu)/(\sqrt{n}\sigma)\). --- 🗂️ Exam Traps Choosing PMF for a continuous RV – answer will be wrong because \(P(X=x)=0\). Confusing “in distribution” with “equal a.s.” – a distractor may state the two are equivalent; remember they differ. Omitting absolute value in Jacobian when transforming PDFs – leads to negative densities. Assuming independence for convolution – if dependence is hinted, convolution does not apply. Using LLN for a single observation – LLN concerns averages of many i.i.d. variables, not a single draw. Applying CLT without standardizing – the limit is standard normal only after centering and scaling. ---