Hilbert space - Orthogonal Complements and Projections

Orthogonal Complements and Projections Introduction In Hilbert spaces, we can decompose any vector into two perpendicular parts: one lying in a given subspace and one orthogonal to it. This is one of the most powerful ideas in functional analysis, with deep connections to optimization, solving equations, and understanding the structure of linear operators. The tools for doing this decomposition are orthogonal complements and orthogonal projections. Orthogonal Complements: The Perpendicular Parts Definition and Basic Properties Let $V$ be a subspace of a Hilbert space $H$. The orthogonal complement $V^{\perp}$ consists of all vectors in $H$ that are orthogonal to every vector in $V$: $$V^{\perp} = \{x \in H : \langle x, v \rangle = 0 \text{ for all } v \in V\}$$ A key fact is that $V^{\perp}$ is always a closed subspace of $H$, regardless of whether $V$ itself is closed. This closure property is crucial: it means $V^{\perp}$ is itself a Hilbert space, inheriting the inner product from $H$. Why this matters: The orthogonal complement gives us a way to identify all vectors perpendicular to a subspace. This is the first step toward decomposing arbitrary vectors. Orthogonal Projections: Mapping to the Closest Point Definition The orthogonal projection onto $V$, denoted $PV : H \to H$, is an operator with the following property: for any vector $x \in H$, it maps $x$ to a vector $PV(x) \in V$ such that $$x = PV(x) + w$$ where $w \in V^{\perp}$. In other words, $PV(x)$ is the unique vector in $V$ that, when subtracted from $x$, leaves a vector orthogonal to $V$. Key Properties of Projections The operator $PV$ has several important properties: Self-adjoint: $PV^ = PV$, meaning $\langle PV(x), y \rangle = \langle x, PV(y) \rangle$ for all $x, y \in H$. Idempotent: $PV^2 = PV$, meaning applying the projection twice gives the same result as applying it once. This makes intuitive sense: once you've projected onto $V$, you're already on $V$, so projecting again doesn't change anything. Bounded: $\|PV(x)\| \le \|x\|$ for all $x$, so the operator norm $\|PV\| \le 1$. In fact, for any non-zero closed subspace, $\|PV\| = 1$. Geometric Interpretation: Minimizing Distance There's an elegant geometric property: $PV(x)$ is the unique vector in $V$ that minimizes the distance to $x$. That is, $$\|x - PV(x)\| = \min{v \in V} \|x - v\|$$ This is what makes orthogonal projection useful in optimization: it finds the best approximation of $x$ by vectors in $V$. The Decomposition Theorem: Everything Falls Apart Orthogonally The Main Result The fundamental theorem of orthogonal projections states that every vector in $H$ can be written uniquely as the sum of two orthogonal pieces: $$x = v + w \quad \text{where} \quad v \in V, \quad w \in V^{\perp}$$ Moreover, these pieces are: $v = PV(x)$ (the projection of $x$ onto $V$) $w = P{V^{\perp}}(x)$ (the projection of $x$ onto $V^{\perp}$) The Direct Sum This decomposition means that the Hilbert space can be written as an internal direct sum: $$H = V \oplus V^{\perp}$$ In other words, $V$ and its orthogonal complement are complementary subspaces that together cover all of $H$ without overlap. Characterizing Projections Through Ranges and Kernels For any orthogonal projection $PV$: The range is $\text{range}(PV) = V$ (the image is exactly $V$) The kernel is $\ker(PV) = V^{\perp}$ (vectors mapping to zero are exactly those in $V^{\perp}$) The Correspondence Between Projections and Subspaces One-to-One Correspondence There's a beautiful connection: closed subspaces of $H$ are in one-to-one correspondence with bounded self-adjoint idempotent operators. That is, an operator $E$ satisfies $E^ = E$ and $E^2 = E$ if and only if $E = PV$ for some unique closed subspace $V$ (which equals $\text{range}(E)$). This correspondence goes both ways: Given a closed subspace $V$, you get the projection $PV$ Given a self-adjoint idempotent operator $E$, its range is a closed subspace and $E$ is the projection onto that subspace Why this matters: This establishes that projections and subspaces are essentially the same thing in Hilbert spaces. Studying one is equivalent to studying the other. Operations on Projections: When Do Combinations Give Projections? A natural question is: when are sums and products of projections again projections? Sums of Projections The sum $PU + PV$ is itself a projection if and only if $U$ and $V$ are orthogonal, meaning $U \perp V$ (equivalently, $PU PV = 0$). When this condition holds: $$PU + PV = P{U+V}$$ The projection onto the sum of two orthogonal subspaces equals the sum of their individual projections. Products of Projections The product $PU PV$ is a projection if and only if $PU$ and $PV$ commute (i.e., $PU PV = PV PU$). When they commute: $$PU PV = P{U \cap V}$$ The product gives the projection onto the intersection of the two subspaces. Note on commutativity: For projections, $PU PV = PV PU$ is equivalent to $U$ and $V$ being "aligned" in a certain sense. This is not always the case for arbitrary subspaces. Properties of Orthogonal Complements Double Orthogonal Complement An important property is that taking the orthogonal complement twice almost recovers the original subspace: $$\overline{V} = V^{\perp \perp}$$ That is, the closure of $V$ equals the orthogonal complement of its orthogonal complement. This shows that orthogonal complements "see" only closed subspaces; they automatically close things up. Monotonicity Orthogonal complement is monotone: if $U \subseteq V$, then $V^{\perp} \subseteq U^{\perp}$. Smaller subspaces have larger orthogonal complements. Equality $V^{\perp} = U^{\perp}$ holds if and only if $V = \overline{U}$ (they're the same up to closure). Complements of Sums For a collection of closed subspaces $Vi$: $$\left(\sumi Vi\right)^{\perp} = \bigcapi Vi^{\perp}$$ The orthogonal complement of a sum is the intersection of the orthogonal complements. This is the orthogonal dual of the formula $\overline{\sum Vi} = \sum \overline{Vi}$.