Vectors in Linear Algebra and the Cauchy-Schwarz Inequality

Problem Statement

In high school, we typically visualize vectors as arrows pointing in 2D or 3D space. However, in higher mathematics and computer science, vectors are generalized as finite sequences of real numbers, e.g., $\mathbf{a} = (a_1, a_2, \dots, a_n)$. This generalisation forms the basis of Linear Algebra, where vectors are used to represent data, change coordinate systems, and perform complex statistical modelling.

One of the most important theoretical bridges between physical geometry and abstract algebra is the Cauchy-Schwarz Inequality. It stems directly from the geometric definition of the dot product and provides a fundamental upper bound for the multiplication of sequences.

Consider two non-zero vectors in 3D space: $\mathbf{u} = (u_1, u_2, u_3)$ and $\mathbf{v} = (v_1, v_2, v_3)$.

(a) State the geometric definition of the dot product $\mathbf{u} \cdot \mathbf{v}$.

(b) Using the fact that the maximum value of $\cos\theta$ is 1, derive the vector form of the Cauchy-Schwarz inequality: $|\mathbf{u} \cdot \mathbf{v}| \le |\mathbf{u}| |\mathbf{v}|$.

(c) Expand this vector inequality into its algebraic form to show that for any real numbers $u_1, u_2, u_3$ and $v_1, v_2, v_3$: $(u_1v_1 + u_2v_2 + u_3v_3)^2 \le (u_1^2 + u_2^2 + u_3^2)(v_1^2 + v_2^2 + v_3^2)$

(d) When does the equality hold true?

Hints

• Part (a): The geometric definition links the dot product to the magnitudes of the vectors and the angle $\theta$ between them.
• Part (b): Start with the definition from (a). Take the absolute value of both sides. What is the maximum possible value for $|\cos\theta|$? Use this to establish a "less than or equal to" relationship.
• Part (c): Take the inequality $|\mathbf{u} \cdot \mathbf{v}| \le |\mathbf{u}| |\mathbf{v}|$ and square both sides. Then, substitute the algebraic definitions for the dot product ($\mathbf{u} \cdot \mathbf{v} = u_1v_1 + u_2v_2 + u_3v_3$) and the square of the magnitudes ($|\mathbf{u}|^2 = u_1^2 + u_2^2 + u_3^2$).
• Part (d): Equality happens when $\cos\theta = \pm 1$. What does this mean geometrically for the vectors $\mathbf{u}$ and $\mathbf{v}$?

Solutions

Part (a): Geometric Definition

1. The geometric definition of the dot product of two vectors $\mathbf{u}$ and $\mathbf{v}$ is: $\mathbf{u} \cdot \mathbf{v} = |\mathbf{u}| |\mathbf{v}| \cos\theta$ where $\theta$ is the angle between the two vectors ($0 \le \theta \le \pi$).

Part (b): Deriving Vector Cauchy-Schwarz

1. Start with the definition: $\mathbf{u} \cdot \mathbf{v} = |\mathbf{u}| |\mathbf{v}| \cos\theta$
2. Take the absolute value of both sides (since lengths are positive, we only need to apply it to the left side and the cosine term): $|\mathbf{u} \cdot \mathbf{v}| = |\mathbf{u}| |\mathbf{v}| |\cos\theta|$
3. We know from trigonometry that the cosine function is bounded between $-1$ and $1$. Therefore, its absolute value is bounded by $1$: $|\cos\theta| \le 1$
4. Substitute this maximum value into our equation to create an inequality: $|\mathbf{u} \cdot \mathbf{v}| \le |\mathbf{u}| |\mathbf{v}| (1)$ $|\mathbf{u} \cdot \mathbf{v}| \le |\mathbf{u}| |\mathbf{v}|$ This is the vector form of the Cauchy-Schwarz inequality.

Part (c): Expanding to Algebraic Form

1. Square both sides of the vector inequality to remove the absolute value and square root signs: $(\mathbf{u} \cdot \mathbf{v})^2 \le |\mathbf{u}|^2 |\mathbf{v}|^2$
2. Recall the algebraic definition of the dot product for 3D vectors: $\mathbf{u} \cdot \mathbf{v} = u_1v_1 + u_2v_2 + u_3v_3$
3. Recall the definition of the squared magnitude: $|\mathbf{u}|^2 = u_1^2 + u_2^2 + u_3^2$ $|\mathbf{v}|^2 = v_1^2 + v_2^2 + v_3^2$
4. Substitute these algebraic definitions back into the squared inequality: $(u_1v_1 + u_2v_2 + u_3v_3)^2 \le (u_1^2 + u_2^2 + u_3^2)(v_1^2 + v_2^2 + v_3^2)$ This is the algebraic form of Cauchy-Schwarz for three variables. (This logic easily extends to $n$ variables!).

Part (d): Conditions for Equality

1. Looking back at step (b), the inequality was formed because we replaced $|\cos\theta|$ with its maximum value, $1$.
2. Therefore, the equality holds exactly when $|\cos\theta| = 1$.
3. This occurs when $\theta = 0$ or $\theta = \pi$.
4. Geometrically, this means the vectors are either pointing in the exact same direction or exact opposite directions. In other words, they are parallel (or collinear).
5. Algebraically, equality holds when one vector is a scalar multiple of the other: $\mathbf{u} = k\mathbf{v}$ for some real number $k$.

Takeaways

• Beyond Geometry: The Cauchy-Schwarz inequality is a perfect example of a theorem rooted in visual geometry ($\cos\theta$) that yields a profoundly powerful tool for pure algebra and inequalities.
• Generalisation: While we proved it for 3D vectors, because a vector can be defined as an $n$-dimensional sequence, this inequality holds true for any number of variables. This is heavily used in statistics and data science.
• Equality Conditions: In inequality proofs, determining when the equality holds is just as important as the inequality itself. For Cauchy-Schwarz, it's always when the "vectors" are proportional.

Further Readings

• HSC Inequalities: https://vumaths.com/booklets/hsc-inequalities/
• HSC Vectors: https://vumaths.com/booklets/hsc-vectors/
• HSC Proofs: https://vumaths.com/booklets/hsc-proofs/

Connect with me

If you want to master the bridge between vectors and algebraic inequalities, check out the comprehensive booklets on Vu's Maths Hub. I post detailed walkthroughs of tricky proofs on my YouTube channel. For more mathematical insights, follow me on Instagram and subscribe to my Substack.