NoBS Linear Algebra and Vector Geometry

Linear algebra is often taught as a collection of rules for manipulating grids of numbers. This is the wrong approach. Linear algebra is the mathematics of data and space. It is the language used to describe rotation, scaling, and translation in computer graphics, the training of neural networks in AI, and the stress points in bridges. "NoBS" Linear Algebra strips away the tedious computation and focuses on the geometry: understanding what is actually happening to the vectors.

What is a Vector?

In physics, a vector is an arrow pointing in space with a specific length (magnitude). In computer science, it is a list of ordered numbers. In NoBS Linear Algebra, we accept both definitions simultaneously.

A vector is an object that exists inside a vector space. If you are in 2D space, a vector is a pair of coordinates [x, y]. If you are in 3D, it is a triplet [x, y, z]. The fundamental operations are:

• Vector Addition: Placing the tail of one vector at the tip of another.
• Scaling: Stretching or shrinking a vector by a scalar (a single number).

Understanding these two operations allows you to grasp the concept of Linear Combinations. If you take two vectors, say v and w, and scale them by any numbers a and b, then add them, the result is a linear combination:

If you visualize every possible vector that can be created by every possible linear combination of v and w, you trace out a shape. If the vectors point in different directions (i.e., they are not parallel), that shape is a flat plane passing through the origin. This set of all possible outputs is called the Span.

Linear Transformations

This is the core of the subject. A transformation is a function that takes a vector as input and spits out a new vector as output. Usually, we visualize this as moving all the vectors in space to new positions.

A transformation is linear if it satisfies two rules:

1. Grid lines must remain parallel and evenly spaced.
2. The origin must remain fixed at (0,0).

This implies that any linear transformation can be fully described by where it sends the basis vectors. In 2D, the basis vectors are i-hat ([1, 0]) and j-hat ([0, 1]). If you know where i-hat lands and where j-hat lands, you can figure out where any vector lands.

This is where the Matrix comes in. A matrix is not just a table of data; it is a transformation encoded in numbers. The columns of the matrix are the transformed versions of the basis vectors.

The first column [a, b] tells you where i-hat goes. The second column [c, d] tells you where j-hat goes. When you multiply a matrix by a vector, you are calculating the coordinates of that vector in the transformed space.

The Dot Product and Geometry

While matrix-vector multiplication handles interactions between dimensions, the Dot Product handles the relationship between two vectors. Algebraically, you pair up the coordinates, multiply them, and add the results.

Geometrically, the dot product is a measure of how much two vectors align. It is the product of the length of the first vector and the projection of the second vector onto the first. It effectively tells you:

• If vectors point in the same direction, the result is positive.
• If they are perpendicular, the result is zero.
• If they point in opposite directions, the result is negative.

The Determinant: Scaling Space

Every matrix transformation changes the area of the unit square (in 2D) or the volume of the unit cube (in 3D). The factor by which the area or volume scales is called the Determinant.

If the determinant is 2, the area of any region in space doubles after the transformation. If the determinant is 0.5, the area shrinks by half. If the determinant is 0, space is squashed into a lower dimension (a line or a point), and the matrix is not invertible. A negative determinant implies that the orientation of space has been flipped (like turning a right hand into a left hand).

Vector Geometry: Cross Products

In three dimensions, the dot product doesn't give us a vector perpendicular to two inputs. For that, we need the Cross Product. Taking the cross product of two vectors v and w produces a new vector that is perpendicular to both.

The length of this new vector is equal to the area of the parallelogram formed by v and w. This makes the cross product incredibly useful in physics for calculating torque and in computer graphics for calculating surface normals (which direction a face is pointing) to determine lighting.

Eigenvectors and Eigenvalues

During a linear transformation, most vectors rotate off their original span. However, there are often special vectors that only get stretched or shrunk; they do not rotate. They remain on their own span. These are the Eigenvectors.

The factor by which an eigenvector is scaled during the transformation is the Eigenvalue.

Think of a rotation matrix. In 2D, almost every vector changes direction unless the rotation is 0 or 180 degrees. However, in 3D, when you spin an object (like the earth), there is an axis of rotation (the line through the poles). Vectors along this axis do not move; they are eigenvectors with an eigenvalue of 1.

Why This Matters

In data science, specifically in Principal Component Analysis (PCA), we use eigenvectors to find the axes along which a dataset varies the most. In physics, the eigenvectors of a stress tensor represent the principal directions of stress. In quantum mechanics, observables are represented by operators where the eigenvalues are the possible measurement outcomes.

Conclusion

NoBS Linear Algebra is about seeing the geometry behind the arithmetic. A matrix is a movement; a determinant is a change in size; an eigenvector is a stable axis. By internalizing these geometric concepts rather than memorizing arithmetic routines, you gain the intuition necessary to apply these tools to physics, engineering, and machine learning.