🔶Intro to Abstract Math Unit 9 – Vector Spaces and Linear Algebra

Key Concepts and Definitions

• Vector space consists of a set of vectors and two operations (addition and scalar multiplication) that satisfy certain axioms
• Field is a set of scalars with addition and multiplication operations that satisfy specific properties (commutativity, associativity, distributivity, identity elements, and inverses)
• Vector is an element of a vector space, typically represented as an ordered list of numbers or a directed line segment
• Scalar is an element of the underlying field used to scale vectors, usually a real or complex number
• Subspace is a subset of a vector space that is closed under vector addition and scalar multiplication, forming a vector space itself
  • Must contain the zero vector and be closed under linear combinations of its elements
• Linear combination is a sum of vectors multiplied by scalars, expressed as $a_1v_1 + a_2v_2 + ... + a_nv_n$ where $a_i$ are scalars and $v_i$ are vectors
• Span is the set of all possible linear combinations of a given set of vectors, forming a subspace

Vector Space Fundamentals

• Vector space axioms ensure consistency and structure for vector operations, allowing for meaningful mathematical analysis
  • Closure under addition and scalar multiplication, associativity, commutativity, existence of identity elements, and existence of inverses
• Examples of vector spaces include $\mathbb{R}^n$ (real coordinate space), $\mathbb{C}^n$ (complex coordinate space), and the set of all polynomials with real coefficients
• Zero vector is the additive identity element in a vector space, denoted as $\vec{0}$ or $\mathbf{0}$, and has all components equal to zero
• Scalar multiplication distributes over vector addition: $a(\vec{u} + \vec{v}) = a\vec{u} + a\vec{v}$ for any scalar $a$ and vectors $\vec{u}$ and $\vec{v}$
• Vector addition is commutative and associative: $\vec{u} + \vec{v} = \vec{v} + \vec{u}$ and $(\vec{u} + \vec{v}) + \vec{w} = \vec{u} + (\vec{v} + \vec{w})$ for any vectors $\vec{u}$, $\vec{v}$, and $\vec{w}$
• Additive inverse of a vector $\vec{v}$ is denoted as $-\vec{v}$ and satisfies $\vec{v} + (-\vec{v}) = \vec{0}$
• Scalar multiplication by zero always yields the zero vector: $0\vec{v} = \vec{0}$ for any vector $\vec{v}$

Linear Combinations and Span

• Linear combination expresses a vector as a weighted sum of other vectors, providing a way to construct new vectors from existing ones
• Coefficients in a linear combination are the scalars that multiply the vectors, determining the contribution of each vector to the resulting sum
• Span of a set of vectors is the smallest subspace containing all linear combinations of those vectors
  • Can be thought of as the "reach" or "coverage" of the set of vectors within the vector space
• Spanning set is a set of vectors whose span is the entire vector space, allowing any vector in the space to be expressed as a linear combination of the spanning set
• Trivial subspace is the subspace containing only the zero vector, denoted as $\{\vec{0}\}$
• Nontrivial subspace is any subspace other than the trivial subspace and the entire vector space itself
• Generating a subspace refers to finding a set of vectors whose span is that subspace, providing a basis for the subspace

Linear Independence and Dependence

• Linearly independent set of vectors has no vector that can be expressed as a linear combination of the others, ensuring each vector contributes uniquely to the span
  • Removing any vector from a linearly independent set reduces the span
• Linearly dependent set of vectors has at least one vector that can be expressed as a linear combination of the others, resulting in redundancy within the set
  • Removing a linearly dependent vector does not change the span of the set
• Trivial linear combination is a linear combination where all coefficients are zero, always resulting in the zero vector
• Nontrivial linear combination is a linear combination where at least one coefficient is nonzero, potentially producing a nonzero vector
• Testing for linear independence involves solving a homogeneous system of linear equations and checking if the only solution is the trivial solution (all coefficients equal to zero)
• Linearly dependent vectors introduce redundancy and can be expressed in terms of other vectors in the set
• Linearly independent vectors are essential for constructing minimal spanning sets and bases for vector spaces and subspaces

Basis and Dimension

• Basis is a linearly independent spanning set for a vector space or subspace, providing a minimal and unique representation for every vector in the space
  • Allows for efficient and unambiguous representation of vectors using coordinate vectors
• Standard basis for $\mathbb{R}^n$ is the set of unit vectors $\{\hat{e}_1, \hat{e}_2, ..., \hat{e}_n\}$ where $\hat{e}_i$ has a 1 in the $i$-th position and zeros elsewhere
• Coordinate vector represents a vector in terms of a basis, with each component indicating the coefficient of the corresponding basis vector
• Change of basis is the process of expressing a vector in terms of a different basis, often used to simplify calculations or gain insights into the vector space structure
• Dimension of a vector space is the number of vectors in any basis for that space, representing the "size" or "degrees of freedom" of the space
  • All bases for a vector space have the same number of vectors
• Finite-dimensional vector space has a finite basis and dimension, while an infinite-dimensional vector space has an infinite basis and dimension (e.g., the space of all polynomials)
• Rank of a matrix is the dimension of the vector space spanned by its columns or rows, indicating the number of linearly independent columns or rows

Linear Transformations

• Linear transformation is a function between vector spaces that preserves vector addition and scalar multiplication, mapping vectors from one space to another while maintaining linear structure
  • Denoted as $T: V \to W$ where $V$ and $W$ are vector spaces and $T(\vec{u} + \vec{v}) = T(\vec{u}) + T(\vec{v})$ and $T(a\vec{v}) = aT(\vec{v})$ for any vectors $\vec{u}, \vec{v} \in V$ and scalar $a$
• Domain of a linear transformation is the vector space of inputs, while the codomain is the vector space of outputs
• Range or image of a linear transformation is the set of all output vectors, forming a subspace of the codomain
• Kernel or null space of a linear transformation is the set of all input vectors that map to the zero vector in the codomain, forming a subspace of the domain
• Injectivity or one-to-one property holds when a linear transformation maps distinct input vectors to distinct output vectors, implying a trivial kernel
• Surjectivity or onto property holds when a linear transformation maps the domain onto the entire codomain, meaning the range equals the codomain
• Isomorphism is a bijective (injective and surjective) linear transformation, establishing a one-to-one correspondence between vector spaces while preserving linear structure

Matrix Representations

• Matrix representation of a linear transformation encodes the transformation as a matrix, allowing for compact representation and computation
  • Columns of the matrix represent the images of the basis vectors under the transformation
• Matrix-vector multiplication computes the output vector of a linear transformation given an input vector, using the matrix representation of the transformation
• Composition of linear transformations corresponds to matrix multiplication of their respective matrix representations, enabling chained transformations
• Inverse of a linear transformation, if it exists, is another linear transformation that "undoes" the original transformation, represented by the inverse matrix
• Determinant of a square matrix is a scalar value that provides information about the invertibility and volume scaling properties of the associated linear transformation
• Eigenvalues and eigenvectors of a square matrix represent special scalar-vector pairs where the transformation only scales the vector, providing insights into the transformation's behavior
  • Eigenvalue equation: $A\vec{v} = \lambda\vec{v}$ where $A$ is the matrix, $\vec{v}$ is an eigenvector, and $\lambda$ is the corresponding eigenvalue
• Diagonalization is the process of expressing a matrix as a product of a diagonal matrix (containing eigenvalues) and a matrix of eigenvectors, simplifying computations and analysis

Applications and Examples

• Linear algebra is fundamental to many areas of mathematics, science, and engineering, providing a framework for modeling and solving problems involving linear relationships
• Computer graphics uses linear transformations (e.g., rotations, reflections, scaling) to manipulate and render 2D and 3D objects, with matrices representing these transformations
• Machine learning and data analysis rely on linear algebra concepts for tasks such as dimensionality reduction (principal component analysis), clustering, and classification
  • Eigenfaces use eigenvectors of covariance matrices to represent and recognize faces in computer vision
• Quantum mechanics employs complex vector spaces (Hilbert spaces) to describe the states of quantum systems, with linear operators representing observables and transformations
• Fourier analysis decomposes functions into linear combinations of sinusoidal basis functions, enabling signal processing and frequency-domain analysis
• Markov chains use transition matrices to model and analyze stochastic processes, with eigenvectors and eigenvalues providing long-term behavior insights
• Optimization problems often involve linear constraints and objectives, with linear programming techniques (e.g., simplex method) used to find optimal solutions
• Cryptography utilizes linear algebra concepts for encryption and decryption, such as matrix-based ciphers and lattice-based cryptosystems