🧮Physical Sciences Math Tools Unit 13 – Linear Algebra & Matrix Operations

Key Concepts and Definitions

• Scalars represent single numerical values without direction or orientation (3, -7.5, π)
• Vectors are mathematical objects possessing both magnitude and direction, often represented by arrows or ordered pairs/triples
• Matrices are rectangular arrays of numbers arranged in rows and columns, used to represent linear transformations and solve systems of equations
• Linear independence means a set of vectors cannot be expressed as linear combinations of each other, forming a basis for a vector space
• Span refers to the set of all possible linear combinations of a given set of vectors, forming a subspace
• Eigenvalues are special scalar values associated with a matrix, satisfying the equation $Av = λv$ for some nonzero vector $v$
  • The corresponding nonzero vectors $v$ are called eigenvectors
• Determinants are scalar values associated with square matrices, used to determine matrix invertibility and calculate volumes and areas in higher dimensions

Vector Operations and Properties

• Vector addition follows the parallelogram law, where the resultant vector is obtained by placing the tail of one vector at the head of the other and drawing a new vector from the free tail to the free head
• Scalar multiplication of a vector involves multiplying each component of the vector by a scalar value, resulting in a new vector with the same direction but scaled magnitude
• The dot product (scalar product) of two vectors is the sum of the products of their corresponding components, resulting in a scalar value
  • Geometrically, the dot product represents the product of the magnitudes of the vectors and the cosine of the angle between them: $a \cdot b = |a||b|\cos θ$
• The cross product of two 3D vectors results in a new vector perpendicular to both original vectors, with magnitude equal to the area of the parallelogram formed by the vectors
• Vector spaces are sets of vectors closed under addition and scalar multiplication, satisfying specific axioms (associativity, commutativity, identity, inverse)
• Orthogonality refers to two vectors being perpendicular to each other, characterized by a zero dot product

Matrix Basics and Notation

• Matrices are denoted using capital letters (A, B, C), with elements represented by lowercase letters with subscripts indicating row and column indices ($a_{ij}$ for the element in the $i$-th row and $j$-th column)
• The main diagonal of a square matrix consists of elements where the row and column indices are equal ($a_{11}, a_{22}, ..., a_{nn}$)
• A matrix is symmetric if it equals its transpose ($A = A^T$), where the transpose is obtained by interchanging the rows and columns
• Identity matrices have 1s on the main diagonal and 0s elsewhere, denoted by $I_n$ for an $n \times n$ matrix
  • Multiplying a matrix by its corresponding identity matrix results in the original matrix: $AI_n = I_nA = A$
• Diagonal matrices have nonzero elements only on the main diagonal, with all other elements being zero
• Triangular matrices have nonzero elements only on and above (upper triangular) or on and below (lower triangular) the main diagonal

Matrix Operations and Algebra

• Matrix addition is performed element-wise, adding corresponding elements from two matrices of the same size
• Scalar multiplication of a matrix involves multiplying each element of the matrix by a scalar value
• Matrix multiplication is a binary operation that produces a matrix from two matrices, following the rule $(AB)_{ij} = \sum_{k} a_{ik}b_{kj}$
  • The number of columns in the first matrix must equal the number of rows in the second matrix for multiplication to be defined
• Matrix multiplication is associative $((AB)C = A(BC))$ and distributive over addition $(A(B+C) = AB + AC)$, but not commutative $(AB \neq BA)$ in general
• The inverse of a square matrix $A$, denoted $A^{-1}$, satisfies $AA^{-1} = A^{-1}A = I$
  • A matrix is invertible if and only if its determinant is nonzero
• The rank of a matrix is the maximum number of linearly independent rows or columns, determining the dimension of the vector space spanned by its rows or columns

Systems of Linear Equations

• A system of linear equations is a collection of linear equations involving the same variables, often represented in matrix form $Ax = b$
• The coefficient matrix $A$ contains the coefficients of the variables in the system, the vector $x$ represents the variables, and the vector $b$ contains the constant terms
• A solution to a system of linear equations is an assignment of values to the variables that satisfies all equations simultaneously
• Systems can have no solution (inconsistent), a unique solution, or infinitely many solutions, depending on the ranks of the coefficient matrix $A$ and the augmented matrix $[A|b]$
  • If $rank(A) = rank([A|b]) = n$ (the number of variables), the system has a unique solution
  • If $rank(A) = rank([A|b]) < n$, the system has infinitely many solutions
  • If $rank(A) \neq rank([A|b])$, the system has no solution (inconsistent)
• Gaussian elimination is a method for solving systems of linear equations by applying elementary row operations to transform the augmented matrix into row echelon form

Determinants and Their Applications

• The determinant of a square matrix is a scalar value that encodes important properties of the matrix, denoted $det(A)$ or $|A|$
• Determinants can be calculated using various methods, such as cofactor expansion or Gaussian elimination
  • For a $2 \times 2$ matrix $A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}$, the determinant is given by $det(A) = ad - bc$
• Properties of determinants include:
  • $det(AB) = det(A) \cdot det(B)$
  • $det(A^T) = det(A)$
  • $det(A^{-1}) = \frac{1}{det(A)}$ for invertible matrices
• Determinants can be used to check matrix invertibility, as a matrix is invertible if and only if its determinant is nonzero
• In geometry, determinants can be used to calculate areas and volumes in higher dimensions
  • The absolute value of the determinant of a $2 \times 2$ matrix represents the area of the parallelogram formed by its column vectors
  • The absolute value of the determinant of a $3 \times 3$ matrix represents the volume of the parallelepiped formed by its column vectors

Eigenvalues and Eigenvectors

• Eigenvalues are scalar values $λ$ associated with a square matrix $A$, satisfying the equation $Av = λv$ for some nonzero vector $v$
  • The corresponding nonzero vectors $v$ are called eigenvectors
• The eigenvalue equation $Av = λv$ can be rewritten as $(A - λI)v = 0$, which has a nontrivial solution if and only if $det(A - λI) = 0$
  • This equation, $det(A - λI) = 0$, is called the characteristic equation of the matrix $A$
• Eigenvalues and eigenvectors have numerous applications, such as in matrix diagonalization, systems of differential equations, and principal component analysis
• A matrix is diagonalizable if it can be written as $A = PDP^{-1}$, where $D$ is a diagonal matrix containing the eigenvalues, and $P$ is a matrix whose columns are the corresponding eigenvectors
• The geometric interpretation of eigenvectors is that they represent directions in which the linear transformation described by the matrix acts as a scaling operation

Real-World Applications in Physical Sciences

• In quantum mechanics, the Schrödinger equation $Hψ = Eψ$ is an eigenvalue problem, where $H$ is the Hamiltonian operator, $E$ represents the energy eigenvalues, and $ψ$ represents the corresponding eigenstates (wavefunctions)
• In classical mechanics, the equations of motion for coupled oscillators can be written as a matrix differential equation $M\ddot{x} + Kx = 0$, where $M$ is the mass matrix, $K$ is the stiffness matrix, and $x$ is the vector of displacements
  • The natural frequencies and modes of vibration can be found by solving the eigenvalue problem $(K - ω^2M)v = 0$
• In electrical engineering, the admittance matrix $Y$ relates the currents and voltages in a circuit through the equation $I = YV$, where $I$ is the current vector and $V$ is the voltage vector
• In computer graphics, linear transformations such as rotations, reflections, and shears can be represented by matrices acting on vectors representing points or vertices in 2D or 3D space
• In data analysis and machine learning, principal component analysis (PCA) uses eigenvalues and eigenvectors of the covariance matrix to identify the most important features and reduce the dimensionality of datasets
• In structural engineering, the stiffness matrix relates the forces and displacements in a structure through the equation $F = Kx$, where $F$ is the force vector, $K$ is the stiffness matrix, and $x$ is the displacement vector
  • Eigenvalue analysis of the stiffness matrix can help determine the buckling loads and mode shapes of the structure