Lecture 1: Introduction to Linear Algebra and Matrix Theory Matrix Multiplication Example

Lecture 1: Introduction to Linear Algebra and Matrix Theory

1. Introduction

Linear Algebra is the foundation of many machine learning algorithms. It deals with vectors, matrices, and linear transformations. Understanding these concepts is essential because data in machine learning is usually represented as vectors and matrices, and most algorithms involve operations on them.

2. Vectors

A vector is an ordered list of numbers. It can represent a data point (features of a person, pixel values of an image, etc.). Example:

  v = [3, 5, 7]   # a 3-dimensional vector

In machine learning, a vector often represents a single observation in the dataset. For example, [height, weight, age] of a person.

3. Matrices

A matrix is a rectangular array of numbers arranged in rows and columns. Example:

\[ A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix} \]

In machine learning, a dataset with m observations and n features is represented as an m × n matrix.

4. Matrix Operations

Addition: Add corresponding elements.
Multiplication: Combine rows and columns (used in neural networks).
Transpose: Flip rows into columns.
Determinant: Tells about invertibility of a square matrix.
Inverse: Needed for solving linear equations.

5. Applications in Machine Learning

Data Representation: Dataset stored as a matrix (rows = samples, columns = features).
Linear Regression: The formula y = Xβ uses matrix multiplication.
Neural Networks: Forward propagation involves multiplying weight matrices with input vectors.
Dimensionality Reduction: PCA uses eigenvalues and eigenvectors of covariance matrices.
Recommendation Systems: Use matrix factorization (Netflix, Amazon product recommendations).

\section*{6. Example} Suppose we have two matrices: \[ A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}, \quad B = \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix} \] \textbf{Matrix Multiplication:} \[ A \times B = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix} = \begin{bmatrix} 1 \cdot 5 + 2 \cdot 7 & 1 \cdot 6 + 2 \cdot 8 \\ 3 \cdot 5 + 4 \cdot 7 & 3 \cdot 6 + 4 \cdot 8 \end{bmatrix} \] \[ = \begin{bmatrix} 19 & 22 \\ 43 & 50 \end{bmatrix} \]

7. Interactive Matrix Calculator

Enter matrices in JSON-like format, e.g. [[1,2],[3,4]]