Linear Algebra as a Mathematical Foundation for Machine Learning Algorithms

Machine Learning (ML) is no longer a novel academic domain or even a niche within a specific technology but has become a technological engine that transforms many sectors of life: healthcare and finance, transportation and entertainment, etc. The effectiveness of such algorithms in analysing complex data, recognising patterns and giving precise predictions is frequently viewed as a kind of computational alchemy. This impression is however, false and does not represent a hard mathematical framework which is fundamental to both the knowledge and development of the field. Linear algebra forms the basis of this foundation by a very huge margin. In this paper, a detailed assessment will be given explaining the inseparable nature of linear algebra as the language of machine learning. The goal is to systematically break down the important ML algorithms, including both simplistic linear regression, but also more complicated deep learning models, and explicitly trace their basic mechanisms to the underlying linear algebraic operations, including: matrix multiplication, transformations of vectors spaces, and manipulations of tensors. This review will help to dispel the mystique of the black box nature of ML by showing that data representation, model operation, and optimization are all intrinsically linear algebraic operations. This synthesis is valuable to students in need of a more conceptual grasp, researchers in need of creating new algorithms, and practitioners in need of debugging, optimizing, and innovating their ML pipelines. Detailed understanding of these mathematical foundations is not only academic but a requirement to be able to master the practice and innovate in the sphere of artificial intelligence.