Jiayi Li, UCLA
Optimization in Polynomial Neural Networks: Insights from Algebraic Geometry
Abstract
With neural networks serving as a cornerstone in modern artificial intelligence, understanding their optimization process becomes important for improving the accuracy and efficiency of the model. In particular, neural networks with polynomial or rational activation functions offer rich structures that extend beyond traditional architectures. We study the geometry of solution spaces and the nature of critical points in the optimization landscapes. By leveraging concepts such as Gröbner bases, algebraic varieties, and ideals, we analyze the structure of the loss surface and characterize the critical points on the function space. Through theoretical insights and practical examples, we will demonstrate how these algebraic techniques can be extended to study large, high-dimensional models.