Created Friday 24 January 2025
I came across a series of blog posts about manifolds:
One place where I want to use "optimization on manifolds" is actually regular optimization of functions with singularities. Singularities may arise in physical systems when some parameter is getting close to a physically impossible value; or they may arise as a consequence of an "inner penalty function" technique for constrained optimization. Such optimization problems may be theoretically appropriate for the use of a gradient-descent method (the functions may be smooth and local optima may be appropriate for the practical problem at hand), but the large numbers arising from the singularities of the functions involved may make the numerical errors of the usual floating-point computations unacceptable. My hope is that we could reformulate the optimization problem such that the optimization domain is the graph of the objective function, rather than the usual Euclidean space of the independent variables. With such reformulation we may avoid the peculiarities of the floating-point computations, and keep all the theoretical advantages of the problem.