Exploring Reformulations of Neural Network Training

Deep neural networks are most commonly trained via back-propagation and gradient-based optimisation. While this paradigm has been highly successful, it remains sensitive to gradient instabilities, non-smooth activations, and an inherently sequential structure that limits parallelisation. This talk presents recent advances that view neural network training from an alternative perspective. By reformulating learning as a higher-dimensional constrained optimisation problem, this approach improves the conditioning of the training process and opens new avenues for distributed updates. I will describe how this framework accommodates a range of architectures and algorithmic strategies, and demonstrate its applications to inverse problems.