Optimization for ML (OML)

Description

This course introduces key fundamental concepts and techniques of optimization applied to machine learning problems. It covers a range of optimization methods, from classical approaches to recent advancements, emphasizing their applications in designing and training machine learning models. The curriculum bridges theoretical concepts with practical implementation, providing students with the tools to solve complex optimization problems encountered in real-world machine learning tasks. The student will learn when to use which method, which tooling is appropriate in which situation, and the connections between the different methods. We will also show how these methods fit within the broader framework of the mathematical foundations of optimization methods and their importance in machine learning and apply various optimization techniques to train machine learning models effectively to analyse the importance of convergence properties of optimization algorithms and their impact on model performance. Finally, in the tutorials and labs, students will gain hands-on experience through several coding assignments and by participating in a project.