Description
Popular Neural Networks have various types of layers such as feedforward, residual, multi-head attention, bias, and normalization layers. Each layer represents a statistical summary (embedding) of the input data. The goal of this project is to understand how different types of layers of such architectures summarize input data distribution for specific purposes. Merging together concepts, approaches and cultures of theoretical machine learning and statistics, we will develop new methods and theories to investigate the embedding of data as a way to compress and specialise information. The embedded sufficient statistics achieves the (almost) minimum description that optimally encodes the data distribution, which is a long-standing open problem in Machine Learning.
Specific project requirements
-
Master degree in computer science, statistics, machine learning, data science, mathematics, physics, engineering, or other relevant field
-
Programming experience with deep neural networks (e.g. PyTorch, Tensorflow) is an advantage but is not necessary, as it can be learned.
Supervisors
-
Associate Professor Ali Ramezani-Kebrya, ali@ifi.uio.no contact person for inquiries about the position