Professor Steinwart: AI Colloquium der Informatik

Title: “A Sober Look at Neural Network Initializations”

Abstract:
Initializing the weights and the biases is a key part of the training process of a neural network. Unlike the subsequent optimization phase, however, the initialization phase has gained only limited attention in the literature. In the first part of the talk, I will discuss some consequences of commonly used initialization strategies for vanilla DNNs with ReLU activations. Based on these insights I will then introduce an alternative initialization strategy, and finally, I will present some large scale experiments assessing the quality of the new initialization strategy.

AI Colloquium UP COMMING TALKS