|Zeit:||09.04.19, 15:00 – 16:00 Uhr|
|Download als iCal:||
Title: “A Sober Look at Neural Network Initializations”
Initializing the weights and the biases is a key part of the training process of a neural network. Unlike the subsequent optimization phase, however, the initialization phase has gained only limited attention in the literature. In the first part of the talk, I will discuss some consequences of commonly used initialization strategies for vanilla DNNs with ReLU activations. Based on these insights I will then introduce an alternative initialization strategy, and finally, I will present some large scale experiments assessing the quality of the new initialization strategy.