Файл:Dropout - A Simple Way to Prevent Neural Networks from Overﬁtting Srivastava14a.pdf
Nitish Srivastava email@example.com Geoﬀrey Hinton firstname.lastname@example.org Alex Krizhevsky email@example.com Ilya Sutskever firstname.lastname@example.org Ruslan Salakhutdinov email@example.com Department of Computer Science University of Toronto 10 Kings College Road, Rm 3302 Toronto, Ontario, M5S 3G4, Canada. Editor: Yoshua Bengio
Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overﬁtting is a serious problem in such networks. Large networks are also slow to use, making it diﬃcult to deal with overﬁtting by combining the predictions of many diﬀerent large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of diﬀerent “thinned” networks. At test time, it is easy to approximate the eﬀect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This signiﬁcantly reduces overﬁtting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classiﬁcation and computational biology, obtaining state-of-the-art results on many benchmark data sets.
Keywords: neural networks, regularization, model combination, deep learning
Нажмите на дату/время, чтобы просмотреть, как тогда выглядел файл.
|текущий||16:21, 22 декабря 2016||0 × 0 (2,74 МБ)||Slikos|
- Вы не можете перезаписать этот файл.