Файл:3048-greedy-layer-wise-training-of-deep-networks.pdf
YoshuaBengio,PascalLamblin,DanPopovici,HugoLarochelle Universite´deMontre´al Montre´al,Que´bec {bengioy,lamblinp,popovicd,larocheh}@iro.umontreal.ca
Abstract
Complexity theory of circuits strongly suggests that deep architecture scan be much more efficient (sometimes exponentially) than shallow architectures, in terms of computational elements required to represent some functions. Deep multi-layer neural networks have many levels of non-linearities allowing them to compactly represent highly non-linear and highly-varyingfunctions. However, until recently it was not clear how to train such deep networks,since gradient-based optimization starting from randominitialization appears to often get stuck in poor solutions. Hintonet al. recently introduced a greedy layer-wise unsupervisedlearningalgorithm for Deep Belief Networks (DBN), a generative model with many layers of hidden causalvariables. Inthecontextoftheaboveoptimizationproblem,westudythisalgorithmempiricallyandexplorevariantstobetterunderstanditssuccessandextend it to cases where the inputsare continuousor where the structureof the inputdistributionisnotrevealingenoughaboutthevariabletobe predictedina supervised task. Ourexperimentsalsoconfirmthehypothesisthatthegreedylayer-wiseunsupervisedtrainingstrategymostlyhelpstheoptimization,byinitializingweightsina regionnearagoodlocalminimum,givingrisetointernaldistributedrepresentations thatarehigh-levelabstractionsoftheinput,bringingbettergeneralization.
История файла
Нажмите на дату/время, чтобы просмотреть, как тогда выглядел файл.
Дата/время | Размеры | Участник | Примечание | |
---|---|---|---|---|
текущий | 17:52, 23 декабря 2016 | 0 × 0 (183 КБ) | Slikos (обсуждение | вклад) |
- Вы не можете перезаписать этот файл.
Использование файла
Следующая 1 страница ссылается на данный файл: