Файл:3048-greedy-layer-wise-training-of-deep-networks.pdf

Материал из Материалы по машинному обучению
Перейти к: навигация, поиск
3048-greedy-layer-wise-training-of-deep-networks.pdf(0 × 0 пикселей, размер файла: 183 КБ, MIME-тип: application/pdf)

YoshuaBengio,PascalLamblin,DanPopovici,HugoLarochelle Universite´deMontre´al Montre´al,Que´bec {bengioy,lamblinp,popovicd,larocheh}@iro.umontreal.ca

Abstract

Complexity theory of circuits strongly suggests that deep architecture scan be much more efficient (sometimes exponentially) than shallow architectures, in terms of computational elements required to represent some functions. Deep multi-layer neural networks have many levels of non-linearities allowing them to compactly represent highly non-linear and highly-varyingfunctions. However, until recently it was not clear how to train such deep networks,since gradient-based optimization starting from randominitialization appears to often get stuck in poor solutions. Hintonet al. recently introduced a greedy layer-wise unsupervisedlearningalgorithm for Deep Belief Networks (DBN), a generative model with many layers of hidden causalvariables. Inthecontextoftheaboveoptimizationproblem,westudythisalgorithmempiricallyandexplorevariantstobetterunderstanditssuccessandextend it to cases where the inputsare continuousor where the structureof the inputdistributionisnotrevealingenoughaboutthevariabletobe predictedina supervised task. Ourexperimentsalsoconfirmthehypothesisthatthegreedylayer-wiseunsupervisedtrainingstrategymostlyhelpstheoptimization,byinitializingweightsina regionnearagoodlocalminimum,givingrisetointernaldistributedrepresentations thatarehigh-levelabstractionsoftheinput,bringingbettergeneralization.

История файла

Нажмите на дату/время, чтобы просмотреть, как тогда выглядел файл.

Дата/времяРазмерыУчастникПримечание
текущий17:52, 23 декабря 20160 × 0 (183 КБ)Slikos (обсуждение | вклад)
  • Вы не можете перезаписать этот файл.

Следующая 1 страница ссылается на данный файл: