Файл:Long short-term memory Hochreiter97 lstm.pdf

Материал из Материалы по машинному обучению
Перейти к: навигация, поиск
Long_short-term_memory_Hochreiter97_lstm.pdf(0 × 0 пикселей, размер файла: 406 КБ, MIME-тип: application/pdf)


Neural Computation 9(8):1735{1780, 1997

Sepp Hochreiter

Fakult?at f?ur Informatik

Technische Universit?at M?unchen

80290 M?unchen, Germany



J?urgen Schmidhuber


Corso Elvezia 36

6900 Lugano, Switzerland




Learning to store information over extended time intervals via recurrent backpropagation takes a very long time, mostly due to insufficient, decaying error back flow. We briey review Hochreiter's 1991 analysis of this problem, then address it by introducing a novel, efficient, gradient-based method called "Long Short-Term Memory" (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete time steps by enforcing constant error floow through "constant error carrousels" within special units. Multiplicative gate units learn to open and close access to the constant error floow. LSTM is local in space and time; its computational complexity per time step and weight is O(1). Our experiments with artifficial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with RTRL, BPTT, Recurrent Cascade-Correlation, Elman nets, and Neural Sequence Chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artifficial long time lag tasks that have never been solved by previous recurrent network algorithms.

Keywords: Long Short-Term Memory (LSTM), Back-Propagation Through Time (BPTT), Real-Time Recurrent Learning (RTRL), Elman network (ELM), Recurrent Cascade-Correlation (RCC), Neural Sequence Chunker (CH)

История файла

Нажмите на дату/время, чтобы просмотреть, как тогда выглядел файл.

текущий18:08, 23 декабря 20160 × 0 (406 КБ)Slikos (обсуждение | вклад)
  • Вы не можете перезаписать этот файл.

Следующие 2 страницы ссылаются на данный файл: