A Novel Fractional Gradient-Based Learning Algorithm for Recurrent Neural Networks

Shujaat Khan; Jawwad Ahmad; Imran Naseem; Muhammad Moinuddin

doi:10.1007/s00034-017-0572-z

A Novel Fractional Gradient-Based Learning Algorithm for Recurrent Neural Networks

Shujaat Khan, Jawwad Ahmad, Imran Naseem^*, Muhammad Moinuddin

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

45 Scopus citations

Abstract

In this research, we propose a novel algorithm for learning of the recurrent neural networks called as the fractional back-propagation through time (FBPTT). Considering the potential of the fractional calculus, we propose to use the fractional calculus-based gradient descent method to derive the FBPTT algorithm. The proposed FBPTT method is shown to outperform the conventional back-propagation through time algorithm on three major problems of estimation namely nonlinear system identification, pattern classification and Mackey–Glass chaotic time series prediction.

Original language	English
Pages (from-to)	593-612
Number of pages	20
Journal	Circuits, Systems, and Signal Processing
Volume	37
Issue number	2
DOIs	https://doi.org/10.1007/s00034-017-0572-z
State	Published - 1 Feb 2018
Externally published	Yes

Bibliographical note

Publisher Copyright:
© 2017, Springer Science+Business Media New York.

Keywords

Back-propagation through time (BPTT)
Fractional calculus
Gradient descent
Mackey–Glass chaotic time series
Minimum redundancy and maximum relevance (mRMR)
Recurrent neural network (RNN)

ASJC Scopus subject areas

Signal Processing
Applied Mathematics

Access to Document

10.1007/s00034-017-0572-z

Cite this

@article{1af151807362449b861716d21d77b313,

title = "A Novel Fractional Gradient-Based Learning Algorithm for Recurrent Neural Networks",

abstract = "In this research, we propose a novel algorithm for learning of the recurrent neural networks called as the fractional back-propagation through time (FBPTT). Considering the potential of the fractional calculus, we propose to use the fractional calculus-based gradient descent method to derive the FBPTT algorithm. The proposed FBPTT method is shown to outperform the conventional back-propagation through time algorithm on three major problems of estimation namely nonlinear system identification, pattern classification and Mackey–Glass chaotic time series prediction.",

keywords = "Back-propagation through time (BPTT), Fractional calculus, Gradient descent, Mackey–Glass chaotic time series, Minimum redundancy and maximum relevance (mRMR), Recurrent neural network (RNN)",

author = "Shujaat Khan and Jawwad Ahmad and Imran Naseem and Muhammad Moinuddin",

note = "Publisher Copyright: {\textcopyright} 2017, Springer Science+Business Media New York.",

year = "2018",

month = feb,

day = "1",

doi = "10.1007/s00034-017-0572-z",

language = "English",

volume = "37",

pages = "593--612",

journal = "Circuits, Systems, and Signal Processing",

issn = "0278-081X",

number = "2",

}

TY - JOUR

T1 - A Novel Fractional Gradient-Based Learning Algorithm for Recurrent Neural Networks

AU - Khan, Shujaat

AU - Ahmad, Jawwad

AU - Naseem, Imran

AU - Moinuddin, Muhammad

PY - 2018/2/1

Y1 - 2018/2/1

N2 - In this research, we propose a novel algorithm for learning of the recurrent neural networks called as the fractional back-propagation through time (FBPTT). Considering the potential of the fractional calculus, we propose to use the fractional calculus-based gradient descent method to derive the FBPTT algorithm. The proposed FBPTT method is shown to outperform the conventional back-propagation through time algorithm on three major problems of estimation namely nonlinear system identification, pattern classification and Mackey–Glass chaotic time series prediction.

AB - In this research, we propose a novel algorithm for learning of the recurrent neural networks called as the fractional back-propagation through time (FBPTT). Considering the potential of the fractional calculus, we propose to use the fractional calculus-based gradient descent method to derive the FBPTT algorithm. The proposed FBPTT method is shown to outperform the conventional back-propagation through time algorithm on three major problems of estimation namely nonlinear system identification, pattern classification and Mackey–Glass chaotic time series prediction.

KW - Back-propagation through time (BPTT)

KW - Fractional calculus

KW - Gradient descent

KW - Mackey–Glass chaotic time series

KW - Minimum redundancy and maximum relevance (mRMR)

KW - Recurrent neural network (RNN)

UR - https://www.scopus.com/pages/publications/85040836202

U2 - 10.1007/s00034-017-0572-z

DO - 10.1007/s00034-017-0572-z

M3 - Article

AN - SCOPUS:85040836202

SN - 0278-081X

VL - 37

SP - 593

EP - 612

JO - Circuits, Systems, and Signal Processing

JF - Circuits, Systems, and Signal Processing

IS - 2

ER -

A Novel Fractional Gradient-Based Learning Algorithm for Recurrent Neural Networks

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this