Accelerating Levenberg-Marquardt Algorithm using Variable Damping Parameter (가변 감쇠 파라미터를 이용한 Levenberg-Marquardt 알고리즘의 학습 속도 향상 )

Young-Tae Kwak (곽영태)

Accelerating Levenberg-Marquardt Algorithm using Variable Damping Parameter

Journal of The Korea Society of Computer and Information
Abbr : JKSCI
2010, 15(4), pp.57-63
Publisher : The Korean Society Of Computer And Information
Research Area : Engineering > Computer Science

Young-Tae Kwak ¹

¹전북대학교

Accredited

ABSTRACT

The damping parameter of Levenberg-Marquardt algorithm switches between error backpropagation and Gauss-Newton learning and affects learning speed. Fixing the damping parameter induces some oscillation of error and decreases learning speed. Therefore, we propose the way of a variable damping parameter with referring to the alternation of error. The proposed method makes the damping parameter increase if error rate is large and makes it decrease if error rate is small. This method so plays the role of momentum that it can improve learning speed. We tested both iris recognition and wine recognition for this paper. We found out that this method improved learning speed in 67% cases on iris recognition and in 78% cases on wine recognition. It was also showed that the oscillation of error by the proposed way was less than those of other algorithms.

KEYWORDS

오류역전파학습, Levenberg-Marquardt 학습, Damping Parameter, Variable Decay Rate

Citation status

* References for papers published after 2023 are currently being built.

[book] D.E.Rumelhart / 1986 / Parallel Distributed Processing / MIT Press : 318~362

[journal] 김광백 / 2007 / Recognition System of Car License Plate using Fuzzy Neural Networks / 한국컴퓨터정보학회논문지 / 한국컴퓨터정보학회 12(5) : 313~320

[journal] John Moody / 1989 / Fast Learning in Networks of Locally-Tuned Processing Units / Neural Computation 1 : 281~294

[journal] T.P.Vogal / 1988 / Accelerating the convergence of the backpropagation method / Biological Cybernetics 59 : 256~264

[journal] T.Tollenaere / 1990 / SuperSAB:Fast adaptive back propagation with good scaling properties / Neural Networks 3(5) : 561~573

[journal] M.Kordos / 2008 / Variable step search algorithm for feedforward networks / Neurocomputing 71 : 2470~2480

[journal] C.Charalambous / 1992 / Conjugate gradient algorithm for efficient training of artificial neural networks / IEEE Proceedings 139(3) : 301~310

[journal] Wei Chu, / 2005 / An improved conjugate gradient scheme to the solution of least squares SVM / IEEE Transactions on Neural Networks 16(2) : 498~501

[journal] M.T.Hagan / 1994 / Training feedforward networks with the Marquardt algorithm / IEEE Transactions on Neural Networks 5(6) : 989~993

[book] Jorge Nocedal / 2006 / Numerical Optimization / Springer

[book] Rolnald E.Miller / 2000 / Optimization / John Wiley & Son,INC. : 358~362

[journal] Chen T / 2003 / Acceleration of Levenberg-Marquardt training of neural networks with variable decay rate / International Joint Conference on Neural Networks Proceedings 3 : 1873~1878

[journal] Amir Abolfazl Suratgar / 2005 / Modified Levenberg-Marquardt Method for Neural Networks Training / Proceedings of world academy of science, engineering and technology 6 : 46~48

[journal] 곽영태 / 2009 / Improving Levenberg-Marquardt algorithm using the principal submatrix of Jacobian matrix / 한국컴퓨터정보학회논문지 / 한국컴퓨터정보학회 14(8) : 10~18

[web] / Machine Learning Repository / http://archive.ics.uci.edu/ml/

KJCKorea
Journal Central

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65