Verlagslink DOI: 10.1103/PhysRevE.111.054208
Titel: Weight dynamics of learning networks
Sprache: Englisch
Autorenschaft: Sharafi, Nahal 
Martin, Christoph 
Hallerberg, Sarah 
Schlagwörter: Dynamical systems; Artificial neural networks; Chaos & nonlinear dynamics
Erscheinungsdatum: 12-Mai-2025
Verlag: American Physical Society
Zeitschrift oder Schriftenreihe: Physical review / publ. by The American Institute of Physics. E 
Zeitschriftenband: 111
Zeitschriftenausgabe: 5
Anfangsseite: 054208-1
Endseite: 054208-14
Zusammenfassung: 
Neural networks have become a widely adopted tool for tackling a variety of problems in machine learning and artificial intelligence. In this contribution, we use the mathematical framework of local stability analysis to gain a deeper understanding of the learning dynamics of feedforward neural networks. We derive equations for the tangent operator of the learning dynamics of three-layer networks learning regression tasks. The results are valid for an arbitrary number of nodes and arbitrary choices of activation functions. Applying the results to a network learning a regression task, we investigate numerically how stability indicators relate to the final training loss. Although the specific results vary with different choices of initial conditions and activation functions, we demonstrate that it is possible to predict the final training loss by monitoring finite-time Lyapunov exponents during the training process.
URI: https://hdl.handle.net/20.500.12738/17863
ISSN: 2470-0053
Begutachtungsstatus: Diese Version hat ein Peer-Review-Verfahren durchlaufen (Peer Review)
Einrichtung: Department Maschinenbau und Produktion 
Fakultät Technik und Informatik 
Dokumenttyp: Zeitschriftenbeitrag
Hinweise zur Quelle: article number: 054208
Enthalten in den Sammlungen:Publications without full text

Zur Langanzeige

Google ScholarTM

Prüfe

HAW Katalog

Prüfe

Volltext ergänzen

Feedback zu diesem Datensatz


Diese Ressource wurde unter folgender Copyright-Bestimmung veröffentlicht: Lizenz von Creative Commons Creative Commons