IF Colloquium No.30 – Localization Lengths in Random Recurrent Neural Networks

——————————————–

📌 Title: ” Localization Lengths in Random Recurrent Neural Networks ”

📌 Speaker : Thiparat Chotibut (Ph.D. in Theoretical Physics from Harvard University) Head of Chula Intelligent and Complex Systems Research Unit, Department of Physics, Faculty of Science, Chulalongkorn University, Thailand

📌 Abstract : In this talk, we will discuss two critical aspects of the localization lengths of eigenfunctions associated with linearized recurrent neural networks (RNNs). In the first part, we will demonstrate how RNNs trained with moderate additive noise exhibit more robust short-term memory maintenance, aligning with recent findings in experimental neuroscience. This robust memory maintenance can be attributed to the emergence of delocalized left eigenstates of RNNs that are close to the edge of instability. In the second part, we will examine the localization-delocalization transition in the Amir-Hatano-Nelson model of random asymmetric ring recurrent neural networks. It is known that, in a strong asymmetry limit, there exists a phase transition between the localized and delocalized phases of the eigenstates. Understanding this transition precisely could inform good initialization strategies in RNNs-based models. Using methods from topological field theory, we will show that this transition can be understood as a topological phase transition, where the spectral winding number around the origin changes its value. Our methodology can be generalized to study delocalization transition in other non-Hermitian random matrix models.

📌 Keywords : Neural Networks

📌 15:00 – 16:00 p.m. Friday, June 21, 2024

📌 Venue: Academia Room (TA212), 2nd Floor, Mahadhammaraja A Building

📌 IF Channel : https://www.youtube.com/watch?v=YkuxXH_gp7k
——————————————–
#NeuralNetworks #Physics #TheoreticalPhysics #IFNU #PhysicsNaresuan #IFColloquium