Coherent activity in excitatory pulse coupled networks - davidar/scholarpedia GitHub Wiki

An excitatory pulse-coupled neural network is a network composed of neurons coupled via excitatory synapses, where the coupling among the neurons is mediated by the transmission of Excitatory Post-Synaptic Potentials (EPSPs). The coherent activity of a neuronal population usually indicates that some form of correlation is present in the firing of the considered neurons. The article focuses on the influence of dilution on the collective dynamics of these networks: a diluted network is a network where connections have been randomly pruned. Two kind of dilution are examined: massively connected versus sparse networks. A massively (sparse) connected network is characterized by an average connectivity which grows proportionally to (does not depend on) the system size.

Neural collective oscillations have been observed in many contexts in brain circuits, ranging from ubiquitous $\gamma$-oscillations to $\theta$-rhythm in the hippocampus. The origin of these oscillations is commonly associated to the balance between excitation and inhibition in the network, while purely excitatory circuits are believed to lead to “unstructured population bursts” (Buzsàki, 2006). However, coherent activity patterns have been observed also in “in vivo” measurements of the developing rodent neocortex and hippocampus for a short period after birth, despite the fact that at this early stage the nature of the involved synapses is essentially excitatory, while inhibitory synapses will develop only later (Allene et al., 2008). Of particular interest are the so-called Giant Depolarizing Potentials (GDPs), recurrent oscillations which repeatedly synchronizes a relatively small assembly of neurons and whose degree of synchrony is orchestrate by hub neurons (Bonifazi et al., 2009). These experimental results suggest that the macroscopic dynamics of excitatory networks can reveal unexpected behaviors.

On the other hand, numerical and analytical studies of collective motions in networks made of simple spiking neurons have been mainly devoted to balanced excitatory-inhibitory configurations (Brunel, 2000), while few studies focused on the emergence of coherent activity in purely excitatory networks. Pioneering studies of two pulse coupled neurons have revealed that excitatory coupling can have desynchronizing effect, while in general synchronization can be achieved only for sufficiently fast synapses (van Vreeswijk et al., 1994; Hansel et al., 1995). Van Vreeswijk in 1996 has extended these analysis to globally (or fully) coupled excitatory networks of Leaky Integrate-and-Fire (LIF) neurons, where each neuron is connected to all the others. This analysis has confirmed that for slow synapses the collective dynamics is asynchronous ( Splay States ) while for sufficiently fast synaptic responses a quite peculiar coherent regime emerges, characterized by partial synchronization at the population level, while single neurons perform quasi-periodic motions (van Vreeswijk, 1996).

In the recent years, following the seminal study by van Vreeswijk, the robustness of the partially synchronized regime has been examined by considering the influence of external noise and the level of dilution in networks of different topologies. Partial synchronization survives to the introduction of a moderate level of noise (Mohanty and Politi, 2006) and it appears to be quite robust also to dilution.

In particular, for neurons connected as in a directed Erdös-Renyi graph (Albert and Barabàsi, 2002) it has been shown that the coherent activity always emerge for (sufficiently) high connectivities. However, while for massively connected networks, composed by a large number of neurons, the dynamics of the collective state (apart some trivial rescaling) essentially coincide with that observed in the fully coupled system (Olmi et al., 2010; Tattini et al., 2012), for sparse networks this is not the case (Luccioli et al., 2012). This is due to the fact that, for sufficiently large networks, the synaptic currents, driving the dynamics of the single neurons, become essentially identical for massively connected networks, while the differences among them do not vanish for sparse networks.

Sparse and massively connected networks reveal even more striking differences at the microscopic level associated to the membrane potentials' dynamics. As a matter of fact, for finite networks chaotic evolution has been observed in both cases. However, this chaos is weak in the massively connected networks, vanishing for sufficiently large system sizes, while sparse networks remain chaotic for any large number of neurons and the chaotic dynamics is extensive.

In a fully coupled network of $N$ neurons, the membrane potential \(u_i(t) \) of the \(i− \)th neuron evolves according to the following ordinary differential equation

\[\dot{u}_i(t)]

where all variables and parameters are expressed in adimensional rescaled units. According to the above equation, the membrane potential \(u_i \) relaxes towards the value \(a + gE(t) \), but as soon as it reaches the threshold value \( u_i = 1\), it is reset to \( u_i = 0 \) and a spike is simultaneously sent to all neurons. This resetting procedure is an approximate way to describe the discharge mechanism operating in real neurons. The parameter \( a > 1 \) is the supra-threshold input DC current and \(g > 0 \) gauges the synaptic coupling strength of the excitatory interaction with the neural field \( E(t)\). This field represents the synaptic current injected in each neuron and is given by the superposition of all the pulses emitted by the network in the past. Following (Abbott and van Vreeswijk, 1993), it is assumed that the shape of a pulse emitted at time \(t=0\) is given by an $\alpha$-function \(s(t)= \frac{\alpha^2 t}{N} {\rm e}^{-\alpha t} \), where \( 1/\alpha \) is the pulse-width. For this choice of the pulse shape it is easy to show that the field evolution is ruled by the following second order differential equation

]\ddot E(t) +2\alpha\dot E(t)+\alpha^2 E(t)= 

  \frac{\alpha^2}{N}\sum_{n|t_n<t}></t}> 0\), as shown in Fig. 6. This kind of deterministic irregular behavior vanishing in the large system size limit has been identified as ''weak chaos'' for coupled phase oscillators (Popovych et al., 2005).

Sparse networks represent a peculiar exception, since they remain intrinsically inhomogeneous and chaotic for any system size. In order to examine the influence of this kind of topology it is sufficient to examine a random network with constant connectivity $K$, which is independent of the network size $N$. At a macroscopic level, also in this case a transition from AS to PS can be observed. In particular, the collective dynamics can be characterized in terms of the standard deviation of the average field $\bar E$, namely \( \sigma_E= \sqrt{<\bar{E}^2>- <\bar{E}>^2} \). For an AS the standard deviation vanishes as $\sigma_E \propto 1/\sqrt{N}$, while in the presence of collective motions it stays finite, as shown in Fig. 7a. Similarly to what observed for massively connected networks, above a finite critical connectivity \(K_c\) a coherent collective dynamics emerges even in sparse networks, as shown in Fig. 7a.

Standard deviation of the mean field, \(\sigma_E\), versus \(K\) for \(N=1,000\) (black) circles, \(N=5,000\) (red) squares, \(N=10,000\) (green) triangles. The inset shows the macroscopic attractors for \(N=5,000\) and $K=3$ and \(K=200\). (b) Lyapunov exponent spectra (in the lower inset a zoom of the largest values) for \(K=20\) and \(N=240-480-960\). (c) Maximum Lyapunov exponent, \(\lambda_{1}\), versus N is shown, the (red) line represents the nonlinear fit \(\lambda_{1}=0.0894-2.3562/N\) and the (green) dashed line marks the asymptotic value.The parameters of the model are $g=0.2$, $a=1.3$ and $\alpha=9$. (Modified from Luccioli et al., 2012)

The most striking difference with respect to massively connected networks concerns the microscopic dynamics, as shown in Fig. 7c the maximal Lyapunov exponent converges to an asymptotic limit for increasing system sizes, therefore these networks will remain chaotic irrespectively of the network size. Furthermore, the dynamics is characterized by extensive high-dimensional chaos (Ruelle, 1982; Grassberger, 1989), i.e. the number of active degrees of freedom, measured by the fractal dimension, increases proportionally to the system size. Extensive chaos has been usually observed in diffusively coupled systems (Livi et al., 1986; Grassberger, 1989; Paul et al., 2007), where the system can be easily decomposed in weakly interacting sub-systems. Whenever the system is chaotically extensive the associated spectra of the Lyapunov exponents \(\{\lambda_i\}\) collapse onto one another, when they are plotted versus the rescaled index \(i/N\), as shown in Fig. 7b (Livi et al., 1986). Fully extensive behavior in sparse neural networks has been observed for the Theta neuron model in (Monteforte and Wolf, 2010) and the LIF model in (Luccioli et al., 2012). The previous results are obtained by assuming that all nodes are characterized by the same connectivity \(K\), but the same scenario holds assuming a Poisson degree distribution with average connectivity \(K\), as in Erdös-Renyi graphs.

The extensivity property is highly non-trivial in sparse networks, since in this case the dynamics is not additive. Contrary to what happens in spatially extended systems with diffusive coupling, where the dynamical evolution of the whole system can be approximated by the juxtaposition of almost independent sub-structures (Grassberger, 1989; Paul et al., 2007). Extensive chaos has not been observed in globally coupled networks, which exhibit a non-extensive component in the Lyapunov spectrum (Takeuchi et al., 2011).

• Abbott L. F. and van Vreeswijk C. (1993). Asynchronous states in networks of pulse-coupled oscillators. Phys. Rev. E 48: 1483.
• Albert R. and Barabàsi A. L. (2002). Statistical mechanics of complex networks. Rev. Mod. Phys. 74: 47–97.
• Allene C.; Cattani A.; Ackman J. B.; Bonifazi P.; Aniksztejn L.; Ben-Ari Y. and Cossart R. (2008). Sequential generation of two distinct synapse-driven network patterns in developing neocortex. The Journal of Neuroscience 26: 12851-12863.
• Ashwin P.; King G.P and Swift J.W. (1990). Three identical oscillators with symmetric coupling. Nonlinearity 3: 585.
• Bonifazi P.; Goldin M.; Picardo M. A.; Jorquera I.; Cattani A.; Bianconi G.; Represa A.; Ben-Ari Y. and Cossart R. (2009). GABAergic Hub Neurons Orchestrate Synchrony in Developing Hippocampal Networks. Science 326: 1419-1424.
• Brunel N. (2000). Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comput. Neurosci. 8: 183.
• Brunel N. and Hansel D. (2006). How noise affects the synchronization properties of recurrent networks of inhibitory neurons. Neural Comput 18: 1066-1110.
• Bressloff P. C. (1999). Mean-field theory of globally coupled integrate-and-fire neural oscillators with dynamic synapses. Phys Rev E 60: 2160-2170.
• Buzsàki G. (2006). Rhythms of the Brain. Oxford University Press, New York.
• Chow C. C. and Kopell N. (2000). Dynamics of spiking neurons with electrical coupling. Neural Comput 12: 1643-1678.
• Dipoppa M.; Krupa M.; Torcini A. and Gutkin B. S. (2012). Splay states in finite pulse-coupled networks of excitable neurons. SIAM J Appl Dyn Syst 11: 864-894.
• Golomb D.; Hansel D. and Mato G. (2001). Mechanisms of Synchrony of neural Activity in large Networks, in Handbook of biological physics. 887-967, Eds. Gielen S and Moss F, Elsevier, Amsterdam.
• Grassberger P. (1989). Information content and predictability of lumped and distributed dynamical systems. Physica Scripta 40: 346.
• Hadley P. and Beasley M. R. (1987). Dynamical states and stability of linear arrays of Josephson junctions. Appl. Phys. Lett. 50: 621.
• Hansel D.; Mato G. and Meunier C. (1995). Synchrony in Excitatory Neural Networks. Neural Computation 7: 307.
• Kuramoto Y. (1984). Chemical oscillations, waves, and turbulence. Springer-Verlag, Berlin.
• Jin D. Z. (2002). Fast convergence of spike sequences to periodic patterns in recurrent networks. Phys. Rev. Lett. 89: 208102.
• Livi R.; Politi A. and Ruffo S. (1986). Distribution of characteristic exponents in the thermodynamic limit. J. Phys. A: Math. Gen. 19: 2033.
• Luccioli S.; Olmi S.; Politi A. and Torcini A. (2012). Collective dynamics in sparse networks. Phys. Rev. Lett. 109: 138103.
• Mohanty P. K. and Politi A. (2006). A new approach to partial synchronization in globally coupled rotators. J. Phys. A 39: L415.
• Monteforte M. and Wolf F. (2010). Dynamical Entropy Production in Spiking Neuron Networks in the Balanced State. Phys. Rev. Lett. 105: 268104.
• Olmi S.; Livi R.; Politi A. and Torcini A. (2010). Collective oscillations in disordered neural networks. Phys. Rev. E 81: 046119.
• Olmi S.; Politi A. and Torcini A. (2012). Stability of the splay state in networks of pulse-coupled neurons. The Journal of Mathematical Neuroscience 2: 12.
• Paul M. R.; Einarsson M. I.; Fischer P. F. and Cross M. C. (2007). Extensive chaos in Rayleigh-Bénard convection. Phys. Rev. E 75: 045203(R).
• Popovych O. V.; Maistrenko Y. L. and Tass P. A. (2005). Phase chaos in coupled oscillators. Phys. Rev. E 71: 065201(R).
• Popovych, O. V. and Tass, P. A. (2011) Macroscopic entrainment of periodically forced oscillatory ensembles, Prog. Biophys. Molec. Biol., 105: 98.
• Rosenblum M. and Pikovsky A. (2007). Self-Organized Quasiperiodicity in Oscillator Ensembles with Global Nonlinear Coupling. Phys. Rev. Lett. 98: 064101.
• Ruelle D. (1982). Large volume limit of the distribution of characteristic exponents in turbulence. Commun. Math. Phys. 87: 287-302.
• Takeuchi K. A.; Chatè H.; Ginelli F.; Politi A. and Torcini A. (2011). Extensive and Subextensive Chaos in globally Coupled Dynamical Systems. Phys. Rev. Lett. 107: 124101.
• Tattini L.; Olmi S. and Torcini A. (2012). Coherent periodic activity in excitatory Erdös-Renyi neural networks: the role of network connectivity. Chaos 22: 023133.
• Temirbayev A. A.; Zhanabaev Z. Z.; Tarasov S. B.; Ponomarenko V. I. and Rosenblum M. (2012). Experiments on oscillator ensembles with global nonlinear coupling. Phys. Rev. E, 85: 015204.
• Tsodyks M.; Mitkov I. and Sompolinsky H. (1993). Pattern of synchrony in inhomogeneous networks of oscillators with pulse interactions. Phys. Rev. Lett. 71: 1280.
• van Vreeswijk C.; Abbott L. F. and Ermentrout G. B. (1994). When inhibition not excitation synchronizes neural firing. Journal of Computational Neuroscience 1: 313.
• van Vreeswijk C. (1996). Partial synchronization in populations of pulse-coupled oscillators. Phys. Rev. E 54: 5522.
• Wiesenfeld K.; Bracikowski C.; James G. and Roy R. (1990). Observation of antiphase states in a multimode laser. Phys. Rev. Lett. 65: 1749.
• Zillmer R.; Livi R.; Politi A. and Torcini A. (2007). Stability of the splay state in pulse-coupled networks. Phys. Rev. E 76: 046102.

• Carmen C. Canavier and Srisairam Achuthan (2007) Pulse Coupled Oscillators Scholarpedia, 2(4):1331.
• Stephen Grossberg (2013) Recurrent Neural Networks Scholarpedia, 8(2):1888.
• Michael Okun and Ilan Lampl (2009) Balance of excitation and inhibition Scholarpedia, 4(8):7467.
• Edward Ott (2008) Attractor Dimensions Scholarpedia, 3(3):2110.
• Antonio Politi (2013) Lyapunov exponent Scholarpedia, 8(3):2722.
• Arkady Pikovsky and Michael Rosenblum (2007) Synchronization Scholarpedia, 2(12):1459.
• Anatoly M. Samoilenko (2007) Quasiperiodic oscillations Scholarpedia, 2(5):1783.

• Biological neuron model
• Alessandro Torcini’s website
• Laboratory of Computational Neuroscience in Florence
• PhD Thesis of Simona Olmi