The Network Model

The neural network consists of 100 simulated excitatory neurons which, in the majority of simulations, are fully recurrently connected by single synapses except for self connections. Although the level of recurrent connectivity present in the CA3 region is estimated as 5–15% (and is non-random), full recurrent connectivity has most often been employed in previous computational models of auto- associative learning [16]–[19], [39]. However, all simulations described here were also performed using networks with more realistic levels of recurrent connectivity (15 separate pre-synaptic connections per simulated neuron, chosen from a random uniform distribution that excludes self-connections) and no significant differences were observed (data not shown).

The Neuron Model

Simulated pyramidal cells operate according to the Izhikevich spiking model [61], which can replicate the firing patterns of all known types of cortical neurons with minimal computational complexity. The membrane potential (v) and a membrane recovery variable (u) are dynamically calculated based on the values of four dimensionless constants (a, b, c and d) and a dimensionless current input (I) according to Equations 1.1–1.3.(1.1)(1.2)(1.3)The parameter values used to replicate firing of a standard excitatory neuron are [a = 0.02, b = 0.2, c = −65, d = 6]. Under these conditions, simulated neurons fire single spikes at low levels of stimulation, but produce complex bursts that are representative of hippocampal pyramidal cells (i.e. several action potentials at a spontaneous rate of ∼150Hz) at higher levels of stimulation [2], [62]. Further details of the dynamics produced by single simulated neurons in response to various forms of applied current can be found in Izhikevich (2004).

Each simulated neuron has an axonal delay (D_i) randomly assigned from a uniform distribution in the range [1ms : Dms] with D = 5 in the majority of simulations (this being realistic of the CA3 region [63]). At the beginning of each millisecond time step, before the parameters v and u are updated, any membrane potential values that exceed threshold are reset according to Equation 1.3. The corresponding neuron(s) are considered to have fired in that time step (t*), and the corresponding spikes arrive at their post-synaptic targets at time t*+D_i.

External Input during Theta Coded Learning

The hippocampal EEG is dominated by both theta and gamma oscillations during stereotype learning behaviour [39], [43], [64], [65]. Here, we include only a minimal model of theta frequency inhibition. A variable θ, which oscillates sinusoidally in the range [0 : 1] at a rate of 8Hz throughout all learning simulations, is used to dynamically represent the theoretical local field potential (LFP). Inhibitory input to every simulated neuron at each millisecond time step is randomly sampled from a Gaussian distribution with mean I_inh = −15θ and standard deviation σ_inh = 2. Neural noise at a rate of ∼0.1Hz (this being realistic of the CA3 region) is generated in the network by the constant application of excitatory current, randomly sampled from a uniform distribution in the range [0 : I_noise] where I_noise = 0.8 in all simulations [66]. The interplay between afferent inhibitory and excitatory currents means that the majority of firing due to neural noise tends to occur around the peak of the LFP, as defined by the value of θ.

Place cells are most often studied in the dorsal CA1 region of the hippocampus, although some data is available from CA3 and, importantly, significant differences can be observed [14], [67]–[69]. Approximately 30% of CA3 pyramidal cells are active in a typical environment, each of which can encode for several (occasionally overlapping) place fields of ∼30cm in size (although this varies along the septotemporal axis) [10], [12], [70]. The phase precession of place cell firing can cover a full theta cycle, but typically changes by 180° between entry and exit, and is correlated with both the relative distance travelled and time spent within a place field (i.e. the rate of phase precession is positively correlated with running speed) [10], [12], [71]. The firing rate of active place cells follows a Gaussian distribution, such that maximum firing rate occurs around the centre of the place field [10]. In CA3, the mean in-field firing rate of place cells is ∼15Hz, although this is strongly modulated by various non-spatial cues [14], [66], [68].

There is considerable debate regarding the mechanisms of phase precession in place cells, fuelled by apparently contradictory empirical findings [72]. Here, we are more directly concerned with the manner in which theta coded neural dynamics interact with local plasticity rules in order to mediate the learning and recall of auto- and hetero- associative connections between active neurons. Hence, the phase at which simulated neurons in our network model fire is primarily dictated by external excitatory input, although it is important to note that phase precession has been empirically observed in both the dentate gyrus and entorhinal cortex, which constitute the two principal synaptic inputs to CA3 [12], [73]. Furthermore, detailed biophysical simulations suggest that input from these afferent structures plays a significant role in dictating the neural dynamics observed in CA3 [74], [75].

During learning simulations, each place field is arbitrarily divided into eight equally sized sub-sections, and theta oscillations in the LFP (as defined by the value of θ) are similarly divided into subsections of π/4 (between π/8 and 15π/8). At each millisecond time step, the theoretical position within a place field dictates the theta phase window at which the corresponding place cell receives external excitation, randomly sampled from a normal distribution with mean I_ext and standard deviation σ_ext (Figure 1a, b). This phenomenological model dictates that the mean phase of (stochastic) activity in place cells recedes in a step-wise fashion as the corresponding place field is traversed. In the majority of simulations, values of I_ext = 5 and σ_ext = 22.5 are used to generate a mean in-field firing rate of ∼15Hz, with active place cells tending to fire bursts at the peak of the LFP (as defined by the value of θ) and single spikes on the ascending and descending slope [10]. In other simulations, values of σ_ext = [12.5 ; 32.5 ; 42.5 ; 52.5] are used to generate a range of mean in-field firing rates.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. The Phenomenological Phase Precession Model and Theta Coding Mechanism.

(a, b) Each theoretical place field and theta cycle (as defined by the value of θ) are divided into eight equally sized sub-sections. At each millisecond time step, the theoretical position within a place field dictates the theta phase window during which the corresponding place cell receives external excitatory input. Hence, when the theoretical animal enters a place field (segment 1), the corresponding place cell receives external stimulation late in the theta cycle (phase window 1); in the centre of the place field (segment 4), the corresponding place cell receives external, excitatory stimulation in the middle of the theta cycle (phase window 4); and as the place field is exited (segment 7), the corresponding place cell receives external, excitatory stimulation early in the theta cycle (phase window 7). The interplay of this external, excitatory stimulation with the constant, oscillatory inhibitory input to each place cell directs place cells to fire complex spike bursts when theoretical position is near the centre of the place field, and single spikes upon entry to or exit from the place field. Importantly, the random distribution of both inhibitory and excitatory inputs to each place cell produce stochastic firing activity within the corresponding phase window, such that place cells which encode for the same place field will fire with the same mean phase, but not necessarily in the same millisecond time step(s). (c) The phenomenological phase precession model creates a theta coding mechanism, whereby the sequence of place fields being traversed on a behavioural time scale is represented by a compressed sequence of activity in the corresponding place cells, repeated in every theta cycle.

https://doi.org/10.1371/journal.pcbi.1000839.g001

In these simulations, hypothetical place fields are generally 80cm in diameter and traversed at a rate of 10cms⁻¹. Although this place field size is larger than that typically observed in vivo [10], [12], these values are chosen for computational convenience such that active place cells fire stochastically in each theta phase window for a period of 1s before receding. Simulations were also performed using place field diameters of [10cm ; 20cm ; 40cm] – which effectively reduces the duration of time for which each theta coded stage of the learned sequence is applied to the network – and the only significant effect observed was a decrease in the rate of synaptic weight change (data not shown).

The Synaptic Plasticity Model

The phase precession of place cells in the hippocampus produces a compressed, theta coded, sequence of firing within each oscillatory cycle that corresponds to the sequence of overlapping place fields being traversed on a behavioural timescale Figure 1c; [12], [13], [34]. These firing patterns are ideally suited to induce the long-term potentiation (LTP) and depression (LTD) of synapses by spike-timing dependent plasticity (STDP), and there is evidence that synaptic connections between overlapping place cells in rat hippocampus are potentiated during exploration [47]–[49], [76]. Mathematically, with s = t_post−t_pre being the time difference between pre- and post- synaptic spiking, the change in the weight of a synapse (Δw) according to a standard STDP rule can be calculated using Equations 2.1–2.5 [77]–[82].(2.1)(2.2)(2.3)(2.4)(2.5)The parameters A₊ and A₋ correspond to the maximum possible change in synaptic weight per isolated spike pair, while τ₊ and τ₋ denote the time constants that approximate an exponential decay of potentiation and depression increments respectively. The co-efficient ε determines the contribution of an additional potentiation process, which is equal to a trace of the most recent weight decrease at a synapse (with s₊₊ = t_Δw+−t_Δw−) decaying exponentially with a time constant τ₊₊. This term accounts for experimental observations of STDP in the hippocampus obtained using triplets of pre- and post- synaptic spikes, which suggest that depression is suppressed by potentiation within a short temporal window [48]. In accordance with empirical data, coincident pre- and post- synaptic firing elicits maximal depression from all STDP implementations examined here [49].

Previous auto-associative network models of hippocampal mnemonic function have most frequently utilised rate-coded Hebbian learning rules [16], [17], [22], [23], [83] – which typically dictate that changes in synaptic strength are proportional to the product of pre- and post- synaptic firing rates (r_j,i) scaled by a learning rate k (Equation 3.1). This form of synaptic plasticity generates no competition between inputs or outputs of a single neuron, as any increase in synaptic weight produces an increase in post-synaptic firing rate in a positive feedback loop [84]. The BCM model (Equation 3.2) was proposed to address this issue, and postulates the existence of a theoretical modification threshold (θ_m) that distinguishes between depression (at lower firing rates) and potentiation (at higher firing rates). The value of θ_m is itself a function of pre- or post- synaptic activity, generating competition between synaptic inputs by making potentiation more difficult to achieve as the long-term average firing rates increases [85].(3.1)(3.2)Interestingly, it has been demonstrated that STDP can provide inherent competition using only local synaptic variables, and thus stabilise Hebbian learning processes [80], [82]. However, these properties rely on synaptic weights being either depressed or unchanged following an increase in pre-synaptic stimulation, which directly contradicts empirical data and the requirements of rate-coded associative learning. Conversely, several computational studies have described conditions under which STDP can be reconciled with the BCM formulation [77]–[79], [81]. This requires that the plasticity rule exhibit an increasing dominance of potentiation processes as inter-spike intervals (ISIs) are reduced [77]. Pair-based STDP rules, which assume a linear integration of potentiation and depression processes, require constraints to be placed on the nature of spike pair interactions and parameters that define the asymmetric learning window [77], [78], [81]. Triplet-based STDP rules, which explicitly account for the observed non-linear integration of potentiation and depression processes, dictate that mean synaptic weight increases with mean stochastic firing rate irrespective of the finer details of the STDP rule [77], [79].

We examine three different additive STDP implementations here, in order to draw a comparison between the emergent synaptic dynamics produced by each. The ‘BCM type’ pair- and triplet- based STDP rules have parameter values described in previous studies as allowing a reconciliation with rate-coded Hebbian learning (A₊>A₋ and τ₊<τ₋), which also concur with empirical measurements made in the hippocampus [47], [49], [78], [81]. Conversely, the ‘non-BCM type’ pair-based STDP rule has parameter values noted in previous modelling studies for the generation of synaptic competition (A₊<A₋ and τ₊ = τ₋) [80], [82]. For each of these STDP rules, a lax nearest neighbour spike pairing scheme – which dictates that values of P_± are reset to the value of A_± upon afferent or efferent firing – is employed. Values of A_± are also scaled by the value of w_max such that ∼60 spike pairings are sufficient to traverse the range of possible synaptic weight values, in accordance with empirical data [47]–[49]. The full details of each plasticity model examined are given in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Computational details of the STDP rules.

https://doi.org/10.1371/journal.pcbi.1000839.t001

Empirical evidence indicates that the degree of synaptic plasticity incurred by consistent stimulation protocols differs across the theta cycle, with potentiation incurred by burst pairings at the peak and depression (or de-potentiation) incurred by burst pairings at the trough of the LFP [86]–[89]. Here, we examine the effects of three different forms of theta modulated plasticity, for comparison, the details of which are shown in Equation 4.(4)In all simulations, hard limits are placed on the achievable strength of synapses, such that synaptic weights are maintained continuously in the range [0 : w_max]. While there is little clear biological basis to inform the relative scale of synaptic weights, it is known that recurrent synapses in the CA3 region are generally incapable of solely provoking post-synaptic activity [90]. In order to generate an action potential using the neural dynamics employed here, a single synaptic current of I = 16.5 is required, and therefore the value generally assigned to the maximum weight limit in these simulations is w_max = 1. In each simulation, all synaptic connections in the network are initialised with a weight of 0.01w_max.

Neuromodulation and Recall

The neuromodulatory effects of Acetylcholine (ACh) have been hypothesised to separate periods of learning and recall in the hippocampus in order to avoid issues of interference [35], [91], [92]. Cholinergic input from the septum, terminating on local interneurons, can induce theta frequency oscillations in the CA3 region, facilitate LTP and enhance afferent input from the dentate gyrus and entorhinal cortex while suppressing recurrent excitation from intrinsic connections – thereby creating the ideal conditions for learning external associations via theta coding [92]. In the absence of cholinergic input pyramidal neurons in CA3 are disinhibited, synaptic plasticity is suppressed, and neural dynamics enter a state of large-amplitude irregular activity (LIA). During this period, postulated recall activity is observed in the form of sharp wave ripples (SWR) – short periods of high frequency firing in large populations of neurons with fine temporal structure that last ∼100ms and originate in CA3 [50]–[53], [69].

In our model, an abstract, global ACh signal modulates the scale of recurrent excitation and synaptic plasticity in the network throughout all simulations. The hypothetical concentration of ACh maintains a dimensionless value of Φ = 1 during periods of theta coded learning and falls to a lower value during periods of recall. In both cases, the relative magnitude of recurrent synaptic weights in the network is scaled by a factor of 1/Φ, while the magnitude of synaptic weight change is scaled by a factor of Φ. During periods of recall, theta frequency inhibitory input to the network is ceased and superthreshold excitation of magnitude I_cue = 30 is provided to a small number of randomly selected neurons for a single millisecond time step. Subsequent activity - dictated by recurrent excitation alone - can then be compared to the auto- and hetero- associative correlations present in external input during learning and SWR activity observed in vivo. In these simulations, the effective speed of recall is strongly dependent on the size and overlap of place fields, as a reduction in place field size and offset implies a reduction in the total length of the learned route, such that the same temporal compression of recall firing equates to a slower traversal of that route.

We use several different measures to assess the fidelity of putative recall activity in this model. For hetero-associative and dual coded activity patterns, we examine the timing of the first action potential fired by each simulated place cell: firing before the first action potential in any place cell encoding for the following place field on the learned route is considered to be accurate, firing at the same time as the first action potential in any place cell encoding for the following place field on the learned route is treated indifferently, and failure to fire or firing after the first action potential in any place cell encoding for the following place field on the learned route is considered to be erroneous. For auto-associative patterns, we examine firing in all simulated neurons for a period of 20ms following the external stimulation of a subset of ‘cued’ neurons from one of the learned activity patterns. Firing in any of the neurons from that learned pattern which are not externally stimulated (‘uncued’) during this period is considered to be accurate recall, while activity in any neuron that is not part of that pattern is considered to be erroneous. The Mann-Whitney U test is used to assess the significance of differences in the strengths of disparate populations of synaptic connections throughout this paper.

Theta Coded Hetero-Associative Learning

Lesions of the hippocampus have been demonstrated to disrupt the temporal ordering of information in memory, impairing recall of a sequence of locations visited [93], [94], olfactory cues presented [4], [95], [96], and trace eyeblink conditioning performance [97], [98]. This has led to the theory that the hippocampus – which exhibits sparse connectivity, temporally asymmetric synaptic plasticity, and theta coded neural dynamics - is critical for sequence learning and predictive recall [13], [28]–[37]. Hence, in the first set of simulations, we examine the learning of theta coded activity patterns in single neurons.

In these simulations, place cell firing corresponds hypothetically to ten traversals of a route of one hundred equidistant and overlapping place fields of 80cm diameter, each encoded by a single neuron, at a constant speed of 10cms⁻¹ (Figure 2a). However, this form of activity could just as easily correspond to a temporal sequence of non-spatial stimuli encountered on a behavioural timescale [29], [40], [64]. The spike raster shown in Figure 2b is representative of the neural dynamics generated by the phenomenological theta coding model, which replicates the gross features of phase precession observed in the hippocampus in vivo.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Theta Coded Hetero-associative Learning in a Spiking Recurrent Neural Network.

(a) Theoretical details of theta coded hetero-associative learning simulations. 100 equidistant and overlapping place fields of 80cm diameter, offset by 10cm, form a circular route that is traversed repeatedly at a constant speed of 10cms⁻¹. Each place field is encoded by the activity of a single place cell. (b) Typical spike raster in seven representative place cells with consecutive and overlapping place fields, showing theta coded neural dynamics generated by the phenomenological phase precession model. For illustrative purposes, this figure was generated with much smaller place fields (10cm diameter) such that active place cells fire in each theta phase window for one oscillatory cycle only. (c) Typical synaptic weight matrix following learning. Asymmetric connections between place cells which correspond to consecutive place fields on the learned route are selectively and significantly potentiated. Inset: synaptic weight histograms for foreground and background connections (i.e. between a neuron and those that encode for either the three successive place fields on the learned route, or all other neurons in the network respectively). Data illustrated for the triplet based BCM type STDP rule with no plasticity modulation. (d) Mean phase of firing in all place cells at place field entry and exit on successive traversals of the route, averaged over 50 separate simulations. This demonstrates the asymmetric expansion of place fields against the direction of motion during spatial learning. Data illustrated for the triplet based BCM type STDP rule with no plasticity modulation. (e) The relative mean weight of synaptic connections between place cells in typical simulations with every combination of STDP rule and plasticity modulation scheme examined. The value of the post-synaptic neuron index corresponds to the distance – in place fields – between the pre- and post- synaptic place cell. (f) The mean rate of synaptic weight change at synapses connecting each place cell to that immediately ahead of it on the theoretical route averaged over 50 separate simulations, which correlates with mean in-field firing rate for (A) the pair-based BCM type; (B) the triplet-based BCM type; and (C) the non-BCM type STDP rule. Data illustrated for simulations with theta modulated plasticity, and synaptic weight change averaged over all neurons until synaptic weights saturate at the upper bounds.

https://doi.org/10.1371/journal.pcbi.1000839.g002

Figure 2c illustrates the typical asymmetric weight matrix that develops – with connections between each place cell and those that follow it on the theoretical route being selectively and significantly potentiated to create a bi-modal distribution of synaptic strengths (inset). Figure 2d illustrates the asymmetric expansion of place fields that proceeds over the course of these simulations, a phenomenon that has been observed experimentally [99]. This results from an increase in excitatory input to each place cell from those preceding it on the route as recurrent connections are potentiated, and the magnitude of place field expansion is therefore correlated with the value assigned to the maximum excitatory synaptic weight (w_max).

It is important to note that the particular details of the STDP rule utilised here makes little difference to the efficient learning of hetero-associative sequences (Figure 2e). Furthermore, the strength of asymmetric connections saturates at the upper bounds regardless of whether neurons fire bursts or single spikes throughout each theta cycle – although mean in-field firing rate is correlated with the rate of synaptic weight change (Figure 2f). These results demonstrate that the combination of theta coding and STDP in a spiking recurrent network is sufficient to mediate rapid and robust sequence learning, irrespective of the finer details of the plasticity rule, in accordance with several previous models [12], [13], [28]–[37].

Theta Coded Auto-Associative Learning

Although the majority of electrophysiology studies have focused on spatial memory, there is a growing body of evidence to suggest that non-spatial stimuli are also encoded in the activity of single neurons in the hippocampus and can significantly modulate the firing rate of established place cells [4]–[7], [9], [14], [57], [68], [100]. Computational theories of episodic memory function generally posit that discrete patterns of rate-coded activity, corresponding to the conjunctively coded sensory elements that constitute an experience, are auto-associated in the recurrent connections of CA3. This cortical activity can subsequently be fully recreated from partial sensory cues via a process of pattern completion [16]–[23], [68], [101].

However, auto-associative network models of the CA3 region have often been criticised on the grounds of biological realism for failing to include realistic neural and synaptic dynamics [39], [102]. Furthermore, it has been suggested that the inherently asymmetric nature of STDP directly contradicts rate-coded associative learning, which explicitly depend on the development of strong bi-directional connections [13], [82], [ but see 41], [42], [44], [45], [77]. Here, we examine whether auto-associative learning can be achieved in a network model that incorporates the main features of neural and synaptic dynamics observed in CA3 – namely, phase precession and STDP.

In these simulations, input to the network effectively corresponds to ten presentations of ten binary and orthogonal activity patterns, in accordance with previous auto-associative network models [22], [23]. However, this form of input could also correspond to ten traversals of a route of ten non-overlapping place fields of 80cm diameter, each encoded by the activity of multiple place cells, at a constant speed of 10cms⁻¹ (Figure 3a). The phenomenological phase precession model implemented dictates that neurons which are active in the same pattern (i.e. place cells that encode for the same place field) fire stochastically with equal mean phase, the value of which decreases in a step-wise fashion over the course of a single presentation (Figure 3b).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Theta Coded Auto-associative Learning in a Spiking Recurrent Neural Network.

(a) Theoretical details of theta coded auto-associative learning simulations. 10 equidistant but non-overlapping place fields of 80cm diameter, offset by 80cm, form a circular route that is traversed repeatedly at a constant speed of 10cms⁻¹. Each place field is encoded for by ten place cells. This form of input effectively corresponds to repeated presentations of ten binary and orthogonal activity patterns. (b) Typical spike raster in place cells encoding for a single place field. For illustrative purposes, this figure was generated with much smaller place fields (10cm diameter) such that typical activity at each phase of theta can be seen more clearly. (c) Typical synaptic weight matrix following learning with the BCM type STDP rules, illustrating how connections between neurons that encode for the same place field are selectively and significantly potentiated. Data shown for triplet-based STDP with theta modulated plasticity. (d) Synaptic weight matrix following learning with the non-BCM type STDP rule and theta modulated plasticity. Under these conditions, synapses between place cells that encode for the same place field are depressed below the mean weight of other connections in the network. (e) The mean weight of synapses connecting each place cell to those that encode for the same place field (dark grey) and different place fields (light grey) following ten traversals of the theoretical route, averaged over 50 separate simulations, for the pair- and triplet- based BCM type STDP rules (A and B respectively) and the non-BCM type STDP rule (C). (f) The relative mean asymptotic weight of auto-associative connections averaged over 50 separate simulations, illustrating that the relative strength of auto-associative connections is positively correlated with mean in-field firing rate for (A) the pair based BCM type STDP rule (with theta modulated plasticity); and (B) the triplet based BCM type STDP rule (with theta modulated plasticity); but negatively correlated with mean in-field firing rate for (C) the pair based non-BCM type STDP rule (with inversely modulated plasticity).

https://doi.org/10.1371/journal.pcbi.1000839.g003

Our results demonstrate that successful auto-associative learning depends on a plasticity rule that produces net potentiation at the high instantaneous firing rates (i.e. short ISIs) present during near-synchronous firing in bi-directionally connected neurons [77]. Accordingly, both the pair- and triplet- based BCM type STDP rules selectively and significantly potentiate synaptic connections between place cells that encode for the same place field (Figure 3c, Mann-Whitney U-test, p<0.01). Conversely the non-BCM type STDP rule produces net depression of synaptic connections between concurrently active neurons (Figure 3d). This demonstrates that efficient auto-associative learning can be achieved in a spiking recurrent neural network when an STDP rule that can be reconciled with rate-coded Hebbian learning is employed, and that this function is fully compatible with theta coded neural dynamics created by the phase precession of principal cells in vivo.

However, in contrast to the hetero-associative learning simulations described above, the mean weight of auto-associative connections in simulations with BCM type STDP rules generally reaches an asymptote well below the upper bounds - exhibiting a bi-modal distribution (Figure 3c, inset) except where potentiation and depression processes are inversely modulated (Figure 3e). This is a consequence of the persistently alternating temporal order of spike pairings at these synapses, which produces an equilibrium between potentiation and depression processes. The position of this equilibrium is significantly affected by several features of the neural dynamics and synaptic plasticity rule employed. For example, the asymptotic mean weight of auto-associative connections increases with mean in-field firing rate for the BCM type STDP rules, but decreases with mean in-field firing rate for the non-BCM type STDP rule (Figure 3f). In both cases, the rate of synaptic weight change (whether positive or negative) correlates with the mean in-field firing rate.

In these simulations, spike pairing events that dictate changes in synaptic strength do not take place immediately following firing, but rather once an action potential reaches the pre-synaptic terminal. Hence, the range of axonal delays (D) can also have a significant impact on the relative strength of auto-associative connections. In bi-directionally connected neurons exhibiting near-synchronous bursting, longer axonal delays imply that the arrival of spikes at pre-synaptic terminals is more likely to occur after post-synaptic firing and therefore generate depression; while shorter axonal delays imply that afferent spikes are more likely to precede post-synaptic activity and therefore generate potentiation (Figures 4a, b). Accordingly, for the BCM type STDP rules examined here, both the rate of potentiation and asymptotic mean weight of auto-associative connections increase as the scale of axonal delays is decreased (Figure 4c). This also explains why, in Figure 3c, the strength of auto-associative post-synaptic connections formed by some place cells is uniformly weak, as the axonal delay of that neuron is higher than others encoding for the same place field.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Effects of Axonal Delay and Profile of the Asymmetric Learning Window on Auto-associative Learning.

(a) Action potentials in bi-directionally connected neurons are more likely to reach the pre-synaptic terminal before the end of synchronous (but stochastic) complex bursts, and therefore induce the potentiation of inter-connecting synapses, if axonal delays are shorter. (b) Conversely, action potentials in each simulated neuron are more likely to arrive at the pre-synaptic terminal after the end of synchronous (but stochastic) complex bursts, and therefore induce depression of the inter-connecting synapses, if axonal delays are longer. (c) Relative mean synaptic weight (w/w_max) of auto-associative and background connections (i.e. between neurons that are in the same or different patterns respectively) produced by the BCM type STDP rules following ten traversals of the theoretical route described in Figure 3a with a varying scale of axonal delays (D). Data is averaged over 50 separate simulations. (d) Relative mean synaptic weight (w/w_max) of auto-associative and background connections produced by the pair- and triplet- based BCM type STDP rules following ten traversals of the theoretical route described in Figure 3a with varying values of A₊ and therefore different positions of the theoretical modification threshold (θ_m). Data is averaged over 50 separate simulations.

https://doi.org/10.1371/journal.pcbi.1000839.g004

Similarly, for BCM type STDP rules, parameters that dictate the profile of the asymmetric learning window (A_± and τ_±) effectively define the position of the theoretical modification threshold (θ_m, Equation 3.2) that marks the transition between net synaptic depression (at low stochastic firing rates) and potentiation (at high stochastic firing rates) [41]–[43], [64]. Hence, lowering the theoretical modification threshold – by increasing the value assigned to A₊, for example – produces a greater degree of potentiation at set in-field firing rate, and therefore increases the asymptotic mean weight of auto-associative connections in these simulations (Figure 4d).

Dual Coded Learning

It has been proposed that the sequential co-activation of groups of neurons during behaviour can be encoded via Hebbian plasticity [55], [56]. Subsequently, transient activity patterns in the same cell assembly can be initiated by internal cognitive processes and maintained via mutual excitation. Phase precession in ensembles of place cells encoding for overlapping place fields represents a prominent empirical model of cell assembly dynamics in the brain [50], [57], [59]. However, previous models of hippocampal mnemonic function have generally focussed on the learning and recall of either discrete rate-coded or sequential temporally-coded activity patterns, while few studies have attempted to integrate these computational models within a single framework [27], [39], [42]–[44]. Here, we demonstrate that both auto- and hetero- associative learning can proceed simultaneously in our network model, such that repeatedly synchronous firing with weak sequence bias produces bi-directional connections while repeatedly asynchronous firing produces asymmetric connections.

Input to the network during these simulations corresponds to a route of twenty overlapping place fields of 80cm in diameter, each encoded for by five place cells, being traversed at a constant speed of 10cms⁻¹ (Figure 5a). This form of input is equivalent to the repeated presentation of a sequence of binary orthogonal activity patterns [39]. Figure 5b illustrates a representative spike raster observed during these simulations, demonstrating how the phenomenological phase precession mechanism dictates that place cells encoding for the same place field fire stochastically within the same theta phase window while place cells encoding for successive place fields fire in successive theta phase windows.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 5. Dual Coded Learning in a Spiking Recurrent Neural Network.

(a) Theoretical details of dual coding simulations. 20 equidistant and overlapping place fields of 80cm diameter, offset by 10cm, form a circular route that is traversed repeatedly at a constant speed of 10cms⁻¹. Each place field is encoded by five place cells. (b) Representative spike raster in thirty-five place cells encoding for seven separate but overlapping place fields. Place cells encoding for different place fields fire stochastically within different theta phase windows. (c) Typical synaptic weight matrix following ten traversals of the route for the BCM type STDP rules. Synaptic connections between place cells that encode for successive place fields on the theoretical route saturate at the upper weight bounds and synaptic connections between place cells that encode for the same place field are selectively and significantly potentiated. Data illustrated for triplet-based STDP with theta modulated plasticity. (d) Dynamic changes in the relative mean weight (w/w_max) of auto-associative (between place cells encoding for the same place field), hetero-associative (between place cells encoding for a place field and that either one or two steps further along the route), and background (between place cells and those encoding for place fields not within three steps further along the route) connections. Data illustrated is for the triplet-based BCM type STDP rule with theta modulated plasticity. (e) Typical synaptic weight matrix following ten traversals of the route when the non-BCM type STDP rule is employed with theta modulated plasticity. In contrast to (c), auto-associative connections between place cells that encode for the same place field are depressed, while hetero-associative connections between place cells that encode for successive place fields saturate at the upper weight bounds. (f) The relative mean weight of synapses connecting each place cell to those that encode for the same place field (dark grey), the next place field on the learned route (medium grey), and all place fields not within three steps ahead on the learned route (light grey) following ten traversals, averaged over 50 separate simulations, with the pair- and triplet- based BCM type STDP rules (A and B respectively) and the non-BCM type STDP rule (C).

https://doi.org/10.1371/journal.pcbi.1000839.g005

Figure 5c illustrates the typical synaptic weight matrix that develops during simulations with BCM type STDP rules, where synapses connecting each hypothetical place cell to those that encode for the same or successive place fields are rapidly, selectively and significantly potentiated (Figure 5d). Conversely, when the non-BCM type STDP rule is utilised (and potentiation and depression are not inversely modulated), then hetero-associative sequence learning proceeds robustly but strong, bi-directional auto-associative connections are not generated (Figure 5e). In fact, there is no significant difference in the asymptotic mean weight of auto- or hetero- associative connections generated in any of these dual coding simulations and those described above with equal parameter values (Figures 2e, 3e, f, 4c, d; 5f). The experimentally observed asymmetric expansion of place fields against the direction of motion during spatial learning (Figure 2d) also proceeds during dual coded learning simulations (data not shown).

These results again demonstrate that an STDP rule which exhibits a dominance of potentiation at short ISIs – and can therefore mediate rate-coded Hebbian learning – is essential for efficient auto-associative learning to proceed. However, each of the BCM type STDP rules examined here exhibits a significant functional weakness: background connections (i.e. synapses between neurons which are not in the same or immediately successive patterns) undergo slight but continual potentiation throughout all simulations, indicating a lack of inherent synaptic competition (Figure 5d). Effectively, a positive feedback loop arises between the potentiation of a synapse and a reduction in the latency of post-synaptic firing following an identical pre-synaptic input. This lack of competition may be necessary to allow the development of strong, bi-directional connections using the asymmetric STDP rule, as the mean weight of background connections correlates with that of auto-associative connections in all simulations (Figure 5f), but is also reminiscent of the global stability issues commonly encountered by rate-coded Hebbian learning [84].

Recall Phase

Electrophysiology studies have demonstrated that learned routes – corresponding to the theta coded activity patterns observed in place cells during exploration – are pre-played in sharp wave ripples (SWR) at the beginning of (and during) a journey, replayed in reverse order at the end of a journey, and replayed in the original order during sleep [51]–[53]. The temporal order and relative latency of firing observed during exploration is preserved during this rehearsal and replay activity, which suggests a Hebbian learning mechanism on the timescale of STDP [50], [54]. Here, we examine the recall activity generated by recurrent excitation in our network under similar conditions - when theta frequency inhibitory input is ceased, the hypothetical concentration of ACh is reduced (to modulate the magnitude of recurrent excitation and synaptic plasticity), and superthreshold external excitation is applied to small numbers of neurons. This activity can then be compared to both the auto- and hetero- associations created during learning in the simulations described above and SWR activity observed in vivo.

Firstly, we examine sequence prediction following hetero-associative learning. As illustrated by Figure 6a, superthreshold stimulation of a single, randomly selected neuron typically produces accurate sequential firing in all neurons that constitute the original learned pattern over a period of ∼400ms. Over one thousand separate recall epochs, the fidelity of recall activity produced is typically ∼90% for every STDP rule and plasticity modulation scheme examined (Figure 6b). The sequential firing patterns observed in these recall simulations continue indefinitely in the absence of inhibitory input to suppress the effects of recurrent excitation. This is a product of the fact that each neuron has few strong post-synaptic connections, and hence the concentration of ACh must be reduced to a level whereby the relative scale of recurrent synaptic weights allows single synapses to produce post-synaptic firing (Φ = 0.05 in Figures 6a, b for example).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 6. Putative Sharp Wave Ripple Recall Activity Following Theta Coded Learning.

(a) Typical spike raster observed in the network during recall simulations following hetero-associative learning (as described in Figure 2). Externally stimulated firing of a single neuron produces sequential recall activity in all neurons that constitute the originally learned pattern; (b) Statistics relating to hetero-associative recall for each STDP rule and plasticity modulation scheme examined. Figures shown represent data averaged over 1000 randomly initialised recall epochs with Φ = 0.05 following hetero-associative learning simulations with the (A) pair-based BCM type; (B) triplet-based BCM type; (C) pair-based non-BCM type STDP rules. Data illustrated for the relative frequency of neurons that fired before (dark grey); at the same time as (medium grey); and after (light grey) the simulated neuron encoding for the next place field on the learned route. (c) Typical spike raster observed in the network during recall simulations following auto-associative learning (as described in Figure 3). External stimulation of a random subset of (cued) neurons from each learned pattern (five out of ten, in this case) generates selective firing in (uncued) neurons that encode for the same place field/pattern after 5–10ms (depending on the plasticity rule employed during learning, and the concentration of ACh employed during recall). (d) Statistics relating to auto-associative recall for each STDP rule and plasticity modulation scheme examined. Figures shown represent data averaged over 1000 randomly initialised recall epochs following learning with the (A) pair-based BCM type STDP rule, and Φ = 0.05; (B) triplet-based BCM type STDP rule, and Φ = 0.083; (C) pair-based non-BCM type STDP rule, and Φ = 0.05. Data illustrated for the relative frequency of uncued neurons that fire within 20ms of externally cued activity in other neurons within the same pattern (dark grey) and the relative frequency of neurons in different, uncued patterns that fire within the same temporal window (light grey). (e) Typical spike raster observed in the network during recall simulations following dual coded learning (as described in Figure 5). External stimulation of a random subset of neurons from a single pattern (three out of five, in this case) produces sequential recall activity in simulated neurons that encode for each successive place field on the learned route. This neural activity pattern is reminiscent of sharp wave/ripple dynamics observed during putative recall activity in the hippocampus; (f) Statistics relating to dual coded recall for each STDP rule and plasticity modulation scheme examined. Figures shown represent data averaged over 1000 randomly initialised recall epochs with Φ = 0.111 following dual coded learning for the (A) pair-based BCM type; (B) triplet-based BCM type; and (C) pair-based non-BCM type STDP rules. Data illustrated for the relative frequency of neurons that fired before (dark grey); at the same time as (medium grey); and after (light grey) the first action potential in any simulated neuron encoding for the next place field on the learned route.

https://doi.org/10.1371/journal.pcbi.1000839.g006

Secondly, we examine pattern completion following auto-associative learning by providing superthreshold excitation to random partial cues consisting of five out of ten simulated neurons from each learned pattern. As illustrated in Figure 6c, the uncued neurons in each pattern are typically activated by recurrent excitation shortly after externally cued activity while other neurons in the network remain silent. The fidelity of recall activity produced in these simulations reflects the relative strength of auto-associative connections generated during learning (Figures 3e; 6f). However, pattern completion does not rely on an ‘idealised’ weight matrix: >90% accurate recall activity is produced following learning with the triplet-based BCM type STDP rule, which produce a mean auto-associative weight of ∼0.7w_max. Furthermore, no erroneous activity (i.e. firing in neurons that are not part of the cued pattern) is produced following learning with any of the STDP rules over a thousand separate recall simulations (Figure 6d).

Finally, we examine recall activity following dual coded learning by applying superthreshold stimulation to a randomly selected subset of simulated neurons (three out of five) that encode for a single theoretical place field on the learned route. As illustrated in Figure 6e, this generates sequential recall activity in all neurons encoding for each consecutive place field on the route over a period of ∼33ms. This activity is self-terminating and on approximately the same timescale as sharp wave ripples observed in vivo. Interestingly, strong auto-associative connections are not necessary to generate these sequential activity patterns in encoded place cell assemblies. Consistently high recall fidelity is produced following learning with the non-BCM type STDP rule, when only strong hetero-associative connections are generated (Figure 6f). In fact, the fidelity of recall activity is generally inversely correlated with the relative strength of auto-associative synaptic weights, regardless of the concentration of ACh employed.

However, further simulations demonstrate that the relative scale of background synaptic connections contributes more significantly to erroneous recall activity than that of auto-associative connections – as arbitrarily setting the weight of all background connections to zero following dual coded learning generally eliminates all incorrect firing activity during subsequent recall (Figure 7a). Furthermore, the temporal error in recall activity following dual coded learning with BCM type STDP rules is generally low (Figure 7b), such that correct sequence prediction might be produced if one considers only the mean time of firing in all neurons that encode for a single place field. It is also interesting to note that recall fidelity consistently decreases over time, with the vast majority of erroneous recall activity occurring in the final ∼15ms of each putative sharp wave ripple event (Figure 7c). Intuitively, the effective speed of putative SWR activity – calculated using the time taken for sequential activity to progress through place cells encoding for the entire length of the 2m track – is significantly affected by the concentration of ACh present in the network (Figure 7d), which dictates the magnitude of recurrent synaptic currents. The effective speed of recall following hetero-associative learning simulations is significantly slower (∼25ms⁻¹), due to the fact that fewer strong pre-synaptic connections (and therefore weaker recurrent synaptic currents) exist for each simulated place cell.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 7. Further Details of Putative Sharp Wave Ripple Recall Activity.

(a) Statistics relating to dual coded recall following learning with the triplet-based BCM type STDP rule, when all background connections (i.e. between place cells and those encoding for all place fields that are not within three steps on the learned route) are set to 0 following learning. Data shown for Φ = 0.111 and averaged over 1000 randomly initialised recall epochs, illustrating the relative frequency of neurons that fired before (dark grey); at the same time as (medium grey); and after (light grey) the first action potential in any simulated neuron encoding for the next place field on the learned route. This can be directly compared with Figure 6f. (b) Histogram of temporal magnitude for every erroneous spike fired during 1000 randomly initialised dual coded recall epochs with Φ = 0.111 following learning with the triplet-based BCM type STDP rule and theta modulated plasticity (that being the lowest recall fidelity displayed in Figure 6f). (c) The mean percentage of incorrectly timed recall spikes observed during sharp wave ripple recall activity, displayed in terms of the distance along the learned route, in place fields, from the externally stimulated place cells. Data is averaged over 1000 randomly initialised dual coded recall epochs for the BCM type STDP rules with Φ = 0.111. (d) The effective speed of SWR activity – calculated using the time interval between the first spike caused by superthreshold external stimulation and the first subsequent spike in a place field encoding for the same place field following the propagation of activity along the entire length of the learned route – for different concentrations of ACh. Data is averaged over 1000 randomly initialised dual coded recall epochs, following learning with theta modulated plasticity.

https://doi.org/10.1371/journal.pcbi.1000839.g007