Spontaneous activity is structured in space and time.

As a first step, we want to get a better understanding of the spontaneous network dynamics by visualizing the activity of the 200 simulated excitatory neurons in three dimensions. For this, we apply principal component analysis to project the 200-dimensional spontaneous and evoked binary activity vectors to the first three principal components (PCs) of the evoked activity. These three components usually account for 40–50% of the variability. The evoked activity (points) captures the properties of the input by forming one activity cluster for each position in the input sequences (“A” and “E”, “B” and “F”, …) (Fig 3a). We believe that the very similar transition structure is responsible for this: both “D” and “H” transition to either “A” or “E”. Since the neurons coding for “D” and “H” will only have a small subset of overlapping projections but these postsynaptic neurons have to code for both “A” or “E”, “A” and “E” should look similar. A similar argument applies for the following letters. Despite the overlap in the first PCs, both sequences can be clearly separated in later PCs (S3 Fig). We observe a trend that letters later in the word separate “easier”, i.e. in earlier PCs. For example, “H” and “D” separate in PC4, “G” and “C” in PC9, “B” and “F” in PC11, and “A” and “E” in PC 15 for this simulation. This is probably due to small differences at the beginning accumulating through the recurrent structure to larger differences towards the end of the word. In addition to the learnt structure, the spontaneous activity (lines) closely follows the structure of the evoked activity. This observed spontaneous replay of evoked sequences is similar to [12, 14, 43]. The authors of [12] showed with optical imaging in cat area 18 that spontaneous activity is highly structured and smoothly varies over time (in their case it smoothly switches between neighbouring orientations). They also observe that spontaneous activity preferentially visits states that correspond to features that occur more often in nature (in their case horizontal and vertical bars). We demonstrate an abstract version of the latter point below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Spontaneous and evoked activity align through self-organization.

The network was stimulated with the words “ABCD” (67%) and “EFGH” (33%). (a) The spontaneous activity follows the spatiotemporal trajectories of the evoked states in the PCA projection. (b) In the multidimensional scaling projection, the evoked activity (red) follows the spontaneous outline (black) and avoids the shuffled spontaneous states (blue) (cp. Fig 6c of [13]). (c) The evoked states are closer to the spontaneous states than to the shuffled spontaneous states: As in Fig 6 of [13], the distance from evoked states to the closest spontaneous states (D_spont) is smaller than the distance to the closest shuffled spontaneous state (D_shuff). The red dashed line shows equality. (d) Spontaneous activity becomes more similar to evoked activity during learning: After self-organizing to “ABCD” and “EFGH” with identical probabilities, spontaneous activity was compared to the evoked activity from the imprinted sequences (natural) or the two control sequences “EDCBA” and “FGH” (control) with Kullback-Leibler divergence. New networks were generated for each training time and condition. Error bars represent SEM over 50 independent realizations. ⋆ indicates p < 0.05 for a t-test assuming independent samples and identical variances.

https://doi.org/10.1371/journal.pcbi.1004640.g003

Spontaneous activity outlines sensory responses.

Next, we tested if the SORN model captures the finding of [13] that “spontaneous events outline the realm of possible sensory responses”. To model the conditions of the original experiment, we compared spontaneous activity to evoked activity from only 5 randomly selected letters from the original words. This captures the fact that only a subset of the “lifetime experience” of stimuli was presented during the experiment. As in [13], 150 randomly selected spontaneous activity vectors of the excitatory units, their shuffled versions, and 150 of the just described evoked events were then reduced from the high-dimensional activity patterns of excitatory neurons to 2D by multidimensional scaling (MDS). Simply put, this method uses all variability in the data to represent the distances between data points in their high-dimensional space (neural activity vectors) in the plotted two dimensions as well as possible. This is in contrast to the previous 3D-PCA plots, which only consider the variability in the first three principal components. Our results indicate that the spontaneous activity outlines the evoked activity while the shuffled activity does not capture its structure (Fig 3b). Similar to the experimental study, we confirm this by showing that evoked activity states are significantly closer to the spontaneous activity states than to the shuffled spontaneous ones (Fig 3c).

Spontaneous activity adapts to evoked activity.

Given the previous results, the question arises how these phenomena are related to the network self-organization. For this, we compared the effect of learning to results from [16]. They showed that during development, the distance between the distribution of spontaneous activity and the distribution of evoked activity decreases. Interestingly, the distance decreases both for stimuli that the animal was exposed to during development (natural movies) and to artificial stimuli (gratings and bars), but the reduction is less pronounced for the artificial stimuli. We try to capture the essentials of this experiment by presenting the same two sequences during the self-organization phase. After learning, we then either present the same sequences (natural condition) or the two control sequences “EDCBA” and “HGF” (control condition) for as many steps as the number of bins in the original paper. We chose this specific control condition because it has a different structure while still stimulating the same input units, similar to the control conditions in the original study. In both conditions, the sequences are presented with equal probability. The evoked network states were then compared to the spontaneous states using the KL-divergence. Our model shows a qualitatively similar behaviour in that the KL-divergence between the distribution of evoked responses in the natural condition and the distribution of spontaneous responses decreases during learning (Fig 3d). This decrease is larger compared to the decrease observed for the reversed sequence.

Taken together, these results show that in our simple model the spontaneous network activity outlines the possible sensory responses after self-organization. It is important to note that none of these effects occur in a random network without plasticity (see [37] for sample network dynamics of the SORN without plasticity).

Self-organization captures stimulus probabilities.

Having compared our model dynamics to qualitative features of spontaneous activity, we next performed a quantitative analysis on how the learnt sequences are represented in the network. In order to do so, we assign to each spontaneous state a stimulus letter based on the stimulus of the closest matching evoked network state (see Methods for details). From this, we compute the frequency of word-occurrence in the spontaneous activity. The spontaneous states resemble the evoked states in two ways (Fig 4a and 4b): First, the letters that were presented more often in the evoked activity also occur more often in the spontaneous activity. Second, the transition between states occurs in the correct temporal direction while reversed transitions rarely occur in the spontaneous activity. By varying the probability of each sequence during self-organization, we can quantify how these priors are captured by the spontaneous activity. As one can see in Fig 4c and 4d, the probability of the words and letters approximate their frequency during learning. However, we observe a tendency to overrepresent the more frequent stimuli. Additionally, the frequency of letters increases towards the end of the more frequent word while it decreases towards the end of the less frequent word. This is due to spontaneous switching between the two words: the more frequent word is more robust and activity switches less often to the less frequent word than the other way round. We further explore this issue in the section Network Analysis below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Different priors are learnt by the network.

During self-organization, the sequences “ABCD” and “EFGH” were randomly interleaved with frequencies of 67% and 33%, respectively. This is reflected in the relative occurrence of (a) each letter and (b) each word in the spontaneous activity. For different priors during self-organization, this results in the frequencies in (c) for each letter and in (d) for each word. Both show overlearning effects in that their frequencies are biased in favour of the word that was shown more often. The reversing trend and high variance for the extreme priors (0.1 and 0.9) can be accounted for by pathological network dynamics for some simulations with these priors. The letter frequency is the observed frequency in the evoked activity while the word frequency was normalized over the total number of observed words (“ABCD”, “EFGH”, “DCBA”, and “HGFE”) to yield better comparison over different realizations. Error bars represent SEM over 20 independent realizations.

https://doi.org/10.1371/journal.pcbi.1004640.g004

Stimulus onset quenches variability.

First, we replicate the “widespread cortical phenomenon” that the “stimulus onset quenches variability” [10]: While the neural activity and the activity in this model is in general highly variable and shows signatures of a Poisson process (cp. Fig 1), we observe a drop in variability measured by the Fano factor (FF) in response to stimulus onset (Fig 5). We also found that this effect is significantly stronger when the respective stimulus had a higher presentation probability (Fig 5c and 5d). Furthermore, the stimulus that was presented more often also elicits a higher mean firing rate. A similar effect was observed for MT responses to moving gratings [44]: the Fano Factor decreased most for the preferred direction, i.e. for the direction with the highest mean rate.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Stimulus onset quenches variability.

(a) and (b) Sample spike trains from two representative neurons aligned to the stimulus presentation (shaded area). (c) and (d) The population average of the Fano factor (FF) decreases with stimulus onset. The FF is only computed for units that do not receive direct sensory input. These results mimic Fig 5 of [10]. FFs, mean rates and variances are for causal moving windows of 5 time steps. (c) was computed for the presentation of stimulus “AXXX_ _ _ …” during the test phase after being presented with a probability of 0.1 during self-organization. (d) In turn, “BXXX_ _ _ …” had a probability of 0.9 in the same experiment. Error bars represent SEM over 20 independent realizations.

https://doi.org/10.1371/journal.pcbi.1004640.g005

Across all stimuli, the drop of the FF is accompanied by a rise of the mean firing rate (Fig 5c). To control for this, we performed a “mean matching” analysis as suggested in [10] (see Methods for details). S4 Fig shows a mean-matched decrease of the FF at stimulus onset.

Spontaneous activity predicts evoked activity and decisions.

Next, we test our model on two similar findings: First, the authors of [8] found that the optical imaging response evoked by simple bar stimuli is almost identical to the sum of the spontaneous activity prior to stimulus onset and the average stimulus-triggered response. Second, the original study that we model here [20] found that spontaneous activity prior to stimulus onset predicts the decision of the subjects. Thus, spontaneous activity influences the evoked response at the level of neural responses and behavioural decisions.

We model these experiments by training simple linear classifiers either to predict the evoked spiking of individual cells from the spontaneous activity immediately before stimulus presentation, or to predict the decision of the network after the presentation of stimuli with different ambiguities. These are compared to a baseline prediction from a “control classifier” that is based on shuffled spontaneous activity.

We find that the spontaneous activity prior to stimulus onset allows prediction of the evoked activity (Fig 6a) and the final decision (Fig 6b) in a linear manner. These two complementary experiments demonstrate that the network’s spontaneous activity prior to the stimulus contains significant information about subsequent evoked responses and perceptual decisions as demonstrated experimentally in [8] and [20].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Spontaneous activity predicts evoked activity and decisions.

The network self-organizes during repeated presentations of the sequences “AXXX_ _ _ …” (33%) and “BXXX_ _ _ …” (67%) with blank intervals in between. In the test phase, an ambiguous mix of cue “A” and cue “B” is presented (“A/BXXX_ _ _ …”, shaded area) and the network decides at the first “_” with a linear classifier if either A or B was the start of the sequence. (a) Trial-to-trial variability of the evoked activity patterns during the test phase is well predicted from activity prior to stimulus onset. The figure shows the correlation between the variable evoked activity patterns at different time steps after stimulus onset and a linear prediction of these based on the stimulus and either the spontaneous activity state prior to stimulus onset (blue) or trial-shuffled spontaneous activity (baseline, grey). Similar to Fig 4 in [8], the decay of the correlation is exponential-like. The correlation drops further as new stimuli are presented (hatched area). Due to variable inter-trial-intervals, the hatched area covers the entire area were stimulation is possible. (b) The decisions of the network can be predicted from spontaneous activity before stimulus onset. The plot shows the accuracy of predicted network decisions (“A” vs. “B”) at the green dashed line from activity surrounding the decision. Separate classifiers were trained for each of the 11 ambiguity classes (e.g. 20%A) and time step surrounding the decision. The grey line corresponds to predictions from trial-shuffled activity. Predictions in (b) are averaged over all priors of Fig 7c. Error bars represent SEM over 20 independent realizations.

https://doi.org/10.1371/journal.pcbi.1004640.g006

The initial trial-shuffled baseline prediction in Fig 6a follows a trend similar to the inverted Fano Factor (Fig 5): it initially increases and then decreases. This is because a more stereotypical response will be both predicted more easily and have a lower variability. The tail of the prediction then decays further due to the fading of memory in these networks [37]. As new stimuli are presented (hatched area), the predictability drops even further since the next stimulation is independent of the previous one. In experimental settings, where one does not have access to the full network state, the unobserved states would have a similar effect to our new inputs: the predictability would decay due to variability that one does not have access to. So even for purely spontaneous activity after stimulation, one would expect a quick decay of predictability due to inputs to the circuit from brain areas that were not recorded.

Please note that the baseline in Fig 6b is above 50%. This is because the decisions of the network are usually biased towards “A” or “B” similar to human subjects being biased to perceive more faces or vases in the face-vase experiment. This bias can be exploited by the control classifier by always predicting the more likely decision and thereby getting above-50% accuracy. Also note that the stimulus onset triggers an increase in predictability. This is due to the nature of stimulation: for the same ambiguous stimulus, different neurons are stimulated at each trial, which influences the decision to be predicted.

Taken together, all these results show that the complex dynamics of a very simple self-organizing recurrent neural network suffice to reproduce key features of the interaction of spontaneous and evoked activity.

Network dynamics is reminiscent of sampling-based inference.

The core of Bayesian learning and inference is the probabilistically correct combination of the hitherto observed experience—which we have already identified as prior in the previous spontaneous activity experiment—with a current stimulus s, which is the new instantaneous evidence in Bayesian terminology. To which extent does the self-organized learning of our model and the subsequent response evoked by a stimulus reflect the Bayesian inference capabilities of the brain as investigated in a number of experiments (see, e.g. [45], but also see [46])? Specifically, we want to investigate to which extent the input-output behaviour of SORN is reminiscent of sampling-based inference.

Before comparing the behaviour of the SORN to a probabilistic inference process we have to reason about the idea where we locate the stochastic component in the deterministic recurrent network. The definition of the SORN implies that its state is deterministic given the current input and the full internal state of the previous time step. From the perspective of an external experimenter, however, the internal state is unknown, which leads to a pseudo-stochastic relation between the actual stimulus and the later readout. In the Methods we derive a probabilistic model that reflects this source of uncertainty by formalizing the input stimulation as noisy binary channels. The probabilistic inference is then modelled as a Naive Bayes classifier that gets its evidences from these noisy channels. Note that the probabilistic model is conceived such that it has no a priori knowledge about the labels of the channels or their relation to the population “A” or “B”, but it gains this knowledge from a learning phase similar to the SORN model with its subsequent linear classifier. As the result of the probabilistic model we use the posterior probability of the response variable “A” given the ambiguous stimulus s with respect to the learned prior p(A), or more precisely the probability function p(A|f_A) that directly relates the fraction f_A of ambiguity in s to that posterior probability.

Importantly, we are not looking for an explicit representation of this posterior probability in our SORN model (as suggested by [47]). Instead, we follow the idea of the sampling hypothesis [45, 48] and interpret the binary decision of the SORN for “A” or “B” as a one-shot sample from this posterior probability distribution p(A|f_A) upon the one-shot presentation of the stimulus s. Upon every new presentation of the same stimulus s we expect to get a new sample from the same posterior. Thus we can experimentally measure the (binary) empirical distribution of the network’s stochastic answers and compare those results with the posterior according to the probabilistic model above.

The experimentally observed fractions of decisions for either “A” or “B” (blue and green lines in Fig 7a) approximate the Bayesian posterior p(A|f_A) and p(B|f_A) (grey dashed lines) for different ambiguity fractions f_A after convergence with learned priors of p(A) = 0.33 and p(B) = 0.67. We compare the results with an identical setting in which the STDP was turned off, but IP left on, to disentangle the respective effects (Fig 7b). The parameters of the probabilistic model were independently fitted to the simulation without STDP. In order to get a compact overview on the inference behaviour of the model across different values of the priors we especially observe the point of intersection of the blue and the green line and ask, which is the mixture fraction f_A of the two stimuli “A” and “B” that is necessary in order to counterbalance the prior such that the probability of the model’s response for “A” and “B” are both 50%. We plot these neutral values of f_A across different priors from p(A) = 10% to p(A) = 90%, both for the full model (blue line in Fig 7c) and for the case where STDP was turned off during the learning phase, leaving only IP for the adaptation of the excitabilities (Fig 7d). We compare them to the corresponding solutions of p(A|f_A) = 0.5 from the probabilistic model (grey dashed line). Note that we always train the readout classifiers for the classes “A” and “B” with an equal number of positive and negative examples regardless of the prior, in order to avoid a trivial classification bias.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Prior and ambiguous information is combined in network decisions.

(a) As in Fig 6, the network self-organizes during repeated presentations of the sequences “AXXX_ _ _ …” (33%) and “BXXX_ _ _ …” (67%) with blank intervals in between. In the test phase, an ambiguous mix of cue “A” and cue “B” is presented (“A/BXXX_ _ _ …”) and the network decides at the first “_” with a linear classifier if either A or B was the start of the sequence. The fraction of decisions for “A” or “B” at the first blank state “_” approximates the integration of cue likelihood and prior stimulus probability as expected from the probabilistic model (grey dashed line) for the given prior (p(A) = 0.33). The probabilistic model was fitted by grid search over its two parameters to the parameters that had the smallest accumulated error over all priors. (b) For the “Only IP” condition, STDP was deactivated during the self-organization phase and a new probabilistic model was fitted. (c, d) The intersections of the decisions for different priors for our simulations (blue line) and the probabilistic model (grey dashed). The performance was evaluated for the same simulation as Figs 5 and 6. Error bars represent SEM over 20 independent realizations.

https://doi.org/10.1371/journal.pcbi.1004640.g007

The result clearly reflects the correct incorporation of the prior—learned by self-organisation with STDP—into the activity upon an ambiguous stimulus. The homeostatic adaptation alone (IP only) only accounts for a small part of this adaptation capability, but provides a bias towards the right behaviour.

Two factors interact in the network in order to learn the above likelihoods and the model. First, firing thresholds of the input units are regulated by intrinsic plasticity. This entails that the “A”-neurons will not fire every time “A” is presented because they might have fired too frequently in the past. Second, input units (and neurons further downstream) for stimulus “A” can be active when the network is presented with stimulus “B” due to ongoing spontaneous activity. These two mechanisms ensure that ambiguous activations of the input populations are already present during the self-organization and learning phase. During the test-phase with ambiguous stimuli, the ongoing spontaneous activity will again lead to suppression or enhancement of “A” or “B” by either adding additional activity or suppressing activity. This can happen either directly through enhanced ongoing excitation or inhibition or indirectly through previous over- or under-activity leading to higher or lower thresholds. This effect of spontaneous activity on network decisions is the reason the decisions can be predicted from spontaneous activity in Fig 6.

Together, these results demonstrate that the network not only captures the experimentally observed interactions of spontaneous and evoked activity, it also integrates prior and ambiguous information while doing so. While the resulting behaviour looks sampling-like it is not clear if this model can learn more complex probability distributions. However, the ability of SORN models to solve much more complex sequence learning tasks is well documented (see, e.g., [37, 38] and Discussion for details).

Single-unit analysis.

We can see in Fig 8a that the conditional probability of neuron x_i spiking given that neuron x_j spiked at the previous time step (p(x_i(t + 1) = 1|x_j(t) = 1)) grows roughly linearly with the synaptic weight between both neurons except for saturation effects: . This relation, as simple as it might seem, immediately breaks down if IP or SN are deactivated (S6 Fig): While the general and intuitive trend persists that high weights imply higher conditional firing probabilities, the linear relation vanishes. The conditional probabilities in these figures were always computed for the spontaneous phase after self-organization to avoid effects from the input.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Self-organization imprints stimulus properties in the network.

(a) The conditional probability of x_i spiking given that x_j just spiked (p(x_i(t + 1) = 1|x_j(t) = 1)) is roughly proportional to their synaptic connection except for saturation effects for very large weights. (b) The firing probabilities of each neuron relative to stimulus onset. The network develops sequential activity patterns during the presentation of both sequences (top and middle) and during spontaneous activity (aligned to spontaneous states corresponding to “C”, bottom). Neurons were sorted according to their maximal firing probability relative to the sequence “ABCD”. (c) The prediction of transition probabilities during spontaneous activity from the singular value decomposition of W^EE. (d) The actual transition probabilities during spontaneous activity show intermediate switching between the two sequences probably due to variability introduced by intrinsic plasticity (see Discussion for details).

https://doi.org/10.1371/journal.pcbi.1004640.g008

This relation between connection strength and firing probability interacts with STDP (but does not result from it, see S6 Fig): Given two reciprocal weights W_ij and W_ji with W_ij > W_ji, the average weight update will be (given the target firing rate from the intrinsic plasticity, H_IP) (1) (2) (3) (4) (5)

We observe that the expected weight update is directly proportional to the weight difference. In the likely case of the reciprocal weight being 0, it is directly proportional to the weight. This leads to the rich-get-richer behaviour of synaptic strengths as observed in a prevous version of the SORN [39]. Basically, a high weight increases the firing probability of the postsynaptic neuron upon presynaptic activation, which further increases the weight Eq (4), and so on. Another key factor for the rich-get-richer behaviour is synaptic normalization, as described in [39].

Apart from the skewed weight distribution, it eventually also leads to a sequential activation of neurons as observed in neocortex [28], in a previous version of the SORN [49], and in the current study (Fig 8b).

Next, we analyse the impact of input structure on the STDP dynamics. For this, we consider the case where an excitatory neuron x_A receives external input with the frequency p(A). Furthermore, we assume that this neuron projects to a second excitatory input-receiving neuron x_B with a very small weight, i.e. , so that the impact of the weight on the firing probabilities can be neglected. Finally, we assume that there is no reverse projection from x_B to x_A, i.e. x_B(t) is assumed to be approximately independent from x_A(t + 1). This neuron receives input from the next letter in the input word, “B”. We further assume that the stimuli are presented infrequently enough so that the intrinsic plasticity does not interfere with the activation of input-receiving neurons when the corresponding input is presented, i.e. p(A) ≪ H_IP. This ensures that whenever x_A is activated by the input, x_B will be active in the subsequent time step with a probability close to 1. In this case, we have (6) (7) (8) (9)

The first term in Eq 7 corresponds to the conditional probability of firing when the firing of x_A is due to the stimulus, which will by assumption also lead to an activation of x_B with probability 1, or due to the chance event that a spontaneously driven spike in x_A is followed by a spontaneously driven spike in x_B, which is H_IP. These spontaneously driven spikes can either be due to recurrent activations from spontaneous activity in presynaptic units or due to the threshold of the unit being lowered below the current drive by intrinsic plasticity. Both probabilities are combined according to their relative occurrence: for the stimulus-driven case and () for the recurrently driven one.

We observe that the expected weight change is directly proportional to p(A). As a result of this, if we consider two stimuli with different frequencies, the weights will grow stronger for the more frequent stimulus. In fact, they are directly proportional so that a stimulus that is twice as frequent will have a twice as strongly growing weight in the network. Taking into account the proportionality between the weight and the conditional firing probability and assuming initially random weights, this means that on average the more frequently presented stimulus during learning will also be associated with stronger weights and have a higher probability of recall. It is important to keep in mind that this specific case assumes that x_A receives direct input. Weights between pairs with only indirect input on x_A will be corrupted from unreliable activation of the presynaptic direct-input neurons of x_A.

The above proportionality will break down if the weights become strong enough to have a strong impact on firing probabilities. This case will again result in rich-get-richer behaviour. This will lead to the overrepresentation of probabilities (“overlearning”) as observed in Fig 4. Nevertheless, the Figure also shows that despite this overrepresentation, the network does not completely abolish the more infrequent stimulus at convergence since the input still has an effect in driving the neuron and thereby the learning.

Population analysis.

Having analysed the interactions of single neurons, it is important to know if these results generalize to the population as a whole. More specifically:

1. Can STDP imprint the sequential input structure in the recurrent excitatory connectivity?
2. Does the excitatory connectivity correspond to the observed transition probabilities of the network states when it is running without input?

We analyse these question by simplifying the excitatory network dynamics to a linear dynamical system:

(10)

We then apply singular-value decomposition (SVD) to the recurrent weight matrix:

(11)

Because U and V are orthonormal and Σ diagonal, we have (12) where u_i and v_i are column-vectors of U and V and σ_i is the corresponding singular value in Σ. By comparing Eqs (10) and (12), one can see that the vectors v_i and u_i σ_i define transitions similar to x(t) and x(t + 1). We therefore analyse the behaviour of learned connections by matching each vector v_i and u_i to their closest matching evoked state. This is done by taking the maximum of the dot product between the vector and the last 2500 evoked activity states of the training phase. Thereby we get from transitions between vectors to transitions between input letters. By scaling each transition by its singular value σ_i and normalizing to 1, we can predict the transition probabilities.

In Fig 8c, the result of this analysis is applied to the sequence learning task. When compared to the actual transitions during spontaneous activity in Fig 8d, one can see that the SVD-analysis approximates the actual transition probabilities. This entails that STDP can indeed imprint the input structure in the weight matrix and that these probabilities are correctly represented during spontaneous activity. The corresponding results for the inference tasks can be found in S5 Fig. Scatter plots for the spontaneous and predicted transitions can be found in S7 Fig.

Taken together, these analyses explain why and how the network activity acquires the stimulation structure in Fig 3 and how the input priors can be both imprinted into the network connectivity (cp. Fig 4) and utilized during testing (cp. Fig 7).

Network setup.

The network consists of a population of N^E = 200 excitatory and N^I = 0.2 × N^E = 40 inhibitory McCulloch & Pitts threshold neurons [68]. The connections between the neurons are described by weight matrices where, for example, is the connection from the j^th inhibitory neuron to the i^th excitatory neuron. We model the excitatory to excitatory connections as a sparse matrix with a directed connection probability of p^EE = 0.1 and no excitatory autapses . W^EI as well as W^IE are dense connection matrices. We do not model connections within the inhibitory population. All weights are randomly drawn from the interval [0,1] and then normalized by the synaptic normalization described below.

The input to the network is modelled as a series of binary vectors u(t) where at each time step t during input presentation all units are zero except for one. For better readability we assign an arbitrary letter to each such state when describing different input sequences later on. The letter “_” corresponds to presenting no input. Each input unit u_i projects to N^U = 10 excitatory neurons with the constant weight wⁱⁿ = 0.5. These randomly selected and possibly overlapping projections are represented by the input weight matrix W^EU.

At each discrete time step t, these variables contribute to the binary excitatory state x(t) ∈ {0, 1}^NE and inhibitory state y(t) ∈ {0, 1}^NI as follows:

(13)

(14)

Here, Θ(a) is the element-wise heaviside step function, which maps activations a to binary spikes if the activation is positive. T^E(t) and T^I are the thresholds of all neurons and are crucial in regulating the spiking. They are evenly spaced in the interval (0, 0.5) for the excitatory thresholds and for the inhibitory ones.

directly influences the amount of inhibition. presents a trade-off between fine-grained inhibition and a broad dynamic range: for a low , the additional excitation needed to activate one more inhibitory neuron is small, leading to a more fine-grained response to excitatory fluctuations, which is in general favourable because it helps to avoid pathological behaviour. However, a low sets a low bound on the maximal amount of inhibition: it is easier to activate all inhibitory neurons and thereby saturate the possible inhibitory response. Therefore, in cases with high fluctuations in excitation, such as trial-like settings with a high rate increase at stimulus onset, a high is necessary. Please note that the average amount of inhibition will have no effect on the average amount of excitation since the average excitatory activity in the network is regulated by the intrinsic plasticity. is set to for the first tasks and to for the inference task with its trial structure. If is set to 1, excitation and inhibition are balanced: For a given fraction of excitatory activity , each inhibitory neuron receives on average excitatory input because synaptic normalization ensures that the summed excitatory incoming weight is 1. This in turn activates the fraction of inhibitory neurons, because the thresholds of the inhibitory units are uniformly distributed in the interval (0, 1) by assumption. It is important to note that at no point any noise is added to the network. The only external variability is the random alternation of input words, as described in the Results.

The excitatory thresholds as well as W^EE(t) are adapted over time by three plasticity mechanisms. The state x(0) is randomly initialized with probabilities T^E(0) and y(0) is initially set to 0.

Plasticity mechanisms.

The network employs three different kinds of plasticity: Spike-timing dependent plasticity (STDP) [69–71] extracts structure from the input by shaping the weights within the excitatory population. This is counterbalanced by two forms of homeostatic plasticity: intrinsic plasticity (IP) [72, 73] and synaptic normalization (SN) [32, 33, 74]. All these mechanisms are known to co-occur in the hippocampus and neocortex.

STDP strengthens the connection from unit j to unit i whenever a spike in i directly follows a spike in j (i.e. j helped to trigger i) and is weakened whenever a spike in i precedes a spike in j. This results in the following update equation (with η_STDP = 0.001):

(15)

The authors of [33] showed in an electron microscopy study that the summed synaptic area per μm of dendrite is similar before and after long-term potentiation while the synaptic area per synapse increases and the number of synapses per μm decreases. This indicates that plasticity redistributes weights to avoid uncontrolled growth. This is modelled by normalizing all incoming connections of a neuron to 1—a process called synaptic normalization (SN). In addition to this postsynaptic normalization, there is also evidence for presynaptic normalization or “conservation of axonal arbor”[54, 55]. These studies suggest that all outgoing connections are redistributed to achieve a constant total weight. This is modelled by normalizing all outgoing connections of a neuron to 1. Together, these mechanisms result in the following iterative weight update:

(16)

At the same time, there are a variety of regulatory mechanisms that control neural firing at different time scales such as absolute and relative refractory periods, spike rate adaptation and intrinsic plasticity [32, 73]. Here, we abstract from those by using a simple homeostatic regulation of the spike threshold at a single time scale (η_IP = 0.001). This intrinsic plasticity (IP) rule regulates the individual thresholds so that on average, the excitatory neurons will fire according to their target rates H_IP:

(17)

The target rates are uniformly drawn from the interval (H_IP − ϵ_IP, H_IP + ϵ_IP) = (0.1 − 0.01, 0.1 + 0.01). The resulting diversity in firing rates helps to prevent pathological network activity.

The inhibitory connections are scaled during the initialization phase so that the sum of excitatory weights received by each inhibitory unit and the sum of inhibitory weights received by each excitatory unit is 1.

The STDP and IP rules are only operating on the excitatory neurons for two reasons. First, by restricting plasticity to only the excitatory population, the model becomes simpler and thereby easier to understand and interpret. Second, there are fewer data on plasticity in inhibitory neurons (but see [39]).

Stimulation paradigm.

The stimulation paradigm is very similar across all tasks and can be divided into three phases:

In the self-organization phase, the network is stimulated with input patterns or sequences for T_plastic = 50000 steps. During this time, all plasticity mechanisms are active and the network can self-organize in the presence of the given input. We represent the input vectors by letters. For example, the stimulation paradigm can then be a random alternation of the words “ABCD” and “EFGH”. After this, STDP is switched off so that the properties of the learnt connections can be studied without interference from continued changes to the weights. This is done during a training and testing phase.

In the training phase, the stimuli are kept identical to observe the properties of the network under the self-organization conditions and training appropriate readouts for T_train = 20000 steps. For the model of the sequence prediction of [31], we do not show stimuli in this phase corresponding to the “sleep”.

Then, during the testing phase, we either observe the spontaneous activity while the network is not stimulated, or we test our readout on data generated by stimulating the network with appropriate stimuli for T_test = 50000 steps.

For these phases, the intrinsic plasticity is still active to ensure stable average activity. After each phase, the activity state is shuffled in accordance with [75]. To simulate the trial-like structure of the inference task, we chose to have blank periods between each stimulus presentation. The length of these periods was drawn from a uniform distribution over the interval [10, 15] time steps. For the model of the sequence learning study (Fig 2), we used a fixed delay of 10 time steps because the experiment only had fixed inter-trial delays [31].

Principal component analysis and multidimensional scaling.

For the principal component analysis (PCA) of the spontaneous and evoked states, we did a PCA of the last 2500 steps of the training phase. We then projected these states in the subspace defined by the first three components of the PCA and also projected the same amount of spontaneous activity states in the same subspace. The spontaneous states were taken from the end of the testing phase to avoid artifacts due to the re-adaptation of the thresholds after cutting the input.

Multidimensional scaling (MDS) was performed as closely as possible to the method used in [13]. As in that paper, we used 150 points of spontaneous, shuffled spontaneous and evoked samples. Further following the paper, we shuffled the spontaneous activity of each neuron individually over time so that neurons kept their rate but lost their timing. The samples were chosen randomly from the end of the training and testing phase. We did not subsample our neurons to the 45 that they recorded from because the distances between individual activity patterns become too similar in that case. This did not happen for their study because they used rates instead of spikes. To account for the fact that [13] only showed a subset of all the stimuli that the network was exposed to during its development we also only used a randomly chosen subset (5 letters) of the stimuli that we presented in the self-organization phase. We used the same Matlab function for MDS as the one used in the original paper (with Kruskal’s normalized stress1 criterion).

KL-divergence of spontaneous and evoked activity.

To replicate the results of [16], we recorded spontaneous and evoked activity for the same number of time steps (750.000) and randomly subsampled 16 units from our network. Please note that due to an exponential increase in possible patterns with the number of units, more units would have soon become infeasible to analyse with conventional methods. We also excluded the first 5000 steps of each phase to account for adaptation after changes in the stimulation paradigm. To compute the KL-divergence we first have to estimate the probabilitiy p(x) for each pattern x from the set of the 2¹⁶ possible patterns X. In order to do so, we created a bin for each pattern and simply counted the occurrence of each pattern. Additionally, we started with a non-informative prior by assuming that each pattern x ∈ X was already observed once. This initial prior is necessary since KL-divergence is only defined for non-zero probabilities. After normalizing, we then computed the KL-divergence between evoked and spontaneous activity according to (18) (19)

Pattern analysis.

Evoked and spontaneous activity patterns may rarely be exactly equal. In order to relate spontaneous activity patterns to evoked activity patterns, we use the following method. We assigned to each spontaneous state the letter corresponding to the best-matching evoked activity state. If, for example, x(t) had the smallest Hamming distance to an evoked state when the letter “A” was shown, then x(t) was labelled as corresponding to input letter “A”. To avoid biases, the collection of evoked and spontaneous states used for the comparison was always obtained from the end of the training and testing phase (the last 2500 steps). The data was further reduced by ignoring blank periods and ensuring that each letter was represented equally often in the data set.

Inference analysis.

To quantify the network’s behaviour in the inference task, we trained output units based on the randomly alternating presentation of the stimuli “AXXX_ _ _ …” and “BXXX_ _ _ …”, where “A”, “B”, and “X” each refer to a subpopulation of excitatory neurons that are stimulated whenever the letter is presented. At the presentation of “_”, no neurons receive external inputs. These stimuli model a decision task where subjects were presented with the ambiguous face-vase stimulus and had to decide whether they perceived a face or a vase after a mask [20]. “A” and “B” represent the face or vase, the following “X”s are identical for both stimuli and thereby act as a mask and delay.

To read out the model decisions after the mask, we trained a linear readout in a supervised way with least-squares regression. We performed two regressions from all neurons at the step when “_” is presented to predict either “A” or “B” (based on the recurrently maintained information of “A” or “B” that should still be present in the network). In the test phase, when ambiguous mixtures of “A” and “B” are presented, the network decision is set to “A” when the readout for “A” is larger than for “B” and the other way round. The regression target was set to 0 for all other letters from the training data, since these should not correspond to a “sampling” of “A” or “B”. The regression was performed on 20000 steps of evoked activity while STDP is turned off. To avoid any biases, we took equal amounts of activity samples for each letter from the end of training.

In the test-phase, the ambiguous face-vase stimulus is modelled by stimulating the network with a mix of “A” and “B”. This is done by using input units of “A” and input units of “B”. Here, we assume that stimulus ambiguity can be modelled by the fraction of activated “A” units, f_A. The specific units selected for the ambiguous stimulus are redrawn at every stimulus presentation. To have well-defined ambiguities, we ensure for this task that the input populations of “A” and “B” do not overlap. As in the original study, the delay between stimuli is random. We model this by randomly adding between 0 and 5 delay steps to the fixed delay of 10 steps during testing (see stimulation paradigm for details).

Probabilistic model.

A detailed inspection of the input populations for the inputs “A” and “B” reveals the root of the stochastic behaviour of SORN: One input neuron, say the first neuron of the population P_A representing input “A”, may be stimulated, i.e. its input set to wⁱⁿ = 0.5, but still that very neuron does not fire in this time step due to a stronger inhibition it receives via the recurrent connections and/or a higher activation threshold due to previous activity. Obviously, the information that the neuron was stimulated by the input in this time step is lost. On the other side the neuron might get active due to the excitatory recurrent input from the network and/or a drop of its threshold due to long inactivity, no matter if there is an external stimulation or not. Also in this case the stimulus had no changing effect on the evolution of the future states of the network and thus that information is lost. In a little more complex but similar way we can reason about the propagation of the effect of this one input stimulus through the network over time until the delayed readout tries to identify if population P_A or population P_B was stimulated. From an information theoretic viewpoint we can formalize this uncertainty as a noisy binary channel, which is fully characterized by two parameters θ_0|0 and θ_1|1, or shorter θ₀, θ₁, denoting the probability that a non-stimulus (0) and a stimulus (1) is received correctly. The probabilities of a wrong transmission are implicitly defined as well, since θ_1|0 = 1 − θ₀ and θ_0|1 = 1 − θ₁.

Based on this channel perspective we can define a simple Naive Bayesian classifier that receives those noisy inputs for population P_A and P_B, denoted by a = {a₁, …, a₁₀} and b = {b₁, …, b₁₀}, during both the learning and the testing phase, assuming N^U = 10 input units as defined in the SORN description above. It is straightforward to derive the parameters of such a Naive Bayes analytically, as there are only two training cases: When the sequence “AXXX___…” is presented in the learning phase, all neurons that belong to the population P_A receive an input stimulation, whereas all neurons of the P_B population do not receive any stimulus. Thus, according to our channel model all a_i have a probability of θ₁ of being 1 and all b_i have a probability of 1 − θ₀ of being 1, i.e. p(a_i = 1|A) = θ₁ and p(b_i = 1|A) = 1 − θ₀. In the second case, when the “BXXX___…”-sequence is presented, the whole P_B-population is stimulated and the P_A-population is left alone, thus in our probabilistic model p(b_i = 1|B) = θ₁ and p(a_i = 1|B) = 1 − θ₀. The prior p(A) just reflects how often the “AXXX___…”-sequence is presented to the model during learning as compared to the “BXXX___…”-sequence. The conditional probabilities together with the prior p(A) and p(B) = 1 − p(A) fully define the Naive Bayes model up to the two free parameters θ₁ and θ₀. We will derive these parameters by fitting the probabilistic model to the experimentally derived results from the SORN network.

Given certain evidence vectors a and b we can derive the posterior probability of the Naive Bayes model according to Bayes’ rule as (20) where the likelihood p(a,b|A) can be expressed explicitly in terms of the above defined parameters (21) (22) where and are the sums of active evidences received from population P_A and P_B, respectively. Note that a and b and thus also n_a and n_b refer to the evidences as seen by the Naive Bayes classifier which result from the hypothetical noisy transmission channels of the external activations. The posterior in terms of n_a and n_b reads (23)

Note that the binomial factors that would appear in p(n_a, n_b|A) are reduced in the above fraction. In order to evaluate the whole probabilistic model we also have to evaluate the noisy channels. In our experiments we always stimulate a certain (random) fraction f_A of neurons of population P_A and a (random) fraction of (1 − f_A) neurons of population P_B. The resulting evidence vectors a and b are characterized by the respective distributions of n_a and n_b: (24) (25) where B(., .) is the binomial distribution. We obtain the resulting probability from the probabilistic model upon stimulation with a stimulus with ambiguity f_A by taking the expectation of the posterior Eq (23) over the possible realizations of n_a and n_b, i.e. (26)

This function still depends on θ₁ and θ₀. In order to determine those parameter values we consider the full experiment in the SORN model and evaluate the decision curves over f_A (Fig 7a) for all different values of the prior during training (10% to 90%). Every such curve is the result of an average of 20 realizations of the—stochastically created—SORN model. Then we find those two parameter values θ₁ and θ₀ that fit all those 9 curves the best, in the sense of the least sum of the mean squared deviations. For the sake of simplicity the optimization is carried out by an exhaustive grid search in the range of 0.05, …, 0.95 with step size 0.05. The result of the search delivered the values θ₁ = 0.85 and θ₀ = 0.45 for the full SORN and θ₁ = 0.95 and θ₀ = 0.3 for the “Only IP” condition.

Fano factor analysis.

The Fano factor is defined as . When applied to spike trains, the windowed variance σ² and mean μ are taken over trials for each neuron, condition and time step. It is then interpreted as the variability of the data. For a Poisson process, which describes the irregular spiking of neurons in many cases quite well, the Fano factor is 1 due to the variance and mean being equal for this process.

We tried to keep our analysis of the Fano factor close to [10]. As in the original paper, the FFs were obtained by weighted regression between the spiking mean and variance over trials. To compute these two, we used a causal sliding window with a width of 5 time steps, i.e. the sum of the spikes in the past four time steps and the time step to be analysed. Together with the IP parameters, this entails a rate of 0.5 spikes per bin on average. This is comparable to the conditions of the original paper: most experiments have a rate on the order of and a bin size of 50ms. One should note that due to the weighted regression and the averaging over many neurons, the resulting Fano Factor does not have to be identical to the ratio of the mean variance and the mean firing rate.

To control for an effect of the mean firing rate on the Fano factor, we also performed the “mean matching” analysis from [10]. We found that this method discards around two thirds of the data, comparable to the original study. Therefore, we only see it as a control and focussed on the real FF for the discussion.

Prediction of evoked activity and decisions.

For our last comparisons, we aimed to show that the spontaneous activity in this network can be used both to predict the following evoked activity [8] and to predict the decision of the network as for example in [19, 20].

The former was done in a similar way as the inference: For each combination of stimulus condition c and time step Δt_a after stimulus onset t_on, we trained a linear readout that maps x^c(t_on − 1) to x^c(t_on − 1 + Δt_a). The match between the predicted evoked activity and the actual evoked activity x^c(t_on − 1 + Δt_a) was then calculated by taking the pearson correlation between both states. We allowed a bias term in the regression by appending a constant to each x^c(t_on − 1) vector. To asses the impact of the spontaneous activity on the prediction, we compared this performance to doing the same regression from shuffled spiking patterns over trials. By shuffling over trials, the statistics of the individual patterns persist, but their relation to the decision vanishes. Since this still includes the bias term, this control can capture average effects but has to deal with the same number of parameters as the original regression. Therefore, if the spontaneous activity prior to stimulus onset contains information about the following evoked activity, the prediction based on the spontaneous activity should be better, i.e. correlate more with the true activity, than the one based on the shuffled spike trains. Also, both predictions should overlap for very late states since the information from the spontaneous activity should be “washed out” by then.

To predict the decision of the network, we also used this regression approach. For this, we divided the T_test steps of network activity with decisions into two halves—one for training the readouts to predict the decision for “A” and “B” and one for testing its performance. For each step before stimulus onset and a given stimulus, an individual readout was trained. The prediction was then defined as the higher readout. We evaluate the quality of this prediction by comparing it to the actual decisions of the network and computing the agreement between both for the testing data. These actual decisions are usually biased towards one of the two alternatives, either due to the prior of the stimulus in the self-organization phase or due to the initial structure of the network. This bias can be exploited by the bias term in the readout mentioned above to get above-50% performance for the baseline comparison, where again trial-shuffled spiking data plus a bias term is used to predict decisions.