One population model

The one-population model used in this study (Fig 1A) was based on that of Yamazaki and Tanaka [15]. It consisted of N_z = 1000 granule cells, each receiving excitatory afferent inputs I_i(t) derived from the external signal x(t), and recurrent inhibitory inputs from other cells. The model neurons were firing-rate (i.e. non-spiking), and the output z_i(t) of the i-th neuron at time t was given by (1) (here the bracket notation []⁺is used to set negative values to zero, preventing the firing-rate of a neuron from becoming negative). This equation describes (see Fig 1A) memory-less rate-neurons connected by single-exponential synaptic process with time constant τ_w so that neuron i sums past inputs z_j(s − 1),1≤ s ≤ t from other neurons, exponentially weighted by distance s − t into the past. Neuron j has synaptic weight A_ij w_ij on neuron i where A_ij was set to 1 with probability a and 0 otherwise, hence the parameter a controls the sparsity of the connectivity. The connectivity strengths w_ij were drawn from a normal distribution with mean w and standard deviation v_ww, normalized by population size Nz, and constrained to be positive, so that . Each neuron received an excitatory input I_i(t) with additive noise nN_i(t) (here N_i(t) is a discrete white noise process with std(N) = 1/2 so that the added noise is smaller in magnitude than the noise amplitude n 95% of the time).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Granular layer models and filter construction procedure.

A: Diagram of one-population granular layer model based on Yamazaki and Tanaka [15] consisting of mutually inhibiting granule cells and input signals coded by push-pull input. B: Diagram of two-population granular layer model consisting of granule cells (GC) and Golgi cells (GO). GC innervate GO using glutamatergic excitation (u) and inhibition by mGluR2 activated GIRK channels (m). GO inhibit GC by GABAergic inhibition (w). All synaptic connection simulated using single exponential processes (see Methods). C: Input signal x(t) consisting of colored noise input for training, test and impulse response input. D: Diagram of filter construction test procedure. The output signals z_i(t) of all granule cells during the training sequence were used to construct exponential (leaky integrator) filters of increasing time constants τ_j = 10ms (blue line), 100ms (green line) and 500ms (red line) using LASSO regression (see Methods). The goodness-of-fit (R²) of this filter construction was evaluated during the noise test sequence.

https://doi.org/10.1371/journal.pcbi.1004515.g001

In the simulations, unless otherwise specified, we used the following default values for the parameters above. The population size was Nz = 1000. The probability of connectivity was a = 0.4 (close to the value 0.5 in Yamazaki and Tanaka [15]), and synaptic variability was set to zero (v_w = 0). The default input noise level was n = 0.

This model had a single time constant which Yamazaki and Tanaka [15] took to be equal to the membrane time constant of Golgi cells in their simulations of granular layer dynamics. However it is not clear that this is the relevant time constant for a firing-rate model since the dynamics of the sub-threshold domain cannot be easily carried over into the supra-threshold (spiking) domain and are often counter intuitive. While a still prevailing misconception is that long membrane time-constants are equal to a slow spike response, the exact opposite is the case: integrate-and-fire with an infinite time constant (perfect integrators) have the fastest response time to a current step and can respond almost instantaneously [33]. Since temporal dynamics of neurons in a network are primarily determined by the time course of the synaptic currents [33–36] we have ignored membrane time constants in this and following models and instead related τ_w to the synaptic time constant of recurrent inhibition in the network.

We further want to note that the values for the synaptic time constants were not directly adjusted to replicate results for individual electrophysiological studies but rather kept at general values to study the effect on network output of interaction between different magnitudes of time constants. This issue is considered further in the Discussion.

Two population model

To allow for more realistic modeling of the dynamics of the granular layer we extended the one-population network of granule cells above to include inhibition via a population of interneurons corresponding to Golgi cells (see Fig 1B). In this model the firing-rates z_i(t) of granule cells and q_i(t) of Golgi cells were given by (2) (3)

The default sizes for the two populations were Nz = 1000 and Nq = 100. As before, the excitatory afferent input into a granule cell i was given by I_i(t), however the two-population model also had direct afferent excitation gI_i(t) of Golgi cells. The factor g setting the level of excitation was set to 0 in the initial simulations, resulting in no afferent excitation for Golgi cells. The output-feedback L(t) was 0 until later simulations (see below). The connectivity between the two populations was given by the random binary connection matrices W and U, however in this model the connectivity was not defined by a probability but by the convergence ratios c_w = 4 between Golgi and granule cells and c_u = 100 vice versa. Thus exactly 4 randomly selected Golgi cells inhibited each granule cell and 100 randomly chosen granule cells were connected to each Golgi cell. The weight of GABAergic inhibition between Golgi and granule cells was drawn from a normal distribution and normalized with (default v_w = 0) and the time constant of inhibition was given by τ_w. Besides the glutamatergic excitatory connections between granule and Golgi cells with weight (default v_u = 0) and time constant τ_u the model was extended to emulate the inhibitory effect of mGluR2 activated GIRK channels [31] with (default m = 0, v_m = 0) and time constant τ_m = 50ms. Note that mGluR2 inhibition was not used until later simulations with m = 0.003. Additional simulations were conducted with only half of the Golgi cells receiving mGluR2 inhibition i.e. Pr(m = 0) = 0.5.

In all simulations u was set to 0.1 and normalized by the excitatory time constant resulting in u = 0.1/τ_u.

All network simulations were written in C and were integrated into Python by transforming them into dynamically linked extensions with the package distutils. The stepsize in all simulations was dt = 1ms. All results were analyzed using Python. All models, methods and simulation results are available from the github repository https://github.com/croessert/ClosedLoopRoessertEtAl. A snapshot of the model code can also be found on ModelDB: https://senselab.med.yale.edu/modeldb/ShowModel.asp?model=168950. Computational resources for the simulations were partially provided by the ICEBERG cluster (University of Sheffield; access granted by the INSIGNEO Institute for in silico Medicine).

Input

The modulated input to each cell was given by the excitatory input I_i(t) = [I_0i + f ∙ 0.1 ∙ I_0i ∙ x(t)]⁺. Unless noted otherwise the input I_0i was chosen from a normal distribution with mean 1 and default standard deviation v_I = 0.1. To test increased input variability, standard deviation was increased to v_I = 2 in a later experiment. The factor f, randomly picked as either 1 or -1 defined whether the input was inverted or not. This type of input coding, here termed “push-pull” coding can be routinely found for example in the vestibulo-cerebellum where half of the cells are ipsilateral preferring (f = 1, type I) or contralateral preferring (f = −1, type II) [37].

In order to test the ability of the network to construct a linear filter with a given impulse response it is not sufficient to use impulse inputs alone, since this does not test linearity (for example the response to two successive impulse inputs may not be the sum of the individual responses). For this reason we also used random process inputs that mimic behavioral inputs.

The input signal x(t) consisted of 3 parts (see Fig 1C). The first part was a training sequence of a 5 second band-passed white noise signal (low-passed with a maximum frequency of 20 Hz) [38] chosen to mimic head velocity in the behaviorally relevant frequency range of 0–20 Hz [39]. Additionally a 5 second silent signal (x(t) = 0) was added to the training sequence to train a stable response. Training with a segment of null data finds weights which not only give the appropriate impulse response but also produce zero output for zero input data, so that they reject spontaneous modulatory activity in the network. Consecutively the previous signals were repeated with a different realization of the noise signal to test the quality of the filter construction. The third part was an impulse test signal where x(t) = 0 apart from a brief pulse of 50 ms where x(t) = 1. The colored noise signal was normalized to std(x) = 1/2 which ensured that the amplitude 0.1 ∙ I_0i included the input 95% of the time.

Filter construction and quality measure

To assess the ability of the network to implement linear filters that depend on the past history of the inputs, the output signals z_i(t) of all granule cells during the training sequence were used to construct exponential (leaky integrator) filters y(t) = F * x(t) of increasing time constants as linear sums y(t) = ∑β_iz_i(t) of granule cell outputs. This can be regarded as the output of an artificial Purkinje cell that acts as a linear neuron. In matrix terms (writing time series in columns) this expression can be written y −Zβ where the undetermined coefficients β are usually fitted by the method of least squares to minimize root sum square fitting error (4)

However over-fitting of the data, due to the large output population, can make this method misleading and give excessively high estimates of reconstruction accuracy. To avoid this problem we used the method of LASSO regression taken from the reservoir computing literature. This is a robust fitting procedure that includes a regularization term to keep the reconstruction weights small [40,41]. Here, the estimates are defined by which is the least-squares minimization above with the additional constraint that the L¹-norm ||β||₁ = ∑β_i of the parameter vector is also kept small. In practice we find that up to about 90% of weights are effectively zero using this method. In contrast to ridge regression that employs a L² -norm penalty and is commonly used to prevent over-fitting in reservoir computing [27] LASSO regression produces very sparse weight distributions. This corresponds well to the actual learning properties of the Purkinje cell, approximated as a linear neuron, in which optimality properties of the learning rule with respect to input noise force the majority of synapses to silence [42–45].

We fitted three responses y_j(t) = x(t)* F_j(t) with j = 1,2,3 and with F_j(t) = exp(−t/τ_j) being one of three exponential filters τ₁ = 10ms, τ₂ = 100ms or τ₃ = 500ms (see Fig 1D). The regularization coefficient was set to α = 1e⁻⁴ which gave best maximum mean goodness-of-fit results for the one-population model with τ_w = 50ms (not shown). LASSO regression was implemented using the function sklearn.linear_model.Lasso() from the python package scikit-learn [46].

In general the estimated weights β_i take both positive and negative values, which is not compatible with the interpretation of equation (4) above as parallel fiber synthesis by Purkinje cells. The use of negative weights is usually justified by assuming a relay through inhibitory molecular interneurons [42,44]. To test whether learning at parallel fibers alone is sufficient for the construction of filters from reservoir signals we additionally employed LASSO regression with only positive coefficients (positive-LASSO) as a comparison.

As a measure of the quality of filter construction, the weights estimated from the training sequence were used to construct the filtered responses in the test sequence and the goodness-of-fit between expected output and constructed output was computed for each filter using the squared Pearson correlation coefficient (R²) [47] (see Fig 1D). For the final goodness-of-fit measure the mean of 10 networks with identical properties but with different random connections was computed.

Lyapunov exponent estimation

A convenient way to analyze the stability or chaoticity of a dynamic system is the Lyapunov exponent λ. It is a measure for the exponential deviation of a system resulting from a small disturbance [25] and a value larger than 0 indicates a chaotic system. The Lyapunov exponent was measured empirically, similar to Legenstein and Maass [22] by calculating the average Euclidian distance between all granule cell rates z_i(t) from a simulation where x(t) = 0 and the rates from a second simulation where the input was disturbed by a small amount at one time step, i.e. x(0) = 10⁻¹⁴. This state separation simulation was repeated for 10 randomly connected networks but otherwise identical parameters and λ was estimated from the mean average Euclidian distance with . To estimate the transition between stability and chaos we were mainly interested in the sign of the Lyapunov exponent. Although taking the mean of a 100 ms period and using a relatively large Δt of 2s [24] decreases the accuracy of the Lyapunov estimation, it was used here to prevent errors in the estimation of the sign. The edge-of-chaos was defined as the point where λ crosses 0 for the first time when traversing in the direction of strong inhibition w to weak and therefore from high λ to low.

Output feedback

To model putative output-feedback to the reservoir via the nucleocortical pathway the signal L(t) = f ∙ o_i ∙ −∑β_iz_i(t) was injected into 20% of all granule and Golgi cells in the last simulations. The factor f was randomly picked as either 1 or -1 to model 50% excitation and inhibition and the weight was drawn from a normal distribution with o_i = [1e⁻⁴±1e⁻⁵]⁺. In these simulations only the case for output-feedback of the slowest filter signal is shown. Thus β_i are the weights needed to construct the filter with τ₃ = 500ms.

As noted in the reservoir computing literature [27,48,49] output-feedback in general is a very difficult task since it leads to instability. Therefore the weights β_i were not learned online but a method called teacher forcing with noise was applied [27]. The weights β_i were learned in a prior step by using the teacher signal L′(t) = f ∙ o_i ∙ −y₃(t) ∙ N(t) instead of the feedback signal L(t) to uncouple the instable learning. Here y₃(t) is the target response for the slowest filter (Fig 1D) and N(t) is a discrete white noise process that helps to increase the dynamical stability [27]. The quality of filter construction and the Lyapunov exponent were estimated in a second simulation using the previously learned weights β_i for filter construction and the feedback signal L(t).

One-population model

In the first part of this study we focused on the one-population rate-neuron model previously published by Yamazaki and Tanaka [15]. While in this previous study the model was used to represent the passage of time, i.e. an internal clock, we now show that it is also possible to use its output to construct exponential filters with various time-constants.

To illustrate the dependence of network stability regime on the amount of feedback we begin by presenting sample impulse responses (Fig 2, second row) for a network (Fig 2, top) with intermediate time constant τ_w = 50ms and with three values of the recurrent inhibition: w = 0.01, lying in the highly stable region, w = 1.4, close to the edge-of-chaos, and w = 3, in the chaotic region. When the weight w was low, (Fig 2A, w = 0.01) the network was highly stable to perturbations and showed no long lasting responses. Close to the edge-of-chaos (Fig 2B, w = 1.4) complex, long lasting responses were present. For larger weights (Fig 2C, w = 3) the network entered a chaotic state in which cells showed random activity without further input modulation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Responses of constructed filters and individual granule cell rates.

A,B,C: Individual responses of randomly selected granule cells with weights w = 0.01 (A), w = 1.4 (B), w = 3 (C). Black bars indicate duration of pulse input. D,E: Responses of filters (τ₁ = 10ms (D1,E1), τ₂ = 100ms (D2,E2) and τ₃ = 500ms (D3,E3)) constructed from network with inhibitory time constant τ_w = 50ms and inhibitory weight w = 1.4 (blue, green and red lines), w = 3 (light blue, light green and light red lines) or w = 0.01 (dotted light blue, light green and light red lines) to colored noise input (D) or pulse input (E). Responses of corresponding ideal filters shown as black lines. To construct the shown filters of 10/100/500ms the percentage of weights equal 0 and mean absolute weights > 0 was 90/53/34% and 20.6/60.4/86.1 for w = 0.01, 76/70/65% and 4.2/13.6/26.5 for w = 3 and 91/86/75% and 5.5/11.7/52.6 for w = 1.4, respectively.

https://doi.org/10.1371/journal.pcbi.1004515.g002

We further illustrate this dependence in the last two rows of Fig 2 which shows filter constructions (see Methods) for three target exponential filters with time constants τ_i of 10 ms (Fig 2D1 and 2E1), 100 ms (Fig 2D2 and 2E2) and 500 ms (Fig 2D3 and 2E3) (chosen to cover the range of performance required for e.g. VOR plant compensation [9]; filter construction of intermediate time constants are not shown, but are generally of similar quality). It is clear that in the highly stable regime only fast and intermediate time constant responses could be reconstructed (dotted light lines). Near the edge-of-chaos acceptable reconstructions were possible at all three time constants (dark lines), and in the chaotic regime reconstruction was always inaccurate and showed oscillatory artifacts (solid light lines).

While Yamazaki and Tanaka [15] argued that this chaotic network state is the preferred network state to implement an internal clock (compare Fig 2C with Fig 1 from [15]) these results show that it is disadvantageous when a filter of a continuous signal has to be implemented (see Discussion).

We have noted above (Methods) that accurate reconstruction of the impulse response of a linear filter does not imply that the output for other inputs is correct; this requires linearity of the reconstructed filter. Linearity of the reconstructed filters is investigated in the second row of Fig 2 by comparing their effects on a band-passed noise signal with that of the exact filter (plotted in black), again for time constants τ_j of 10 ms (Fig 2D1), 100 ms (Fig 2D2) and 500 ms (Fig 2D3), It is clear that the reconstruction in the stable regime or the chaotic regime (light lines) were much less accurate than in the edge-of-chaos-regime (dark lines). Note these plots show the response to a test input (rather than the training input, see Methods).

The regularized fitting method used (LASSO regression, see Methods) tends to use weights that are as small as possible. This property is clear in our example, to construct filters from granule cell signals at the edge-of-chaos only a small subset of granule cell responses were necessary. For the filters with 10, 100 and 500 ms (Fig 2D and 2E; w = 1.4), the percentage of weights being equal to zero was 90%, 86% and 75%, respectively, and the mean of non-zero weights was 5.5 and 11.7 and 52.6, respectively. The high proportion of silent synapses is consistent with experimental findings (see Discussion)

As discussed previously, the value of w corresponding to the edge-of-chaos can be identified using the Lyapunov exponent (see Methods). We illustrate this property by investigating the dependence of filter reconstruction accuracy on the Lyapunov exponent (Fig 3). Results are shown for three networks with different time constants for the recurrent inhibition: τ_w = 10ms (column 1), τ_w = 50ms (column 2) Fig 2B and τ_w = 100ms (column 3) approximately corresponding to the ranges of membrane and synaptic time constants present in the granular layer.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Construction of basis filters using a randomly connected network with feed-forward inhibition using LASSO regression.

A1,B1,C1: Lyapunov exponent of randomly connected networks with increasing weight w. Networks with three different feed-forward inhibition time-constants are considered: τ_w = 10ms (A1), τ_w = 50ms (A2) and τ_w = 100ms (A3). Vertical black lines visualize the “edge-of-chaos” as defined in Methods. A2,B2,C2: Goodness of fit (R²) of three exponential filters (τ₁ = 10ms (blue), τ₂ = 100ms (green) and τ₃ = 500ms (red)) constructed from responses of corresponding networks above.

https://doi.org/10.1371/journal.pcbi.1004515.g003

The top row of Fig 3 shows the Lyapunov exponent of each network plotted against the amount of recurrent inhibition w. In each case there was a point at which the exponent crossed the zero axis, corresponding to the edge-of-chaos value for that network time constant. It can be seen that the amount of recurrent inhibition needed decreased as the time constant increased.

The bottom row shows the effect of w on reconstruction accuracy (measured by R² goodness-of-fit) for exponential filters with the three time constants considered previously: τ_j = 10ms (blue lines), 100ms (green lines) and 500ms (red lines) for each network. Performance strongly depended on the weight of the recurrent inhibition. The goodness-of-fit was best, especially for filters with time constants longer than the internal inhibitory time-constant, for networks close to the edge-of-chaos, just before the transition from stable to chaotic behavior.

Other observations were that while, as expected, the goodness-of-fit for slow filters, e.g. 500ms, increased with the (inhibitory) time constant, the performance for fast filters decreased slightly (Fig 3C2). Furthermore the performance was best if the inhibitory time constant was equal to the time-constant of the filter (Fig 3A2, τ₁ = 10ms blue line; Fig 3C2, τ₂ = 100ms green line).

Fig 4 investigates the robustness of the properties described above to moderate levels of additive noise and to variability in input signal levels and synaptic weights. While white noise with amplitude of a = 0.01 (noise amplitude equal to 10% of the input modulation amplitude) lead to a reduction in goodness-of-fit (Fig 4A1) the principal mechanism of filter construction was not disrupted and the edge-of-chaos was only shifted to larger weights w (Fig 4A2). Increasing the between-neuron variability of the mean input excitation to a high value of e.g. v_I = 2 (i.e. 95% of constant input increased to 0–5 from 0.8–1.2 for default value v_I = 0.1) (Fig 4B1 solid dark lines) had almost no benefit for the goodness-of-fit while shifting the edge-of-chaos to larger weight values.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Filter construction is sensitive to various parameters.

As previously, filters used are τ₁ = 10ms (blue), τ₂ = 100ms (green) and τ₃ = 500ms (red). Default inhibitory time constant: τ_w = 50ms. Goodness of fit (R²) (A1,B1,C1) and Lyapunov exponent (A2,B2,C2) are compared to a control shown as light solid lines in all subplots. A: Effect of additive white noise (n = 0.01) in the network (dark lines). B: Effect of increased input variability (v_in = 2) (dark solid lines) and increased variability of the inhibitory weight (v_w = 2) (dotted lines). C: Effect of increased sparseness by reducing probability of connectivity to a = 0.04 for network size N = 1k (dotted lines) and N = 10k (dark solid lines). For a better comparison x-axis was normalized with the probability of connectivity a.

https://doi.org/10.1371/journal.pcbi.1004515.g004

In contrast, imposing larger variability in the inhibitory weight with v_w = 2 (i.e. 95% of weights between 0 and w+4w) shifted the edge-of-chaos in the opposite direction—towards lower weights (Fig 4B2, dotted lines), and the quality of filter construction was increased (Fig 4B1, dotted lines). This phenomena may be caused by a proportion of input signals or weights being driven to zero due to the positive cut-off which effectively leads to some cells receiving no input and a reduction of connectivity, respectively. To test the effect of reduced connectivity we examined the direct effect of increased sparseness on reservoir performance (Fig 4C).

Two methods were used to increase sparseness: the first was to decrease the convergence of inhibition to 40 cells (Fig 4C1, dotted lines) by decreasing the network connectivity from a = 0.4 to a = 0.04 while keeping the network size at Nz = 1k. The second way was to increase the network size to Nz = 10k while keeping convergence constant at 400 cells (Fig 4C2, solid dark lines) with a = 0.04. While both cases resulted in an improvement of filter quality, a smaller convergence slightly outperformed an increased network size suggesting that a sampling from less cells is more beneficial since it leads to a higher diversity and variability.

An important requirement for filter construction turned out to be push-pull coding, found for example in the vestibulo-cerebellum, where half of the input signals are inverted (see Discussion). When the input did not include inverted signals the responses from individual granule cells showed almost no variety in damped oscillations in response to pulse input (Fig 5A). This consequently lead to an impairment of filter construction performance especially for larger filter time-constants and a shift of the edge-of-chaos to lower weights w (Fig 5B1, dark lines) when compared to the control case (light lines). Although filter construction performance was only slightly reduced when using regression with positive coefficients only (see Methods) (Fig 5C, light lines) when push-pull input was present, without push-pull input filter construction quality was heavily reduced (Fig 5C, dark lines).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Push-pull coding is beneficial for filter construction.

A: Individual responses of randomly chosen granule cells with w = 1.4 but without push-pull input coding. Black bars indicate duration of pulse input. B: As previously, filters used are τ₁ = 10ms (blue), τ₂ = 100ms (green) and τ₃ = 500ms (red). Default network: τ_w = 50ms and push-pull input coding. B1: Default goodness of fit (R²) (light lines) is compared to network without push-pull input coding (dark lines). B2: Goodness of fit for default network (light lines) is compared to network without push-pull input (dark lines) using regression with positive coefficients only. B3: Corresponding Lyapunov exponent for network with (dark orange line) and without push-pull coding (light orange lines).

https://doi.org/10.1371/journal.pcbi.1004515.g005

Two population model

While the previous model was able to show the principles of filter construction from a simplified model of the granular layer with recurrent inhibition, it did not take into account the fact that inhibition in the granular layer is relayed via a second population of cells, i.e. Golgi cells. To investigate the effects of this arrangement we extended the one-population model to a two-population model.

The connectivity of the extended model was based on plausible convergence ratios of c_w = 4 between Golgi and granule cells and c_u = 100 vice versa [50]. Additional parameters were excitatory time constant τ_u and the weight of excitation u (Fig 6, top). Increasing τ_u while keeping the inhibitory time constant at τ_w = 50ms showed that the performance of the two-population model was very similar to the one-population model if the excitation is fast (Fig 6A1). However, increasing the excitatory time constant improved the quality of the constructed slow filter (τ = 500ms) at the expense of the faster filters (τ = 10ms and τ = 100ms) (Fig 6B1 and 6C1). Additionally, this leads to a lowered gradient of the Lyapunov exponent (Fig 6B2 and 6C2). We therefore focus in the following on the best-case scenario of τ_u = 1ms and τ_w = 50ms. As in the one-population model before (Fig 2B), responses of single granule and Golgi cells in networks close to the edge-of-chaos featured complex but stable, long lasting damped oscillations (not shown).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Construction of basis filters using a randomly connected network with feed-forward inhibition via a second population mimicking Golgi cells.

As previously, filters used are τ₁ = 10ms (blue), τ₂ = 100ms (green) and τ₃ = 500ms (red). A,B,C: Goodness of fit (R²) (A1,B1,C1) and Lyapunov exponents (A2,B2,C2) for three networks with τ_w = 50ms, τ_u = 1ms (A), τ_w = 50ms, τ_u = 50ms (B) and τ_w = 50ms, τ_u = 100ms (C). Results for previous one-population network with τ_w = 50ms shown as light lines. D: Effect of increased sparseness in a two-population network with τ_w = 50ms, τ_u = 1ms: 1. Increase of granule cell population size from Nz = 1k (dark solid lines) to N = 10k (dotted lines). 2. Decrease of convergence to c_u = 10 while keeping network size at Nz = 1k (light solid lines). E: Effect of increased weight variability in a two-population network with τ_w = 50ms, τ_u = 1ms. Compared to control network without weight variability (dark solid lines). 1. Increase of variability for weight of inhibition w (v_w = 4) (dotted lines). 2. Increase of variability for weight of excitation u (v_u = 4) (light lines)

https://doi.org/10.1371/journal.pcbi.1004515.g006

In Fig 6D we show that increased sparseness, achieved by reducing the convergence onto Golgi cells from c_u = 100 to c_u = 10 (light lines) increased the quality of constructed filters as in the previous model. However, this time, increasing the granular cell population size to Nz = 10k (dotted lines) has almost no beneficial effect, which can be attributed to the bottleneck effect of the small Golgi cell population of Nq = 100 (compare to Fig 4C, dotted lines). Here, many granule cell responses converge onto a lower dimension of signals, which decreases the fidelity. On the contrary increasing the granule cell as well as the Golgi cell population size to Nq = Nz = 10k increased the filter construction performance similar to before (results not shown).

Equally, further reducing the Golgi cell population to Nq = 10 for the default case (Nz = 1k, c_u = 100) enforced the bottleneck effect and strongly decreased the construction quality of slow filters (results not shown).

The effects of synaptic-weight variability in the two-population model differed for excitatory and inhibitory weights (Fig 6E). Adding a large variability to excitatory weights v_u = 4 increased the goodness-of-fit (light lines) just as seen in the model before. However, adding variability to inhibitory weights v_w = 4 decreased the quality of constructed filters (dotted lines). This can be explained by the low number of connections between Golgi and granule cells of c_w = 4. Using equal convergence of c_w = c_u = 20 gave equal effects in increased filter quality with increased variability for excitatory and inhibitory weights (results not shown).

Putative biological features increase reservoir performance.

Finally, we investigate modifications to the network configurations that significantly influence filter construction (Figs 7 and 8). Default values for synaptic time constants were τ_u = 1ms and τ_w = 50ms and results are compared to a control network without modifications (light lines).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Effects of Golgi cell afferent excitation on filter construction performance.

As previously, target filters used are τ₁ = 10ms (blue), τ₂ = 100ms (green) and τ₃ = 500ms (red). Default synaptic time constants: τ_w = 50ms, τ_u = 1ms. A,B: Goodness of fit (R²) (A1,B1) and Lyapunov exponents (A2,B2) for default network (light lines) and network with external excitation of Golgi cells (dark lines) with either default excitation of u = 0.1 (A) or increased excitation u = 50 (B).

https://doi.org/10.1371/journal.pcbi.1004515.g007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Effect of mGluR2 receptor activated GIRK channel inhibition of Golgi cells on filter construction performance.

For clarity only most affected target filter τ₃ = 500ms (red) is shown. Default synaptic time constants: τ_w = 50ms, τ_u = 1ms. A,B: Goodness of fit (R²) (A1) and Lyapunov exponents (A2) for default network (light red lines) and network with mGluR2 receptor activated GIRK channel inhibition of Golgi cells (m = 0.003) either without (v_m = 0) (dotted red lines) or small (v_m = 0.1) (dashed red lines) weight variability. Additionally results with mGluR2 dynamics in only 50% of the Golgi cells (solid red lines) (Pr(m = 0) = 0.5, v_m = 0.1). B shows results for networks without push-pull input coding.

https://doi.org/10.1371/journal.pcbi.1004515.g008

The first influential configuration is excitatory input to Golgi cells. While this property is often omitted for simplicity in granular-layer networks [20] it had a substantial impact on filter construction (Fig 7). Simply adding excitation to the Golgi cell population (g = 1, see Methods, equation (3) and keeping all other parameters identical prevented the system from entering into the edge-of-chaos regime (Fig 7A2) by inhibiting any activity due to the strong direct activation of Golgi cells. The only possible filter construction, which was however very poor, could be achieved during low inhibition with w < 0.015 (Fig 7A1, dark lines). There are several ways to increase filter construction performance during Golgi cells afferent excitation. The most obvious case being the increase of afferent Granule cell excitation, which counteracts Golgi cell inhibition and prevents Granule cells from being silenced (not shown). Another interesting possibility however is to increase recurrent Golgi cell excitation which shifts the edge-of-chaos to lower values of w. Increasing this weight of excitation to u = 50 resulted in an effective excitatory weight from granule cell to Golgi cell that was equal to the afferent Golgi cell excitation (, see Methods) and lead to an interesting effect on filter construction performance (Fig 7B, dark lines). While the same performance as without afferent excitation (light lines) was achieved, the decline in goodness-of-fit was postponed until larger values of w. The afferent excitation that effectively counteracted the generation of chaotic behavior kept the state of the network closer to the edge-of-chaos for a wider range of w. This can be seen in the disrupted rise in the Lyapunov exponent (Fig 7B2, dark orange lines).

A second property that also had a large influence on the filter construction was the inhibition of Golgi cells by granule cell input via mGluR2 receptor activated GIRK channels [31,32]. This property was modeled by a slow inhibitory process with τ_m = 50ms in addition to the short excitatory process (τ_u = 1ms) at the connection from granule to Golgi cell (Fig 8, top). For clarity only results for the most affected slow filter (τ₃ = 500ms) will be shown. While initially this concurrent inhibition resulted in a strong reduction of filter construction performance (Fig 8A, dotted red lines) adding weight variability (v_m = 0.1) lead to a strong improvement of the overall goodness-of-fit (dashed red lines). Since some Golgi cells have been shown not to express mGluR2 [31,51] we further tested the effect of only 50% of the Golgi cells receiving mGluR2 inhibition (Fig 7A, solid red lines). While this did not massively improve filter quality compared to the baseline condition (compare to light lines) it delayed the development of chaotic behavior. The greatest beneficial effect of mGluR2 however was observed in the absence of push-pull coding (Fig 8B). Here, but only for the case of mGluR2 inhibition to half of the Golgi population, push-pull coding input is almost made redundant (solid red lines). However even with these improvements, as before, filter construction was strongly reduced with positive-LASSO regression (not shown). While the addition of variable mGluR2 inhibition did not change the qualitative behavior of single cell responses it increased the variability of damped oscillations for granule and Golgi cell responses (not shown).

Effect of output feedback.

For some reservoir computing applications, e.g. pattern generation, the learned output signal is fed back into the reservoir leading to a recurrence between the reservoir and the trained readout [27,48,49]. In the cerebellum a corresponding connection that would allow for a similar recurrence is the nucleocortical pathway that consists of inhibitory and excitatory connections from cerebellar nucleus to the granular layer [52–54]. This connection thus supports a putative recurrent loop that sends the trained signals from Purkinje cells via cerebellar nucleus neurons back to the granular layer reservoir.

While information on the exact distribution of source and target neuron properties for the nucleocortical pathway is sparse we tested the effect of output-feedback onto granule and Golgi cells using 50% excitatory and inhibitory connections (Fig 9). When using the output of the slowest filter (τ = 500ms) as feedback (Fig 9A, dark lines) the maximum performance increased for the slowest filter but slightly decreased for the two other filters (compared to case without feedback, light lines). This effect can be explained by the increase of delayed signals that are now being fed back through the slow filter output. Furthermore the maximum of filter construction quality is now well shifted away from the edge-of-chaos into more stable regimes. The preference for more stable networks with output-feedback becomes even more evident when push-pull input is removed (Fig 9B, dark lines). Here, even in the absence of any connections between Golgi and granule cells (w = 0) the slow filter (dark red line) can be well constructed due to the memory created by the recurrent loop between output and input, similar to a neural integrator [55]. The goodness of fit for the other two filters is however closer to the previous results (Fig 9B1, compare dark blue and green to light lines). In this regime (w<0.01) the reservoir with output-feedback is very stable (Fig 9B2) resulting in Lyapunov exponents of negative infinity (i.e. the Euclidian distance between perturbed and unperturbed simulation decays to 0). Interestingly however the filter performance reaches goodness-of-fit values above 80% for all three filters (Fig 9B1, dark lines) only when the Lyapunov exponents have values closer to 0 and thus the reservoir is not completely stable (Fig 9B2, dark orange dots).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Effect of output-feedback on filter construction performance.

As previously, target filters used are τ₁ = 10ms (blue), τ₂ = 100ms (green) and τ₃ = 500ms (red). Default synaptic time constants: τ_w = 50ms, τ_u = 1ms. A,B: Goodness of fit (R²) (A1,B1) and Lyapunov exponents (A2,B2) for default network (light lines) and network having output-feedback of the slowest filter response (τ₃ = 500ms) to granule and Golgi cells (dark lines) with either push-pull input (A) or without (B).

https://doi.org/10.1371/journal.pcbi.1004515.g009

Method of simulation and analysis

Difference between one- and two-population models

The one-population model demonstrated the properties of a homogeneous reservoir dominated by recurrent inhibition. However in the cerebellum recurrent inhibition is implemented as a relay via a second population of Golgi cells. To test the effect of this layered recurrence we considered a two-population model and found that reservoir behavior was generally preserved. We did observe three differences between the two types of networks: 1. A slow synaptic time constant for Golgi cell excitation decreases the quality of fast filter construction (Fig 6C). 2. The lower number of Golgi cells can create a bottleneck in filter fidelity when few Golgi cells sample input from a large number of granule cells (Fig 4C). 3. Direct afferent Golgi cell excitation can decrease chaotic behavior and is beneficial for filter construction by prolonging the edge-of-chaos regime. These issues are further discussed below.

Theoretical findings

Early models of the cerebellar microcircuit focused on its ability to adaptively process static patterns. Subsequently Fujita [3] described a cerebellar model that could adaptively process time-varying analogue signals, based on the adaptive linear filter used in signal processing and control engineering [61]. In Fujita’s model the granular layer recurrent neuronal network (RNN) generated a set of parallel-fiber outputs for a given mossy-fiber input which by linear summation at the Purkinje cell allowed the microcircuit to adaptively transform time-varying inputs into the specific time-varying outputs required for any given signal-processing task.

Fujita’s adaptive-filter model required the granular layer to generate computationally adequate sets of parallel-fiber outputs in a biologically realistic way [4]. Subsequent extensions of his work initially focused on computational adequacy, suggesting suitable basis functions for transforming mossy-fiber inputs that included tapped delay lines, spectral timing, and sinusoids (references in [8–10]). However, it was not usually made clear how these functions could be generated by a plausible neuronal network, raising concerns that the approach was unrealistic [11] and excessively ‘top-down’ [14]. The preferred alternative was ‘bottom-up’ modeling in which temporal-processing properties emerged from biologically detailed models of the granular-layer RNN rather than being imposed by a priori theoretical considerations [13,14,62]. These models successfully learned an eyeblink-conditioning task by generating a suitably delayed output timed to coincide with the arrival of the unconditioned stimulus.

Yamazaki and Tanaka [15] investigated the generic time-coding properties of simplified RNNs, with (usually) 1000 rate-coding neurons receiving excitatory afferent inputs and recurrent inhibitory inputs from other neurons (cf. first model here). These RNNs could generate a sequence of activity patterns that never recurred, a sequence that could be triggered reliably by a strong transient input signal. Such networks could therefore be used to encode the passage of time for any task that required it. Related results were found for RNNs of integrate-and-fire model neurons [16], and for a more realistic two-layer RNN embedded in a spiking model of the entire cerebellar microcircuit [63].

Experimental findings

The main theoretical conclusion is that with appropriate network parameters the granular-layer RNN can in principle generate the outputs needed for an adaptive filter. It also indicates what those outputs might look like. We now consider experimental evidence bearing on these two points. For convenience, evidence from detailed models of the granular-layer that are concerned primarily with its electrophysiology (as opposed to the functional models discussed above) is also included in this experimental section.

Implications of findings

In all networks and configurations considered the single cell activity of granule and Golgi cells at the edge-of-chaos showed complex but stable, long lasting damped oscillations that were the effect of inhibition and dis-inhibition (only shown for the one-population model, Fig 2B). The similarity between Golgi and granule cell responses is easily graspable when one considers that for the default two-population network Golgi cells merely relay signals from granule cells und thus have to show similar activity. On the other hand for Golgi cells with added mGluR2 dynamics they themselves become prone to inhibition and dis-inhibition. This study thus suggests that one indicator for the presence of reservoir computation in a certain area of the cerebellum would be the similarity in heterogeneity and timing of the responses for granule and Golgi cells.

This comparison however must not be made based on the spike activity but on the modulated signals. One explanation for the often-reported bursting responses in granule cells compared to Golgi cells could be the lower baseline/background activity of the former cells [77]. While the signal is carried and hidden by the higher spike rate in Golgi cells, granule cells would ultimately only spike during phases of strong dis-inhibition, which would effectively resemble bursting behavior. This would be even further increased if the operation point of the network is not close to the edge-of-chaos but in the chaotic regime. A further evaluation of these properties will however require the inclusion of spiking neurons in future studies.

While the present study only focuses on the interaction between granule and Golgi cells the inclusion of other identified neurons might improve filter construction properties. Glycinergic Lugaro cells which have been found to increase the long-lasting depression of Golgi cells [32] and various other non-traditional interneurons like globular [84] or perivascular neurons [85] might further improve the reservoir performance by contributing to the inhibitory circuit. Furthermore, in some areas of cerebellar cortex, particularly in the vestibulo-cerebellum (e.g. [86]), a substantial proportion of mossy-fiber input is processed and relayed by unipolar brush cells (UBC) which are thought to prolong and diversify the input signals [87]. In addition a recently discovered timing mechanism intrinsic to Purkinje cells [88] would potentially further increase the heterogeneity of the granular layer reservoir signals.

Although the edge-of-chaos criterion is not a universal predictor of maximal computational performance [22] we find that it applies for most of the configurations considered here. With this requirement however the question arises how the granular layer network can be adjusted to operate in this computationally powerful regime. While the easiest way to achieve this is to change properties inside the loop like weights at Golgi cell—granule cell synapses or the convergence ratio of Golgi cell excitation we show that also external mechanisms like noise, input variability and especially mossy fiber—Golgi cell exaction can shift the network state. The question however remains how and if the cerebellar granule layer network can be automatically tuned to operate close to the edge-of-chaos. Possible mechanism that could help achieve this are synaptic long or short-term plasticity.