The authors have declared that no competing interests exist.
Performed the experiments: BML ERO NAS JR BE SVD SAS TJB KM AZ NGH LEM. Analyzed the data: IHS. Wrote the paper: IHS KPK.
How interactions between neurons relate to tuned neural responses is a longstanding question in systems neuroscience. Here we use statistical modeling and simultaneous multi-electrode recordings to explore the relationship between these interactions and tuning curves in six different brain areas. We find that, in most cases, functional interactions between neurons provide an explanation of spiking that complements and, in some cases, surpasses the influence of canonical tuning curves. Modeling functional interactions improves both encoding and decoding accuracy by accounting for noise correlations and features of the external world that tuning curves fail to capture. In cortex, modeling coupling alone allows spikes to be predicted more accurately than tuning curve models based on external variables. These results suggest that statistical models of functional interactions between even relatively small numbers of neurons may provide a useful framework for examining neural coding.
The number of simultaneous neurons that electrophysiologists can record is growing rapidly, and a central goal of computational neuroscience is to develop statistical methods that can make sense of this growing data. Here we present a unified statistical analysis of 10 different datasets recorded from several different species and brain areas. We show how functional interactions between neurons may be used to predict spiking in each of these different areas, and find that, in many cases, modeling interactions between a small number of neurons yields better spike predictions than modeling each neuron's relationship to the outside world using tuning curves. Although these statistical results cannot be linked to specific network architectures, since the measured interactions between neurons are purely functional rather than anatomical, they suggest that modeling interactions between neurons will be a useful approach to understanding neural coding as electrophysiologists record from increasing numbers of neurons.
One of the central tenets of systems neuroscience is that the functional properties of neurons, such as receptive fields and tuning curves, arise from the inputs that each neuron receives from pre-synaptic neurons. Over the past few decades, a number of experimental techniques have been developed to study exactly how interactions between neurons determine receptive field structure, including
To understand how interactions between neurons drive neural activity, recent model-based statistical methods attempt to predict the activity of each neuron based on the activity of other simultaneously observed neurons in addition to any external variables, such as the orientation of a visual stimulus or the direction of hand movement
Statistical models of interactions between neurons have been used to describe many different aspects of multi-electrode data in retina
To make our analysis as broad as possible, we collected ten multi-electrode spike train datasets with at least 30 simultaneously recorded neurons. Datasets were obtained from six different brain areas across four different species performing a variety of tasks. By modeling typical tuning curves for neurons in each area as well as interactions between neurons we determine how much these two factors contribute to spike prediction. We find that including information about the activity of other observed neurons improves both spike prediction and decoding accuracy substantially. By capturing noise correlations and unmodeled features of the external world models of interactions between even a relatively small number of recorded neurons can complement and, in some cases, surpass, models of tuning curves alone.
Although neurons are often characterized by how their firing rate relates to external stimuli or movement variables, the functional properties of most neurons are byproducts of the input they receive from other neurons (
A) Schematic illustrating how tuning properties may be related to functional connectivity. The tuning properties of observed neurons (black) are a direct result of input they receive from peripheral neurons (blue and red). Even when observed neurons do not have a direct relationship to external variables, each neuron may have apparent tuning caused by the input it receives from peripheral neurons and shaped by interactions with other observed neurons. B) A linear-nonlinear-Poisson model that includes both tuning properties as well as interactions or coupling between neurons. The firing rate of each neuron is modeled as a weighted sum of external variables as well as the activities of other observed neurons passed through an exponential nonlinearity. C) A toy example where tuning properties are explained away using coupling. In two simulated networks only neuron 1 is directly related to the external world (with Gaussian tuning). However, neurons 2 and 3 have tuning due to the input they receive from neuron 1 (middle). If these interactions are estimated, then coupling can fully explain the observed tuning (bottom).
Tuning curves can be “explained away” if the other observed neurons provide a better explanation for spiking than the external variables (
We fit spike count data from multi-electrode recordings in 6 different brain areas using
A) Hand position during center-out reaching in a monkey M1 dataset (top) and tuning curves to hand direction for typical M1 neurons under the tuning curve-only model (black) and the full model including coupling between neurons (red). All tuning curve plots are rescaled between zero and the maximum tuning curve value for visualization purposes. B) Randomly oriented gratings for the monkey V1 dataset (top) and typical V1 tuning curves to grating direction. C) Random frequency tones for the ferret A1 dataset (top) and typical A1 tuning curves to frequency fit using radial basis functions in the log-frequency domain. D) Hand position during random-target pursuit in the monkey S1 dataset (top) and typical 2-dimensional tuning curves to hand position and velocity under the tuning curve model and full model. E) Head position during free-foraging in the rat hippocampus dataset (top) and place fields for typical hippocampal neurons under each model. Color in E and D denotes firing rate. The color scale is the same for TC and full, but differs across neurons. Note that, for most neurons, the modulation decreases when coupling to other neurons is included (full model), while preferred direction, velocity, and place remain similar.
In the full model, most, but not all, neurons showed decreased modulation to external variables (
A) Tuning curve modulation (max-min) under the tuning curve and full model. Note that the modulation attributed to the tuning curve decreases under the full model on average for all datasets. B) Tuning curve preference under the tuning curve and full model. Preferred direction is shown for M1/V1/PMd/S1 (deg), preferred frequency for A1 (Hz), and preferred place along the x-axis for HC (cm). Gray lines denote equality. C) Correlation coefficient as a function of tuning curve overlap for all pairs of neurons. Note that, in general, correlations increase with increasing tuning curve overlap. D) Coupling strength as a function of tuning curve overlap for all pairs of neurons. Note that, in general, coupling strength does not depend on tuning curve overlap. Red lines denote linear fit. Plots in B–D are organized by dataset as in A.
To quantify how coupling in the full model relates to tuning properties we measured the overlap between tuning curves for each pair of neurons in each dataset using the angle between the tuning curve parameter vectors (cosine similarity). An overlap of zero corresponds to orthogonal tuning (i.e. cosine tuned neurons with preferred directions of 0 and 90 deg), an overlap of one corresponds to identical tuning, and an overlap of negative one corresponds to exactly opposite tuning (i.e. cosine tuned neurons with preferred directions of 0 and 180 degrees). We find that tuning curve overlap is clearly related to the bulk spike-count correlation across all stimulus/movement conditions (
The structure of the coupling terms, particularly the number of connections that each neuron makes with the other observed neurons (the “degree”) provides some insight into how tuning curves are explained away. In contrast to theories of scale-free neural connectivity
A) Degree distributions, both in-degree (blue) and out-degree (red), for the coupling matrices estimated from each dataset under the full model. B) The fraction of non-zero inputs for each neuron as a function of the fraction of variance explained by tuning. Red lines denotes the linear trend. Plots are organized by dataset as in A.
How these models behave as the number of simultaneously recorded neurons grows is an important consideration for future modeling. Here we fit the coupling alone and the full model, varying the number of neurons used to predict spikes. Under the full model, we find that, in good approximation, the fraction of variance explained by tuning decreases logarithmically as the number of observed neurons increases (
For the full model the fraction of variance explained by the tuning component of the model decays approximately logarithmically (note the log-scale) to 30–90% of the total variance when all neurons are included in the model.
A second metric for studying how these methods scale with the number of observed neurons is spike prediction accuracy (see
A) Mean spike prediction accuracy under the tuning curve (black), coupling (blue), and full (red) models. Where spike prediction accuracy denotes the cross-validated log likelihood ratio relative to a homogeneous Poisson process reported in bits/s. Error bars denote SEM across neurons (tuning curve models) or networks (coupling and full models). In all cortical areas the coupling model out-performs tuning curve models once coupling between 10–30 neurons is included. Note that for spontaneous activity in V1, coupling improves spike prediction accuracy even though stimuli were not displayed and tuning curves cannot be fit. B) Mean decoding accuracy under the tuning curve (black) and full (red) models, as well as accuracy for simulated, conditionally independent neurons with data-matched tuning curves (green). Note that the full model slightly outperforms the tuning curve model alone, but dependencies between neurons degrade decoding performance relative to the simulated conditionally independent neurons. Error bars denote standard deviation across neurons (tuning curve models) or networks (full model).
Hippocampal neurons appear to differ from cortical recordings in that spike prediction accuracy increases approximately linearly. Moreover, modeling coupling alone does not provide more accurate spike prediction than the basic place field model. This may be due to the low correlations between HC neurons. Electrode spacing may also be a factor, since, unlike the 400 µm electrode spacing used in almost all of the intra-cortical arrays, HC recordings had 20 µm vertical electrode spacing. However, the coupling model for spontaneous activity in V1 shows the same hyperbolic behavior despite data being recorded using a polytrode with 50 µm electrode spacing. Scaling of spike prediction accuracy in hippocampus appears to be qualitatively different from that in cortex.
In addition to examining how encoding accuracy scales with the number of recorded neurons, we also examined decoding accuracy for several datasets (
It is important to consider what factors may be driving these scaling phenomena. Although the coupling terms are regularized during estimation and the spike prediction accuracy is cross-validated, it may be the case that tuning curves are explained away as a result of over-fitting or, alternatively, as a simple side effect of stimulus correlations. To test for this possibility we simulated spike counts from the tuning curve model, where the neurons are conditionally independent given the external variables. That is, although there may be stimulus correlations, spiking can be completely predicted by external variables. Here we find that no matter how many neurons are included in the full model, tuning explains between 90–100% of the variance (
A) Mean fraction of variance explained by tuning as a function of network size for simulated, independent neurons whose tuning curves were matched to the recorded data. The fraction of variance explained by tuning remains close to 1, indicating that there is little to no over-fitting. B) Mean fraction of variance explained by tuning as a function of network size for shuffled data. These results provide an additional control for over-fitting, while retaining stimulus correlations. C) Mean spike prediction accuracy under the tuning curve (black), coupling (blue), and full (red) models on shuffled data. Error bars denote SEM across neurons (tuning curve models) or networks (coupling and full models).
Additionally, we can quantify how much stimulus correlation contributes to explaining away by shuffling the data to remove noise correlations. Where possible (M1, PMd, and V1) we shuffle the spike counts within each trial condition (target or grating direction) independently for each neuron. This manipulation retains stimulus correlations while destroying any structure unrelated to the stimulus. Here we find that, in the full model, tuning explains between 85–95% of the variance (
Finally, to examine what drives the shape of these spike prediction accuracy curves we simulated a linear-nonlinear-Poisson neuron receiving sparse, correlated input. As input correlation increases spike prediction accuracy converges more quickly to its maximum (
A) Spike prediction accuracy for simulated neurons receiving correlated input, with different levels of input correlation. For these simulations all neurons contribute to the output (100% non-zero entries). B) Spike prediction accuracy for simulated networks with different input sparseness. For these simulations the input correlation is fixed at 0.25. Percentages denote the fraction of non-zero input.
Linking the strength of common input and sparseness to the spike prediction accuracy curves observed in real data is difficult. Both a weakly correlated, highly connected network and a highly correlated, highly sparse network will have near-linear growth. However, here we find that neurons in cortex (particularly V1 and A1) tend to be more strongly correlated than neurons in hippocampus (
Excepting peripheral neurons such as photoreceptors, the relationship between a neuron's spiking and the external world is a result of the input that each neuron receives from other neurons. Many studies have examined how pre-synaptic input determines receptive field structure and tuning properties both experimentally
The extent to which the activity of simultaneously observed neurons or tuning properties explain spiking likely depends on a number of factors including the timescales on which we model spiking, the stimulus or task parameters, and the external variables being used to describe tuning. The coarse, instantaneous coupling models used here cannot distinguish between the many possible hidden causes of correlated neural activity. Since the models used here reflect pair-wise dependencies on a long timescale of 100 s of milliseconds, it is likely that unobserved behavioral variables and internal processes make strong contributions to the coupling terms. Modeling these effects explicitly may yield a more nuanced view of the relationship between tuning curves and interactions between neurons
The datasets used here yield surprisingly similar results considering that they were recorded from different brain areas and species with different electrode configurations. However, in addition to these anatomical differences, it is important to note that the datasets were recorded under a variety of experimental circumstances, which may help to explain some of the remaining differences in the results obtained from each dataset. For data from motor cortices and hippocampus, for instance, the external variables are not controlled in the same way that sensory experiments are. Movement variables such as velocity or body orientation differ even when the monkey's reaches to the same target or when the rat is at the same maze location. These external differences may lead to higher apparent trial-to-trial variability. Additionally, data from V1 was recorded while the animals were under anesthesia, which may lead to higher correlations between neurons
The tuning models used here, despite their wide-spread use, are relatively simplistic. Tuning functions that take into account more external variables are likely to give more accurate spike prediction, and including these variables may change the degree to which tuning properties are explained away as interactions between neurons are added to the model. At the same time, exploring the space of external variables and determining what causes a neuron to fire can be difficult
Neurons receive pre-synaptic input from tens of thousands of other neurons, and each of these inputs, presumably, plays a role in determining the tuning properties of a post-synaptic neuron. How is it possible then that models of interactions between <100 neurons are able to explain spiking more directly than traditional tuning curve models without any guarantee that the neurons are even anatomically connected?
Ultimately, explaining away can only occur when neural activity is not independent. Many studies have examined correlated neural activity
It is important to note, however, that the statistical approaches used here are unlikely to capture anatomical information about the underlying circuitry. These methods still only provide a sketch of the underlying circuit that best explains the observed spiking. The hyperbolic scaling of spike prediction accuracy observed here, for instance, may be a general property of correlated prediction problems
For many years, studies of the relationship between neural interactions and tuning properties have been based on detailed electrophysiology
Understanding how interactions between neurons give rise to tuning properties, will ultimately mean understanding the relative contributions of feed-forward, local, and top-down pre-synaptic inputs, as well as how different subtypes of neurons and neurons with different types of tuning interact. One area where statistical approaches have revealed this type of detailed architecture is in the retina. By recording from dense populations of retinal ganglion cells (RGCs), recent work has shown that RGC receptive fields arise directly and clearly from input received from rods and cones
In most areas of the brain, beyond the retina, recording from a complete neural circuit is experimentally infeasible and the complete network of neurons is immensely under-sampled. In these cases, it is difficult to determine whether potential interactions between neurons are direct (mono-synaptic) or indirect (poly-synaptic), and the estimated interactions are likely to be strongly influenced by unobserved common input
We analyzed 10 multi-electrode spike datasets recorded from 6 different brain areas and 4 different species. Recordings from primary (M1) and dorsal pre-motor cortex (PMd) were made while a macaque monkey performed a center-out reaching task. Recordings from primary sensory cortex (S1) were made while a macaque monkey performed a random-target pursuit task. Recordings from primary auditory cortex (A1) were made while a ferret was exposed to random frequency tone stimuli. Data from primary visual cortex (V1) consisted of recordings of 1) evoked activity while an anesthetized monkey viewed randomly oriented moving gratings and 2) spontaneous activity from an anesthetized, paralyzed cat. Finally, recordings from dorsal hippocampus (HC) were made while a Long-Evans rat was freely foraging for food on a square platform.
All animal use procedures were approved by the institutional animal care and use committees at Northwestern University (M1 & S1), University of Chicago (M1 & PMd), Albert Einstein College of Medicine (V1), University of Maryland College Park (A1), University of British Columbia (V1 spont), or Rutgers University (HC) , and conform to the principles outlined in the Guide for the Care and Use of Laboratory Animals (National Institutes of Health publication no. 86-23, revised 1985). Data presented here were previously recorded for use with multiple analyses. Procedures were designed to minimize animal suffering and reduce the number used.
The aim of our analysis was to examine the relationship between typical tuning curves and receptive fields in each of these brain areas and coupling between neurons. To this end we extracted spike count data from the spike-sorted multi-electrode recordings and focused either on evoked responses for the stimulus and directed movement tasks or binned responses for the foraging and spontaneous tasks. Each dataset contained at least 31 and as many as 107 simultaneously recorded, putative single neurons after spike sorting (
Area | Neurons | #Trials/Bins | Bin Size (ms) | Stimulus/Task |
M1 | 87 | 290 | 200 | Center-out Reaching |
M1 | 101 | 193 | 200 | Center-out Reaching (Wrist) |
M1 | 78 | 315 | 200 | Center-out Reaching |
PMd | 65 | 315 | 200 | Center-out Reaching |
S1 | 61 | 3539 | 200 | Random Target Pursuit |
V1 | 107 | 3200 | 400 | Drifting Sine-Wave Gratings |
V1 | 50 | 3600 | 100 | N/A (Spontaneous) |
A1 | 31 | 165 | 100 | Pure Tones |
HC | 76 | 5000 | 250 | Free Foraging |
HC | 87 | 5000 | 250 | Free Foraging |
Datasets were obtained from the motor cortices of two monkeys (designated K and R). Monkey K was implanted with a 100-electrode Utah array (Blackrock Microsystems, 400 µm spacing, 1.5 mm length) in the arm area of primary motor cortex. Data were recorded during two different tasks: a standard eight-target center-out reaching task, and an isometric eight-target center-out wrist force task. In the first task the monkey was seated in a primate chair, with movement constrained to a horizontal plane, with the arm roughly in a sagittal plane. The monkey grasped the handle of a two link planar manipulandum that moved within a 20 cm by 20 cm workspace. In the second task the monkey used isometric forces about the wrist (with the forearm in a posture midway between pronation and supination) to produce center-out forces. In both tasks feedback about movement or force was given on a computer screen in front of the monkey, displayed as a circular cursor, 1–2 cm diameter. The recordings were made approximately 4 months apart with 87 well isolated single-units recorded during the center-out reaching task and 101 units recorded during the center-out wrist-force task after offline spike-sorting. Trial-by-trial spike counts were collected during the period 100–300 ms following movement onset on 209 and 193 trials, respectively.
For Monkey R, two 100-electrode Utah arrays (Blackrock Microsystems) were implanted in dorsal pre-motor and primary motor cortices. Data were recorded while the monkey performed a randomized, eight-target, center-out reaching task using a KINARM device (BKIN Technologies, Kingston, ON, Canada) in which the monkey's arm rested on cushioned troughs secured to links of a two-joint robotic arm
In each of the M1 and PMd datasets neuronal signals were classified as single- or multi-unit based on action potential shape and minimum inter-spike intervals greater than 1.6 ms. Spike sorting was performed by manual cluster cutting using an offline sorter (Plexon, Inc). All trials for the center-out tasks began with the acquisition of a square center target that the monkey was required to hold for 0.3–0.5 s. Subjects had 1.25 s to acquire the peripheral target and were required to hold this outer target for at least 0.2–0.5 s. Each success was rewarded with juice or water.
Datasets are available for download at
One dataset was recorded from a macaque monkey implanted with a 100-electrode Utah array (Blackrock Microsystems) in primary somatosensory cortex (areas 1 and 2) while the monkey performed a random-target pursuit task. The monkey was seated in a primate chair, with movement constrained to a horizontal plane. The monkey grasped the handle of a two link planar manipulandum that moved within a 20 cm by 20 cm workspace. After each target hit a new target would appear in a random location. A run of 3–4 successes was rewarded with juice or water.
Neuronal signals were classified as single- or multi-unit and spike sorted as above to provide 61 well-isolated units. Trial-by-trial spike counts were collected during the period 100–300 ms following movement onset from 3539 trials for subsequent analyses. See
Data from the primary auditory cortex A1 of an awake, passively listening ferret were recorded during presentations of pure-tones of various frequencies spanning 6.4 octaves (0.18–15.56 kHz) equally spaced in log-frequency presented in random order over 165 trials. Recordings were made with a 32 electrode array (500 µm spacing, 2.5 MΩ, Microprobes Inc.) in Layer IV (depth ∼700 µm). Neuronal signals were classified as single- or multi-unit based on action potential shape and inter-spike intervals greater than 1.6 ms. Spike sorting was performed by manual cluster cutting, providing 31 well-isolated units. Trial-by-trial spike counts were collected during the 100 ms stimulus period. See
Two datasets were obtained from primary visual cortex. The first dataset was recorded while an anesthetized monkey viewed one of eight randomly oriented, drifting sine-wave gratings. Stimuli had a spatial frequency of 1 cyc/deg, drift rate of 6.25 cyc/s, size of 2.9 deg and were presented for 400 ms with a 800 ms delay between stimuli. Recordings were made using a 100-electrode Utah array, 400 µm electrode spacing (Cyberkinetics Neurotechnology Systems). After automatic spike sorting and manual cluster adjustment, 107 single units and small multi-unit clusters with firing rates >1 Hz were used. Trial-by-trial spike counts were collected for the entire 400 ms stimulus period for 3200 trials (400 repetitions for each orientation) for subsequent analyses. See
The second dataset from primary visual cortex was downloaded from the Collaborative Research in Computational Neuroscience (CRCNS,
Two hippocampal datasets were obtained from the Collaborative Research in Computational Neuroscience (CRCNS,
Spike count data were fit using either external variables, the activity of the other recorded neurons, or both
Using this same framework, tuning curves alone were modeled by
Note that, for the coupling model, the spike count for the neuron
For each of these three models – the full model, tuning curve model, and coupling model – we estimated the parameters
Where regularization is used we optimized the regularization hyperparameter via the cross-validated (10-fold) log-likelihood, and in all cases we evaluated the “spike prediction accuracy” of the models using the cross-validated log likelihood ratio relative to a homogeneous Poisson process. For a firing rate
In this case, a spike prediction accuracy of zero corresponds to a model that does no better than predicting the mean spike count. Values were calculated in base-2 and rescaled by time to give units of bits/s
An important component of these models is the choice of basis functions for the external variables. Here we have attempted to choose common tuning models, appropriate for each dataset. For M1 and PMd neurons, for instance,
While activity in M1 has also been shown to covary with speed
Place fields of the neurons in hippocampus have been well described
Specifically we use K = 25 isotropic Gaussian radial basis functions equally spaced on a 5×5 grid with means
Finally, for neurons in A1
In most cases (TC dimensionality <4), regularization was only applied to the coefficients modeling coupling between neurons. To avoid convergence problems
It is important to note that the models used here differ from previous approaches in that they are time-instantaneous — we model coupling between neurons at the same time. This does not pose any difficulties during fitting, since we are modeling only the conditional distributions for each neuron
To quantify the changes in tuning under the full model we evaluate the tuning modulation, tuning preference, and tuning curve overlap between pairs of neurons. Tuning modulation is simply the peak-to-peak difference in firing rate for the tuning curve component of the model, reported in Hz. Tuning preference is defined differently for each dataset: for M1, V1, PMd, and S1 we use the preferred direction, for A1 we use the preferred frequency, and for HC we use the preferred place along the x-axis. Finally, to measure similarity between the tuning curves for pairs of neurons we evaluate the tuning curve overlap between neurons
To quantify network properties we also report the spike count correlation (Pearson's correlation). For two neurons with trial-by-trial spike count observations
To quantify the relative contributions of the tuning curve and coupling components in the full model we summarize the fit using the fraction of variance explained by tuning. For each neuron we calculate
A value of 1 suggests that the coupling terms provide no additional information, while a value of 0 suggests that any tuning information is explained completely by coupling to other observed neurons. It is important to note that there is considerable heterogeneity in how well tuned neurons are to the external variables. Here we analyze all recorded neurons, even those that might be considered un-tuned.
In contrast to the encoding models above, which aim to predict spikes given a stimulus, we can also examine how coupling affects decoding, which aims to predict a stimulus
For the tuning curve models, we assume that the neurons are conditionally independent given the stimulus,
However, for the full model, since we assume that coupling is instantaneous, we cannot make this assumption. In this case we use a variation of Gibb's sampling
For each model we initialize the sampler with
In practice, it is non-trivial to estimate the probability
Although we cannot write down the full joint probability analytically, we can approximate each of the marginal distributions in the chain rule using the set of Gibbs samples
To examine how the scaling of spike prediction accuracy relates to the underlying structure of the inputs we simulated spikes from a linear-nonlinear-Poisson neuron receiving correlated input
In general, producing correlated Poisson random variables with specific marginal distributions and covariance structure is difficult. Here we use a simplified family of covariance matrices where all neurons have the same correlation and simulate spike counts following
Many thanks to Adam Kohn, Gyorgy Buzsaki, and Joshua Vogelstein for helpful comments.