Basic theory of neuronal dynamics

Models in Mathematical biology

The theories of the brain’s working are understood at different levels of rigour for each level of abstraction. Such as, at the cognitive level it is understood that different areas of Orbito-frontal cortex regulate different emotions, at circuit level the pyramidal neurons are stacked in a nearly layered fashion in the visual cortex, at the neuronal level it is known that the ion channels create a threshold system for spike generation through fast-slow dynamics. Any theories on the brain’s structure after basic cognitive experiments are referred to as the Ordinary Language Model (OLM).When these theories are physiologically tested, and near-complete information is provided in its favor, we graduate these theories to Comprehensive Models.

Now comprehensive models are rarely perfect. The observed information can have noise, or inconsistencies, or extrapolating all that evidence into a simple mathematical model is simply not possible. In this case, we make an approximation from the provided evidence and create emperical models. For example, the Hodgkin Huxley neuronal model is a very accurate representation of the neurons, but the number of variables of the system increases with the number of ion channels taken into account. To approximate this, Izhikevich developed a quadratic system that is able to approximate all major spike patterns given a fixed set of parameters. Lastly, there are the canonical models that exist only in critical regimes (such as near bifurcation points). These models help in coming up with canonical models for an unknown system, without knowing the system explicitly. A few canonical models will be discussed in future blogs.

Important Definitions:

  • Excitatory Connection/Effect: When an incoming spike increases the membrane potential difference, it is said to helping in exciting the neuron.
  • Inhibitory Connection/Effect: When an incoming spike decreases the membrane potential difference, it is said to helping in inhibiting the neuron.
  • Dendrites: The long tail of a neuron along which a spike travels away from the nucleus and towards the neuron terminal.
  • Axon: Long protrusions from the nucleus which receive spikes from the dendrites of another neuron and bring it to the nucleus.
  • Spike: Biologically, it is the release of ions after a certain membrane potential difference, which triggers a rapid onset of ion influx. Theoretically (or rather, computationally), it can be understood as a simple threshold function.
  • Synapses: The connections between two neurons are referred to as a synapse.
  • Synapse strength: Theoretically, it describes the ease of passing a spike from the neuron terminal to an axon of the next neuron.
  • Neurotransmitters: Chemicals responsible for inducing the ion influx in the following axon for spike transmission, hence the name.

I will explain the spike generation process, neuronal spiking models, and synapse modulation in a future blog on the Spiking Neural Network.

What is a Weakly Connected Neural Network? - OLM edition

Now that we have some idea how brain models work, we can try and define a neural network. A neural network can be defined as multiple neurons connected by synapses in some specific pattern with a particular set of synaptic strength. Now the WCNN theory states that given a network for a specific state, on providing some arbitrary input, only those neurons are essential which have are near the threshold. We can effectively remove all the neurons far from the threshold and still get the same local analysis in the network.

Neuronal Models

Dale’s principle is an OLM which tried to explain the synaptic connection of a neuron in a neural network. Dale says that if all the synapses of a neuron are excitation, the neuron itself is excitatory.

Dale’s Neurons are defined mathematically as follows

\[\dot{x_i}=f_i(x_1,x_2, ..., x_n), \textrm{ } i=1,...,n\\ f_i:\mathbb{R}^n \rightarrow \mathbb{R}\\ s_{ij} = \frac{\partial f_i}{\partial x_j}, \textrm{ } i \neq j\]

This model addresses how neurons interact with each other, and the notion is synaptic connectivity is defined by the coefficient \(s_{ij}\) denoting connection between the \(i^{th}\) and the \(j^{th}\) neuron and for each neuron the activity for all synapses are either excitatory or inhibitory. This ensures that increasing the activity of \(j^{th}\) neuron has a direct impact on the ith neuron proportional to \(s_{ij}\).

For the notion of a WCNN we bring in the notion of weak connections.

\[\dot{x_i} = f_i(x_i) + \epsilon g_i(x_1,...,x_n), \textrm{ }\epsilon \ll 1, \textrm{ } i=1,...,n\\\]

In the above equation, the function \(f_i\) are the internal neuronal dynamics and function \(g_i\) the overall network dynamics for that neuron. Both \(f_i\) and \(g_i\) are smooth functions. The variable \(\epsilon\) is used to indicate the strength of global dynamics on the \(i^{th}\) neuron’s dynamics. The value of \(\epsilon\) is decided on the basis of the canonical model being used.

Neural Network Models

Now, at the empirical modeling stage, a few neural network models are found to model the neural circuits well.

Additive Neural Networks are the simplest form of continuous time neural networks. They are given as follows: \(\dot{x_i} = -x_i + S(\rho_i + \displaystyle \sum_{j=1}{n}c_{ij}x_j), \textrm{ }x_i\in \mathbb{R}, \textrm{ }i=1,...,n\)

In this formulation, \(c_{ij}\) is given as the synaptic connection between \(i^{th}\) and the \(j^{th}\) neuron. and the saturation gain function \(S:\mathbb{R} \rightarrow \mathbb{R}\) is defined as the sigmoid function. \(S(x)=\frac{1}{1+e^{-x}}\). The S function brings in the notion of non-linearity to our model and helps in making our dynamics more closer to an approximation of neuronal dynamics, further, being easy to compute and being continuous it behaves well for computations. This simple model is able to capture the circuit dynamics of external sense receptors very well.

Wilson-Cowan Model is an advancement on the simple additive neural networks, here instead of independent neurons, we have a neuronal pair, of excitatory and inhibitory neurons in a \textit{neural oscillator} setup. The model is given as follows:

\[\mu_X \dot{x_i} = -x_i + (1-\tau_Xx_i)S(\rho_{x_i} + \displaystyle \sum_{j=1}^{n} a_{ij}x_j - \displaystyle \sum_{j=1}^{n}b_{ij}y_j)\\ \mu_Y \dot{y_i} = -y_i + (1-\tau_Yy_i)S(\rho_{y_i} + \displaystyle \sum_{j=1}^{n} c_{ij}x_j - \displaystyle \sum_{j=1}^{n}d_{ij}y_j)\]

Here \(\mu_X, \mu_Y>0\) are membrane time constants, and \(\tau_X, \tau_Y\) are refractory periods of excitatory and inhibitory neurons, respectively. We can also take the special case where \(\mu_X = \mu_Y = 1\) and \(\tau_X = \tau_Y = 0\) which gives us the special case of simple additive neurons. This kind of network is actually observed in olfactory circuits and neocortex.

Reference:

Weakly Connected Neural Networks - Frank C. HoppensteadtEugene M. Izhikevich