The Brainstem

In the past two weeks, I’ve focused on the implementation of the A1 primary auditory cortex and the early brainstem where the spike trains from the auditory nerve arrive.

Development Activity - https://github.com/akhil-reddy/beads/graphs/commit-activity
Building the brainstem components - https://github.com/akhil-reddy/beads/blob/main/beads/core/eru/hub/audio/early_brainstem.py
Building the auditory cortex components - https://github.com/akhil-reddy/beads/blob/main/beads/core/eru/hub/audio/a1_cortex.py

Please note that some code (class templates, function comments, etc) is AI generated, so that I spend more of my productive time thinking and designing. However, I cross-verify each block of generated code with its corresponding design choice before moving ahead.

Algorithms / Development

Building the Environmental Response System (ERS)

1. Initialize biologically-accurate cell constants (such as resting membrane potential, spike current, refractory period, etc)
2. For each channel / tonotopy frequency “hole”
   • For each spike in spike train
     • If spike exists, advance membrane potential by dt given synaptic input current I_syn
     • If voltage crosses threshold, signal

1. Initialize a different set of biologically-accurate cell constants (such as resting membrane potential, spike current, refractory period, etc)
2. For each channel / tonotopy frequency “hole”
   • For each spike in spike train
     • If spike exists, advance membrane potential by dt given synaptic input current I_syn
     • If voltage crosses threshold, signal

1. Initialize a different set of biologically-accurate cell constants (such as resting membrane potential, spike current, refractory period, etc)
2. For each channel / tonotopy frequency “hole”
   • For each spike in spike train
     • If spike exists, advance membrane potential by dt given synaptic input current I_syn
     • If voltage crosses threshold, signal

1. Note: These cells are helpful only if the audio CMU is binaural i.e., if two cochlear units are built, one for each ear
2. For a predetermined set of delays, match the spikes between right and left ear
3. If spikes cross a threshold at any given delay (for example, if right is 100 microseconds slower than the left, then t-d of right will coincide with t of left, and d is determined as 100 microseconds), record the delay and timestamp
4. Feed this info into the efferent feedback

1. Note: These cells are helpful only if the audio CMU is binaural i.e., if two cochlear units are built, one for each ear
2. Compare the spike pattern between both ears
3. At any given point in time, if a spike is recorded in one ear and not in the other, record the timestamp
4. Feed this info into the efferent feedback

1. Define the STRF (spectro temporal receptive field) i.e., a 2D array for each neuron with different spectrotemporal affinities. These group of neurons contribute to population coding
2. (Placeholder logic) Define the dynamics of a biological neuron synapse (now that we've “entered” the brain)
3. (Placeholder logic) Define the mechanics of when a neuron should fire spikes (based on accumulating membrane potential)

Next Steps

Building the Environmental Response System (ERS)

1. Building the visual cortex
2. Building the ERUs
3. Neurotransmitters - Fed by vision’s bipolar and amacrine cells, for example, to act on contrasting and/or temporal stimulus. Neurotransmitters (and their lifecycles) can be added later during ERU development

Deployment

1. Overlaying video frames onto the retina, including code optimization for channel processing
2. Overlaying audio clips onto the cochlea, including optimization for wave segment processing
3. Parallelization / streaming of cellular events via Flink or equivalent

The (Early) Brainstem

When (tonotopically arranged) stimulus arrives at the brainstem - the portion of the brain that houses the thalamus and medulla oblongata, for example - it adds context to each channel as follows

1. Bushy cells - These cells (with a low voltage threshold and refractory period) examine the auditory nerve volleys to detect coincidences, and precisely signal when spike trains arrive
2. Octopus cells - These cells examine the channel for a sharp increase in stimulus (or a high voltage change), signaling the beginning of any sound at that frequency, simple or complex
3. Chopper cells - These cells examine the waveform for periodicity i.e., when in a given frequency channel, if the synaptic current is sustained, the implication is that the said frequency keeps repeating in the sound waveform

Additionally, the brainstem also feeds back into the cochlea through efferent fibers (for context, these fibers are used by the outer hair cells to modulate their prestin gain / elasticity) after measuring the binaural differences in energy. This explicit mechanism, much like the neurotransmitters in the retina, sharply contrasts with the implicit (and slow) backprop feedback mechanisms of current AI systems.