Audio Communication Unit and Visual Ganglion Cells

In the past two weeks, I’ve focused on wrapping up the (4 different) ganglion cell implementations, to finish the first complete version of the digital retina. Now I’m implementing the audio CMU (see architecture above), particularly the digital versions of the cochlea and auditory nerve, to supplement visual stimulus processing.

Development Activity - https://github.com/akhil-reddy/beads/graphs/commit-activity
Building the ganglion cells - https://github.com/akhil-reddy/beads/blob/main/beads/core/cmu/transportation/vision.py

Please note that some code (class templates, function comments, etc) is AI generated, so that I spend more of my productive time thinking and designing. However, I cross-verify each block of generated code with its corresponding design choice before moving ahead.

Algorithms / Development

Push implementation and transportation

1. Greedily cluster SACs into groups of approximately 'sac_group_size' based on spatial proximity
2. Integrate the input signal from many amacrine cells (in a similar centrifugal manner as SAC)
3. Generate spikes based on the updated firing rates

1. Greedily cluster bipolar cells by spatial proximity
2. Integrate input from r-g channel in cone bipolar cells, with spatial weighting, as they supporting high-acuity vision and red–green color opponency
3. Generate spikes based on the stimulus

1. Greedily cluster bipolar cells by spatial proximity
2. Integrate input from rod / motion channel and cone brightness channel, with spatial weighting
3. Generate spikes based on the stimulus

1. Greedily cluster bipolar cells by spatial proximity
2. Integrate input from b-y channel in cone bipolar cells, with spatial weighting
3. Generate spikes based on the stimulus

Next Steps

Push implementation and transportation

1. Implementing the audio CMU components - inner ear and auditory nerve
2. Overlaying video frames onto the retina, including code optimization for channel processing
3. Overlaying audio clips onto the cochlea

Building the Environmental Response System (ERS)

1. Building the visual cortex
2. Building the auditory cortex
3. Neurotransmitters - Fed by vision’s bipolar and amacrine cells, for example, to act on contrasting and/or temporal stimulus. Neurotransmitters can be added later during ERU development