The Visual Cortex

In the past two weeks, I’ve focused on building similar structures and mechanisms for vision i.e., bridging the ganglion cells and the ERUs via the visual cortex (V1, V2 and so on).

The Visual Cortex

Architecture

Components

  1. Lateral Geniculate Nucleus (LGN) - Relays center surround retinal outputs in tonic and burst modes - depending on how active the eyes have been in the immediate past. Also, segregates the p-type and m-type ganglion cell streams
  2. V1 Primary Visual Cortex
    • Layer 4 - Cells detect edges and motion
    • Layer 2/3 - Integrates and reinforces lines in the same orientation, essentially stitching together small visual field stimuli
    • Layer 5/6 - Feedback mechanisms to tune LGN / Layer 4 and transmit horizontally normalized signals
  3. V2 Secondary Visual Cortex - Groups edges to detect corners, foreground, background, etc
  4. V4 Tertiary Visual Cortex - Groups corners into curves while mapping hue and color combinations with invariance to lighting and viewpoint (via population coding)
  5. Inferotemporal Cortex (IT) - Houses neurons tuned to complex objects and faces, invariant to position and scale (again, via population coding). Note: the brain’s associative memory starts here, from the IT cortex itself!

Algorithms / Development

Building the Environmental Response System (ERS)

LGN implementation v1

  1. Define the tonic and burst modes
  2. Define a center surround gaussian kernel which convolves on the incoming spike stream
  3. Apply a mode based on the neurotransmitters from later stages, essentially creating a systemic modulatory loop
V1 / V2 / V4 / IT implementation v1

  1. Define the STRF (spectro temporal receptive field) i.e., a 3D array for each neuron with different contrast-colour-direction affinities. These group of neurons contribute to population coding
  2. Hierarchically, create an array-like field with different affinities (Needs better logic)

Development Activity - https://github.com/akhil-reddy/beads/graphs/commit-activity
Building the visual cortex components - https://github.com/akhil-reddy/beads/blob/main/beads/core/eru/hub/visual/v_cortex.py

Please note that some code (class templates, function comments, etc) is AI generated, so that I spend more of my productive time thinking and designing. However, I cross-verify each block of generated code with its corresponding design choice before moving ahead.

Next Steps

Building the Environmental Response System (ERS)

  1. Refining the cortices
  2. Building the ERUs
  3. Neurotransmitters - Fed by vision’s bipolar and amacrine cells, for example, to act on contrasting and/or temporal stimulus
  4. Focus - Building focus and its supporting mechanisms (of which acetylcholine is one)

Deployment

  1. Overlaying video frames onto the retina, including code optimization for channel processing
  2. Overlaying audio clips onto the cochlea, including optimization for wave segment processing
  3. Parallelization / streaming of cellular events via Flink or equivalent
Created Aug 02, 2025