Yash Shah
Nov 13, 2026

Recently, myself and Daniel Yamins put out a preprint titled The transformer as a candidate canonical cortical microcircuit. You might find several interesting kinds of empirical analyses we performed to present the transformer architecture as a compelling candidate for the brain’s canonical cortical microcircuit! To help break it down for quicker digestion, I'm starting a series of blog posts that I will release at a weekly cadence. Let me know your thoughts as you read through them :)


Over the past decade, deep neural networks have become powerful tools for modeling various sensorimotor and cognitive regions of the brain such as vision, language, audition, motor control, somatosensation etc.

Within vision neuroscience, convolutional neural networks (CNNs), in particular, have served as de-facto models of the visual system. Their strong inductive biases of hierarchical organization and locality resemble several well-established properties of visual cortex.

Then, in 2017, came the transformer.

Originally developed for language, transformers have become a dominant architecture across artificial intelligence, including computer vision. Like CNNs, transformers retain a hierarchical architecture. Unlike CNNs, transformers relax the locality constraint and use learned multiplicative interactions to dynamically integrate information across different parts of an input.

Could this (back then new) microcircuit extend the kinds of brain computations captured by CNNs?

In this series, we (metaphorically) cut open the putative black box architecture of the transformer and analyze interactions between model units at the unit, population, and systems level and how they correspond to interactions at similar scales in the brain.

Convergent evolution between natural and artificial intelligence

Part 1 — Are transformers that different from convolutional networks?

Part 2 — Transformers capture the fine details of local cortical circuitry

Part 3 — The issue of extraneous “brain-unlike” dimensions in models and how to deal with them

Part 4 — Simultaneously achieving high task performance and building wiring efficient systems

Part 5 — It is all about the multiplications!

I hope this is intellectually stimulating!


Reach me by
email: ynshah [at] stanford [dot] edu
bluesky: @ynshah.bsky.social
linkedin: @ynshah3
x: @itsysxx