← Blogs
Complete Since March 30, 2026 4 days · 10 min
Search Images with Words
How do pixels become a word, in the brain and in a vision-language model? From the neuroscience of naming what we see, to CLIP, to a working on-device text-to-image search prototype.
Coverage
4 of 7 strands covered
Day
1 Words in the brain
2 Two encoders
3 Contrastive learning
4 Shared embedding space
5 Zero-shot classification
6 Text-to-image search
7 On device
01 CLIP: the paper that married text and images
02 CLIP's contrastive loss: how two encoders learn one space
03 Building CLIP from scratch: reading the paper properly
04 Zero-shot classification: from embeddings to probabilities
Syllabus
9 of 12 topics written up
- 1 Words in the brain 0/1
- How the brain ties what we see to words
- 2 Two encoders 2/2
- 3 Contrastive learning 4/4
- 4 Shared embedding space 1/1
- 5 Zero-shot classification 2/2
- 6 Text-to-image search 0/1
- Ranking images by a text query
- 7 On device 0/1
- Running the encoders on a phone
Index
Everything written, tangents included, A to Z
C
I
N