Several theories have been proposed to explain the evolution of specie
s differences in brain size, but no concensus has emerged. One unresol
ved question is whether brain size differences are a result of neural
specializations or of biological constraints affecting the whole brain
. Here I show that, among primates, brain size variation is associated
with visual specialization. Primates with large brains for their body
size have relatively expanded visual brain areas, including the prima
ry visual cortex and lateral geniculate nucleus. Within the visual sys
tem, it is, in particular, one functionally specialized pathway upon w
hich selection has acted: evolutionary changes in the number of neuron
s in parvocellular, but not magnocellular, layers of the lateral genic
ulate nucleus are correlated with changes in both brain size and ecolo
gical variables (diet and social group size). Given the known function
s of the parvocellular pathway, these results suggest that the relativ
ely large brains of frugivorous species are products of selection on t
he ability to perceive and select fruits using specific visual cues su
ch as colour. The separate correlation between group size and visual b
rain evolution, on the other hand, may indicate the visual basis of so
cial information processing in the primate brain.