Early onset blindness allows one to investigate how the human brain adapts to sensory experience in infancy and early childhood. Over the past decade, lesion, functional and structural imaging studies have accumulated evidence that severe perturbations to visual experience alter the functional and structural organization of the human brain. Visual deprivation can induce plastic changes not only in the visual system, but also in the remaining intact sensory-motor system, secondary to altered experience using these spared modalities. In particular, occipital, usually visual, areas are reorganized and recruited by the remaining senses and higher cognitive tasks primarily through cortico-cortical connectivity. Importantly, these plastic changes vary as a function of timing and are most pronounced in early onset blindness. Thus, sensory experience shapes functional and structural brain organization during sensitive periods in neurodevelopment.