Mind Reading

Researchers have been able to scan people's brains to "read off" images of
letters and shapes that people are watching at the time

BRAIN-scanning technology has been used for the first time to recreate
simple images by decoding the brain activity of people looking at them.
Previously, the best that had been achieved was to pick which of a
predetermined set of images the subject was observing.


Using functional magnetic resonance imaging, Yukiyasu Kamitani at ATR
Computational Neuroscience Laboratories in Kyoto, Japan, initially scans
his subjects' brains while they look at various images made up of a
10-by-10 array of black and white pixels. Software developed by Kamitani's
team analyses the scans to find patterns of activity that are associated
with certain pixels being blacked out. It then uses this information to
discover signature patterns of brain activity for each pixel.


Kamitani then shows the subjects fresh patterns while their brains are
being scanned, and feeds the scans to his software. By comparing the
patterns in the scan with its list of signature patterns, the software can
reconstruct the 10-by-10-pixel images the subjects are watching (Neuron ,
DOI: 10.1016/j.neuron.2008.11.004).


In June, Jack Gallant and colleagues at the University of California,
Berkeley, showed that they could tell from a brain scan which of a set of
images someone was looking at by comparing patterns of brain activity with
those produced by looking at a separate set of "training" photos (Nature ,
vol 452, p 352). Kamitani's work goes further by recreating the images from
scratch. "It really is a very significant step forward," says John-Dylan
Haynes of the Max Planck Institute for Human Cognitive and Brain Sciences
in Leipzig, Germany.


The images that Kamitani recreated are quite primitive, but by combining
several scans, the researchers were able to read off the word "neuron" and
several numbers and shapes that people were shown. It is an important proof
of principle, Haynes says.


As brain-scanning technology improves, Kamitani says it should be possible
to recreate higher-quality images with many more pixels. He also wants to
find out if it is possible to create images from shapes that people are
imagining rather than actually seeing. Haynes suggests this might make it
possible to "make a videotape of a dream".