Computational creativity and the future of AI

Scientists read peoples' brains to identify letters


August 21, 2013

Reconstructed images of letters viewed by the test subjects

Reconstructed images of letters viewed by the test subjects

If someone were looking at a letter of the alphabet that was blocked from your view, would you be able to accurately guess what that letter was? Well, if you were at Radboud University Nijmegen in The Netherlands, you might not have to guess or call in a psychic. Scientists there have used an MRI scanner and a mathematical model to read observed letters, right out of test subjects’ brains.

The researchers started by virtually (not physically!) dividing each subject’s visual cortex up into a matrix of 1,200 cubic sections known as voxels – each voxel, in this particular study, measured 2 mm per side. Using the MRI, the scientists then noted how those voxels electrically responded to visual stimuli. When the subjects were subsequently shown hand-written letters, the overall pattern of all the responsive voxels (as viewed via the MRI, and processed using the mathematical model) could be used to roughly reconstruct an image of each letter.

The problem was, the images were still quite indistinct, each one appearing as a “fuzzy speckle pattern.” If someone were to look at one of those patterns without being told what it represented, they likely wouldn’t recognize it as any particular letter.

To sharpen the images up, the researchers provided the model with prior knowledge of what the letters looked like. In other words, instead of just seeing the pattern that corresponded to the letter "S" as nothing but one random arrangement of fuzzy speckles, the model now realized what form the pattern should take. The model then helped things along a little, by visually “pushing” each pattern towards the appearance of the letter that it represented.

While this may sound almost like cheating, one of the main purposes of the study was to get a feel for the ways in which prior knowledge is combined with sensory information. For instance, when we see the word “speckle,” we don’t actually stop and identify each and every letter in turn – instead, we briefly get a sense of the whole word, then our brain more or less says “Oh, I know what this is, it’s that ‘speckle’ word I’ve seen before.”

The research team, led by Dr. Marcel van Gerven, now plans on using a more powerful MRI that can scan up to 15,000 voxels at once. Instead of letters, however, they will be attempting to reconstruct images of faces viewed by test subjects. Scientists at the University of California, Berkeley have already conducted a similar experiment, in which images from movie trailers could be “read” from participants’ brains.

Source: Radboud University Nijmegen

About the Author
Ben Coxworth An experienced freelance writer, videographer and television producer, Ben's interest in all forms of innovation is particularly fanatical when it comes to human-powered transportation, film-making gear, environmentally-friendly technologies and anything that's designed to go underwater. He lives in Edmonton, Alberta, where he spends a lot of time going over the handlebars of his mountain bike, hanging out in off-leash parks, and wishing the Pacific Ocean wasn't so far away.   All articles by Ben Coxworth
1 Comment

But are the fuzzy speckle patterns unique to each individual person, or is my letter "A" fuzzy speckle very similar to your letter "A" fuzzy speckle? The latter would be most useful, the former.... well

22nd August, 2013 @ 03:28 am PDT
Post a Comment

Login with your gizmag account:

Or Login with Facebook:

Related Articles
Looking for something? Search our 31,353 articles
Recent popular articles in Science
Product Comparisons