How video game assists the visually impaired

Courtesy of Lotfi Merabe

A visually impaired study participant plays a computer game while a researcher looks on. Such games could visually impaired people navigate real buildings.

There are two things that make optometrist and neuroscientist Lotfi Merabet's new computer game unusual: The various rooms and corridors in the game exactly mirror a real place, and players aren't able to navigate by looking at graphics on-screen. Instead, players must rely entirely on different sounds that tell them where doors, walls, jewels and monsters lay in wait.
Merabet and his colleagues at Harvard Medical School and the University of Chile designed the game for — and tested the game with — visually impaired people.
And they found something interesting about their players when they later took the gamers to the real building on which the game was based. Although the researchers hadn't told the study participants to remember the game's layout, they found that those who played the game excelled at independently navigating the real building.
The researchers' study shows the promise of audio maps helping to solve a real problem for the blind, said Gordon Legge, who directs the Laboratory for Low-Vision Research at the University of Minnesota, is visually impaired, and was not involved in Merabet's study.
"People who are visually impaired are often anxious about going to new places. That's a big issue," Legge said. "If there were software methods to explore and learn a place before they go, it could be quite advantageous." [10 Profound Innovations Ahead]
The study also hints at the possibilities of using audio games to teach visually impaired people in a way that sticks better than other methods, including a guided tour of the computer game's layout, which Merabet also studied.
"Gaming sort of gives you a problem-solving sense that you don't have through directed navigation," said Merabet, who runs studies and sees patients at Harvard's Massachusetts Ear and Eye Infirmary.
Although there are still important software problems to resolve, in the future, major buildings might be able to offer audio maps or games to help visually impaired visitors prepare before they get there.
From game to real lifeStarting last year, Merabet began testing his game on people aged 15 to 45 who had been blind since birth. Players wore headphones while they played the game and navigated by pressing keys on a keyboard. Whenever their avatars touched walls or doors, they heard characteristic sounds. A knock in the left ear meant a door to the left; a knock on the right, a door to the right. Knocking in both ears indicated a door in front. "The best way for you to get a sense of that room is by feeling around," Merabet said. [Video Game Improves Vision for Adults Born with Cataracts]

He told his study participants they were supposed to collect jewels hidden in the game's rooms, while avoiding monsters that would steal the jewels. Both the gems and monsters emitted sounds that increased in volume as the game players approached them — or as they approached the players. He didn't say anything about remembering the map, nor did he mention that the rooms' layout was exactly the same as the two-story, 23-room Carroll Center for the Blind, a school in Newton, Mass.
Yet when he took the game-players to the Carroll Center afterward, he found they could apply what they'd learned from the game to navigating the real building. When researchers told gamers to go from one specific room to another, the gamers could finish navigating the real-life building in about a minute and 15 seconds. When taken to a random room and told to find the nearest exit, the gamers could do so in a minute and one second.
In fact, the players performed better at the exit-finding task than visually impaired study volunteers who had taken a strict guided audio tour of the computer game's layout, sans monsters and jewels. (The gamers and the guided tour-takers performed equally well at walking from room to room.) Tour-takers generally chose longer routes than gamers did, the researchers found.
"Both groups can learn the layout of the building whether you teach them explicitly or whether they learn it implicitly through the game," Merabet told TechNewsDaily. But gamers, he said, "have a better sense of how the rooms are connected with each other."
Does it have to be a game?Merabet's game wrapped up many characteristics that help with learning in one package: It forced people to explore independently and learn the hard way, by bumping into things. People enjoyed playing it and felt motivated to collect lots of jewels. ("I can tell you, at least anecdotally, it's remarkable how much people liked the game," Merabet said.) And the game may have forced people to walk certain paths that are important to learning a building's layout.

More experiments are needed if researchers want to pinpoint which characteristics are important to better learning from audio games and other audio maps, Legge said. It would be especially helpful to know if independent exploration by itself is enough, so that people could use software to wander through buildings virtually without having to contend with monsters every time, Legge thinks. "From a practical point of view, I'm not sure people would want to play a game every time they wanted to learn a new building," he said.
Merabet is now working to expand on his computer game research in several ways. One way is improving his software so that any building's blueprint can get automatically rendered into an audio map. Another is a brain-scan study that examines what parts of blind people's brains are activated when they play the game. He has finished the study, he said, but didn't want to reveal the results until they're published in a peer-reviewed journal.
Getting the software to work for all buildings is technologically challenging, Merabet said. Right now, his software works well for one-story buildings with straight corridors. For more complicated buildings, however, it's difficult for the computer program to remember to play sounds in the right place while users continually walk and change their position.
If Merabet is able to make an audio map generator, it could become exactly the tool Legge said would be so advantageous to those with vision impairments. Merabet compared the potential maps he could make to flight simulation software for pilots. Both would mentally prepare people for challenging tasks. "The idea of mental rehearsal is a very, very powerful one," he said.