How This New App Will Help The Blind "See"

by Lara Rutherford-Morrison

You might think of your smartphone as a device whose primary purpose is to let you play Candy Crush on the bus and take endless selfies with your dog (or is that just me?), but a new app reminds us of just how much potential mobile technology has to change how people live and function, even in the most basic of ways. BlindTool is a free Android app that helps visually impaired people to “see” by identifying and naming the everyday items around them. The app is still in development, with a few obstacles to work out, but it nevertheless provides a glimpse into how smartphones might function as auxiliary “eyes” for the blind.

BlindTool was developed by Joseph Paul Cohen, a Ph.D. Candidate at the University of Massachusetts Boston. He told FastCoDesign that he was inspired to create the app after working with a blind programmer a few years ago. “I had the idea then to have some sort of technology to see for [visually impaired people],” he explained, “but that technology didn’t exist at that point to be able to do that.”

Advancements in image recognition technology mean that today’s computers are able to recognize images with impressive precision, but, as FastCoDesign points out, those systems are generally too bulky and slow to be of practical use to a blind person in his or her day-to-day life. BlindTool, in contrast, can be housed inside a smartphone and works quickly, taking only a second to identify an object. A demo video of the app shows how BlindTool works: The user simply points the phone at an object, and the app names it aloud and in text.

However, BlindTool isn’t always so accurate. In order to make the app work quickly, Cohen used a database of about one thousand categories of images (compare that to large-scale systems that have as many as 37,000 categories of images). That relatively small database allows users to get almost instant feedback, but it also means the app is frequently wrong. Mark Wilson at FastCoDesign reports that when he tried the app in his apartment, it identified his Christmas tree as a feather boa and a door as an armoire. When Andrew LaSane at Mental Floss tested it out, the app named a toothbrush as a letter opener and insisted that a trashcan was a toilet seat.

Cohen designed the app with these inaccuracies in mind, and built in measures to let users know how confident (or unconfident) BlindTool is in particular identifications. The app only identifies an object verbally if it’s at least 30 sure that the identification is correct. The app also vibrates according to its confidence level, and will only vibrate at full intensity when it’s 90 percent sure that it’s right.

The app may not be perfect, but it still holds much potential as a tool for people with visual impairments, giving them one more way to access the world around them. Cohen explained to FastCoDesign,

If someone just wants to look around, and get the gist of what something looks like so they can add a whole other dimension of sight, experience of the world, maybe [BlindTool is technically] wrong, but it’s right in intuition about something. I still think it gives a sense of independence, which is a big goal of doing this.

You can find the BlindTool app free for Android here.

Images: BlindTool (3); YouTube