Facebook's New Way to 'See' Photos for the Blind, Visually Impaired

Facebook offers its users a window into the worlds of their friends, families, and colleagues. However, for visually impaired and blind users, that window is frequently opaque due to the fact that people post non-accessible images. However, a new iOS technology may help to virtually crack that window open a bit and let users like me see what we've been missing.

Last week Facebook came out with something pretty cool, combining artificial intelligence technology, and VoiceOver, for its iOS application which provides basic descriptions of the photos that people post.

When a user comes to a photo of a forest near a lake, for instance, VoiceOver (the text-to-speech engine on the iPhone) will say “photo. May contain trees, water, outdoors." If it recognizes that there are people in the photo, it will say “May contain: two persons, smiling." Though primitive in its current state -- especially as it can’t specifically describe the myriad of poses and expressions most subjects of Facebook photos have -- it presents many possibilities for accessible photo access. This is vastly exciting.

With a few caveats.

The possibility of having access to the most popular use of -- but least accessible aspect of -- Facebook is really exciting. Most people I know post photos regularly without providing any type of description -- despite my best efforts to train them. Having an alternative is fantastic.

However, machines are never as good as humans in describing things. “Image may contain trees and water” is not quite the same as a more eloquent description of a natural scene. Images of people are not really described at all, apart from a statement that there are people in the shot. Text, such as in a screenshot, is not translated. “Image may contain text” is as good as it gets.

Though I admit my ignorance to the technical details, I would hope that an optical character recognition engine wouldn’t be too much of a stretch if artificial intelligence which could decipher objects is available.

There is one concern with the message that this type of technology can send. To some, this type of application can imply that the only means to make photos useful to the blind and visually impaired is through complex technology. My worry is that, in some ways, this software may allow people to feel that they can get 'off the hook' for using their words to describe things. Already it's a challenge because most people don't make their photos accessible -- so there's a very real worry that people may feel that they don't even need to make an attempt towards accessibility. After all, if the program will do it for you -- regardless of how vague the result may be -- why bother manually adding in descriptions?

There are also concerns -- and this might resonate more greatly with the mainstream user group -- around privacy. In order to make images useful to me, or other blind and visually impaired users, there would need to be greater detail in the description. Would that mean technology might require facial recognition and geolocation to identify places and people in a photo? Though it's an interesting concept, it doesn’t exactly help the reputation of a platform whose privacy record is already suspect in some people's minds.

Those are questions that will have to be addressed if applications like Voiceover are going to continue to grow and become more useful. For now though, the prospect is pretty neat.

And I look forward to seeing what my contacts are up to on Facebook as I never have before!