Facebook trains AI computers to describe photos for the blind

Facebook is training its computers to become seeing-eye guides for blind and visually impaired people as they scroll through the pictures posted on the world’s largest online social network. The feature rolling out Tuesday on Facebook’s iPhone app interprets what’s in a picture using a form of artificial intelligence that recognises faces and objects. The iPhone’s built-in screen reader, VoiceOver, must be turned on for Facebook’s photo descriptions to be read. For now, the feature will only be available in English. The man behind the development is Matt King, a Facebook engineer who lost his sight as a result of retinitis pigmentosa - a condition which destroys the light sensitive cells in the retina.

On Facebook, a lot of what happens is extremely visual. And, as somebody who’s blind, you can really feel like you’re left out of the conversation, like you’re on the outside.

Facebook engineer Matt King

The descriptions initially will be confined to a vocabulary of 100 words in a restriction that will prevent the computer from providing a lot of details.  Facebook is being careful with the technology, called “automatic alternative text,” in an attempt to avoid making a mistake that offends its audience. Google learned the risks of technology last year when an image recognition feature in its Photos app labeled a black couple as gorillas, prompting the company to issue an apology. Eventually, though, Facebook hopes to refine the technology so it provides more precise descriptions and even answers questions that a user might pose about a picture. The vocabulary of Facebook’s photo-recognition program includes “car”, “sky”, “dessert”, “baby”, “shoes”, and, of course, “selfie”.

Facebook blind