Facebook's Artificial Intelligence Can Describe Photos to Blind Users
The AI can even tell if you're smiling in a photo.
— -- Facebook debuted a new artificial intelligence system today that can describe photos in stunning detail, making the social network even more accessible to visually impaired users.
"Worldwide, more than 39 million people are blind, and more than 246 million have a severe visual impairment. As Facebook becomes an increasingly visual experience, we hope our new automatic alternative text technology will help the blind community experience Facebook the same way others enjoy it," a Facebook blog post today from Facebook's Head of Accessibility Jeffrey Wieland and his team said.
Using Facebook’s object recognition technology, automatic alternative text generates information about a photo. Visually impaired people using screen readers on iOS devices will hear a photo described to them with more context. For instance, Facebook says its artificial intelligence system can even provide such detail as: "Image may contain two people, smiling, sunglasses, sky, outdoor, water."
The update is a huge step up from the previous experience, which would say the name of the person who shared a photo and the word "photo" without adding more context.
The object recognition technology is based on a neural network, which is a computer system modeled after the human brain that gets smarter as it processes more information. Facebook's neural network has billions of parameters and millions of examples, according to the company.
The new audio photo captions will begin with describing the number of people in a photo, whether they are smiling and then lists each object detected in the photo, ordered by the algorithm's confidence in what it is seeing. The image's properties, such as whether its indoors, a selfie or a meme, will be announced at the end of the description.
However, it's not perfect and will continue to learn, the company said, noting that for now, Facebook's automatic alternative text will begin with the words "image may contain" to convey uncertainty.
The experience is currently only available on iOS devices and in English. However, Facebook plans to add the automatic alternative text option to other platforms in the future with plans to support more languages.