Tomas strolls through Rock Bridge Hy-vee looking to fill his shopping list. He stops in the soup aisle and pulls out his iPhone to scan the shelves until he locates the Campbell’s chicken and wild rice soup. Utilizing Microsoft’s Seeing AI app to narrate the world around him, he gathers the necessary items to make dinner tonight.
The Seeing AI app is designed for the blind and low vision community and uses artificial intelligence (AI) to open up the visual world by describing scenes, people, text and objects in front of the viewer. Tomas uses the “short text” function to read the variety of soup labels that line the shelves, as well as check use-by dates and identify ingredients. Being able to easily carry out tasks, like grocery shopping, gives Tomas more independence to tackle everyday challenges.
Tomas first heard about the app through his boss at work and thought it sounded like something that would benefit him. He contacted his Community Skills Specialist, Christi, and they began researching the app and accessibility benefits of the iPhone.
Tomas said, “Working with Christi has been really great, she’s trained me in a lot of things and I’ve learned from all the trainings we’ve been through. She taught me to use the bus, and I know how to get places now. Back in my old place, she used to train me how to cook and make meals. I never had an iPhone before, (I thought) maybe this is something Christi can train me to use.”
Christi and Tomas researched accessibility features of smartphones as well as the
effectiveness of the Seeing AI app. After careful consideration, Tomas decided to purchased an iPhone and download the free app, and began using both in his everyday life.
Christi said, “His iPhone comes with a lot of accessibility features, shortcuts, voiceover, verbal commands, etc. He’s very independent and has learned all kinds of settings I didn’t even know about it. He was able to figure out how to have a curtain on his phone, so it’s private and no one has to know what he’s doing. He doesn’t have to pull out his wallet with cash showing, or get his card out. The other goal was being able to use the AI app — use it to read things to you, tell you a color, show you a scene, read money.”
The app has a color recognition function that, for instance, can say whether a berry is red, purple, or blue. It also helps blind people use touch to experience a photo. They can touch different parts of an image and the AI describes exactly what they are touching, including objects and people. So far, Tomas continues to find new ways to use the app, and benefit him in his everyday life.
“The trainings that I have received have definitely paid off,” said Tomas. “Every time I do the BCFR survey, I put 5 stars on everything because this is one program that I’ve felt like I’ve done really well, there haven’t been that many places that I can say that about. I’ve learned a lot from Christi.”
Tomas is looking forward to his next training with Christi, which is a refresher on bus routes since he is moving.