1. Computing

VizWiz App Answers Blind iPhone Users’ Spoken Questions on What’s Around Them

Click Photo; Ask Question; Hear Web-Sourced Answer in Minutes Via VoiceOver

By

VizWiz App Answers Blind Users’ Spoken Questions on What’s Around Them

The VizWiz Social app enables blind and visually impaired iPhone users to send photos with recorded questions to get quick answers about what's around them from various online sources.

Apple iTunes Store

VizWiz Social is a free app that enables blind and visually impaired iPhone users to receive quick answers to questions about their surroundings.

A user takes a picture with the iPhone camera, records a question, and presses send. The app routes the question to one of four online resources: anonymous web workers, object recognition software, a user’s Twitter followers, or an email contact.

Responses are sent back in near real time. Users listen to answers using VoiceOver, Apple’s built-in screen reader.

VizWiz Social 1.0 works with the iPhone, iPod Touch 4G, and -- with a 2x display -- the iPad 2.

With VizWiz, a blind or visually impaired person can learn many useful things, including:

  • Clothing: Does this shirt match my pants?
  • Location: Is this the right entrance?
  • Cooking: Which of these cans is chicken broth?
  • Mail: Who sent me this letter?
  • Equipment: Where’s the USB port on this device?

Who Answers VizWiz Questions?

VizWiz uses automatic image processing, anonymous web workers, and a user’s contacts to answer questions. Users select which of the four is the most viable answer source.

For its web workers, VizWiz draws on a pool of volunteers, some of whom are members of Amazon's Mechanical Turk marketplace. VizWiz gathers multiple responses to help users gauge an answer’s reliability.

IQ Engines, which also powers the oMoby iOS app, is image recognition software that immediately identifies objects in photos. When an image is submitted, IQ Engines attempts to label the primary object. If it cannot, workers make the identification.

Questions can also be emailed to a friend. When sent, the contact is directed to a response page where they can answer the question. This option enables users to select someone they know.

Images and questions can also be sent to a user’s Twitter followers. As with the email option, Twitter recipients provide answers from a response page.

Using VizWiz Social

To use VizWiz Social, your phone must be connected to the Internet, either with WiFi, or cellular network.

To send a question, you first access the app’s home screen. Once there, you take a photograph, record a question, and choose an answer destination.

To take a picture:

  • Navigate to the “Camera” button
  • Point the iPhone towards the subject of your question
  • Double-tap the screen to capture the image.

Once the photograph is taken, VizWiz opens a screen where you can record a question.

To record a question:

  • Navigate to the “Record” button
  • Double-tap the screen to start recording
  • Ask your question in a slow, clear voice
  • Double-tap the screen to finish recording.

Once your question is recorded, you then select who you’d answers from. Your options are: Web Workers, IQ Engines, Email, and Twitter.

To send your question:

  • Navigate the toggle switches
  • Select your destination
  • Double-tap the appropriate toggle switch
  • Press “Send.”

Note: Selecting Twitter opens two fields for you to enter your Twitter username and password. Selecting Email opens your iPhone address book where you choose which contact to send to.

Never send any photo you wish to keep private as anonymous workers may be viewing the image.

The application enables you to return to the previous screen to redo that step.

Receiving an Answer

Once your query is submitted, you can wait for a response on the “View Answers” screen. When a response is posted, the phone will say, “New answers available.” Navigate down to listen to those answers. VizWiz will continue to alert you when answers are received.

Rochester Human Computer Interaction (RHCI) in the University of Rochester's School of Engineering & Applied Sciences developed VizWiz to help blind people recruit remote sighted workers to help them with vision-related problems. The website has a video tutorial.

©2014 About.com. All rights reserved.