Editor’s note: Links to a video and high-resolution images for download appear at the end of this release.

David Ruth
713-348-6327
david@rice.edu

Mike Williams
713-348-6728
mikewilliams@rice.edu

VEST helps deaf feel, understand speech

Rice University, Baylor College of Medicine project allows subconscious translation 

HOUSTON – (April 8, 2015) – A vest that allows the profoundly deaf to “feel” and understand speech is under development by engineering students and their mentors at Rice University and Baylor College of Medicine.

Under the direction of neuroscientist and best-selling author David Eagleman, Rice students are refining a vest with dozens of embedded actuators that vibrate in specific patterns to represent words. The vest responds to input from a phone or tablet app that isolates speech from ambient sound.

Eagleman introduced VEST – Versatile Extra-Sensory Transducer – to the world at a TED Conference talk in March. He is director of the Laboratory for Perception and Action at Baylor College of Medicine and an adjunct assistant professor of electrical and computer engineering at Rice, of which he is also an alumnus. His lab studies the complex mechanisms of perception through psychophysical, behavioral and computational approaches as well as neuroscience and the law.

The Rice students working on VEST, all electrical and computer engineering majors, call themselves the Eagleman Substitution Project (ESP) team. They include seniors Zihe Huang, Evan Dougal, Eric Kang and Edward Luckett and juniors Abhipray Sahoo and John Yan.

They are aiding Scott Novich, a doctoral student in electrical and computer engineering at Rice who works in Eagleman’s lab. Novich devised the algorithm that enables the VEST to “hear” only the human voice and screen out distracting sounds.

The low-cost, noninvasive vest collects sounds from a mobile app and converts them into tactile vibration patterns on the user’s torso. Haptic feedback supplants auditory input.

The first VEST prototype put together by the team has 24 actuators sewn into the back. A second version, already in production, will include 40 of the actuators Eagleman calls “vibratory motors.” He described the experience, at least for a hearing person, as “feeling the sonic world around me.”

“Along with all the actuators, the system includes a controller board and two batteries,” said Gary Woods, the team’s adviser and a Rice professor in the practice of computer technology. “The actuators vibrate in a very complicated pattern based on audio fed through a smartphone. The patterns are too complicated to translate consciously.”

With training, the brains of deaf people adapt to the “translation” process, Eagleman said. Test subjects, some of them deaf from birth, “listened” to spoken words and wrote them on a white board. “They can start understanding the ‘language’ of the vest,” he said.

“We’ve already run some simple experiments with both hearing and deaf people,” Novich said. “As they use the vest more, they get feedback and know whether they are right or wrong and start to memorize patterns. People are able to identify words they have never encountered before.”

The project has also prompted students to learn skills they wouldn’t necessarily acquire in engineering classrooms. Huang became the team’s tailor when he learned to sew via YouTube. “I’m an electrical engineer,” he said. “I didn’t know anything about sewing.” But the teammates’ quick-study abilities have paid dividends already.

Last November, ESP placed second in the sixth-annual Undergraduate Elevator Pitch Competition sponsored by the Oshman Engineering Design Kitchen at Rice. In February, the team placed third in the third-annual Owl Open, the Rice student startup competition sponsored by the Rice Alliance for Technology and Entrepreneurship. The team will also present its work this month at the annual Design of Medical Devices conference in Minnesota.

“We see other applications for what we’re calling tactile sensory substitution,” Sahoo said. “Information can be sent through the human body. It’s not just an augmentative device for the deaf. The VEST could be a general neural input device. You could receive any form of information.”

-30-

Watch a video about the project here: https://youtu.be/JTSGX6RYuiM

Follow Rice News and Media Relations via Twitter @RiceUNews

Related Materials:

Eagleman Lab: http://www.eaglemanlab.net

David Eagleman at 2015 TED Conference: https://www.ted.com/talks/david_eagleman_can_we_create_new_senses_for_humans?language=en

Gary Woods: http://www.ece.rice.edu/garywoods.aspx

Oshman Engineering Design Kitchen: http://oedk.rice.edu

Images for download:

https://news2.rice.edu/files/2015/04/0406_VEST-1-web.jpg

Rice University engineering students and their mentors at Rice and Baylor College of Medicine are creating a vest that allows the profoundly deaf to “feel” and understand speech. Showing prototypes at neuroscientist David Eagleman’s Baylor office are, from left, students John Yan, Eric Kang, Abhipray Sahoo and Edward Luckett, graduate student Scott Novich, student Evan Dougal, advisers Eagleman and Gary Woods, and student Zihe Huang. (Credit: Jeff Fitlow/Rice University)

https://news2.rice.edu/files/2015/04/0406_VEST-2-web.jpg

David Eagleman (Credit: Jeff Fitlow/Rice University)

https://news2.rice.edu/files/2015/04/0406_VEST-3-web.jpg

Scott Novich (Credit: Jeff Fitlow/Rice University)

https://news2.rice.edu/files/2015/04/0406_VEST-4-web.jpg

Custom circuitry translates signals from an app to vibrations in the VEST designed by Rice University and Baylor College of Medicine to help the hearing impaired “feel” and understand speech. (Credit: Jeff Fitlow/Rice University)

https://news2.rice.edu/files/2015/04/0406_VEST-5-web.jpg

Rice University engineering student Edward Luckett pieces together components of the VEST, which allows the profoundly deaf to “feel” and understand speech. The project is a collaboration with Baylor College of Medicine. (Credit: Jeff Fitlow/Rice University)

Located on a 300-acre forested campus in Houston, Rice University is consistently ranked among the nation’s top 20 universities by U.S. News & World Report. Rice has highly respected schools of Architecture, Business, Continuing Studies, Engineering, Humanities, Music, Natural Sciences and Social Sciences and is home to the Baker Institute for Public Policy. With 3,888 undergraduates and 2,610 graduate students, Rice’s undergraduate student-to-faculty ratio is 6-to-1. Its residential college system builds close-knit communities and lifelong friendships, just one reason why Rice is ranked among some of the top schools for best quality of life by the Princeton Review and for best value among private universities by Kiplinger’s Personal Finance. To read “What they’re saying about Rice,” go to http://tinyurl.com/AboutRiceU.