RAS SIGHT Project, “Speak Up!” Wins Two Prizes


The project “Enhancing the Speak Up! Suite of Voice Powered Games,” was completed by Amal Nanavati, M. Bernardine Dias and Aaron Steinfeld, Carnegie Mellon University (PA), USA. This project received two awards at the Carnegie Mellon’s undergraduate research symposium.

  • The Osher Lifelong Learning Institute’s award for “Best Multi-Year Project,” which awards projects that have “positive impacts on individuals’ quality of life and health.”
  • The Dietrich Humanities Prize, which awards projects that “best exemplify the humanities as they are understood at Carnegie Mellon.”

“Speak Up!” was created to support and train the deaf community in developing areas of the world. The “Speak Up!” suite of voice-powered games includes games where: student voice propels a vehicle forward, students vary their volume to move a bird up and down, and student voice gradually reveals a flashcard. The games had been in use at a school for the deaf in India for 2 years. he team conducted fieldwork to understand their strengths and shortcomings, and further improve upon them.

We had the pleasure of interviewing the project lead, Amal Nanavati, and gained insight on the motivation and experiences learned from the project. Read on to learn more about what Amal had to say about this project. 

  1. What inspired you to start this project?

This project builds upon a decade-long partnership between Carnegie Mellon’s TechBridgeWorld research group and the Mathru Educational Trust for the Blind. In summer 2015, myself and other TechBridgeWorld research interns traveled to Bengaluru, India to develop low-cost, accessible, culturally appropriate, and sustainable educational technologies at the Mathru Centre for the Deaf and Differently-Abled. Our methodology followed TechBridgeWorld’s Compassionate Engineering process. During our initial needs assessment at the Centre for the Deaf, we learned that teachers found it difficult to engage students during Speech class, due to the time-intensive and individualized nature of speech therapy. Therefore, we worked with the community to create “Speak Up!,” a suite of games designed to help students understand, explore, and strengthen their voices during Speech class. More information about the 2015 research can be found here.

Over the following two years, I had been periodically in-touch with teachers and administrators at the school. They had been using the games regularly, and had requests for multiple new features and modes, due to improvements in students’ vocal skills as well as curricular changes. Further, I was intrigued by the research question: how has this two-year deployment of the games shaped the community’s norms (i.e. views about technology, views about speech therapy, teaching practices, etc.), how have the community’s norms shaped their usage of the technology, and how can I incorporate that information into the iterative technology development process? To explore this question and to address the community’s requests, I applied for the RAS-SIGHT funding.

  1. What was the greatest challenge you faced while working on the project?

One challenge we faced multiple times during the project was addressing teacher requests while accounting for the community’s context and adhering to broader goals of the project. For example, multiple teachers had requested a game that could tell students whether or not they pronounced a word correctly. However, due to technological resource constraints, local accents, and the fact that many students were pre-verbal, state-of-the-art speech recognition tools would not be appropriate for the school’s context. Therefore, I created a game where teachers specify which words students must say, students say them, and teachers then tell the computer whether the student pronounced the word correctly or not. This not only addressed the teachers’ requests in a way that accounted for the community’s context, but it also furthered one of the broader goals of this project — to increase teachers’ familiarity with computers.

  1. Do you have any advice for other SIGHT groups you would like to share?

Ensure that the research equitably balances the community’s interests as well as the researchers’ interests. This is crucial, since the central goal of many humanitarian technology project is to support the community and many such projects would not be feasible without the help and support of the community. Ways that we have ensured equitable partnership in our works include:

  • Supporting the Community Even Beyond Our Direct Research Goals: There were many times when I removed viruses from computers, found animations to teach particular concepts, provided editorial feedback on English communications, etc. This is all part of the crucial work of building community trust and forming a respectful, mutually beneficial partnership.
  • Leaving the Community With a Tangible and Sustainable Outcome: We made sure to leave usable and bug-free copies of the games with the community. We also trained teachers, students, and administrators on how to use the games, left documentation that could be easily understood by community members in multiple locations, and provided multiple backup copies of the games to various community members. This ensures that the community continues to receive the benefits of the research even if the researchers are unable to return for a while.
  • Providing Positive Publicity: We have made it a point that news releases, academic publications, and conference presentations about this work speak about the Mathru Educational Trust for the Blind and provide links to its website.


Additional information regarding this project:

If you would like to obtain the CHI 2018 published article based on this project, here is a link: https://dl.acm.org/citation.cfm?id=3173892

Community Partner Information

The Mathru Educational Trust for the Blind is a nonprofit, founded in 2001 by Ms. Gubbi R. Muktha. Ms. Muktha had been in an unfortunate road accident in 1986, which resulted in losing mobility for almost three years, during which she regularly visited the Vocational Rehabilitation Centre of the Disabled in Bangalore. At the rehabilitation centre Ms. Muktha came into contact with many blind people, who were struggling with mobility and were extremely dependent on family and friends. She was moved. In her own words, “Just because a child is blind does not mean he or she should become a burden to their family and society. Just because a child is blind does not mean they are worthless. They have so many other strengths and talents which can be harnessed for their benefit and for the benefit of their family and society. I realized that God had a bigger plan for me.” In 2001, she founded the Mathru Educational Trust for the Blind with the original goal of creating a residential school for blind children in and around Bangalore. The Mathru Educational Trust has since expanded, and it now runs two residential schools for disabled children and countless other charitable projects. The site of this project was the Mathru Centre for the Deaf and Differently-Abled, which was established in 2012 to serve students with hearing impairments and multiple disabilities. Learn more about all the Mathru Educational Trust’s projects at http://www.mathrublindschool.org/