Mandayam, Schussheim code stuttered speech recognition algorithm
Benji Schussheim, one of the algorithm’s co-creators, codes a prototype entitled Personalized Automatic Stuttered Speech Recognition (PASSR).
It was summertime when Rishabh Mandayam ’21’s phone lit up with a text from Benji Schussheim ’20. “I want to create a machine learning algorithm for stuttering,” it said. “Are you in or are you out?”
In the months following the text exchange, Mandayam and Schussheim coded a prototype algorithm entitled Personalized Automatic Stuttered Speech Recognition (PASSR). The algorithm’s purpose is to understand stuttering and create a personalized user experience – a tool that hasn’t ever been widely accessible.
The algorithm prompts a user to adjust their speaking patterns – for instance, asking a user to speak quieter or louder. Through gathering information about the characteristics of sound, it becomes easier for the network to classify speech as stuttering.
“It’s kind of a paradigm shift,” Schussheim said, “because it used to be that in order for a speech recognition algorithm to understand you, you had to speak fluently.”
According to Schussheim, most stuttering recognition programs today use tools that edit stuttered speech out of audio clips. But Schussheim and Mandayam hope to find a different approach, prioritizing inclusivity.
“The existing algorithms out there use harmful mentalities about stuttering because the people who are making them don’t stutter,” Schussheim said, noting that many programs refer to audio without stuttering as normal speech. “It’s perfectly normal to stutter, it’s not really something that has to be fixed.”
When Schussheim first approached Mandayam with the idea, Mandayam felt excited by the opportunity and the hurdles that coding such a program presents.
“Since audio is super complicated compared to most things, immediately there was that challenge aspect that came to me,” Mandayam said. “It’s a really useful application that I didn’t think of before, and once [Schussheim] brought it up, I was like ‘hey, why hasn’t anyone done this already?’”
Harrison Sholler ’20 is a member of the Staples coding club and feels enthusiastic about the potential for a speech recognition algorithm.
“The development of this program sends a positive message to stutterers and to the world,” Sholler said. “Stuttering is not something to be hidden, it is not a defect. And to non-stutterers who perpetuate its negative image, it says that stutterers deserve to be heard.”
In addition to appreciating the project’s mission, Sholler acknowledges the complicated nature of the endeavor.
“Language is one of the most complex things humans have created and it takes a lot to get a computer to understand it,” he said.
Claire Lee ’21 is also a member of the coding club and is similarly impressed by Mandayam and Schussheim’s work. Lee added that she appreciates their work ethic and receptivity to new ideas.
“I’ve been programming for three years and I can’t even fathom being able to program that,” Lee said.
Mandayam hopes he can continue to work with Schussheim through the summer and beyond to ensure their algorithm becomes widely accessible.
“It’s important that people who have stuttering are given a tool to better communicate with other people,” Mandayam said. “It would defeat the purpose if we made the code, but then never distributed it to anyone.”
