Published on July 21st, 2014 | by SMUMN0
Giving real voice to computers
While in high school, she read computer books and tediously wrote out programming language in a notebook. When she started school at Saint Mary’s, simply typing her research papers was a struggle, as she had never used a keyboard.
During her college career, Mishra not only taught herself to type and double-majored in computer science and math, she also interned with the Mayo Clinic and assisted with a research grant that would come to shape her career as a computer speech scientist.
“My senior year, I ended up doing research through a grant with Ann Smith, (retired) then-chair of the computer science department and my advisor. It was an NSF grant, which looked at how to teach computer science to someone who is visually impaired. We had to use a lot of computer-generated speech,” she said. “This was my first introduction to getting computers to produce words and speech, and I was hooked. It ended up becoming my career. I’m very glad I got that amazing opportunity; it changed my life.”
Mishra earned her Ph.D. in computer science and is now a senior member of research staff at AT&T in New York City, a position she has held since 2008.
Mishra works on speech and language technologies to improve communication and reduce accessibility barriers. She builds synthetic voices to embody different characters and emotions. As someone who speaks five languages, she has always been fascinated by dialect.
“My work involves two pieces, analysis and synthesis,” she explains. “Synthesis takes text and gets the computer to produce speech. This field has been around for a few decades but the challenge is discovering how to produce not just intelligible and speech, but also expressive speech. My particular focus is making computer-generated speech more natural sounding, more expressive, with human-live melody and expression to it.
“The analysis aspect of my job uses computational methods to analyze human speech, looking at both the words that are said and how they are said, to identify emotion. Are they happy, sad, angry, disgusted? We also identify intent. By looking at emphasis, we determine which part of what a person says they care the most about.”
One of the applications she is working on is a storyteller application geared toward children.
“I focused on children’s stories for two reasons. My interests lie in developing child-directed applications for people with accessibility needs. This goes back to the first project (in college), using technology to meet accessibility needs. That is my passion, the focus of my work.
“Children love stories. An app that reads out children’s stories in a variety of character-appropriate voices so that kids are seeing and hearing the words at the same time could be both entertaining and educational. This app may also serve the needs of children with learning disabilities, who need information presented in a different way than just printed on a page.”
Children, she says, are the most difficult customers. With a 2- and 4-year-old at home, she can speak from personal experience. If a story isn’t read in an interesting, fun and engaging way, they will quickly lose interest.
“This is a very difficult genre,” she said. “It immediately points out all the ways we need to improve. But any success in underlying technology easily generalizes to other tasks.”
Mishra said she credits some amazing mentors for helping her get where she is today. “I don’t think I would be where I am without the advisors and the teachers I had in college, and before that in high school. Students should identify good mentors and ask them lots of questions. I would have never gotten a Ph.D. without my advisor’s encouragement.”
Mishra came to Saint Mary’s because her caring and protective parents — sending their then 18-year-old daughter to study overseas — wanted her to study at a smaller Catholic school with individualized attention. After some acclimation time, they told her she could still transfer to a larger school.
“After a year, I checked it out, but I had gotten used to the close community at Saint Mary’s, and the largeness and the anonymity of a larger school didn’t appeal to me anymore,” she said.
Mishra believes successful students are self-motivated. “A mentor can encourage you but you still have to do the work. Identify what you want, where you want to go and work hard at it; give it everything. If you fail, it’s not because you didn’t try, and you will learn lots of things in the process.”
This article, written by Deb Nahrgang, was previously published in the Spring 2014 Saint Mary’s Magazine.