Sushant Kafle

sushant @ mail.rit.edu
Ph.D. Candidate,
Computing and Information Science,
GCCIS, RIT
I am a fourth year Ph.D. student in Computing and Information Science at the Golisano College of Computing and Information Science, Rochester Institute of Technology (RIT) under the advisement of Prof. Matt Huenerfauth. My research aims to inform the evaluation and the design of Automatic Speech Recognition (ASR) technology for use in captioning for people who are deaf or hard of hearing (DHH).

Research Statement: I am interested in creating machine learning (ML) models that are able to understand the meaning behind natural language text and speech to enhance human-to-machine interaction. This often involves evaluating and validating these ML-based systems through real-world studies and observation with the users.


Some research projects I'm associated with..

2018
Behavioral Changes in Speakers who are Automatically Captioned in Meetings with Deaf or Hard-of-Hearing Peers
Matthew Seita, Khaled Albusays, Sushant Kafle, Michael Stinson and Matt Huenerfauth Annual SIGACCESS Conference on Computers and Accessibility (ASSETS'18)
Modeling the Speed and Timing of American Sign Language to Generate Realistic Animations.
Sedeeq Al-khazraji, Larwan Berke, Sushant Kafle, Peter Yeung and Matt Huenerfauth. Annual SIGACCESS Conference on Computers and Accessibility (ASSETS'18)
Best Paper Award
A Corpus for Modeling Word Importance in Spoken Dialogue Transcripts.
Sushant Kafle, Matt Huenerfauth. International Conference on Language Resources and Evaluation (LREC'18)
Methods for Evaluation of Imperfect Captioning Tools by Deaf or Hard-of-Hearing Users at Different Reading Literacy Levels.
Larwan Berke, Sushant Kafle, Matt Huenerfauth. ACM Conference on Human Factors in Computing Systems (CHI'18)
Best Paper Honarable Mention
Modeling and Predicting the Location of Pauses for the Generation of Animations of American Sign Language.
Sedeeq Al-khazraji, Sushant Kafle, Matt Huenerfauth. International Conference on Language Resources and Evaluation (LREC'18)
[to appear]
2017
Evaluating the Usability of Automatically Generated Captions for People who are Deaf or Hard of Hearing.
Sushant Kafle, Matt Huenerfauth. Annual SIGACCESS Conference on Computers and Accessibility (ASSETS'17)
Best Paper Award
2016
Effect of Speech Recognition Errors on Text Understandability for People who are Deaf or Hard of Hearing.
Sushant Kafle, Matt Huenerfauth. Speech and Language Processing for Assistive Technologies (SLPAT'16)


Timeline and Events

October, 2018

Presented our paper on "Modeling the Speed and Timing of American Sign Language to Generate Realistic Animations."at the ASSETS 2018 conference which was also recognized with the Best Paper Award in the conference.

June, 2018

Participating in the summer internship program at Google in Seattle office till September, 2018.

June, 2018

Two of our papers from the lab, which I am pleased to have contributed to, has been accepted at the ASSETS 2018 conference.

June, 2018

Sucessfully defended my Ph.D. thesis proposal. Officially a Ph.D. candidate now (yay!).

January, 2018

Our workshop paper "Modeling and Predicting the Location of Pauses for the Generation of Animations of American Sign Language" was accepted at the Sign Langague Workshop at LREC 2018.

December, 2017

Our paper "Methods for Evaluation of Imperfect Captioning Tools by Deaf or Hard-of-Hearing Users at Different Reading Literacy Levels" was accepted at the CHI 2018 conference and was nominated for a Best Paper Honarable Mention award (ranked among the top 5% of all submissions to the SIGCHI 2018 conference).

December, 2017

Our paper "A Corpus for Modeling Word Importance in Spoken Dialogue Transcripts." was accepted at the LREC 2018 conference.

November, 2017

Our ASSETS 2017 paper won the "Best Paper Award".

September, 2017

We announce the creation of the Corpus of Word Importance Annotations, more details here.

July, 2017

Our paper "Evaluating the Usability of Automatically Generated Captions for People who are Deaf or Hard of Hearing" was accepted at the ASSETS 2017 conference and was nominated for a Best Paper Award.

May, 2017

Helped facilitate the Research Experience for Undergraduates (REU) program at the CAIR lab.

October, 2016

Participated and presented at ASSETS Doctoral Consortium 2016.

July, 2016

Our paper "Effect of Speech Recognition Errors on Text Understandability for People who are Deaf or Hard of Hearing" was accepted at SLPAT 2016 workshop.

May, 2016

Sucessfully defended the PhD Research Potential Assesment.

August, 2015

Joined RIT for doctoral studies in the Golisano College of Computing and Information Sciences. Started working as a research assistant at the CAIR Lab.


Other projects I'm associated with..



Word Importance Labeler
Developed a tool with a suite of metrics for evaluating the quality automatically generated transcripts of classroom lectures based on word importance information. The tool was developed as a part of a research project at National Technical Institute for Deaf (NTID) which investigated the usability of automatic captioning for classrooms.


Speech Analysis for Word Importance Modeling
Investigated various acoustic-prosodic features from human speech to see if they provide clues about the importance of word being spoken; importance defined in terms of the contribution of the word in understanding the meaning of a spoken utterance.


Speech Recognition Error Analysis
Categorized and analyzed different types of errors produced by Sphinx4 Speech Recognition System on 100-hrs of speech recordings from LibriSpeech Corpus. Implemented novel output alignment modules to account for fuzzy time-stamp matching and, one to many and many to one substitution errors. Created a local compute cluster to make speech recognition faster.