Research
Research Topics
Listening to someone talk typically feels like effortless and automatic understanding. With little conscious effort in commanding our mind to comply, an interlocutor can provide information, evoke emotion, or exchange social pleasantries. Despite the ease with which hearing humans verbally communicate, the process of transforming human-articulated sounds into an infinite possibility of novel and complex meanings -- i.e., speech comprehension -- is a computationally intricate, and currently unsolved, challenge.
What computational solution does the brain implement to overcome this challenge? This is the big question that guides the lab's research. It is our goal to delineate the processing architecture upholding speech comprehension in terms of what representations the brain generates from the auditory signal, and what computations are applied to those representations during the timecourse of processing. Describing these components of the human processing architecture is key to understanding auditory, speech, and language processing, which, we believe, require complementary insight from linguistics, machine learning and neuroscience in order to be successful.
Keeping track of the past and the future in processing speech
How the human brain processes words in continuous speech depends on which words came before and which come after. Using AI models for contextualized speech processing, we find that distinct neural populations in the brain are specialized for integrating past, future and surrounding context when listening to a narrative.
Why the brain misunderstands speech after stroke
This study reveals reliable decoding of speech sound properties, known as phonetic features, from EEG recordings in older adults. We also find decreased phonetic processing in individuals with a language disorder called aphasia. Most importantly, we demonstrate that healthy controls, but not individuals with aphasia, encode phonetic features for longer when uncertainty about word identity is high, indicating that this mechanism—encoding phonetic information until word identity is resolved—is crucial for successful language processing.
Measuring speech comprehension, moment by moment
We designed, developed, and validated a custom slider device that participants can use while listening to continuous speech, allowing them to report how well they understand what they are hearing in real time. This allows researchers to align the time course of the input speech signal with the listener’s changing experience of comprehension, creating a path toward linking behavioral dynamics to neural dynamics.
Investigating how the brain processes the rapid sequences of speech sounds in continuous speech
Rather than applying a fixed and static filter on the input signal, the brain passes information between neural populations as a function of time. This allows for joint encoding of both speech content (e.g., /b/ vs. /p/) and relative order (e.g., 1st vs 2nd vs 3rd). It is precisely this process that allows you to tell the difference between "melons" and "lemons" or "pets" and "pests"!
2023 Lab Research Summary
To learn more about the research in the lab, and our general scientific principles, read this short paper!