Predicting Sound from Video

In the Laboratory for Computational Audition, Joshua McDermott PhD ’07 operates at the intersection of psychology, neuroscience, and engineering. An assistant professor in the Department of Brain and Cognitive Sciences, McDermott and his team work to understand how humans derive information from sound, to improve treatments for hearing impairment, and to enable the design of machine systems that mirror human abilities to recognize and interpret sound. In one such project, McDermott collaborated with colleagues from the Computer Science and Artificial Intelligence Lab (CSAIL) on an algorithm that can learn how to predict sound from video footage and produce the expected sound realistically. Such an algorithm could strengthen machines’ ability to understand the physical properties of objects.