Interdisciplinary Initiatives Program Round 6 - 2012

Michael C. Frank, Psychology
Fei-Fei Li, Computer Science

What is it like to be a baby? Lab experiments allow researchers to ask focused questions about children's social development, but experiments often leave us ignorant about children's social experiences in their day-to-day life. Recent technical improvements such as light-weight cameras and microphones offer the opportunity to go beyond this previous work and measure what children actually see and hear (especially during unscripted social interactions), but analyzing idiosyncratic records of a child's experience is at best slow and labor-intensive. Our project addresses the growing need for analysis tools by developing publicly-available, easy-to-use analytic methods for extracting signal from large, naturalistic datasets. We will adapt current algorithms from computer vision for use with data from head-mounted cameras and wearable eye-trackers, producing a set of tools that can be applied to find social signals like hands and faces. In addition, we will conduct validation studies on this method using data collected with children. The resulting software will be of broad use to the psychology and artificial intelligence communities and will help researchers begin to make sense of these exciting new data sources.