|
The CoCoCo Lab studies how our sensory and cognitive abilities are determined by experience and culture. Our methodologies employ machine learning techniques such as deep generative synthesis algorithms alongside a significant data-intensive expansion of the scale and scope of experimental research both by means of massive online experiments and fieldwork in locations around the globe.
Our research program has three foci:
1) Internal representations, 2) The universality and diversity of human perception, & 3) Simulated virtual worlds and social networks
1) Internal Representations
Human perception is rich, multi-dimensional and contextual. (Consider, for example, the ways in which emotion is conveyed by the voice: by pitch, volume, and many other parameters.) Yet behavioral methods, biased by their limitation to one-dimensional and simplified stimulus spaces, typically produce an impoverished understanding of human perception. Inspired by Monte Carlo Markov Chain techniques borrowed from machine learning and physics, our research program addresses this gap by developing new adaptive sampling methods, in which each successive stimulus depends on the subject's response to the previous stimulus. Such processes allow us to sample from the complex and high-dimensional joint distribution associated with internal representations and obtain high resolution maps of perceptual spaces.
Related publications:
•
Gibbs
sampling with people. Oral presentation, Advances in Neural Information Processing Systems (NeurIPS), 34.
• Timbral effects on consonance disentangle psychoacoustic mechanisms and suggest perceptual origins for musical scales. Nature Communications vol. 15: 1482.
• Serial reproduction reveals the geometry of visuospatial
representations. PNAS 118 (13) e2012938118.
Another related area of interest is comparing human and machine representations. With the rise of Large Language Models and foundational multimodal machine learning, it is crucial to understand how machine learning models align with human cognition. This is essential not only for improving machine learning models but also for enhancing their interpretability and safety.
Related publications:
• Large language models predict human sensory judgments across six modalities. Nature Scientific Reports. 14, Article number: 21445 (2024).
• Words are all you need? Capturing human sensory similarity with textual descriptors. The 11th Int. Conf. on Learning Representations (ICLR), 2023. ArXiv: 2206.04105.
• Passive
attention in artificial neural networks predicts human visual selectivity. Oral presentation, Advances in Neural Information Processing
Systems (NeurIPS), 35.
2) Understanding the Universality and Diversity of Human Perception
Traditional psychology experiments recruit participants with access to computer technology located in industrialized countries such as India and the USA. This sampling constraint severely limits our understanding of the roles of nature and nurture in human perception, as the similarities we find between participants may stem either from universal biological mechanisms or from comparable exposure. To overcome this limitation, we apply computational methods and analysis to data obtained in field research with diverse populations around the world. We also study vast cultural datasets from around the world and design new infrastructures for large-scale online experiments involving participants globally.
Related publications:
• Cross-cultural commonalities and variation in mental representations of music revealed by a large-scale comparison of rhythm priors from around the world.
Nature Human Behaviour.
• Universal and non-universal features of musical pitch
perception revealed by sung reproduction. Current Biology 29,
1-15.
• Extreme precision in rhythmic interaction is enabled
by role-optimized sensorimotor coupling: Analysis and modeling of West African
drum ensemble music. Phil. Trans. B 376(1835).
We also study vast cultural datasets from around the world and design new infrastructures for large-scale online experiments involving participants globally:
Related publications:
• Global music discoveries reveal cultural shifts during the war in Ukraine.
• Studying the Effect of Globalization on Color Perception using Multilingual Online Recruitment and Large Language Models.
3) Simulated virtual worlds and social networks
With our newly developed experimental toolsets, we can now conduct experiments that would be impossible in traditional lab settings. These experiments allow participants to be immersed in virtual worlds and social networks, where we can experimentally control the rules of social interactions. For the first time, this enables us to study collective behavior on a large scale in a controlled environment. This approach goes beyond agent-based simulations (which don't involve humans) and the analysis of real social networks (where ethical or practical limitations prevent experimental control over network structures).
We were recently awarded an NSF grant to explore how collectives can become more creative, intelligent, and what happens when they can choose the rules governing their interactions. We are currently recruiting joint postdoctoral fellows for a collaborative team involving PIs at Cornell, UC-Davis, Princeton, and CUNY.
Related publications:
• Large-scale iterated singing experiments reveal oral transmission mechanisms underlying music evolution. Current Biology 33.8, 1472-1486.e12.
• Incentivizing free riders improves collective intelligence in social dilemmas. Proceedings of the National Academy of Sciences, 120.46, p. e2311497120.
I am recruiting PhDs and Postdocs for my new lab at
Cornell!
|
|
|
|
|
|
|
|
|
