Multisensory Integration (MSI) + Synesthesia

Background

Synesthesia and multi-sensory integration both deal with aspects of perception and our senses. Our lab was interested in how these two topics influence our ability to learn/succeed in educational environments (like school), and how they interact with other cognitive functions, like working memory.

Perception is our ability to recognize that we are experiencing something through our senses

 

Multi-sensory Integration (MSI) is how our brains perceive and combine information received from different senses (like touch, sound, vision) to produce a neural response (and eventually a behavioral one)

Synesthesia is a condition where the activation of one sense or cognitive pathway also activates another, unrelated sense at the same time. An example of Synesthesia is hearing music and simultaneously sensing the sound as swirls or patterns of color.

 

Working Memory: Memories we have for topics or information we are currently working with. It also allows us to combine bits of information together. Our working memory can only hold limited amounts of information for a short period of time. Some examples of activities that we use working memory for are: Remembering what someone said to us a couple minutes ago, solving a math problem, taking notes, or remembering a phone number as you get ready to write it down.

 

Questions of Interest

Do multi-sensory stimuli provide a benefit to working memory processing?

• How is this influenced by audio and visual modalities?
  • How does the spatial positioning of the audio and visual stimuli affect MSI and working memory?

Past Projects

Multi-Sensory Integration (MSI)

Our team studied how multi-sensory integration affects working memory with behavioral experiments. Multi-sensory integration was found to enhance perception, and especially for congruent stimuli. We examined the audiovisual integration effect by manipulated the spatial locations and duration time of simple and complex stimuli, including visual and auditory modalities. Furthermore, we tested how this integration of information from different modalities influence working memory capacity with existing paradigms. In future steps, we will focus on how multisensory information of numbers affect numerical learning in children.

Selected Publications

Electronic versions are provided as a professional courtesy to ensure timely dissemination of academic work for individual, noncommercial purposes. Copyright and all rights therein reside with the respective copyright holders, as stated within each paper. These files may not be reposted without permission.

Toomarian, E.Y., Gosavi, R.S., Hubbard, E.M. (2019). Implicit versus explicit interference effects in a number-color synesthete. Consciousness and Cognition. 75: 102806 doi:10.1016/j.concog.2019.102806

Gosavi, R.S. & Hubbard, E.M. (2019). A colorful advantage in iconic memory. Cognition, 187:32-37 doi: 10.1016/j.cognition.2019.02.009

Simner, J. & Hubbard, E.M. (Eds; 2013 and 2018). The Oxford Handbook of Synesthesia. Oxford, UK: Oxford University Press. Hardcover version (2013): ISBN 978–0-19-960332-9. Reissue as paperback (2018): ISBN 978–0–19–883627–8.

Simner, J., Carmichael, D.A., Hubbard, E.M., Morris, Z. & Lawrie, S.M. (2015). Rates of white matter hyperintensities compatible with the radiological profile of multiple sclerosis within self-referred synesthete populations. 21(3):322-330. doi:10.1080/13554794.2014.892625

Hubbard, E.M. (2007). Neurophysiology of synesthesia. Current Psychiatry Reports. 9(3): 193–199.

Hubbard, E.M. & Ramachandran, V.S. (2005). Neurocognitive mechanisms of synesthesia. Neuron, 48(3): 509–520.

 

See Our Publications for a chronological list of all of our published articles.