Sound Hologram

Sound Hologram is a creative platform for 3D sound-based Immersive Art. The system generates, positions, and moves sound sources in space in response to people’s actions and object movements. With Sound Hologram, artists can create immersive experiences for people to interact with holographic sounds placed directly next to their ears without needing headphones. For example, a person gesturing to pour with an empty bottle could produce the sound of flowing water emerging directly from the bottle without speakers wired inside it.

Sound Hologram combines Wave Field Synthesis (WFS) and Motion Tracking (MT) technology using Spatial Computing for integration, control, and visualization (Unity). Although a few WFS systems have been built, Sounds Hologram is the only 3D sound-based platform that supports real-time interaction, i.e., the sound sources are dynamically positioned and transformed in response to the movement of people and objects.
We have built an initial working prototype using a 64-channelWFS array in a 30×40′ room on campus. Using this setup, we have developed multiple live demonstrations to explore and showcase novel applications for Immersive Art. We have created 3D sound sources that appear and disappear, fly around people, and emerge from and move between physical objects, such as mugs, glass bottles, pieces of cardboard, plants, and plastic toys. We seek to spark collaborations and inspire others to use our system in their artistic, educational, and research projects.

We have showcased our platform to the author Mo Willems, the artist Mikael Owunna, and the Director of the Rube GoldbergInstitute, Jennifer George. Faculty across campus have also visited and started using our system. For example, Annie Hui-Hsin Hsieh, an Assistant Professor at the School of Music, used our platform to teach her students how to create interactive soundscapes.

Sound Hologram is a new medium for expression. Our ultimate goal is to develop a platform for immersive artists, musicians, and technologists to create a form of sound-based immersion never experienced before, pushing the envelope of what is possible.

Sound Hologram was made possible in part by the Frank-Ratchye Further Fund Grant #2023-002. Additional images can be found here.