August 20, 2019
The sound is unmistakable among the low rumble of passing traffic and rattling construction equipment in the distance. It’s a consistent high-pitched hum coming from somewhere on your right about 20 feet away, presumably on the other side of the street. Presumably, because you’re just listening to a 360-degree audio file and there’s no visual element to tell you exactly what is between you and the hum. You can’t see it, but you can almost feel footsteps going by, some idle chatter and a vehicle or two passing between you and the hum. It’s unlike anything you’ve heard before.
What is that? You ask Dinny Aletheiani, an Indonesian language studies professor who made the recording earlier this year on a trip to her hometown during Winter break.
Professor Aletheiani smiles. “It’s the sound of a bamboo stove that is being used to make putu ayu, a local coconut snack made out of rice flour,” she says. “It’s a sound that you won’t hear anywhere in the world.”
You close your eyes and let the sound envelop you—and it does thanks to the 360-degree ambisonic microphone that the professor brought with her to record the soundscapes of Demak, an emerging city in central Java. Though when Professor Aletheiani lived there as a little girl, Demak was just a small village 500 kilometers from Jakarta on the Java Sea.
The recording is part of the Sound Storytelling Blended Reality project that Aletheiani created to help record the sounds that have been lost as one-time Indonesian villages explode into modern, growing cities. A linguistics and curriculum studies researcher, Professor Aletheiani believes that listening is a crucial component of learning a new language. Along with writing, speaking and reading, listening allows students to immerse themselves in a language and understand the context of how dialects developed over time.
She describes the local slang for “let’s go” and how it mimics the putt-putting sound of the thousands of 50cc motorbikes that swamp the city. Reading the phrase in a book or even hearing a local enunciate the words wouldn’t provide the same experience as hearing the actual sound of motorbikes swarming around you through the city.
The soundscape with the humming bamboo stove is just one of several two- to three- minute, 360-degree recordings and video that Professor Aletheiani created on her trip. There’s the market soundscape with crowds of shoppers pushing past. There’s a bird store filled with chattering parakeets in bamboo cages. And a woman on the outskirts of town harvesting enceng gondok, a local crop harvested in the wetlands along the banks of Lake Rawa Pening.
Upon her return to Yale, she gave the recordings to two graduate students who partnered with her on the Yale Blended Reality project: Seth Wenger from the Divinity School and Sara Abbaspour, a visual artist in the MFA program. Wenger has a background in video and sound editing, so he took the raw files and edited them into two- to three- minute soundbites. He then imported them into Unity where Abbaspour combined the audio and video to create the cohesive immersive experiences. Abbaspour was able to use gaze control to match directional sound to where the viewer is aiming his eyes, allowing you to vary perspective across the 3D sound experience.
Professor Aletheiani has used the experiences in her classroom, allowing her students to put on a headset and immediately be transported to Indonesia with the context they need to truly understand the local dialect. She says that the soundscapes make learning the language in conjunction with the culture possible and lead to a richer educational experience.
Professor Aletheiani hopes to capture additional ambisonic sound and 360-degree video on future trips to Indonesia in hopes of capturing more nuance into Indonesian culture and language during this transformative period in the county’s history. Soon, small villages will be swallowed up by modern cities, losing much of the flavor unique to that corner of the world.
Perhaps future generations will access Professor Aletheiani’s soundscapes and wonder at a world when people still rode motorbikes and used bamboo stoves to make sweets.