By Finbarr Moynihan, Vice President, Corporate Marketing.
We’re at a revolutionary moment in TV audio. It’s a moment that’s ushering in a new era of multichannel sound that feels deeply immersive and highly personalized. And it could see televisions evolve into multifaceted entertainment hubs, with companies like MediaTek and visionary composers like Hildur Guðnadóttir blazing trails to redefine the audio experience.
For Guðnadóttir, an Icelandic composer whose award-winning works include the scores to “Chernobyl,” “Women Talking,” “Joker,” and the upcoming “Joker 2: Folie a Deux,” the next chapter of audio harkens back to her creative process – a buildout between script and sound, character and composer.
"You’re trying to imagine the big picture,” Guðnadóttir says. “This might be the orchestration of the characters’ whole world – but then we have the smaller worlds of each character within that."
In the same way, MediaTek chips are enabling technologies like Dolby Atmos FlexConnect to engage listeners in the back-and-forth process of creating their own soundscape. The system, launched last fall, adapts to the speakers located throughout your space, using object-based multichannel sound to bring these movies – and Guðnadóttir’s scores – to life, with audio steered perfectly around each listener.
Soon, you’ll even be able to speak through your TV’s audio system to faraway friends viewing the same programming, as if they’re with you in your home. Thanks to audio-based object technology, your co-watching friend’s voice can be projected to the couch as if your friend is right there sitting next to you.
It takes audience-powered audio to a whole new level.
Technology So Good Ears Can Almost See
Object-based audio represents the latest frontier of multichannel sound. Audio is recorded, mixed, and presented across multiple channels in a sound field, placing different frequencies in strategic locations.
Whether it’s Guðnadóttir’s “Joker” or Ennio Morricone’s “Cinema Paradiso,” exceptional artistry still needs the right technology to bring it to the masses. That’s especially true at home.
Leveraging high-speed connectivity and spatial awareness of audiences and speakers, the technology steers the sound of a car speeding past the camera, or even the frenetic patches of violin in “Women Talking,” from one side of you to the other, moving audio as if it were an object from speaker to speaker in your room. The effect makes you feel you’re in the middle of a scene, no longer a passive watcher.
That experience is now possible thanks to a maturation in connectivity that coincided with that of audio. Once limited to hardwired speakers and cinematic settings, object audio first came into consumers’ homes on Blu-ray in the mid-2010s, with streaming services following in subsequent years. With Wi-Fi 7, the latest Wi-Fi standard, Dolby Atmos FlexConnect synchronizes audio with less than 150 microseconds between speakers and cuts latency down to less than 40 milliseconds between TV and the speakers – or 1/25th of a second.
That’s the unimaginably – and undetectably – short amount of time between something happening onscreen and your ears picking up the corresponding audio in front, behind, or beside you. MediaTek enables this unprecedented audio precision with its TV system-on-chip designs, including speaker and Wi-Fi chipsets. It’s no easy feat given humans’ aural sensitivity.
“Our visual system tends to be more tolerant to single visual artifacts,” says Alfred Chan, Vice President of TV in MediaTek’s Smart Home Business Group. “Our brains process video images by objects, so we integrate the pixels on screen into objects. At high resolutions like 4K, a single pixel drop isn’t noticeable.”
Audio, however, is a different story, where latency is far more noticeable than momentary visual latency. “A small audio drop, leading to a gap or silence, can be detected by human hearing as a distortion of sound continuity,” says Chan. “But MediaTek’s technology conceals continuity errors with predictive audio sampling that evens out the sound and eliminates momentary choppiness in real-time.”
With Dolby’s new system, users position their FlexConnect-enabled speakers throughout the room, and the technology does the rest. Similar to how people use two eyes to realize parallax and recognize the depth of visual objects, FlexConnect uses two mics during initial system calibration to detect sound from wireless speakers. In less than a minute, the technology can precisely identify the speakers’ various locations, which the TV uses to create a spatial map of the room. Based on this rendering, the system places each audio object in different speakers around the audience to create a realistic, immersive, and spatial 360-degree audio experience.
With FlexConnect, the first system to make this level of audio precision accessible, that entire process “happens seamlessly,” says Vikram Shrivastava, MediaTek’s Director of Marketing.
In the new era of audio, sound no longer fills the room; it moves within the space to spine-tingling effect.
"When you look at the history of recording techniques, you go from this single speaker into two speakers…and when you multiply that by more speakers around you, it just opens totally different dimensions." – Hildur Guðnadóttir
The Artistry of Absorption
If FlexConnect is steering audio through a given space, Hildur Guðnadóttir is drawing from the rich experience of her own space: New York City, a place she finds unlikely solitude.
“I absorb the various sounds of the city,” Guðnadóttir says. “All those elements blend into their own white noise. When we think about solitude, it’s easy to imagine a person walking through the forest. But I find it inspiring to be by myself among lots of people and observing them without words. As much as I love going for a walk in the forest, I equally love walking up and down Manhattan.”
Such observations have made their way into Guðnadóttir’s creative process.
“When you’re reading a script, you set the pacing, the tempo, the scenario, the whole environment,” she says. “That internal visualization of the world becomes quite musical, sonic even. I get a pretty good read of what I think the characters are, and then I have a dialog with the director and writer. That way, the music can be in dialogue with what’s happening on set. We can have this back-and-forth between the acting, the cinematography, the movement, the tempo, the overall feel.”
This plays out in “Bathroom Dance,” the infamous “Joker” scene where the central character, Arthur Fleck, realizes his initial Joker embodiment through a solo dance sequence. The actor Joaquin Phoenix was reportedly inspired after hearing Guðnadóttir’s music, and his Fleck weaves about the grime to the movie’s iconic, stripped-down strings.
Indeed, Phoenix would go on to win the Oscar for Best Actor for his performance, just as Guðnadóttir’s accompaniment won the Oscar for Best Original Score.
A Convergence of Art and Technology
For masterpiece scores – whether it’s Guðnadóttir’s haunting “Joker” or Ennio Morricone’s woodwind-flecked “Cinema Paradiso” (a personal favorite of mine) – it’s clear that exceptional artistry still needs the right technology to bring it to the masses. That’s especially true at home, where so many screens are competing for viewers’ attention.
But as TVs become hubs for increasingly complex and personalized entertainment, this convergence of technological and artistic innovation is poised to reshape how we experience sound and offer a rich alternative to video on a smartphone. The next era of on-screen audio will not only project cinematic-quality sound, but also detect and respond to individual voice commands amid multichannel activity onscreen, putting you at the heart of a new conversation with your TV.
Guðnadóttir has seen this journey from the recording side and is eager to work with what comes next.
“When you look at the history of recording techniques, you go from this single speaker into two speakers, and it just opens up a whole other universe,” she says. “And when you multiply that by more speakers around you, it just opens totally different dimensions. I’m excited to learn about the possibilities that are opening through MediaTek advancements.”
To learn more about how MediaTek's technology drives immersive experiences, check out our Visionaries on Vision series with Hildur Guðnadóttir.
