A few days ago, the AI dubbing company Dubformer published an essay entitled “From Craft to Algorithm”. It’s a vision for dubbing’s future. It’s written by Dubformer co-founder Elena Chernysheva, and it’s worth reading. She calls for a conversation and engagement, and it’s truly time to talk. What actually struck me first, before anything else connected to dubbing itself, was a quote from one of the responses she invited. Mark Howorth, who runs a media and management consulting company, writes: “Traditional dubbing is a melting iceberg.”
Firstly, “traditional dubbing” doesn’t exist. Neither does “AI dubbing.” There is only dubbing. Dubbed material is produced following an enormous variety of workflows, bande rythmo or take-by-take, with or without a translation, with an automatic translation or one prepared by humans, with a dubbing script produced by an AI, plus human qc, or fully human. Synthetic voices or human voices. Fully automatic or with varying degrees of human involvement. Written on a computer or entered into an online platform. But it’s always dubbing. AI dubbing vs. traditional dubbing is an artificial dichotomy unless you think about dubbing done fully automatic and dubbing done exclusively by humans. Everyone in dubbing knows that neither one exists. But let’s for argument’s sake say that there is something called “AI dubbing”, which means that the process utilizes AI at many stages, most crucially for the creation of synthetic voices, and that there is something called “traditional dubbing”, which means that humans helm the creative process.
Secondly, Howorth is right. The iceberg is melting.
After all, we all know about the climate catastrophe. The localization environment is changing indeed. It has always changed. I don’t use a VRC anymore, actors don’t stand in the studio together when they record a dialogue, and a director might be joined to the session from home via a link. Audiences are more aware of international productions and might want to see the new season of their favorite series faster. They might hear interviews with actors and know what their voices sound like, and might compare those with their dubbing voices.
And arguably, the speed of change has increased. Technology is turning production processes upside down. Synthetic voices sound incredibly natural. AI is used at several stages of the workflow, like translation or in contracts management.
So yes, again. The environment is heating up and the icebergs are melting.
My vision of the present, however, is slightly different. I see the Titanic approaching the iceberg of “traditional dubbing.” On that Titanic are all the AI dubbing evangelists, dancing in the fancy ballrooms, hoping that global warming will happen fast enough to melt the iceberg before the Titanic hits it.
We could stand by and watch this scenario play out to see who is first. But there will be no winners. My vision of how people interact with one another is different.
But let’s see what we would all be missing after the iceberg is gone.
Howorth’s opinion of “traditional dubbing” is rather low: “Even today’s BEST dubbing is a poor facsimile for seeing the media created in its original language” (original emphasis).
I’ve heard this before. Someone from an English-language country insisting that the only way that one can truly appreciate anything is in its original language. Forget about Kurosawa or the Odyssey. If you don’t speak Japanese or ancient Greek, you are getting a second-rate experience out of these works of art at best. I have nothing against “The Emoji Movie” or “Gigli”, but without Pippi Longstocking or Dante’s Inferno, you’re really missing out on something in your life. And unless you know Swedish or Italian, you will need a translation. The world would be a much poorer place without translation.
The language barrier will always be there. But we have a ladder across it and that ladder is called translation. Unless you know every language in the world, you’re going to need it.
What’s sad, however, is what this description of dubbing says about audiences all over the world who have enjoyed dubbed films and series for close to a century now. That they are happy with the second best.
But in any case, it’s not as if “AI dubbing” will change this. Howorth again: “… it will be close enough, just as it is today.” I do actually see technological advances (including AI) make dubbing better, but that’s a matter for another article.
But what will likely have changed is that “AI dubbing” is cheaper – not for the audience, of course, but for the producers of dubbing. It will still be dubbing, only made by a machine, plus a few people shoveling coal into the engine and a few others making sure that the machine stays on course. Those people will not be creatives and I truly wonder dubbing can survive at all without creativity. In this case, the icebergs will melt and we will ALL drown.
In the vision of “AI dubbing” that is proposed most frequently today, not only will the language be adapted, but the face will be changed as well. This means even more change to the original than “traditional dubbing”. But will it be better because the audience is now hearing the actor’s original voice which has been squeezed through a machine to make it sound German or French or Hindi?
Dubbing might be something that people in the English-speaking countries that dominate the world of film production still are not used to, and they understandably get hung up on mismatching lip movement. But research has shown that in the case of dubbing, familiarity breeds acceptance, and that audiences have long ago developed coping mechanisms to adjust for the fact that lip movements don’t perfectly match the spoken words in a dubbed version. They watch a film for the emotional connection that they get from dubbing, not for the mechanics of a bilabial.
Years ago, I heard an “AI dubbing” expert say that dubbing scriptwriters “hack the language just to fit” the mouth movements, and that he felt that “AI dubbing” (including changes to the face of the original actors) is honoring filmmakers by being closer to their creative intent. But the fact is that “AI dubbing” as it is practiced today isn’t moving dubbing any closer to the original filmmaker’s vision. If it did, it would be even more expensive and time consuming than it already is.
This year, I’m celebrating 30 years in the dubbing industry. If there is one thing that I have learned throughout this time, it’s that dubbing is based on empathy. Just like the audience, I want to feel. I want to live with a character, wrap myself into a blanket on the sofa and cry, or join my fellow cinema-goers with a drink and a bucket of popcorn and be taken into a world that is fantastical, or scary, or just plain escapist fun.
Dubbers might be standing on a melting iceberg. Very possibly true. I don’t have a career in “helping companies navigate through turbulence,” so I’ll just take Howorth’s word on that one. But I know dubbing. And the dubbing industry has adaptation written into its DNA. We are standing on a melting iceberg? Maybe we just need a bigger boat that we all fit into.
