If 2020 was the year deepfakes went mainstream, 2021, so far, has been the year that they became inescapable. Within the past week, a set of eerily accurate deepfakes on TikTok gave Tom Cruise his most viral moment since the couch jump (or at least since his COVID-19 tirade last December), and a company called MyHeritage debuted a “Deep Nostalgia” feature that let anyone with a JPEG give a deceased celebrity or relative the Harry Potter portrait treatment. Although the animated head shots were weird-looking and limited and the more convincing Cruise videos were the creations of a professional visual effects artist who was working with a Cruise impersonator, the latest deepfake-driven news cycle renewed rampant fears of a Black Mirror–esque nightmare world where deepfakes can’t be distinguished from the real thing, no footage can be trusted, and anyone with widely available image-alteration tools could more easily spread misinformation or incite scandals and conflicts.
On the plus side—and no, this may not totally make up for the growing risk of dystopian terror—the increasing ease of doctoring footage of real-life figures is good news for fans of Ronald D. Moore’s Apple TV+ alternate history of the space race, For All Mankind. “From an entertainment perspective, I have to say it’s really, really amazing what is possible now,” says For All Mankind writer and executive producer Ben Nedivi. As Nedivi notes, “The better it is for us, for our show, the scarier it seems this technology is for the world.” Hey, at least there’s some silver lining.
For All Mankind, now in its engrossing second season, presents one vision of how history might have unfolded had the Soviet Union beaten the United States to the moon. (If you’re just joining the series because of its strong second-season reviews, welcome to the FAMily.) In Season 1, which starts in 1969 and proceeds through 1974, the discovery of water ice inside the moon’s dark craters causes both superpowers to intensify their efforts to establish permanent presences on the previously lifeless satellite. In Season 2, which picks up in 1983, the cold warriors have expanded their moon bases, and competition over nearby resources has heightened tensions to the point that the two sides are arming and threatening to start World War III on another world.
These deviations from our reality reverberate back on earth: Ted Kennedy ousts Richard Nixon from the White House in 1972; the Equal Rights Amendment passes; the Vietnam War ends early and the Soviets opt not to invade Afghanistan; the Camp David Accords are never signed; Ronald Reagan gets elected four years ahead of schedule; Three Mile Island doesn’t melt down (thanks to technology developed for the moon base); John Lennon and Anwar Sadat survive assassination attempts but John Paul II doesn’t; the Miracle on Ice never happens; Charles marries Camilla instead of Diana; and so on.
As The Ringer’s Michael Baumann explained last month, the series’ creators maintain an exhaustive account of their alternate history, complete with a detailed timeline that wraps around the walls of the writers’ room. But it’s easier to lay out those imagined developments in a show bible than it is to make them look credible onscreen. Although part of the pleasure of watching the show is picking up on ways in which its history differs from the version we know, Nedivi says that “we also wanted this version of our history to feel as close to reality as possible. And I think the way to do that is to use real figures. When I’ve seen shows like this and everyone’s made up, it takes you out of it a little bit.”
Part of the producers’ plan for packaging their fiction in a persuasive way involves dressing up unreality in the iconography of authentic-looking newspaper headlines and news broadcasts. A fake presidential address rings less hollow when a believable anchor cues it up. “With the amount of news events and historical events that are changing, we felt like we needed fictional anchors to really sell that stuff,” says Matt Wolpert, another writer and EP. “And it was very important that we found people that felt like the Walter Cronkite or the Roger Mudd or the Connie Chung or the Ted Koppel types. So it’s a very exhaustive process in that regard as well, because if you don’t believe them as anchors, then you’re not really going to be buying into what they’re saying. You’re going to be focusing on them not being as realistic as they should be.”
That dissonance would be devastating for a series that depends on verisimilitude. “The nature of our show, the what-if alternate history, really makes moments like this even more important,” Nedivi says. Real-life character encounters, he continues, “ground the show in a way that is really necessary to pull off the more sci-fi elements. If you have that down, then it’s much easier for people to buy the more outlandish stuff that we’re also doing in the same episodes, so that balance is a real important element to the show.”
However, the alt-history hack of blurring the boundaries between real and unreal can backfire. If implemented poorly, the presence of politicians and celebrities we recognize may make it more difficult for viewers to suspend their disbelief. And because the audience knows that the cameos from those figures must be the products of digital trickery, they’re hyper-attuned to the uncanny valley (which can be a bit distracting even if the effects pass muster). “Using the actual figure is one thing, making it believable is another,” Nedivi says. In the first and second seasons, actors play some real-life figures, such as Sally Ride, Lee Atwater, or various NASA officials. But “casting someone to play somebody like Reagan, it doesn’t read,” says Wolpert. “The audience always kind of has that, ‘This isn’t real’ thought in the back of their mind.”
In instances when For All Mankind’s timeline largely overlaps with ours, its creators can repurpose unaltered or archival footage, sometimes taken out of context. But when they want, for instance, Nixon to vent his spleen about the Soviets beating NASA to the moon—which the real Nixon didn’t do—they have to play puppet master. In the show’s timeline, the Nixon tapes that played a prominent part in the real-life Watergate investigation never leave the White House, and Nixon continues to record himself, with the For All Mankind writing staff supplying his lines. “With someone like Nixon, we felt like we had a lot of license because he’s literally on tape saying horrible things all the time,” Wolpert says. “So we felt like, ‘Well, he’s shown everyone that he is that kind of person, so we can also have him say some pretty bad things.’”
Deciding what Nixon would say wasn’t an issue, but figuring out how he would say it was. In the first season, the series was largely restricted to using impersonators who recorded fabricated, audio-only rants that played over a background of b-roll or still images. On rare occasions, video of a real Nixon speech was altered to make it look like he was saying something else, using a painstaking, old-fashioned, Forrest Gump–era method called lip flap. “You’d have to manipulate his mouth frame by frame to make it look like he was saying that, which is really exhaustive and never really looks perfect,” Wolpert says.
Finding the right people to play those parts was challenging enough. “We’re not casting a president, but casting the voice of a historical figure is almost as difficult, because that voice is sometimes just as familiar as the physicality of the character,” says Nedivi. The producers estimate that they listened to roughly 30 candidates for the parts of household names (and voices) like Lennon and Reagan, rejecting those who sounded like they were doing caricatures of those famous figures more so than straight-up, lifelike imitations. “It’s the people who more take on the feeling and the tone of the figure that really are able to capture it for our purposes,” Nedivi adds. Some experienced, prolific impersonators played multiple parts: Jeff Bergman took on Reagan and Johnny Carson, and Jim Meskimen played Nixon, Kennedy, and Lennon.
In Season 1, though, the voice-impersonated historical figures didn’t interact directly with the series’ original characters (aside from an occasional voice-only call), which prevented the familiar and fictional timelines from mingling in a more palpable way. “We knew if we were going to expand this element of the show in Season 2, we needed something better,” Wolpert says. That something was deepfaked footage that might not have been feasible when Apple ordered the first season in 2017 or when shooting commenced in the summer of 2018. “It’s been incredible to see the progress from the beginning, where we felt like it was going to be impossible, to today, where the technology is actually getting better with every year of the show,” Nedivi says.
To achieve the desired result, For All Mankind visual effects supervisor Jay Redd worked with a company called Canny AI, which is located in Israel and specializes in a deepfake-style technique called Video Dialogue Replacement. VDR permits a target video (such as a real Reagan address) to be altered in such a way that the lips and facial expressions of the speaker sync up with those of a source video (such as a Reagan impersonator recording original lines). Canny AI—which also provides a face-swapping service, in which a video’s dialogue stays the same but another speaker’s face is substituted for the original’s—demonstrated the potential of the technology in a 2019 initiative called In Event of Moon Disaster, for which the MIT Center for Advanced Virtuality commissioned the company to create a VDR-altered video in which Nixon delivers the speech he would have given in the event of an Apollo 11 disaster.
Filming for the first season of For All Mankind concluded months before that MIT project appeared, but the series’ producers teamed up with the company to create the effects for Season 2. Because of the coronavirus, the actors playing the parts of Reagan, Carson, Lennon, and others couldn’t come into a recording studio, so the producers sent them microphones and video cameras that they set up in front of themselves. (In some cases, voices and faces were done by different actors.)
Merging grainy, low-res footage from the ’80s with modern, high-res video made the process more challenging, as did the idiosyncrasies of some of the speakers. Gary Hart (played by Jon Briddell) was both tough to cast (because Gary Hart impersonation isn’t a booming business) and tough to VDR (because the former senator and presidential candidate was something of a mumbler). “We’re going from looking at people on the lunar surface, to spaceships in earth’s orbit, to doing close-ups of Gary Hart’s mumbling lips,” Wolpert says. “There’s just such a wide variety of visual effects in the show.”
Ultimately, the producers were able to use the deepfake technology on all of the altered speeches in the second season, including the brief Reagan Rose Garden address about a fictional hostage situation in Panama in the latest episode, “Rules of Engagement.”
Most of these scenes are subtle, but the second season includes at least two attention-getting deepfake-assisted set pieces. In the Season 2 premiere, astronaut Tracy Stevens (played by Sarah Jones) guests on The Tonight Show Starring Johnny Carson and announces that she’s gotten remarried (much to the dismay of her blindsided ex-husband, Gordo, who’s watching at home). Later in the season, Tracy appears on the talk show remotely, but in this scene, she’s on set, sitting right next to Johnny. “We created a part of the Johnny Carson set, had to have her sit exactly where a guest would sit there, and then had to merge the two, the reality and the tape from back in the day, together,” says Nedivi.
The producers got permission to use footage from The Tonight Show, and a staffer was assigned to binge old episodes in search of one where Johnny made physical contact with a guest in a way that For All Mankind could use. (“Lucky guy,” Wolpert says of the PA who drew the assignment.) The producers struck gold with a 1983 appearance by an 18-year-old Diane Lane, in which Johnny reaches out to examine the recent high school graduate’s class ring. In the sci-fi show, he makes the same motion to scrutinize Tracy’s wedding ring.
“The hand that is taking Sarah Jones’s hand in that shot is actually an actor’s hand, and then above the arm is Johnny,” Wolpert says. “That’s one of the visual effects shots I’m most proud of, because it looks seamless to me.”
In Episode 8, which will air in April, three recurring characters conduct a video call with Reagan. The VDR rendering isn’t done in real time, so the actors had to sell the fake Reagan their characters would be talking to sight unseen. As a spectator, Wolpert says, “You just kind of go with it. ‘Oh, they’re just talking with Ronald Reagan.’ And that to me shows the real promise of this, that you can bring real characters into the world of the show in a seamless way.” (No word on whether Moore will be deepfaking Walt Disney for his Magic Kingdom universe.)
It seems unlikely that the producers of a series stuffed with flashier effects—rocket and missile launches, space flight, firefights, moon walks, and more—would take such pride in plausible deliveries of fake dialogue. (Then again, as Nedivi and Wolpert admit, it also seems unlikely that two guys who started as staff writers on Entourage would be working on a sprawling, sci-fi, alt-history epic.) “Even though it’s not as in-your-face as some of the big ships or the Sea Dragon, I’m almost more impressed by this, the subtlety of it,” Nedivi says. “A lot of times when you’re creating something with visual effects that doesn’t have a historical reference, it’s easier. Whereas here, everybody knows Ronald Reagan, how he sounds, how he looks. And so really selling those moments can be just as difficult, and honestly more rewarding when we pull it off.”
For All Mankind was renewed for a third season ahead of its second-season premiere. Season 3 will take another time jump, which will create additional complications and possibilities as the actual and fictional timelines diverge. “In adapting history, there’s always going to be that challenge of making sure we don’t go too far, but at the same time having the ability to not limit ourselves in terms of the storytelling,” Nedivi says. “And as we move forward, I think that challenge will probably increase, because more people are going to be living that we’re talking about. But also I think more of our world is changing, so … I think we’ll become a lot more flexible in terms of our creative choices.”
The visual effects will have to become more flexible too. In Season 2, Lennon pops up frequently on news broadcasts—partly out of wish fulfillment for Moore, Nedivi, and Wolpert (big Beatles fans, all), and partly because they believe he would have been a valuable voice for peace amid the brinkmanship of their timeline. For a season set in the early ’80s, the producers could get away with using Lennon’s real likeness, but that wouldn’t work for Season 3. (“Now we have to find an older John Lennon that never actually existed in our history,” Nedivi says—which wouldn’t be a first.) And if the president in the show’s version of the 1990s is a real-life figure who didn’t actually become president and leave a library of televised presidential addresses to draw from—which is possible, considering that George H.W. Bush isn’t Reagan’s VP in Season 2—then it would take even more visual wizardry to make them look presidential. (Unless the president is a new character, or one of the series’ returning characters becomes commander in chief.)
In most respects, our solar-system explorers lag well behind the precocious space program of For All Mankind, but the series’ events still resonate today. More water is turning up on the moon. NASA is trying to protect astronauts from solar storms like the one that endangered the crew of the moon base in the second-season premiere. The U.S. and Russia are clashing over the use of weapons in space. Thus, the series comments on our present and previews our future even as it revises our past, in a way that wouldn’t be as powerful without the latest visual advances.
The real Reagan once said, “We’ve grown used to wonders in this century. It’s hard to dazzle us.” The real Reagan never saw himself deepfaked. For All Mankind isn’t the only screen project that’s drawing on deepfakes to make its magic happen, but its blend of events that never occurred and events that (one expects) are still to come makes it an exciting and disturbing demo of the bleeding edge. As Nedivi says, “I feel like we’re riding the wave of the technology with this show in a way that, as the show continues, this will only get even more effective.” And maybe more scary.
Correction: An earlier version of this piece incorrectly stated who played Richard Nixon and John F. Kennedy on the show. The impersonator was Jim Meskimen.
