Generative AI Couples Up with Video: Cute or Cringeworthy?
Last year, I went on a bit of a dating show binge. Love Is Blind. Married at First Sight. The Ultimatum. Love Island. You name it. Each show has its own unique formula for how to inspire love to blossom on camera.
During my binge, I watched flavors of dating shows from all around the world. But I was hesitant to enter the realm of dubbed dating shows. Why? There is a little something lost in translation when the words being spoken don’t originate from the speaker themselves, like listening to an audiobook through a narrator who is someone other than the author who actually lived the story being told.
Enter Deep Fake Love. As a dubbed dating show, it wasn’t at the top of my dating show binge list, but working in AI, I just had to experience the combination of my love for dating shows and AI. So I watched the show … in awe.
If you’re not familiar, the Netflix description for this show is: “Five couples put their trust to the test in this steamy reality series, where deepfake technology blurs the line of truth and lies in a cash prize game.“
So, what are deepfakes? Let’s head to Wikipedia:
“Deepfakes ([a word formed by combining two concepts] “deep learning” and “fake”) are synthetic media that have been digitally manipulated to replace one person’s likeness convincingly with that of another … Deepfakes leverage powerful techniques from machine learning and artificial intelligence to manipulate or generate visual and audio content that can more easily deceive.”
In short: deepfakes are AI-generated imitations.
Deep Fake Love tests the strength of five couples’ loyalty, while each couple is separated into two different houses to live among a slew of singles. So, where does the deepfake technology come into play? At various points in the show, couples are shown video footage of their partners’ conduct in the other house and must decide if the video is real or fake. What’s on the line? A cash prize. (And, of course, the health of the relationship itself.) I can only describe Deep Fake Love like the 2010 reaction to Gmail’s message threading, “like cilantro … you either love it … or you hate it.”
Although I only began looking into deepfake technology after watching Deep Fake Love, the idea behind deepfakes is not new. People have been convincingly manipulating photos, voices, and videos for a long time. Plus, deepfake technology itself has its beginnings in the 1990’s according to Wikipedia (more on this in the next section of this article).
So is the coupling up of generative AI and videos, begetting deepfake technology, cute, or cringeworthy?
Here’s a cute example. My daughter and I played around with video manipulation using the Reface face-swap app. Here’s my daughter’s face as Tom Cruise in the movie Knight and Day:
And here I am as Jim Carrey in the movie Dumb and Dumber (orange suit if you aren’t sure :) ):
But deepfake technology can also be cringeworthy, like in the case of the news story from a couple of months ago about videos that use the AI-generated likeness of actress Scarlett Johansson without her authorization or consent. Seriously cringeworthy! And, even back in our dating show example of Deep Fake Love, at least one author found it to be cruel and manipulative.
So, what makes deepfakes possible? What are the pitfalls of deepfakes? And how can everyday people protect themselves? Let’s take a look!
What Makes Deepfakes Possible?
According to Wikipedia, deepfake technology dates back to the 1990’s. However, while scrolling through the internet, I came to understand that the acceleration of deepfake technology, especially as it relates to synthetic video generation, is credited to the introduction of Generative Adversarial Nets (GANs). And because I work at an AI company, I casually asked my co-worker to confirm whether or not this was true. Turns out, it’s true!
So, what are GANs? GANs involve training two different, competing machine learning models at the same time: a generative model and a discriminative model. In the academic paper that introduced GANs, Generative Adversarial Nets, Goodfellow et al. (2014) provided these helpful analogies to further explain this concept, before diving into the math behind it:
The generative model can be thought of as analogous to a team of counterfeiters, trying to produce fake currency and use it without detection, while the discriminative model is analogous to the police, trying to detect the counterfeit currency. Competition in this game drives both teams to improve their methods until the counterfeits are indistinguishable from the genuine articles.
In the paper, the authors visualize samples from the generative model (shown in the rightmost column) after it was trained on four separate datasets:
- a) MNIST (a dataset of handwritten digits)
- b) TFD (Toronto Faces Dataset that doesn’t seem to exist any longer)
- c) and d) are from different flavors of CIFAR-10, a dataset representing these classes:
airplanes,cars,birds,cats,deer,dogs,frogs,horses,ships, andtrucks
The authors offer this note: “we believe that these samples are at least competitive with the better generative models in the literature”. Although I’m not familiar with the generative models from 2014 (10 years ago!), looking at the images, the AI generated samples are impressive indeed.
Before my research, I was picking up negative vibes about GANs — the word “adversarial” is in the name, and they are at the heart of deepfakes — but I knew my AI engineer colleagues were very excited about them, so I had to find out: why were GANs created?
Turns out, MIT Technology Review reported on the inside scoop. GANs were created out of a friendly dialog between Ian Goodfellow (“the GANfather” and lead author on the GAN paper I’ve been referring to) and his friends who needed “a computer that could create photos by itself.”
Ok, let’s dig in a little further. Why would computers need to create their own photos? I’ll share a quote from the MIT article, but working in AI, this is well-trodden territory for me!
“Today, AI programmers often need to tell a machine exactly what’s in the training data it’s being fed — [such as] which of a million pictures contain a pedestrian crossing a road, and which don’t. This is not only costly and labor-intensive; it limits how well the system deals with even slight departures from what it was trained on.”
Amen! Because the process of generating high quality training data for the machines is expensive, labor-intensive, and sometimes there’s just simply not enough data for your model to produce quality results, creating the technology to enable computers to generate their own data is truly a noble endeavor.
But, just like any technology out there, what was once created from good intentions is reapplied for lesser noble causes. And now, we live in a world where AI-generated content can be both helpful and harmful.
Beware the Dangers of Deepfakes
I started the Raising AI publication that you’re reading as a way to raise our own understanding of AI, and to discuss how to nurture healthy relationships between people and AI for ourselves and our families.
Imagining how deepfakes might maliciously convince and influence the people I love, my first instinct was to begin helping them (and myself) learn to detect deepfakes so they don’t fall for it.
As it relates to deepfakes today, there will be some elements of reality that, just like dubbed dating shows, will be lost in translation during the AI generation. Based on the Norton article “What are deepfakes? How they work and how to spot them”, and the MIT Media Lab article “Detect DeepFakes: How to counteract misinformation created by AI”, I put together a mashup of the advice, adding a little of my own, shown where on the face the generated anomalies will occur. (There aren’t any anomalies in this sketch, I simply used it as the foundation for this informational graphic.)
Want to see if you have what it takes to detect deepfakes? The MIT Media Lab article points to this handy site where you can practice your deepfake detection abilities and see your results. I found out I can be tricked, but the experience was a lot of fun!
In addition to beefing up your deepfake detection skills, there are many other ways people can protect themselves. The National Cybersecurity Alliance recently published 10 ways to protect yourself against deepfakes, including limiting the amount of data you share about yourself and your family online, avoiding phishing, and how to report deepfake content so it can be investigated and eradicated. It’s a great resource and I encourage you to check it out!
Just for Fun
Somewhat related to detecting deepfakes, and just for fun, I tried out the Microsoft Bing Copilot image generator feature by asking it to create an image by passing the prompt a few characteristics I identify with, and here are the options that came back. Can you spot any issues? 🤣
Summary & Conclusion
Let’s quickly review where we’ve been so far on our journey in this post. After watching a dating show, I became inspired to learn about deepfakes. Deepfakes are made possible by generative AI based on the exciting GAN technology created by Ian Goodfellow and fellow researchers/authors who set out to solve a common problem in AI programming. But AI generated video, just like anything, can be used for good or bad (cute or cringeworthy to use our storyline so far). It’s possible to have some fun (like the example of using the Reface app with my kid), but it’s also improper and simply flat out wrong to use someone’s likeness without their consent and to use it maliciously.
What can we do? The good news is, as the saying goes, if something doesn’t feel right, it probably isn’t. So far there are some limitations of AI generated imitations, so please do inform yourself on how to detect deepfakes, and stay abreast of ways to defend against fake content in order to keep yourself, your friends, and your families safe.
The purpose of writing this blog post was share what I learned while researching deepfakes so that any reader can raise their own understanding of the topic, get a glimpse into the implications that fake content has on individual people and society, and get pointers to some resources with tips on how to protect yourself against against misleading and malicious deepfakes.
However, perhaps the most important point of all — we are all on this journey together. As you uncover new insights into deepfakes, please comment on this post or reach out to me on LinkedIn and let’s continue the conversation.
Finally, if you like what you see in this post, I’d be honored to have you follow, clap, and/or comment. Thank you!
