“The story of 2022 was the emergence of AI,” wrote Ben Thompson, the author of the Stratechery newsletter and podcast. “It seems clear to me that this is a new epoch in technology.” Ben and Derek talk about ChatGPT, Stable Diffusion, the state of generative AI, and how the biggest tech companies will try to wrangle this fascinating suite of new tools.
If you have questions, observations, or ideas for future episodes, email us at PlainEnglish@Spotify.com. You can find us on TikTok at www.tiktok.com/@plainenglish_
In the following excerpt, Ben Thompson discusses why he thinks AI was the story of 2022.
Derek Thompson: In a recent article for the Stratechery newsletter, you said that AI was the story of the year in 2022. Why?
Ben Thompson: Well, it’s something that’s been burbling under the surface. Like a lot of things in tech, it takes many years for something to come to the surface. I think it was around 2017 or ‘18. There’s a group of Google researchers that publish this paper—now that name’s escaping me … it’s something very clever, like “Attention Is All You Need,” or something like that—that details this new capability called the transformer that completely transformed the way that data was processed for machine learning models and for AI. I mean, it’s difficult to overstate what a big shift that was. So that’s actually the story, but that’s a good example of something that happened five years ago, didn’t emerge until 2022. What happened in 2022, I think, was a series of things. So the first thing was from the public perception of AI, the first was the emergence of DALL-E 2.
Again, a good example of why it takes a while for this stuff to emerge, it wasn’t even the first version, it was the second version—where you could make these incredible images. And this came from OpenAI, and on one hand it was an amazing capability. On the other hand, it fed into the conventional wisdom around AI, which is that it was going to be centralized, it was going to be these entities that had a ton of power, computing power and data processing ability that would dominate. Then over the summer came Midjourney, which was a startup, an unfunded startup; it self-funded and they had arguably even better or different, very compelling image generation run via Discord. And then the big bombshell in the image generation space was Stable Diffusion, which was open-source and it could run on a single GPU on your own computer.
And that was a really big deal, because it opened up the possibility that this AI capability is maybe going to be more distributed than we thought, which changes all sorts of things like what the competitive landscape might look like. Is this going to be a centralized thing? Is it going to be much more of a commodity? So that’s the image generation space, and the thing with images is, an image is worth a thousand words—which we’ll get to in a moment; that’s actually a very important insight—but images are so evocative, and you could see that and realize that this was a big deal. The biggest release of all, though, was ChatGPT, which I’m sure all your listeners are familiar with. And again, ChatGPT is, underneath the surface, built on GPT-3 and it’s an evolved version of it.
They call it GPT-3.5, but GPT-3 came out in 2020, and at that time if you were plugged in, it was pretty mind-blowing what it could generate. But ChatGPT productized that. It used this method called reinforcement learning with human feedback, where basically humans were in the loop, really guiding it towards not just controlling what it did or did not say, which is obviously a controversial topic, but also how it’s said. Really shaping it, so it would give you high-school-quality essay answers. A lot of GPT answers, you have a topic sentence, you have supporting point one, supporting point two, supporting point three, and the concluding sentence. And it’s easy to mock that, but you realize there’s a reason we teach people to do that: It’s an effective way to communicate, it gets the point across, and it was in this super-easy-to-access chat interface that really woke people up. There’s a lot of interesting lessons. So I think the reason why 2022 is the year of AI is because it was a wake-up publicly about what was coming, even if that stuff that was coming had actually been burbling for a while. And there’s lots of interesting takeaways there about where this can be done, the importance of products. The thing about the language models is they are more complicated. You can’t really run them locally to the extent or quality you can GPT-3. I think ChatGPT, one question runs across 16 GPUs as opposed to one [for DALL-E 2], and part of that is the thousand-words thing—language is more complex, it’s harder to get it right, and so that is still fairly centralized. But anyhow, I’m diving into 47 details in one answer. It was the emergence of this in the popular consciousness, in the realization that this is a lot closer than people realize that made it such a momentous year.
Derek Thompson: You had a great menu there, and we’re going to dive into some of those appetizers and entrées in just a second. I want to begin by saying I have friends who aren’t as interested or aren’t as deep into tech as I am, and one of their questions to me about AI is often “How is this not just crypto again? How is this not just the metaverse?” And when they say that, what they’re saying is how can you ensure, how can you promise me, this isn’t something that people are really excited about for three months in early 2023, but nine months from now it’s not going to be a thing at all? What do you say in response to that “it’s just a lot of hype” case?
Ben Thompson: Well, I think the most damning thing for crypto is the fact that it has been around for 15, 16 years and there has yet to be a single demo or use case that’s as compelling as any of the ones that came out last year for AI. And so, there is an extent where machine learning has obviously been a thing for a long time. Transformers, as I noted, have been around for five or six years, but it is meaningful that the use cases that we have already seen, the demos we’ve already seen, the quality of a demo and the degree to which it grabs people’s attention is, I think, a meaningful signal. And the reality is that ChatGPT is more compelling than basically anything crypto has developed in 16 or 17 years. And a lot of the crypto use cases are compelling to the extent they are in the back end and hidden from users. And even then, there’s not really many good demos.
This excerpt was edited for clarity. Listen to the rest of the episode here and follow the Plain English feed on Spotify.
Host: Derek Thompson
Guest: Ben Thompson
Producer: Devon Manze
Subscribe: Spotify
