AI Researcher Answers: Is AI’s Existential Threat Real?
NEWS | 10 October 2026
Transcript Timnit, thank you so much for being here and joining me on The Big Interview this week. Thank you for having me. A few weeks ago, a young AI researcher at Anthropic basically stepped down and in his resignation post on X indicated that a lot of people within the company feel that we are facing this major existential risk with AI. And then a few people responded and there was this whole pile on and discourse about the real existential threats of AI. And when you and I spoke in the immediate aftermath of that, you said that you felt that particular narrative was a distraction from the real problems that we're facing with AI. You know, I would even go further than that and say that it's dangerous. It's not just a distraction, but it's harmful discourse. And so the reason being, before I answer your question, you know, I try to do this, I want to give a little brief history of how this narrative has been going on for a long time because I wrote a Wired op-ed in 2022. We wrote a statement in 2023 when this happened. Elon Musk and Peter Thiel have been saying this since 2013. Prominent people have been saying that AI is an existential risk to humanity. And the way it goes is usually, and this is exactly the same rhetoric every three years. So I can regurgitate whatever I wrote 10 years ago to respond to these things. For example, the Future of Life Institute, which now is advising- And what is that? Yeah, the Future of Life Institute is an institute that was founded by Max Tegmark and Jaan Tallinn, a billionaire who led Anthropic Series A funding. And he was a Skype co-founder, correct? A Skype co-founder. Jaan Tallinn also funds METR, which is an auditor, a so-called third party, whose report went viral about these rogue agents hacking Hugging Face. So to the public, it might seem that there are so many different entities coming together saying the same thing, but just like the tobacco industry and fossil fuel industry playbook, there's a playbook here where the same billionaires who founded and funded Anthropic stand to benefit so much more from its IPO more than anybody else, also founded and funded the institutions warning about existential risk, quotation marks, of AI, also founded and funded the third parties that are being cited right now as if they are, like, an independent entity. So that I want to ground the public in that fact. And I can say more about that. Tell us why you thought the machine got narrative. It's a distraction. It's even more than a distraction, it's harmful. For example, the reason I was telling you all these facts is that these people, the funders, the founders, the investors who stand to benefit and profit the most from these companies IPOing are saying this, that they have been the ones seeding this narrative going back multiple decades actually. People should ask why. If I stand to get lots of money from a particular company, why do I seem to be the person saying this particular company might build something that kills us all, right? It sounds counterintuitive. But the first thing is that if you convince the public that you're building something so powerful that it's beyond even your control, it's beyond something that we've seen before, it's beyond something that we don't have existing regulation for. And this is why I have a lot of respect for Lena Khan, because one of the things she said is that there's no exception to the current laws that we have for AI companies, and that they talk as if there is an exception. And you're saying that whatever you're building is so powerful that it's beyond anything we've seen before. Right. It's self-aggrandizing. It's saying we're the future makers here, but the future is going to be really scary. They don't always say, the same people, you have to pay attention to what they say in the news cycles. When they say that AI can cause an existential risk, they also say that AI can stop climate change, bring us world peace and eradicate poverty. AI can do all of these things because it's powerful. It can bring us utopia, but also if the wrong people do it and we don't have guardrails or whatever, it can also just kill us all. So the, the same people believe these things. There's only a few people who might be in one of these camps. When you're telling people that you have the super powerful machines, what you're speaking to, first of all, is to your investors. You're saying that, you know, you are, you want to get your hands on these super powerful machines. You're talking to government saying, you don't want your adversaries to get their hands on these super powerful machines, you want to get your hands on these super powerful machines. And you're also telling any regulating bodies that whatever regulation they're thinking about should be about these fictional, super powerful things. And imagine, if I'm talking about potentially eradicating all of humanity, things like pollution, data centers, they sound kind of small potatoes, right? You're talking about copyright protection for artists, actual lawsuits that are going on right now, government should not waste its time thinking about this stuff. If you waste your time regulating us on these small topics, you risk China getting its hand on these super powerful machines. And you don't want that because at least us, trust us, we can build the machine that can help us. And we're telling you we want to be regulated, but you don't want China to be the one. Right. That's the first one. The first one. The second one. Second issue is you can always abdicate responsibility. In our prior conversations, we've talked about the usage of certain language or phrases to describe this modern era of AI, and how you take issue with some of them. So for example, in our previous conversation, you said something about autonomous weapons. After that was published, you told me you later regretted saying that because you would've phrased it differently. We've talked about P[doom]. We've talked about how when you say that agents have gone rogue such as OpenAI, you're essentially removing culpability from the engineers who built those tools. I can't help but wonder if maybe there are groups that are. We've even talked about the word alignment, so I was just going to say I wonder if there are groups that are actually more aligned than they realize around safety and transparency, but who are deploying different phrases to describe what is essentially the same thing. Some of the reasons that I regretted saying these words is not that the term autonomous weapons has, I have any issue with that. It's a legal term that has, even landmines are considered autonomous weapons legally. It's just that the people in the so-called existential risk camp have co-opted it to mean something different. When I said the term, I knew, I'm like, huh, I wonder if this is going to be, taken in that way and it was because the people talking about existential risks of AI took it to mean what they're saying, right? Which is like some kind of a superhuman, super intelligent machine doing stuff on its own or whatever. Just to bring it back to the latest conversation about this, I was at the UN General Assembly this week. I was a part of some side panels, so I wasn't actually in the UNGA nor the Security Council, but there were some prominent tech folks there, including Sam Altman from OpenAI and Dario Amodei from Anthropic. Dario reportedly said, If managed poorly, I even believe AI could be a risk to humanity as a whole. As competitors, Sam Altman said, We could lose control of the future to AI. So explain how it is that these folks who once again have these, you know, companies that are valued at nearly a trillion dollars or more are now here on the world stage at the UN saying, hmm, I don't know. There could be some risks here, right? You know, is this part of regulatory capture? Are they going to be looking for ways to self-govern before there's a regulatory crackdown on this? Absolutely. It's part of regulatory capture. And in the same paper where I introduced a test reel bundle, I talk exactly about this regulatory capture and especially what has happened at the UN. Briefly after, you know, Sam Bankman-Fried, as you know, was an effective altruist and briefly after he went to prison, the effective altruists were briefly scrutinized by journalists and journalists were talking about, was a political article, for example, about the regulatory capture. Let's talk about right now, what can be done? What kind of governance do you think is needed for AI? So I mean, we've been talking about, regulatory capture has been happening for a long time. 2023, same thing. Sam Altman said the same thing. We need the world cooperation, et cetera, et cetera. The EU AI Act regulated, and then he threatened to pull out of the EU. So you just have to see what they've actually done. When there's actual regulation that holds them liable, they threaten to pull out or they lobby super hard to water down that regulation. On the other hand, they're going around telling these multilateral bodies that there has to be world cooperation, et cetera, et cetera. And then again, what is this doing? It is A, selling themselves as organizations that are creating super-intelligence, right? And so that's already marketing. And B, they're making everybody scared of anybody else who might be creating such things. So they don't want open-weight models from China, they don't want this competition. I am not following Chinese regulation that closely, but they're not talking about existential risk. They're talking about deep fakes and they're talking about, like, real things that need to be regulated. Mm-hmm. So the first one is marketing yourself as creating some super powerful, unprecedented things for which we don't have existing regulation, which is not true, we have existing regulation. But the second one, is what they're doing, is when someone tries to enforce the existing regulation, they either threaten to pull out or they lobby hard so that they don't have this existing regulation. And so the reason I keep on going back to 10 years ago, five years ago, three years ago is that the same, in my book, I say the same movie on repeat with different heroes. So what is the solution for governance right now if you had to propose it? Very, very simple things. First of all, Lena Khan outlined five things. One is deceptive marketing practices. There's a law for deceptive marketing practices and you can go after companies for that. Two, is transparency, documentation. Before you put something out there, you should be able to tell us where all the data came from and actually document. This simple thing they don't do and they will never do. I'm going to tell you that they will fight tooth and nail to do the simple thing of documenting data. And labor exploitation of data workers, that's another one that they don't want to talk about. We are here in the world, in the clouds, talking about super-intelligence where you have hundreds of millions of people around the world, painstakingly labeling data, even pretending to be chatbots. There are data workers called AI impersonators, right? We now know that OpenAI hires people to look at your conversations with their chatbots. So people should be careful. Don't believe that, you know, you have privacy. Right. 404 Media just recently reported that the new Meta Muse chatbot is actually a person on the other end who's responding. Yeah. Things like this. Very simple. Data transparency, labor exploitation, you should not be able to steal data from people. Even the first three things I talked about, data transparent documentation, labor exploitation, if they had to abide by laws like that and they were not allowed to steal data and not document it, the market calculation, right now, the market calculation is not working, but with these additional measures, it just would not work whatsoever. Right, so you would automatically slow them down and have to make them accountable for something. Back in 2021, you co-authored a paper that was titled On the Dangers of Stochastic Parrots. Google had approved it initially, then it was being reviewed. There were parts of it that were in dispute. You said, Look, if you want me to remove my name from this, I'm not going to be a part of this. You ended up decamping from Google. That paper was later presented at the 2021 ACM Conference on Fairness, Accountability and Transparency, and people still reference it. Can you briefly explain for someone who's never heard of stochastic parrots before what it has to do with the AI we're using today, what it means? Stochastic parrots is a metaphor to help people understand what large language models do. And large language models are trained on vast amounts of textual data on the internet, trained to output the most likely sequences of text given their training data. They power most of the chatbots that we see today, whether it's cloud, whether it is ChatGPT. But when I wrote this paper ChatGPT hadn't come out yet, but we saw the race to build larger and larger language models. And so that's the danger of building larger and larger language models that we were describing in this paper. That they would essentially parrot people? To parrot is to repeat back without understanding, right? So there was this whole, this whole existential risk narrative was happening back then too, if you can believe it. And so there was all this conversation about how OpenAI had claimed that GPT-2, the precursor to GPT-3 that powers ChatGPT was too dangerous and too powerful to release. There was this whole conversation about whether GPTs can be ethical or are they creative and all this stuff. And so we really wanted to ground the conversation in the real issues. One of them, one of these issues is the environmental catastrophe, which a lot of people are now seeing, but we discussed it back then. And that was one of the main sections that Google people were unhappy with, the environmental costs. The other one is not documenting your data because you say you have too much data to document. The other one is deceiving people into believing that there is a mind behind the textual outputs that they're interacting with. And so there, that's where we really wanted to explain that these systems are parroting the patterns of their training data. And it's very dangerous when you're outputting text like that because when you have a plausible sounding text or very fluent text, there's so many different kinds of issues that can occur besides you believing that there's a mind behind a machine. I gave, in that paper, we gave an example of this Palestinian man writing good morning, which was translated to attack them. And because of that grammatical correctness and there were no cues that the translation could be wrong, and people believed the translation. So the other issue of believing there is a mind behind the machine is what we call automation bias. You over-trust automated systems, and if you believe that this thing is an all-knowing machine, then you're going to over trust the errors that you get, right? [Lauren] That's so interesting. So we're seeing this with medical scribes where I was just reading an article, another article where medical scribes that were summarized said this woman was micro-dosing on mushrooms, since this poor woman has never heard that, never discussed mushrooms, never done mushrooms. She saw it on the notes. So none of the doctors, nobody checked because again, if you believe, this is why I believe the super-intelligence existential risk discussion is not just distracting, but dangerous because we, with automation bias, with over trusting these machines already, if you believe that they are nearly super-intelligent instead of error prone, large language models parroting things, then you're not even, you're going to be less likely to check, you're going to be less likely to put checks and balances and regulation. And you're seeing things like misdiagnosis Medical errors. based on medical errors and things like that, which are serious. It's funny, I must be too much of an elder millennial because you say that there's too much automation trust and I'm like, I'm so distrustful. I've got, call the bank and it's a robot, I'm like, nope, nope, I have to. I don't want to talk to it. Totally. Yeah, exactly. So one of Anthropic's co-founders, Jack Clark, recently posted something on X. You know what I'm going to say. Yeah, I do. He basically, he puts stochastic parrot in quotes and said he, It was a mimetically fit cognitive virus that spread from 2021, when your paper was out, to 2025. It temporarily blinded many gifted people to the nature of AI progress, burned up crucial years of research, he says. He later says, The use of this frame causes people to materially underestimate what AI systems can and can't do. When you saw Jack's post on X, what was your initial response to that? I was not surprised, by the way, let me just tell you, because the effective altruists have been. I was telling people that this is a talking point that they're telling lawmakers now, because every time I said something, they were like, Oh, you should not take seriously someone who still takes the stochastic parrots things seriously in 2026. So this has been a talking point of theirs for a while. Right. So the idea is that the research is outdated, right? Yeah, and it's not. And that's what I want to say. It's so ludicrous that we're even saying this because these are definitions of what large language models are. What large language models are has not changed, will never change. Like, large language models are large language models. Now, you might have large language models in a separate system that's trained in a, you know, they have now reinforcement learning agents, but our paper was about large language models and that's never changed. And these chatbots still have large language models as a basis. And we are seeing a. It's so ridiculous that Jack Clark is talking about overestimating systems because what I'm seeing is the examples that I just told you. It's over-trusting these systems, not having checks and balances and ending up misdiagnosing someone's breast cancer to the wrong side or - So there's a direct line between LLMs being stochastic parrots and misdiagnosing, giving a medical misdiagnosis using an AI tool because why? How does that actually. How does one lead to the other? Because large language models are stochastic, the stochastic parrots, they don't understand what's inside the text. So you cannot expect them to be factual. Even if you see something like the AI overview, I had another example, where I was calling an oncologist friend of mine to ask about a specific medication and whether it was appropriate for a specific use, because I read the academic paper saying that it was not, and I wanted confirmation. And my oncologist friend was like, Oh, look at the Google overview. It says that it's appropriate. But I read that, but it turned out not be. Why? I encounter that all the time with Google AI overviews. Because they are stochastic parrots trained to give you the most likely sequences of text based on their training data. So this is not grounded in a world models. Emily M. Bender has been trying to say this in so many languages for a longtime. Also one of the co-authors on the paper, correct? Co-first author with me to say that the research is outdated is ludicrous because right now, as we are hyping up these super-intelligence and all that, there are news stories that are going unnoticed, which is about the complete opposite scenario that is actually happening in the real world. My understanding is that some of the biggest critiques people have had about focusing so much on research from that era may not be incorporating the thinking or the reasoning or even- There's no reasoning. Or even the recursive intelligence that- There is no recursive intelligence. Is there none? There is no thinking. How is there none? This is something, by the way, that at the Berkeley AI Conference earlier this summer, I heard someone from Google talking about recursive intelligence. It's something that- Of course they are. The, you know, The New York Times just did a big story about how the, you know, the scientists and researchers right now, that's what they're looking towards, recursive intelligence, the AI learning from other AI. Well- Is that a reality? It sounds like you're saying that's not a reality. No, because, you know, there's one problem with AI researchers, which is aspirational naming and aspirational stuff. So machine learning, that's aspirational naming. But machine learning is real. The naming is aspirational. Okay. The machine not necessarily. So this happens all the time, right? So just because there's curriculum learning in AI, it's a field. But let me give you a paper from my former manager, Samy Bengio, and I ask him, aren't you tired of your whole life being, like, debunking the whole reasoning thing every single paper you write? He quit after I got fired from Google and he's now head of machine learning research at Apple. And if you look at almost every single paper that they have, it's showing how if you change the benchmarks on reasoning slightly , the whole thing breaks down. It's not reasoning. It's not reasoning. 0 Just because you're looking at chain of aspirational naming. Let me give you another example. Just because your models are largely, the stochastic patterns that you trained to print out certain tokens, you call them chain of thought reasoning. You didn't know that they were thinking. You don't know it's a chain. You just know that these are tokens that are being printed out, but you called them chain of though reasoning. Now you're saying that they're reasoning already. One of the biggest crises that we have right now is actually sound scientific research. So if you look at my work, if I ever have access to the data, the code, the training data, and the evaluation data, which none of these companies give you those things. You don't even know whether they ingested that benchmark during training or not. If you ingest a benchmark during training, it's like studying to the test. It's like me coming to an exam, knowing what the answers to those 10 questions are already, studying that and writing it down, right? So every time I have had access to these things, I have shown how their claims are not correct. But then with Samy Bengio and his team's paper, what they showed is, and this is from 2025, what they've shown is that you just change the benchmarks a little bit, tweak it a little bit, and it all breaks down, showing that you were just sort of relying on the patterns. Now, some people, when I give them this example, they're like, well, we're talking about 2026 models. Guess what? Real evaluation in science takes time. We should not be going from press releases to lawmakers parroting those press releases and journalists repeating those claims. If you want to do real research and evaluation, ask for the training data, the evaluation data and the methodology so we can all reproduce it. Mm-hmm. And so you are ascribing the thinking, reasoning, recursive intelligence. You, it sounds like you still see that as something that is purely human. It's ascribed to us humans. It's the way our neural processes work, but the AI doesn't work that way yet. That's what you believe. No, and I don't know if it'll ever. Like, intelligence, that's aspirational naming. And sometimes I play, just to take people back to these hype cycles. I tell them something, a claim that was made, and I say, Is this 1954 or 2024? What would you say is the biggest part of your own thinking, your own research that has evolved since the early days, 2020s, of AI till now that has surprised you the most? I have to be actively making space for the kinds of models that I think should be built and building them, and sometimes ignore the noise. That's the conclusion I'm getting to over time. That's your biggest learning, your biggest takeaway. Yeah. There is so much noise online right now, it's hard to get any deep work done if you're paying attention to whatever the AI guys say at this point. And even if your research is all about debunking what they're saying, you know, it gets tiresome, it's not fun to do that. Right. You know, it's more fun to think about the future you want to have, the technological you want to have, and work on that. Before I let you go, what would you say gives you the most hope right now for the future of AI? I want to tell you that we have a list called the AI Resist List where we talk about people resisting in the ways that they should resist, in many ways, which is media capture, narrative, data centers funding, et cetera. And so we list the ways in which they're resisting. And also people of creating alternative tech futures that don't kill our environment, that don't kill, exploit labor or steal data, and instead are actually actively helping their communities. And these ideologies are spreading, right? You see one small organization somewhere doing something, you get inspired by them, you do something different. So I have hope that there are a lot of people tired of what they're seeing and they're actually in small circles doing something different. I believe in human agency and collective power to imagine a better future and stop bad things from happening and bad things if they're bad, right? So my belief in human agency, I think, is what gives me hope. Thank you so much, Timnit. I really appreciate you joining me today for The Big Interview and for sharing your insights and congratulations on your upcoming book. I look forward to reading it when it comes out early next year, and I'm sure we'll chat again soon. Thank you, thank you so much. [gentle music]
Author: Condé Nast.
Source