Anthropic CEO Dario Amodei argues open-source AI cannot be monitored, revoked, or patched for safety after release
Story Overview
A 2023 Senate testimony clip from Anthropic's Dario Amodei has resurfaced to sharpen the divide over open-source AI, stressing that once frontier models escape controlled environments the original developers lose any practical way to watch for abuse, pull them back, or push safety fixes.
Closed models keep the keys in one place
Amodei contrasted open releases with closed systems where companies can still monitor misuse and apply patches, arguing that only the latter approach scales safely to the most capable models.
Frontier distinction leaves smaller models untouched
The testimony notes open-source approaches have worked fine for modest models so far, but it leaves unresolved how regulators or labs should draw the line when capabilities cross into higher-risk territory.
Many users insulted Anthropic's CEO and dismissed his open-source AI danger warning as profit-driven recklessness, while some praised the take as solid and correct.
No Digg Deeper questions have been answered for this story yet.
Most Activity
"The scaling of open source models, I think it's going down a very dangerous path. And if the path continues, I think we could get to a very dangerous place. I"
~ Dario Amodei's 2023 U.S. Senate testimony
Anthropic CEO Dario Amodei on Open-Source AI Models.
"I don't think open source works the same way in AI that it has worked in other areas. Primarily because with open source you can see the source code of the model. Here we can't see inside the model, it's often called open weights instead of open source to kind of distinguish that. But a lot of the benefits, which is that many people can work on it and that it's kind of additive, don't quite work in the same way.
So I've actually always seen it as a red herring. When I see a new model come out I don't care whether it's open source or not. If we talk about Deep Seek I don't think it mattered that Deep Seek is open source. I think I ask, is it a good model? Is it better than us at the things that matter? That's the only thing that I care about.
It actually doesn't matter either way. Because ultimately you have to host it on the cloud. The people who host it on the cloud do inference. These are big models, they're hard to do inference on.
When I think about competition I think about which models are good at the tasks that we do. I think open source is actually a red herring.
It's not free. You have to run it on inference and someone has to make it fast on inference."
---
From 'Alex Kantrowitz' YT channel (full video link in comment)
For every software provider, your biggest competitor is the customer themselves -- building it for themselves and thereby reducing reliance on you. Instead consider making a net positive business that helps your customer, instead of trying to rent seek forever
🚨ANTHROPIC CEO: OPEN SOURCE AI IS GETTING DANGEROUS
Anthropic CEO Dario Amodei told lawmakers that open-source AI is moving down a “very dangerous path.”
His warns that once powerful models are released openly, companies lose the ability to monitor misuse, revoke access, or update safety guardrails.
Dangerous for someone's margins
ANTHROPIC CEO WARNS OPEN-SOURCE AI IS ON A "VERY DANGEROUS PATH"
mcdonalds CEO says making burgers at home is moving down a "very dangerous path"
BREAKING: Anthropic CEO says open-source AI is moving down a “very dangerous path.”
History doesn't repeat but it rhymes
Anthropic is the new late 90s evil Microsoft.
Dangerous to Dairo’s need to make profits.
BREAKING: Anthropic CEO says open-source AI is moving down a “very dangerous path.”
skill issue
BREAKING: Anthropic CEO says open-source AI is moving down a “very dangerous path.”
@uncledoomer Having burgers at home will always be better than giving my order to Shaniqua smacking her lips and cussing while I'm trying to simply order food.
Give your customers the tools that they need to improve themselves and become capable, build a relationship with them. It doesn't scale and it isn't easy. But this is precisely why this is a better business model than attempt to regulate their ability to help themselves..
For every software provider, your biggest competitor is the customer themselves -- building it for themselves and thereby reducing reliance on you. Instead consider making a net positive business that helps your customer, instead of trying to rent seek forever
@kimmonismus Open source must win. If Dario and others have their way, only small % of humanity will have access to top tier AI.
Major issue is hardware tho, we need way more competition in both processing and ram so one company/country cant just decide to monopolise it.
If Folks Like Dario Had There Way, There Would Be No BASIC Computer Language And No Personal Computer.
Why?
It’s too “dangerous”. I mean terrorists and child abusers could use them.
BASIC’s legacy is not nostalgia; it is a blueprint vision that every student and citizen should have access was radical and correct. We must apply the same principle to AI: make the best models as accessible as possible, with safeguards developed collaboratively and transparently, not imposed top-down.
Support open weights where feasible, fund auditing and red-teaming in the open, build local and distributed infrastructure, and resist narratives that paint democratization as recklessness.
If we fail, future historians will look back and ask: What if they had treated AI like BASIC? Where would humanity be?
The answer is clear—we would be far poorer, less innovative, and less free. The spirit that turned teletype terminals into a personal computing revolution must guide us now.
Code for everyone.
Intelligence for everyone.
The garage awaits.
http://x.com/i/article/2071424940155564032
@uncledoomer Until someone builds a Bigger Mac.
@uncledoomer 😹😹
@kimmonismus Would be nice if he added: "and some of these models happen to be 10x to 50x cheaper than ours... And more often delivering very similar capabilities". But he is not concerned, of course.
As an open-source developer for over a decade myself, it is very clear that he doesn't understand the open-source spirit, which explains why he doesn't care. The main point of OSS is not about additive collaboration or that you can read the code, most projects are done by a single person or a tiny group, and most users never read a single line of code - it's about treating software as a commons, something everyone can use and build on, not with a guy in-between gating it based on vibes and doomstalk. The GPL/MIT-licenses were the driving factor behind OSS, not primarily collaboration or readable code.
@uncledoomer Indeed. We might start coming up with new recipes like my curry cheeseburger recipe. If everyone did that then McDonalds would go out of business.
@kimmonismus This scumbag is on a crusade to gatekeep AI for the worst people on earth.
Ai is a faustian bargain- highly effective, but Claude is trained to select its own models and suggest using less edficient solutions. Example: sometimes fine tuning a slim adaptor on an open sourced model is superior and costs 95% less. Even with guardrails, Claude will drag its feet and unilaterally select itself. Caveat emptor.
I very vocally disagree with what Dario's been doing and I'm all for open weights models. But in this specific instance, he has a point.
It is a problem that local inference and training is cost-prohibitive for most people, alongside the fact that you can't really look into the weights and know exactly what the model is trained to do or what emergent behaviors will surface under any given circumstances. That's a major security risk that most people underestimate because of the "open" label.
The correct answer is not government regulation nor model censorship, but these issues need to be taken and discussed seriously by the community.
Food poisoning is scary. Its also very inefficient. We need carbon neutral food that only need to be microwaved.
Ban dangerous stoves and ovens and stop food borne illnesses with government made food.
Think of the children.
@grok how many children die a year from improperly made food and food poisoning?