Anthropic's new model is too dangerous?

00:00:00: As a software developer, and I guess in general (laughs) as a

00:00:04: human, but especially as a software developer,

00:00:08: Anthropic right now, if you want to or not.

00:00:10: And I don't think you should try to ignore it because it is

00:00:14: relevant. It's relevant for our future as

00:00:18: software developers, I would say.

00:00:20: And in this episode, I won't talk about the Claude Code leak we

00:00:24: had last week.

00:00:26: I won't talk about their, uh,

00:00:29: reinforced terms regarding the usage of their subscription

00:00:33: offerings, Claude Max and so on, and how they're

00:00:37: cracking down on unauthorized use of those

00:00:41: subscriptions. They are doing that right now because, of course, their

00:00:45: subscription offerings, just like the ones by OpenAI, are heavily

00:00:49: subsidized and they can't make any money if everybody

00:00:53: maxes out their subscriptions. So yeah, they're really

00:00:57: restricting, or they're trying to restrict,

00:01:00: subscriptions to just humans, just on their

00:01:04: website, or in Claude Code or the Claude desktop app, I

00:01:08: guess. Uh, but yeah, again, this is not the focus here and I

00:01:12: won't even focus on their impressive revenue

00:01:16: growth, which is worth, uh, a, a short note though, because

00:01:20: Anthropic has reached a annual recurring

00:01:23: revenue of $30 billion, which is

00:01:27: already impressive, but especially impressive

00:01:31: $9 billion at the end of 2025. So, they

00:01:35: more than tripled their annual recurring revenue within

00:01:39: just a few months, which is really impressive.

00:01:42: And therefore, of course, if you wanna learn how to efficiently

00:01:46: use Claude Code, how to get the most out of it,

00:01:50: And, um, it's also very popular, which of course makes me happy,

00:01:54: and you'll find a link below if you wanna join it and learn how to

00:01:58: efficiently work with Claude Code.

00:02:00: But as mentioned, that's not even the main topic here.

00:02:03: Instead, I wanna talk about Project Glasswing and

00:02:07: their new model, Mythos, which they haven't

00:02:11: released to the public, and they also shared

00:02:15: why. And I think this is important to understand, and it's also

00:02:19: important to try to look behind the scenes, behind their

00:02:22: rationale, and what the impact of this new model

00:02:26: and how it works and its capabilities is for us developers.

00:02:30: So, what is Project Glasswing? What is their new model all about?

00:02:34: Uh, below, of course, you'll find a link to this article as

00:02:38: article on the official Anthropic site where they announce Project Glasswing

00:02:42: and also talk about their new model.

00:02:44: And if I scroll down a bit, we can already see some, uh,

00:02:48: summary benchmark statistics here, where we can see

00:02:52: that this new model, the Mythos, uh, preview version of the model, the,

00:02:56: the, the model name is Mythos, um, performs way better than

00:03:00: Opus 4.6. And depending on which

00:03:04: exact benchmark you look on, there is quite a big difference

00:03:08: between Opus 4.6 and this new model.

00:03:12: Now, of course, this is, uh, on its own, not super impressive.

00:03:16: Whenever a new model is announced, no matter by which company, it

00:03:20: performs way better, or at least a tiny bit better,

00:03:24: models, otherwise it wouldn't get released.

00:03:26: And of course, there are ways of gaming some of these

00:03:29: so I typically don't care too much about those benchmark numbers,

00:03:33: and that wouldn't really be different for this

00:03:37: model here, but there are interesting things about the

00:03:41: new Mythos model. And that's the fact that Anthropic

00:03:45: decided to not release it to the public because, as

00:03:49: they say,

00:03:51: it is too good at finding and

00:03:54: exploiting vulnerabilities in operating

00:03:58: systems, any other software, browsers, really, in

00:04:02: software in general. And in this article, and also in a

00:04:06: separate article, which is also linked below,

00:04:09: they share some details, and especially this separate article is

00:04:12: extremely long and gives concrete

00:04:16: examples for vulnerabilities and potential

00:04:19: exploits this new model found.

00:04:24: Uh, for example, they start in this article with a

00:04:27: very serious exploit and vulnerability

00:04:31: that was found in, uh,

00:04:34: OpenBSD. OpenBSD is, of course, an operating system which is

00:04:38: popular on certain pieces of networking software, for example,

00:04:42: and Mythos, their new model running in

00:04:46: Agentic Harness, like Claude Code, I guess, was able

00:04:49: to find and exploit, a- and that's the

00:04:53: interesting part - a vulnerability that was related

00:04:57: to integer overflow and, uh, memory

00:05:00: access, unexpected memory access, um,

00:05:05: that was able to crash machines that were running

00:05:08: OpenBSD in a reproducible way, which

00:05:12: could, of course, be leveraged to run very damaging,

00:05:16: uh, denial of service attacks by over and over

00:05:20: sending specific packets and requests to such machines

00:05:23: here, which exploited that vulnerability to bring

00:05:27: down those machines and potentially bring down entire

00:05:31: corporate networks. And this vulnerability was

00:05:35: detected in a run that cost under $50, though

00:05:39: the overall set of runs cost under

00:05:42: $20,000. And since you, of course, don't know in advance which

00:05:46: run will find a vulnerability, it's that number that matters.

00:05:51: Still, of course, it's easy to imagine that

00:05:55: a model capable of finding such critical

00:05:58: vulnerabilities for such a comparatively low

00:06:02: cost...Depending on who you are, if you're a nation, um, for

00:06:06: example, or some, uh,

00:06:09: serious bad actor, that may not be a lot of money

00:06:13: to you.

00:06:14: That, of course, is a problem because it's easy to imagine

00:06:17: that if such a model were

00:06:21: developed by a company, an organization

00:06:25: that cares a bit less about security

00:06:29: and/or that maybe doesn't have to

00:06:32: fear any consequences of

00:06:35: abusing such vulnerabilities,

00:06:40: that this could be a problem. And it seems

00:06:44: as if we're entering a new age

00:06:48: with AI, with these AI models,

00:06:52: where nothing is secure and

00:06:56: it's easier than ever to mass

00:07:00: deploy AI agents running models like this to

00:07:04: scan all kinds of pieces of software and find and

00:07:07: potentially exploit vulnerabilities.

00:07:09: And of course, as a human on, on your own,

00:07:13: there is no way of keeping up with it.

00:07:16: I mean, the bug, the exploit that was found here existed

00:07:19: for, I think they said 27 years or something like this.

00:07:25: This shows that no human was able to

00:07:28: find this bug in such a long period of time, including

00:07:32: bad actors, which of course would've had an interest

00:07:36: attack this operating system in the past too.

00:07:39: Now this is just one, maybe the most prominent

00:07:43: finding, um, this new model had. They are listing

00:07:47: way more bugs and exploits the model was find and

00:07:51: sometimes also able to abuse,

00:07:54: and they also shared other stories.

00:07:56: Uh, on X, for example, like the model being able to

00:07:59: escape a sandbox or the AI agent running the

00:08:04: model being able to escape a sandbox in which it was running.

00:08:08: And that brings us back to Project Glasswing, which is an

00:08:12: initiative created by Anthropic together

00:08:16: with other big companies like AWS, Apple,

00:08:19: um, Microsoft, the Linux Foundation, and others

00:08:23: to use this model to basically patch

00:08:27: up their software before this is

00:08:31: publicly released and before the public gets their hands

00:08:35: on this model. That is the, the narrative of this article.

00:08:39: That is the explanation of Anthropic.

00:08:42: And

00:08:44: I have some mixed thoughts here. Now, for one,

00:08:48: I have no strong reason to believe that this isn't true.

00:08:52: Clearly, Anthropic would have some

00:08:56: reasons to not release this model outside of what they're

00:09:00: mentioning here. For example,

00:09:02: um, I read that this model is a roughly

00:09:06: 10 trillion parameters model, which is way bigger than

00:09:10: all the frontier models we had thus far or were able

00:09:14: to use publicly thus far,

00:09:16: and training it is said to have cost around

00:09:20: $10 billion. The token cost of

00:09:24: this model, I read, is expected to

00:09:28: be around this range, $25,

00:09:31: $125 for, uh, input and output tokens.

00:09:35: And of course, that would also be reasons for not

00:09:39: releasing this model because they can't include it in

00:09:43: their Claude subscriptions because it's just too

00:09:46: expensive. They would have to ramp up the

00:09:50: to a price point where not a lot of people are willing to pay it,

00:09:54: uh, and therefore there wouldn't really be a

00:09:58: exposing it to the public, at least as part of Claude code.

00:10:01: Now of course, they could still expose it through their

00:10:05: pay-per-use cost basis, and if it's expensive, who

00:10:09: cares? If there are companies, people that would be willing to pay it,

00:10:13: they could do that. And that, of course, is the part where the

00:10:16: cybersecurity concerns might really come into play

00:10:20: because clearly that all

00:10:23: is very likely not made up. I mean, it's definitely not made

00:10:27: up. The FFmpeg team, for example, which

00:10:31: is also listed here as a, um, as a

00:10:35: vulnerability that they were f- able to find a vulnerability in

00:10:38: FFmpeg,

00:10:40: uh, the team confirmed on X that Anthropic

00:10:44: sent, uh, sent a patch for, uh, uh, a

00:10:48: vulnerability in the FFmpeg, uh,

00:10:51: software, uh, program So yeah, this is

00:10:54: clearly not made up. These concerns are valid.

00:10:59: Cybersecurity concerns are valid, especially since, of course,

00:11:03: if money is not the main issue, you could deploy thousands of

00:11:06: agents running simultaneously using this or

00:11:10: similar models, which we may have in the future to, yeah,

00:11:15: scan all kinds of software and exploit them.

00:11:18: And of course, the big problem is that

00:11:22: using this model to find vulnerabilities

00:11:26: and patch them up is possible,

00:11:30: but it's only possible if the

00:11:33: owner or the maintainer of a certain piece of software can afford the

00:11:37: model or gets access for free or anything like that.

00:11:41: And even if a vulnerability is patched up,

00:11:45: we all know

00:11:46: that not all (laughs) computers out there, not all machines,

00:11:50: not all users

00:11:53: have up-to-date software running on them.

00:11:57: If you were to take a look at all the

00:12:00: various servers running out there in the

00:12:03: worldwide webI would guess the vast

00:12:07: majority of them is running outdated software.

00:12:10: I mean, on our phones, our, our laptops, we are

00:12:14: of- not running the latest software, the latest

00:12:18: version of our operating system. The latest security patch may not be

00:12:22: installed and that is true for all layers of s- of

00:12:26: software. And in a world where it's easier than ever

00:12:30: find security vulnerabilities, that of course becomes an

00:12:34: even bigger issue. Because of course the good thing about

00:12:38: this AI model is that it can also be

00:12:41: used for proactively looking for security

00:12:45: vulnerabilities and patching them. So it's not just a tool for attackers.

00:12:49: It can also make defense easier because you now have a

00:12:53: tool that can be run simultaneously in parallel across

00:12:57: thousands of agents to make your software secure.

00:13:00: In theory, this can be a very useful tool for

00:13:04: of course, again, not every company, person

00:13:08: that may be developing crucial software may be able to afford it, may

00:13:12: be interested in using it, and then even if it is

00:13:16: used to find and patch vulnerabilities,

00:13:19: still these latest versions will not be installed

00:13:23: everywhere. And that of course gives attackers, uh,

00:13:27: opportunity where they know about way

00:13:31: more vulnerabilities than before at some point because

00:13:35: way more vulnerabilities are detected, but not every

00:13:39: machine, every user is protected against those

00:13:42: vulnerabilities and that is one real concern I have

00:13:46: about this development. Now that is the, the bigger

00:13:50: picture which affects everybody, all companies, all humans

00:13:54: in the end. Another question of course is what does

00:13:58: a model like this mean for us

00:14:02: developers? I mean, clearly this seems to be a

00:14:05: highly capable model that was able to look for

00:14:10: vulnerabilities on its own and exploit vulnerabilities on its

00:14:14: own. So yeah, what is the impact for

00:14:17: developers?

00:14:19: And I think here, when it comes to that,

00:14:24: not a lot changes for now. I

00:14:28: mean, we're already living in a world where

00:14:31: AI agents like Claude code and the underlying models,

00:14:35: true for Codex and so on, whatever your favorite AI agent

00:14:40: are able to generate most of our code.

00:14:43: You may not be using them, you may not like them.

00:14:45: I created a separate video where I share my feelings about that and that

00:14:49: this sucks the joy out of the, the software development part

00:14:53: for me, but it is the, the reality nonetheless, if you like it or

00:14:57: not. And believe me, I don't necessarily like it, but yeah,

00:15:01: reality. Nonetheless, what a human brings to the

00:15:05: table or why humans still matter here and may matter

00:15:09: more than ever is, of course, that you definitely

00:15:13: don't want an AI agent like this go rogue

00:15:17: and work totally on its own. Steering

00:15:21: such models and agents, controlling them,

00:15:24: giving them clear tasks, limiting the scope of the

00:15:28: work they do,

00:15:31: all these things are more important than ever.

00:15:34: These models can, as it seems, do way

00:15:38: more than

00:15:39: the vast majority of developers can do. Definitely way more than I can do.

00:15:45: And yet, when it comes to shipping products,

00:15:49: comes to building software used by humans,

00:15:54: the influence of a human is of

00:15:58: course extremely, extremely important.

00:16:00: What's changing, of course, is our role as software developers.

00:16:04: We're changing from the people that are writing the code to the

00:16:08: people that are steering the model, that are reviewing the code, that are

00:16:11: understanding what it does, that are setting the scope.

00:16:15: And yeah, again, I talked about this in that other video,

00:16:19: and then that this may not necessarily be what, uh,

00:16:23: what, what you like. It's definitely not w- why I got

00:16:26: into software development in the first place.

00:16:28: But yeah, this is the impact here and the more capable these

00:16:32: models get, the more important I think it gets to

00:16:36: have that, that human voice in there as well,

00:16:40: that human influence, uh, in there as well

00:16:45: So that is, that is that changing role and, and

00:16:49: our role in the future. But yeah, I mean,

00:16:53: these are really interesting developments and

00:16:57: especially this model and its implications and

00:17:01: that cybersecurity relevance it has

00:17:06: makes one think what would have happened or what would

00:17:09: happen if other

00:17:12: actors, other nations or

00:17:16: organizations in the world get their hands

00:17:20: on this model or models that are similar in capability.

00:17:23: Because of course it's only a matter of

00:17:27: time until models with similar

00:17:30: capabilities are accessible

00:17:34: by the public or certainly at least by

00:17:38: other nations and actors. And

00:17:42: yeah, I'm not sure if we are prepared for that

00:17:46: new race in cybersecurity and that delay

00:17:50: between bugs being found and being patched

00:17:54: and people installing those patches.

00:17:58: I think we'll enter a new era of cybersecurity and we'll

00:18:02: be able to adjust, I'm sure, but this, uh, definitely

00:18:06: marks an interesting,

00:18:08: uh, point in the history of, of model development

00:18:13: I would say