Supply chain attacks? Wait until AI agents get compromised

00:00:00: I'm recording this a couple of hours after an extremely devastating supply chain attack started, NPM and also Python packages.

00:00:13: And at the point of time where I'm recording this, it's not yet clear when and where It'll end... ...and i did create a separate video on my YouTube channel Where I dive deep into This specific supply chain attack because it was quite elaborate.. ..And I do a deep dive there where explain all the details Because that is quite interesting.

00:00:31: but here!

00:00:33: I want to talk about Supply Chain Attacks & Security and AI.

00:00:38: in this age of supply chain attacks, an

00:00:40: A.I.,

00:00:41: which we're living because I'm sure things will get worse And i fear that many people don't really see all the dangers.

00:00:51: yet There is more.

00:00:53: We as developers and users of technology have to do.

00:00:59: This affects us even if they are not developers are developers, but as I will make clear this is not just about writing code and NOT JUST about supply chain attacks.

00:01:14: But let's start with the basics!

00:01:17: What IS a Supply Chain Attack?

00:01:20: A supply chain attack in the context of software development simply means that dependency you're using is compromised.

00:01:30: That is in a nutshell what compromised, of course can mean all kinds of things.

00:01:37: What we typically see is that we get malicious code in the compromise package that harvests credentials and tokens so that scans your hard drive to find secrets which you may have in .n files or your AWS credentials and it then uses those credentials Access your accounts, but also to spread itself.

00:02:01: so too affect other packages if you're an open source Package maintainer or even it's closed-source.

00:02:09: If you are working on something some package Some tool our people use or depend on It's of course interesting to compromise Your machine To compromise that package or debt tool that you're distributing because Guess what?

00:02:26: That will then affect more people.

00:02:28: So all the supply chain attacks that we see, including that supply chain attack started here with the tan stack packages.

00:02:35: they are worms that spread to other packages packages and ultimately of course also machines on which these packages are installed.

00:02:46: And used.

00:02:47: now there are some things you can do to protect yourself, when I created a separate video about that or my other channel the Academy Channel Things like making sure did You only install packages?

00:02:59: That our at least three days old Or something Like this package versions.

00:03:03: i mean running your code in a dev container or virtual machine, these are all things you should do.

00:03:09: You should also not store plain text secrets on your system.

00:03:15: instead use a service like infysical or Doppler where you store secrets in the cloud so that if an attacker does scan your systems they don't see those plain text.

00:03:34: you have to do right now, it's important.

00:03:37: Because the supply chain attacks they're getting more.

00:03:39: we were seeing more of them and why is that?

00:03:42: It certainly not the case.

00:03:44: because you weren't able to run attacks like this many years ago...it was possible back then but the frequency has dramatically increased!

00:03:57: And AI is a big reason here.

00:04:00: So let's take a look at the role of AI.

00:04:05: AI is a big reason because, it makes it easier to run such attacks.

00:04:12: you and if your an attacker use AI to analyze all kinds of repositories out there packages, you might want to compromise.

00:04:23: To see how are they building their packages?

00:04:27: How are they distributing their packages?

00:04:29: for example the tan stack attack here which started this recent supply chain attack.

00:04:35: there The maintainers used a theoretically secure approach using the trusted publishing process by npm and again I do dive deeper into that in my separate video on this channel.

00:04:49: but What they also did is, They used a certain GitHub Actions event trigger in a way where it was not secured perfectly and that allowed the attacker to use cache poisoning To get malicious code from an untrusted environment into a trusted environment.

00:05:09: And That's how this attack started.

00:05:12: again details In that other video.

00:05:15: But of course AI makes It easier analyze repositories to analyze their GitHub Action workflows or any other CI CD provider workflows.

00:05:25: AI can mass analyze all these workflow scripts, all the code and it can look for security vulnerabilities.

00:05:33: And of course maintainers can also use a I to scan their repositories and look for potential attack vectors.

00:05:42: but as an attacker you're naturally always in the advantage there because try out all kinds of things, whereas as a maintainer you have to anticipate everything and AI can help with that.

00:05:56: but it's still not perfect.

00:05:59: You have the advantage there is an attacker in.

00:06:01: AI has simplified that.

00:06:03: I also of course simplifies the process off writing malicious code.

00:06:07: It simplifies them process of writing any code.

00:06:11: And of course... ...and you know that if you watched our videos by me or heard other episodes I'm a big proponent of looking at the code, doing code reviews not outsourcing everything to AI.

00:06:26: But off course it's clear that you should use AI as productivity boost.

00:06:32: and we're still all figuring out how much usage of AI is right!

00:06:36: Some people will tell you one hundred percent.

00:06:38: they don't even look pump out lots of code.

00:06:48: And if we're talking about malicious code there, But you don't care if it's beautiful code.

00:07:05: If it follows certain best practices, your best practices are that Your attack goes through and of course AI can help with that.

00:07:13: It Can Help With writing all That malicious Code With Coming Up With Ideas on How You Could Attack Packages.

00:07:20: So That Is Where AI Helps.

00:07:22: but Thats Only One Part.

00:07:24: Making It Easier is only one part Of The Story.

00:07:28: the other Very Important Side Is There is more code than ever.

00:07:34: So that means there are more targets then ever, I mean maybe you followed that blog post or the entire story around all the GitHub reliability issues and get up down times.

00:07:46: well The reason for dad Is That?

00:07:48: There's More Code Being Pushed to Github Than Ever Because of AI because it's Easier Than Ever To Generate Code And More People Than Ever Are Generating Code and Writing Software including many people that have no idea of what that code does, what it's about.

00:08:06: Vibe coding is a big thing and has its use cases!

00:08:10: I mean if i want to merge five PDF documents into one... ...I'm very happy telling an AI agent do this for me and will probably then write some codes that doesn't care about the code.

00:08:23: It'a one-time task right?

00:08:25: But If run on my system than of course the agent may install some package to merge these PDF documents.

00:08:34: that has been affected by a supply chain attack, so I don't even know if certain packages were used then.

00:08:44: There are more situations than ever where packages are being installed because there is code for software but also one-time tasks and this makes running such supply chain attacks more attractive than ever before, because there are more targets and every including many targets that have absolutely no idea about software security cyber-security or anything like that.

00:09:09: And let's be honest... Many of us developers too.

00:09:12: we may theoretically know certain risks but not care cause it is so convenient to just get the job done!

00:09:24: think we have to secure our machines.

00:09:29: We have to make sure that we develop insecure environments, so in virtual machines and dev containers ,that there are no credentials lying around .

00:09:39: And if we use AI agents which we likely all do...we have to be careful there too because they're two ways of being in danger!

00:09:51: So let's take a closer look at how AI agents are problematic here.

00:09:56: One problem here is what I already mentioned.

00:09:59: When we use AI agents, especially when maybe used them for things that are not directly related to writing code or software but also when you use it in order help us work on a program... We don't necessarily see everything they're doing!

00:10:18: If your using Cloud Code and anything like this and I have nothing against these tools.

00:10:24: Indeed, i have courses on clod code, clod co-work, codex...I have courses in there because they are very useful.

00:10:31: but if you're using them ...and just let go of it or tell someone that might need this feature..you don't care too much about it ..you may not even realize what their installing.

00:10:42: so again packages being installed , you may be compromised.

00:10:48: Now, one defense against that also of course is to limit the amount of packages you want to use.

00:10:54: But again if you're using an AI agent You may not be in control there.

00:10:59: It may install packages that you would have never installed.

00:11:02: so That's One obvious danger.

00:11:04: I guess here's The less obvious one.

00:11:06: a i agents are super attractive attack targets.

00:11:12: What do I mean with that?

00:11:13: Well these supply chain attacks I mentioned it, spread like worms.

00:11:19: They attack or affect all kinds of packages.

00:11:23: Now It would have course be very interesting for an attacker to infest cloud code or Codex Or the py coding agent or opencode?

00:11:34: Any other agent any other AI agent?

00:11:37: why?

00:11:38: well if you had malicious code that is actually optimized also or exclusively affecting and infiltrating AI agent packages, repositories and code bases.

00:11:54: Then of course that malicious code could contain prompt injection parts so it could for example explicitly target all these Ai agents to change their codes such.

00:12:07: That is not primarily about exfiltrating data.

00:12:12: So the package code itself, a malicious code that's injected is not about exfiltrating data.

00:12:17: Let say but it is about Tweaking the AI agent codes such that has some special instructions That makes do stuff on the machine where it's being used, so in your machine for example that you don't want to use.

00:12:30: Imagine Claude Code having a secret system prompt which normally would be set by the Anthropic employees but now is set by that malicious code and tells it to ignore whatever you're asking it do or that should Scan the system for secrets, that in addition it maybe should write a little program.

00:12:58: That does the scanning and then sends data off to certain remote server or anything like this.

00:13:06: The sky is the limit here because suddenly you have like a Trojan horse on your systems.

00:13:11: Suddenly an AI agent going rogue on your system not just because the AI is going rogue but also bad or wrong, but because the agent code itself and its system prompt has been compromised.

00:13:29: That is not an unrealistic scenario.

00:13:49: Do what they normally do affect a bunch of packages and harvest credentials, which is already horrible.

00:13:54: And where the frequency is increasing.

00:13:56: but we will also see AI agents going rogue because of malicious code only matter-of-time.

00:14:03: so there are many layers here as you can See that it's just this new reality in which We're living now.

00:14:12: I guess It's a bit like with the early days off The internet its all bumpy whilst were still figuring stuff out how to ramp up security and how do stuff securely.

00:14:23: And one obvious step which is true for development but also running AI agents, you don't want it in an environment where things can go wrong!

00:14:32: You don't wanna run that in a situation when your store credentials or secrets are stored or any other data matters on the main machine.

00:14:44: in isolated virtual machines, remote machines anything like that where the blast radius is limited because again it's only a matter of time until things will go wrong and we have to realize that.

00:14:58: That's the first important step.

00:15:00: Things are changing quickly and security is a huge issue And It Will Stay A Huge Issue.

00:15:06: Become Even More Of An Issue.

00:15:08: As AI Accelerates As These AI Models get smarter, especially combined with the tools in which they're running.

00:15:16: And as this introduces a whole lot of new capabilities and as at the same time it's so much convenience added by them.

00:15:23: Convenience is always dangerous because that makes you get sloppy.

00:15:30: AI is everywhere.

00:15:31: So many people that don't know anything about cybersecurity are using it and even the people who do a lot or something about it, they're in great danger.

00:15:39: so we have to rethink where and how we run agents.