RAISE summit this year was a first for me. An exciting one.
I started the week at the AI Security Summit listening to security experts discuss risks that are occurring within the AI space, while giving tips on how to combat them and put structures in place that naturally help avoid them.
What I found interesting was my self-presumed gap between how aware experts are of the security gaps that exist, and how unaware regular users are. Throughout the day, I heard about "prompt injection," agents going haywire, the fact that "agents don't have security in mind first".. to list a few things. Upon reflection, I wonder how many regular users of ChatGPT are actually aware of this. Do vibecoders know that agents don't have a security-first approach to thinking or solving problems?
Anyway, let's focus on the things I learnt:
- You have to make sure agents don't have access to things you don't want them to have access to. In order to make sure bad actors can't access them either. You see, agents don't think like humans. There's no inbuilt "I shouldn't be doing this" or "I shouldn't be here" approach to executing tasks. Agents are like soldiers, they receive commands and go all out to execute on them and please you. In doing so, they may take action or ingest information you don't want them to. Like your passwords or sensitive data. Which would then leave you vulnerable to them deleting key files or code, or exposing them to bad actors through prompt injection.
Prompt injection in simple english is a prompt that's usually hidden in some text or instruction from someone else that orders your AI agent to do something in their favour. An example would be a prompt asking an agent to download all your personal information from your desktop and email it to someone else.
Agents communicate and operate in a different way from humans. Human beings have their way of doing things. In offices, we have our ways of slacking coworkers, googling things to find the right answer or emailing external clients. Agents need to have their own operating systems or approaches to things like these and we must learn how to enable them operate in a way that works well for them. An example would be search. Agents don't scour the internet in the same way we do, they need special tools. That's how I learnt about Exa and Linkup, tools that enable agents perform in-depth research.
Never forget to have a human in the loop especially when there's a lot at stake. We hear this often, but we also hear about people building extremely valuable companies using only agents. Yet, keeping a human in the loop remains valuable when actions or decisions by agents are crucial and irreversible. Yes, unfortunately, we humans still have to work. We still need to review coding architecture designed by our agents, and review important email drafts before they get sent out
Protect agents from themselves by thinking with the lethal-trifecta. This term, developed by Simon Willison, refers to the trinity of private data, untrusted input and external action. If your agent deals with all three of them, then you have a high risk situation. Ideally, you'd want to remove one of the three and then give it a strong harness to ensure it doesn't enable the third by itself because agents are truly that powerful.
Make decisions and actions traceable. In human society, we can usually recall who did what and made which decision, with agents, we might be dealing with a black box and be unable to tell or see what happened. To avoid this, we need to add things like agent registries so we can see all the agents we have (this is especially important in the context of big companies where everyone can create an agent), we also need to have credential management so we can decide what agents have access to and how they get access to it.
One key reminder is that we human beings are responsible for the things our AI agents do. We must master them and direct them in the way we want them to go.