The_TWIML_AI_Podcast_formerly_This_Week_in_Machine_Learning_&_Artificial

0:00I recently attended a major enterprise tech conference speaking on a couple of panels about scaling AI agents. Naturally, the topic of risk came up a lot, and the default answer was usually some combination of static guardrails and human approval. In theory, this sounds simple enough, block the dangerous stuff and when something looks risky, put a human in the loop. But agents put pressure on both sides of this model.

0:24Static rules are hard because agents are creative. They don't just follow a fixed path through software. They plan, improvise, call tools, and find workarounds. And human approval is hard because agents can operate much faster than we can. So the question isn't whether we need guardrails and oversight. Of course we do. The question is what that should look like when agents are operating at scale across high stakes tools, databases, and workflows.

0:51I spoke about this with Dev Rishi, GM of AI at Rubrik, where he and his team are building infrastructure to secure and govern agents in enterprise environments. He shared an example from his personal experience that gets at why this is such a tricky problem. But then we saw some more sophisticated types of things that were going wrong. One thing we noticed was that Claude Code was really trying to post internal source code to a public repo.

1:16rather than private. And so we saw this kind of relatively frequently. And there's even instances where like, if we looked back through the audit logs, we saw like the check, check, check. And so technically it felt that it had gotten the human and the loop approval for it. And we even saw one crazy instance where Claude code tried to get around this like blocking we were doing of like GitHub public just. And rather than like doing this as a text in text out system, where it was like, all right, I'm posting to this URL.

1:42It spun up a browser window, and we just started to see mouse clicks on certain coordinates. And we noticed that one of the coordinates actually was for a public gist.

2:19an expertise challenge. You know, there's lots of ways to think of the challenges, but it does seem to be to a larger degree kind of mindset and a mindset shift that's required. You know, Sam, I do think like a lot of other tech changes, there is like, there's a cultural component, there's a learning component, but I actually don't think that's the biggest piece. I actually think the biggest piece after speaking with a lot of the organizations now is the approach and how to manage risk.

2:45If I thought about what's different from a fast-moving, AI-native organization and what's different from, let's say, a top 10 global bank or a top 20 healthcare company, the way that the healthcare company and the bank have really been brought up is that they have to make sure that the ways and the systems that they have are deterministic, that they have guardrails, that there's real clear downside protection. Have you used an agent like Claude Code or Codex before?

3:14It doesn't exactly feel like there's a ton of downside protection as you're using it all the time. It's incredible. I had one global CIO describe it as it feels like a fast car with no brakes. I'm moving really, really quickly and who knows what exactly is happening on the background. I sympathize with that. I think the biggest thing is that these tools have come out with an incredible capacity to be tools, but they didn't really come out with a perfect way to secure and govern.

3:41them and they're really operating on legacy IT infrastructure. Honestly, the biggest difference I see between the startups and these like global 2000 enterprises, the global 2000 enterprise has maybe a bit more of a legacy mind shift, but really more so just more to lose, like as they take kind of take this approach. And that has been like the number one thing that's hindered the AI adoption. So if you're an AI startup, you can use one of these harnesses out of the shelf and start, yeah, going on it. If you're an enterprise.

4:06you're going to probably bog it down with some AI governance committee meetings, you know, essentially, let's meet this week to define a framework three months later to come up with a V2 of the framework and so forth. And that's what ends up, I think, delaying the site. And I imagine that those conversations and that observation is what led you and the team narrative focus a little bit more on the agentic side of things. Yeah, that's exactly right. So...

4:32We saw both the conversation happening externally, but we also saw it happening internally. Rubrik is an interesting company. It was a startup that was just born 11 and a half, 12 years ago. It's also a public company in data security today backing up the data for some of the most important global 2000 enterprise. So it's like got a little bit of both in the DNA. And what we were noticing was as we were developing AI and agents, We actually ran into these same types of security and got rents bottlenecks ourselves and we found them to be quite frustrating actually. Now, I find it helpful to define agents for a, you know, a quick minute. We define agents as really LLMs or models with access to tools. So you can think about this as like models that can take action on behalf of the user. Well, we missed the tools that increases the blast radius significantly. Exactly.

5:23I know two years ago, everyone was afraid of like, what if I send the wrong data over to ChatGPT? And I think that's like a legitimate concern. But really, I think like, what if ChatGPT gets access to my Salesforce, like my system of record for all financial data, and then starts hallucinating numbers like that's the much bigger concern that exists, right? But at the same time, it's also what makes AI magical. Like we're never going to get to I think the level of like economic productivity we're looking for without giving these models like access to do work internally.

5:51And so A it's gonna be necessary, but B I don't know like say if you've been using these Asian harnesses like frequently But I'm using them and like Claude is coming back to me with like a bunch of requests and I'm just like yep except except except like it's kind of scary because it's moving so quickly and I don't have the ability to really dissect every single thing that it's doing and kind of check over So the first time I was using it I remember I had a thought which was I wish somebody was watching over my shoulder Just to make sure I didn't do anything that screwed up And then I wish if I did screw up that I had a way to be able to, you know, reverse that change. And so that's exactly what we built, basically, at Rebrick. We built something we call the Rebrick Agent Cloud to help organizations secure and govern their agentic rollouts. You know, I also have had that experience where you're like, accept, accept, accept, and you realize how much of it is kind of security theater in the sense of like, first of all, it's a super long command line and you can't really see all the command line and you're like, okay, yeah, that looks fine.

6:50And then it's like you're approving, you know, the agent acting or running some file that the agent can control and put whatever it wants in it. And it's like, what am I really accomplishing here? Earlier when we were talking about wire predabase in rubric join force, as I mentioned, I saw an opportunity to attack a product of the market that no one else was attacking. And like the specific thing that I think I saw was like the legacy way that we approach security was never going to be.

7:18appropriate for agents. And you called it, I think, a little bit of like the security theater. We had a spirited debate on one of our internal Slack channels where somebody from security, one of the engineers was like, hey, do I have to go ahead and hit yes every single time? Can I just go ahead and like say, look, for most apps, just automatically run them. And the security person chimed in and their point of view was, listen, it's important that this agent is acting on your behalf as a user.

7:45So you need to understand every action that the agent's taking and to authorize it. It logically makes sense, but in practice, if you see how quickly these things are operating, the value proposition of the agent is that it can operate 10 times faster than I can. If it can operate 10 times faster than I can, I can't realistically do 10 times the level of review. And so the argument that the engineer was making back is, hey, is this actually becoming less secure? Because I don't have the opportunity. I'm signing off on this.

8:12almost without having a good appreciation for what exactly I'm doing at every point. Because it's like the iTunes terms of service, essentially. Now I, for one, read that diligently, line by line. But not everyone will. And so I think that the trick is how to be able to manage that. My fundamental view is we can't use the legacy approaches for this. We can't rely on rules-based systems. And human-to-loop is something that feels good, but it's not actually going to work at the pace that we're going.

8:41So my view is we actually need to look, I'm an AI person by background. We started an infrastructure company. What did I arrive at? My view is we should use AI and throw AI at the problem. So go from human in the loop systems to AI in the loop systems. I talked about how I wish someone was watching over my shoulder. I think essentially that needs to be a really smart and highly specialized trained domain specific cybersecurity agent. And that's what we've been building internally. Since you're a security person now, I'll ask you this.

9:10you know, the days when we were just worried about distributed systems and, you know, connecting systems to the internet and things like that, we came up with this, you know, this term instead of practices, you know, called zero trust, which is like, in the older days, you know, you would establish trust with the other system and then connect it fewer, you know, gates between because you accept you assume that the systems are trustworthy.

9:37And then we move to this model where, yeah, let's just not trust anything and force policies and things like that around the things that we care about. And I'm paraphrasing because I'm not a security person. But it strikes me that the world you're describing is one of, not only am I going to not trust external things, but I'm not going to trust this agent that's here sitting on my desk, on my computer or wherever, working on my behalf.

10:07I'm wondering if that resonates with you, and if you think or talk about this idea of zero trust extending to agents, and what are the implications of that? Yeah, I like to think of myself as an AI info person, now masquerading, and looking at all the challenges in security. But I spent a lot of my time now thinking about the security implications for AI info, and I do think that there's these...

10:33principles that exist from legacy security, zero trust, secure by design, and others that I think directionally have the right principle. But I actually do think the way that they've been applied, it's a little bit different when it comes towards AI. The main reason is that a lot of historical security principles were baked into static and deterministic systems and policies. So zero trust would mean like I don't have any trust by default. Maybe I'm doing like a just-in-time authorization for the action that you're looking to take. And that is a whole principle I think is like relatively appropriate. But one thing that I think is difficult is like even in a lot of the secure by design like security infrastructure, usually they always made some assessment like they were securing the software that humans were using.

11:18But I actually think of agents as a lot more similar to humans than like the software that legacy secure security solutions were actually securing elaborate on that Yeah, so okay if I gave you an example of like my cloud code or my cloud co-work instance that's running on my laptop So my co-work instance has access to Salesforce. I use it to summarize opportunities. It also has access to my email. These are the send out emails Now in a like pure security design standpoint, I'd be like, all right, check, it can do these things in Salesforce. Check, it can do these things over email. But the really tricky thing is that now it's like one agent harness that has multiple different permissions. So what's really supposed to stop it from like, for example, taking sensitive data from Salesforce and then writing it out in an email to another customer? Like those conventional guardrails that you would have that say like, I've secured each system individually doesn't really work in this agent feature.

12:10And the second thing is like, I'm not actually telling Claude Cowork or Claude Code what are the steps it should go through. I'm just giving it a task and it's coming up with its own execution plan and then it's executing the plan. It's a lot more similar to how a human might operate. So like, you know, I gave you one example where data can kind of like, I think be used across identity and permission boundaries that conventional identity systems would not solve, right? There isn't like one unified way to think about how do I govern access on data from Salesforce going into email.

12:39It's like never been something like usually you'd have like, again, a very deterministic flow. Now you don't. The second example that I think is very present is like these models are very good at circumventing the rules that we put on them. And I think like as soon as I say that, like everyone starts laughing because they know that they've run into this before too. I asked Claude to write me a document and I had the Google Drive MCP connector disabled. So what did Claude say? It's like, looks like the Drive MCP connector is disabled. No problem. Let me try this work around.

13:07opened up a browser window, typed in drive.google.com, used the mouse click button, hit upload file, and it's just like it's creative in the same way that a human might be, much less than a static and deterministic software system might be. I'm laughing because I recently started playing around with the codex, the goal feature, and I thought I would try to get it to act like DSPY and like optimize or prompt.

13:33So I gave it a document with a bunch of URLs, and it was supposed to identify the URLs in the document that I would be interested in. Kind of a recommender type of system or a classifier is probably a better way to think about it. And I was like, OK, here's the goal. Iterate on this prompt until you can.

13:57It was actually doing very well, but it had a lot of false positives. So try to like it, you know, reduce number of false positives. And ultimately it was like, okay, I did it. And then I went and looked at the prompt and it was like, if the URL contains this, or this, or this, or this, or this, you know, then it's good. Otherwise it's not. Exactly. And now think about if you're trying to write a rule.

14:20That was like, oh, do this, don't do that. It would be impossible. It's like a game of whack-a-mole, essentially, to try and prevent every single action it's going to take. And so I think, look, in my view, that's sometimes a little bit either funny or even a little bit frustrating when it comes to you're developing something. It sounds like you're doing a ranking problem as an example, or if you're building a system, it's like, ah, it's funny. It came up with this little side workaround. But if you're a security person, that's terrifying.

14:49Like this system that like I had thought I'd put every best practice into place is now finding unique and novel ways to circumvent it. And I'm on the hook for it because ultimately we're taking this bit of a posture that's like, you know, the tool is just acting on behalf of the user. That's, I think, the terrifying gap that's really the thing slowing down AI adoption in the enterprise today. So you need AI to secure AI. What is your proposed approach for injecting AI into, you know, this landscape to affect?

15:18the solution. Totally. So I think honestly for a good solution, you need three things. And if you're like building your own internal governance and security solution, I'd recommend these three things are things you think about. The first is I think you need some cross platform visibility. And so agents are running roughly everywhere now. So like they're running into cloud, they're running on the endpoint. So you need some way to be able to see what's going on, what kind of access they are.

15:41But a lot of times people get very stuck on this visibility point and then they finally solve visibility and they realize visibility is sort of useless without the ability to do something about it when something goes wrong. So I think visibility is just like the base layer. The second thing that you need is a way to be able to do dynamic runtime security. And my view is that you need to be able to do this with an AI in the loop system. So we built a system that we call Sage. Sage stands for semantic AI governance engine.

16:08It's basically our own agent that uses a small language model at its core. And what Sage does is it runs over every prompt response and tool call that's going through an agent system. Everything you put into an agent, everything the agent's about to do, every tool it's going to call on the parameters of the tool, it looks at every single one of those. And Sage has a lot of the cybersecurity best practices that we know firsthand through our security research you need to be able to do.

16:36So we make sure to prevent all the obvious things that you would want to make sure, but maybe like haven't written down somewhere, like prevent data exfil, prevent dangerous and destructive actions, all of these different things Sage is looking out for. But then the real trick is to customize it to your organization. There's two levels of customization. The first is you want to customize it to your policies. So we allow organizations to like bring your own policies like a doc upload or write your own international language directly.

17:02And so Sage can understand your financial services institution. That means you should not be giving financial advice to end customers as an agent or something along those lines. Your healthcare needs to be very careful about PHI. So we allow organizations to customize. And the second thing we do is we enrich Sage with data and identity context. Because Rubrik is a data security company that backs up data and identity systems, we know things like where sensitive data exists in your organization, what identity has been compromised or others.

17:29So all of this enriches this AI in the loop system that we call SAGE. And it's critical to use a small language model, which is where ProtoBase's infrastructure is instrumental at its core. Because if I told you the way we're going to secure ungoverned AI is by doubling your token count and by doubling your bill and your latency, you'd tell me, no thanks, I'll take the insecure version. And so we need to be able to do it at a very fast and low footprint. So that's the second component.

17:55The first component's visibility, the second component is secure with an and loop system that inspects all traffic, determines whether or not to permit it or not. And then the third component is, look, Rubrik as a company has had this mentality of assume breach, which means at some point something's gonna go wrong. Is this even more true with agents today? You need some sort of undo button when something goes wrong. And the undo button we've built and thought through, and I'd recommend at least other folks think through is tie your observability, with whatever you're using for business resilience and recovery. So if you have like a data backup system somewhere, tie your observability there. So that way if you notice an agent taking a destructive action, like so in pocket OS, for example, there was this incident where, you know, a startup went viral because a coding agent went in and decided to delete the production database. Yeah, exactly. So what I think about is like tie your agent observability with your recoverability story. So if you notice from your observability stack, the agent took some destructive action.

18:54dropped a prod database or so forth, you can also then immediately create a one-click recovery plan that looks through your previous snapshots, determines what's the snapshot that was right before the agent took the destructive action and can rehydrate that system back from the previous healthy snapshot. And so we call this capability like agent rewind. There's a number of different ways I think you could probably think about referring to it. But the three key capabilities I think are monitor what's happening, have a system to be able to constantly like run and enforce your policies, have a way to make sure that you can recover when things go wrong. Bunch of thoughts there. On that second point, I think you answered this. I was going to ask, you know, where does enforcement happen or what's the kind of form factor of sage? Is it like an agent? Is it a shim that is like programmatically inserted? It sounds like it's something that's running over the wire. You know, the reality is it's actually a number of different things depending on where your runtime is hooked in. So this comes back. I like this idea that agents are running all over the place.

19:52And so what we see is, in a lot of cases, Sage is in line with the request. So for example, kind of like a reverse proxy, if you're going into making calls to open error or Claude, we can sit right in the middle of that and we can actually go and determine, you know, whether or not to allow a certain action. But sometimes people want different integration modes. So there's a number of tools that have also exposed things like pre-tool call API hooks. So if you're building the agent in Microsoft's co-pod studio, or you're using something like Claude code.

20:21These agent harnesses themselves have an ability to like phone home to a verifier service like sage and be like Should I allow this action or not? And the brilliant thing is these things can run in parallel You know the request can be like starting to be transacted while the sage system determines whether or not it actually should be fulfilled or not and can block it all together and so it can be you know done in line it can be done via like these different instrumentation hooks once you connect your different agent runtimes in we determine the system in the way that makes the most sense for the different agent one time you've connected. And then on the, what was my question on your third point? Remind me of the third point. Recovery and resilience. High observability to something that allows you to recover quickly so you're not dead in the water once something does go wrong. It strikes me that that in and of itself is a big ask and potential impediment for organizations that are trying to do this like

21:16How many of them really have a recovery and backup system in place, let alone one that can be automatable and tied into an agent? Am I imagining a level of immaturity that has been surpassed, or is it as grim out there as I imagine? I think a lot of organizations, especially large enterprises, do have a data backup and recovery solution in place.

21:43it was mandated a lot of ways. And the reason why was that it used to be that you needed data backup for business continuity in case of like natural disaster fire flood. So think like a couple of decades ago, everyone started buying data backup because it was like if a flood hits the data center, how are you going to come back into business? But those solutions were, I'm not sure what the right word is. Let's just say those solutions were like relatively basic at the time. It was like an insurance check mark that you hope you never had to use. I think also I'm feeling like heavyweight like I'm imagining a world that you're talking about where You know you set agents are everywhere. They're like constantly charting through things if you're getting You know if you're getting kind of you know these alerts or triggers or whatever you would call them where you know Hey, the agent did something, you know, not so right here. You're using a SLM, you know, so maybe not the smartest model like it's gonna let some through

22:42When I think of backup and recovery and even snapshotting, it's like, this is a heavyweight process. It's not something that like, oh, the agent did it. Let's roll back. Oh, let's roll back. Maybe for a small individual database. Most backup and recovery for a long time has been this heavyweight piece of software that you buy and you hope you never have to use. And you're like, if something goes wrong once every however often, well, blow off the dust and figure out how to be able to plug it in.

23:12I think the core observation that we have is like, I think agents are going to change that game. I don't think like that this now the need for recovery and resilience is just going to be the once in a, you know, hopefully never ransomware attack or otherwise that you end up happening. I think it's going to be much more frequent, both from external malicious AI driven attacks and internal inadvertent AI mistakes. The brilliance would be if you can tie a really good recovery system with a really good AI monitoring system. To your point, I don't think many people have that today because you ideally want to bundle and buy this together. Slight plug for Rubrik, which is actually doing this. But I do think that, you know, to your point, the core is like, can you find a system that basically architects both of these things together? Because you probably have something for data back and recovery for like other compliance reason or other. Is it possible to connect it into your observability?

24:05And I will actually say one misnomer is that the SLM is not as good at enforcement on the ALib. Actually, we find that for domain specific tasks, small language models that are tuned for a very specific task tend to outperform a generically prompt engineered larger language model. So like when we benchmark our SLM versus at the time I think we benchmarked GPT-52 as an example, we found that not only were we in order of magnitude faster and cheaper, but we are also actually more accurate on being able to make a binary classification on whether or not to allow or disallow.

24:35And we've relatively consistently seen that fact when you just constrain the outputs of the LLM to be very low cardinality in terms of what it's supposed to do. Just exactly what you want from a guardian agent. Yeah, I could see that. I think what colored my perspective on that is thinking about with open claw or, you know, personal agent. It is frequently said that you should use like a frontier model for your main orchestrator because your SLMs If you expose them to external untrusted data sources, they can be easily manipulated relative to a frontier model that A is tuned better to be able to detect and resist manipulation, but also is just smarter, more parameters. We've done a lot of benchmarking on this exact line of thinking, and we did it even on ProtoBase.

25:27released a paper called Laura Land because all these are like Laura tuned adapters essentially, it was a technical detail. But what we found was that for open-ended, like for open-domain tasks, you're 100% right. Use a large frontier model. That's why the orchestrator or the planner should really be like a larger model. But if the more constrained you get into tasks, the better and better you actually tend to see performance. So if it's just a fixed domain classifier, then if it's a fixed domain classifier, the best you could usually do is post training with like SFT, you know, a small model on it. And then if you have a task like what we're talking about, which is simply should this request be permitted or denied? Like that is actually the ideal type of task for an SLM. And that's where you're able to run at a super low latency. Now, I think to your point, like ultimately,

26:15I think that all prevention mechanisms are going to have some rate of false negatives. And like the agent world is just moving too quickly to be able to catch and block everything, which is why the resilience story is so important as well. We talked about the agents themselves, you know, being incredibly resourceful and trying to get their things done, right? And then we talked about, you know, using an SLM or an LLM as judge. I'm envisioning a scenario where your agent starts trying to hack your SLM to get its request through. I'm imagining that it could probably be pretty good at that if it really figured out what was going on. How do you prevent the agent injecting something into the request that says...

27:03you know, this is a permitted action blah blah blah or you know, you know the thing I'm getting at. Totally. And this is why it's so important for, I've seen a lot of people when they first think about this idea that we need AI to help secure and govern AI. The first thought is like, great, I'm already doing this, I'm putting guardrails into the prompt of my mouth. And that's usually the first, and you're laughing, but I would say in fact, most often, that's actually the usual take. So just like, I'd say, where does the state of the art in the universe?

27:31Step zero is like, do nothing, essentially. Step one is like, I have a bunch of deterministic rules that I like configured in some cloud console somewhere that hopefully works for something or not, but it's very loaded. Step two is like, I know that I need to use AI. I'm going to put these in the prompt of my model. And then step three is you have an external system policing the inputs and the outputs, which is what SAGE is. Now, I think that, you know, even SAGE is something that people might be able to try and attack. We haven't seen, like we processed trillions of tokens.

28:01You know inside of sage and we haven't seen any like incidents to the evidence where that kind of can be circumvented and we also specifically post-train models that are Watching out for this kind of thing But I will say that this is the reason why you need an external system like sage because what a lot of people will do is they'll say I put these 10 guard rails into my model prompt That's the exact type of thing that the models are very good at certain event thing And so even the external system might not be perfect, but it's a lot better that I think everything else people are using today. It's funny because, you know, I see so many of these like open claw in particular, like YouTube videos, and they're like, and security is such a big deal, you really have to tell your model to be secure. Wait, what? Yeah, I see that this is, I think it's gonna be like the clouds and the hyperscalers in some way, which is like...

28:55I think that the labs and other folks will build in some security processes within it, but I think you're going to need someone else to police the infrastructure. Like you aren't going to want the infrastructure to be policing itself. And this idea of like, hopefully the model will do the right thing. It probably will 95% of the time, but that's a huge blast radius where things can go wrong. So you mentioned that Sage, you already have trillions of tokens flowing through this. How long has it been?

29:25around and is it like a, you know, SaaS offering? Is it open source? What's the kind of packaging for it? We went GA with the rubric agent cloud and Sage is the agent security, the agent for agent security inside of the rubric agent cloud. We went GA with that product in February this year, so just a few months ago actually. The reason we're already processing trillions of tokens is if you work with certain organizations that are starting to use these like agent harnesses, you're starting to notice usage go like this.

29:53We saw it even internally here at Rubrik. I think the question around how do we actually package it, we use it as a Rubrik hosted version, can do it directly as something that we host, or we can deploy instead of the customer's environment, which tends to be important for some of our customers that are in more heavily regulated industry. Whether you want a hosted version or something that you host yourself, we actually have both offerings. Talk a little bit about when you've deployed this.

30:25Like you kind of just mentioned like you know if it's you know finds 95% of things that's still a huge blast radius like I guess I'm trying to get a sense of like when you have this running in the wild like You know either anecdotally or percentage wise like how often are you seeing things that otherwise would have just been shocking? But you know you're able to identify and stop those things. Is it rare or is it? Realistically, it's all the time. Deployments at scale inside of an organization. So maybe just like a couple of anecdotes from like our deployment here at Rupert, because I feel that we could more likely speak to those. You know, one thing we noticed pretty early on was we're using Claude code very heavily. And there's a bunch of things that you saw that you're like, yep.

31:16I bet people were doing this and now I see it and actually, you know, catch it in practice. For example, people putting like raw credentials in like requests and responses, right? So like the literal text is just there. It's like the thing that you're supposed to not do in security 101. You see it there and you're like, okay, I want to block these types of actions in the future. So you can do that. But then we saw some more sophisticated types of things that were going wrong. And so we Claude code is a very common use case for what we're securing. And of course, those coding agents have access to GitHub and others.

31:45One thing we noticed was that cloud code was really trying to post things that were internal. This happened multiple times over the month. Internal source code to the wrong repository. And the problem with posting it to the wrong repository was it was posting it to a public repo rather than private. So it was actually taking parts of source code and trying to package it up and put it into these public GitHub gys. And so we saw this relatively frequently. And there was even instances where If we looked back through the audit logs, I think, and I don't want to misquote, but I think this is one of those things where if you saw the check, check, check, at some point, the way that Claude code had communicated what it was requesting was pretty confusing. But technically, it felt that it had gotten the human and the loop approval for it. But I think if we had the reason we were able to catch it, is we were looking at the full context. We were looking at what is the thing that it's actually looking to post? What is the destination?

32:41We have a policy that says anything that looks proprietary and internal should go to a public source. Let's go ahead and catch it. And we even saw one crazy instance where Claude code tried to get around this like blocking we were doing of like get a public just. And rather than like doing this as a text in text out system where it was like, all right, I'm posting to this URL. It spun up a browser window and we just started to see mouse clicks on certain coordinates. And we noticed that one of the coordinates actually was for a public just.

33:09And so we were able to go ahead and catch and stop something like that as well. So this is obviously like one line of I think interesting things we saw which was like more the sensitive data exfiltration standpoint. We see other things like in terms of credentials and others like pretty, I don't want to say like all the time, but it feels like it's all the time. And I think in general kind of reflects what I hear when I speak with leaders at large enterprise organizations, which is they're always surprised when they get like in an audit report of what's actually happening with AI in their ecosystem.

33:39I spoke with somebody who told me last year, I don't think I need the agent security governance thing because I think right now we have three or four agents that are deployed. Then I got dinner with them a few months ago and they were like, you know what? I think I was wrong. We did an audit. I guess how many agents were deployed? And I was like, it wasn't three or four, was it? And he was like, nope, it was 250. And I said, okay, got it. It's just surprising, I think, at the rate at which these tools can get adopted internally. And when they reported that number, was that like a...

34:06was that 250 people using cloud code or are these agents that are like, you know, 24 seven living on some infrastructure in the organization or mix of them. But it was a lot of like agents that were actually basically autonomous background agents. People had dealt like in the cloud, like on copot studios, one example, as a way to be able to just start to run tasks. I'm trying to form a question around it. I think that's essentially like pinging or pushing back on like AI only as a means of securing these agent interactions. And I could ask it from an enterprise perspective, is an enterprise going to feel like they have enough control? Or even from my perspective, I'm a little bit old school, and I want to say, OK, and in fact, it surprised me. I was looking at co-work.

35:03you know, and trying to connect a tool to co-work just the other day. And I was like, why doesn't it just give me the ability to say read only and not, you know, you know, everything. And it's hard to come to terms with, you know, not having that degree of control for people of a certain age. No, no, I think it's hard to come to terms with this for honestly everyone, which is like, I'm going to end up trusting. You're telling me I'm going to end up trusting.

35:31AI model to help me do my security and governance posture. What I think is like one of the things you pick up as AI inference security is there's this concept of defense in depth, which you're usually going to layer in multiple of these solutions. My view is that the link that's missing today is that AI in the loop system. We have plenty of good rules based system. We have plenty of good like other configurations. A lot of people like, you know, I'm not saying. So you're not saying that it needs to be just AI.

35:58or that AI is sufficient necessarily. It's just that if you have anything today, it's probably insufficient at keeping up with the volume and you need AI for. I'm saying two things, right? Like one of them is what you have is good, but you're missing this. The second thing that I actually believe is I actually think that this thing that you're missing is the most important piece of puzzle. Because yes, you want some of those deterministic rules. No doubt about it.

36:25But the example you gave, which was like, well, can I just go ahead and put this in read mode? Totally. You can put it in read mode. And a lot of ways that I've seen people secure certain infrastructure as they like look at my Claude code connecting to Salesforce in an email example, and they're like, don't worry, Doug, that would never happen to us. We disable all access to Salesforce. I'm sitting there and I'm like, how often can you just disable access as like the end solution if you're also getting board level pressure?

36:53to go and adopt AI to enhance productivity. So my point of view, Sam, is like, there's a lot of instances where you might end up saying, I never want AI to touch the system. But I think the majority of the instances are, I actually do need AI to do some of the work. I needed to have that right permission as scary as that is. But I wanted to be able to do it in the context of very secure runtime guardrails. And that's where I think we think about, that's why I think the AI and the loop system is the most important part. Ultimately, if your organization can continue by saying, we can block access for like everything, like the security posture will be blocking for everything. My point of view is you're just going to massively compress the ROI and AI that you get. I think the correct solution is going to be there's a class of things that are fully blocked and you're like, I can't think of a single reason to do this. That can be deterministic and rules. And there's going to be a large gnarly class of things that are going to be all it kind of depends on a policy enforcement and it's based on the intent in the context. And that's where I think the end-of-loop system comes in.

37:50I'm curious to what degree do some of the emerging protocols like MCP and A to A and the many, many other agent to agent and the like protocols change all this? Yeah, I think that they just introduced like a new surface area for where the problem basically shows up. So if I think about MCP or A to A.

38:12They're really good, like, interconnectivity protocols. And, like, they help, for example, MCP, Model Context Protocol, introduced by Anthropic, like, helps give agents sort of like an API they can understand the different applications that you might be running inside of your ecosystem. So I mentioned, for example, my Cloud Code uses an MCP connector to Google Drive. The thing about MCPs is that what we saw internally was, like, Cloud Code was approved for some subset of MCPs.

38:40But if you looked at the MCPs that were connected to Cloud Code, it was decently larger than the subset that were approved. Meaning that people were connecting Cloud Codes to custom MCPs and connecting it via MCP servers to ones that maybe weren't on that initial. So you had a centralized org that approved some MCPs, but...

39:03you had MCP sprawl within your organization. MCP sprawl is a great way to put it. You see some approaches to try and solve this, like an MCP gateway, and then you have to question how to get everything through the gateway traffic or not. But again, there's a number of layers towards this. I think MCP does really help if it's executed perfectly, the centralization of access.

39:25It still doesn't solve this like, even if like, I have the legitimate MCP connector to Salesforce, I have the legitimate MCP connector to my email, but MCP is not preventing me from exfiltrating Salesforce data to email. So it's not so much there, but it's helping me understand what applications should be authorized or not to give my given agent. So I find it like a very helpful protocol. Similar with A to A for like agent-agent like communication, I think that all of these are very useful for like structuring and making the scheme a bit more consistent.

39:54It also makes some of our job in terms of the patterns to look for a bit easier as well. I do think that as like the evolution happens there's an open question on like will MCP continue to be the thing or will these agents to start to use like command line tools with parameters that they find directly in the documentation. So we see a mix of both to be honest today and so we've expanded our definition from what was just MCP tools to just all tools which can include like direct API access in addition to MCP.

40:23injecting yourself over the wire or kind of in that wire conversation doesn't really matter to you because ultimately whether you are on one side, you know, curling a request or using a CLI or using an MCP at some point is over the wire. Yeah, exactly. It doesn't really matter to us. I think, you know, with the slight caveat that sometimes organizations have policies defined for their MCPs, they may not have defined for the other tools. So we want to help people enforce the policies that they may have.

40:53but we're able to see the traffic regardless. Got it. And then observability is a big conversation with folks that are building agents and being able to review traces and things like that. That's often using a similar kind of approach to reverse proxy or something kind of between the agent and the other things that it's interacting with.

41:24There, the noise is even greater than your tool approval box. But do you find that that is, is it valuable? Does it help you do your job? How do you think about observability as a field relating to securing agents? It's funny because observability can be used really for two purposes and I think there's distinct tools for the two purposes.

41:52Observability can be helpful for the people that manage the info or the developers where they're basically like tracking all their traces. They're doing evals over those and they're trying to get a sense of like, are my agents, you know, useful or they do it well or are they working? And then there's observability for security, which is a little bit more of the area where we're oriented towards, which is like our dangerous things happening. And you care about slightly different things in the two different contexts, right?

42:17I think to the extent that an organization's already thinking about agent observability, it's really good for us as a place to embed. Because I think about the Rubik Agent Cloud is again doing three things, like monitoring observability, kind of the consistent runtime enforcement with Sage, our small-image model harness, and the resilience and rewind capabilities. When I talk to a customer, I usually tell them the second and the third I view is differentiated. The first I view is like...

42:43Over time just commoditized like if you have an observability solution, that's great for us because what we want to do is like I view observability invisibility is kind of the base layer of the cake and what you end up doing is actually building out your more security and governance practices on top of what you have And so most organizations tend to only have a like really good observability for like a small portion of their agent stack So like to give you an example a lot of organizations are like, yeah We're building custom agents with Langchain on our cloud. We're using co-pod studio. We're rolling out Claude code. And by the way, like we just introduced Glean and agent for Salesforce. So we've got everything under the sun. Typically, my guess is like that only a first party agent has any kind of observability. Yeah. Probably those Langchain agents have a good observability stack and everything else in the dress. And then I'm standing there and I'm like, wait a second, but like Claude code is like usage is like over here in your organization. Langchain agent might be like down here, but you know, the observability stack is only on one.

43:38So we want to unify all of those into one place, but I think to the extent that organizations like OTEL, for example, like Open Telemetry and others that are standardizing observability metrics have been massively helpful for us. And I think like we'll continue to be a tailwind as we go forward. Because you consume them or because the enterprise has thought about certain things, you know, through the process of getting that level of observability in place that lends itself to willing.

44:04Both, because we consume logs like in an hotel compliant format, so if you've already had that system in process running, it becomes very easy then to start to direct those towards us. And then second, like now you're familiar with this idea, I'm gonna have like a system processing through this data and I'll be able to make sure that I have kind of this immutable trace somewhere. When we talk about agents kind of producing so many actions and decisions that they kind of overwhelm, the human and a loop oriented approach, to what degree is that really primarily an issue only for developers and folks using cloud code versus people using co-work that tend to be more interactive? Do you see that or do you see that independent of use case, people just can't keep up? It's definitely more present in developer workflows today.

45:03But I don't think that's going to be the terminal state. I think the reason it's more present in developer workflows today is because coding agents are A, the most heavily used, but B, they also tend to have the most broad levels of access. Oftentimes, if I think about a realistic cloud co-work use case, for example, it's like, help me make my deck or slides better or run this quick analysis in Excel. And today, I think that's where co-work is because it's newer than cloud code and others.

45:31I think what's going to happen over time is as you start to give this greater and greater level of access. So it's not just like making you a deck, but it's like running through reports and sales. Like similar to how in engineering, we're not just like, hey, make the button blue or like design the entire interface. I think as you start to do that and you're looking at creating, you know, full, full stack plans starting from Tableau, starting from Salesforce into Tableau, writing an email, having it go ahead and update something in your underlying system of record.

46:01I think things are actually going to start to introduce that same level of overhead. Because the fundamental reason for the coding agent asking permission and the co-work agent asking permission are really the same. It's like it's taking an action or operation on your behalf. So right now developers probably have like 10 times as much permission and access on average than the conventional co-work agent might. I don't know if the co-work agents will get all the way to the same page, but I do think they're directly on that trajectory too. Do you find that the SLMs need to be tuned to use case or you know even more narrowly customer or Do you ship them generically and they work the same? We find that it's helpful to tune the SLMs towards use cases being like policy enforcement So like you know understanding session data being able to arbitrate decisions on whether or not to allow or deny actions We find that post training the SLM for

46:58the organizational specific context hasn't been as critical so far versus like inference time customizations that we're able to do by being able to embed certain contexts from the organization at the time that the SLM is making the arbitration. So you know put simply today I think like post-trained SLMs for use case yes, ship the same one across customers and do an inference time customization.

47:23for each customer. I think that's consistent with where the question came from, which is an SLM that's really focused on cloud code probably needs a different way of looking at the data than one that's focused on things that are permissible in email conversations and business user conversations. Yeah, and even higher level than that, an SLM in general is going to be trained to generate all the tokens in universe still.

47:51But being able to go ahead and say, look, what you're really looking to be able to do is be very good at decision boundary between risky and unrisky actions, I think is an area where you get significant lift right out of the box. And we glossed over this earlier, but the broader rubric agent cloud, what is that doing? Yes. So rubric as a company has really two core offerings, rubric security cloud, which is all about our data and cyber resilience. So think about backing up data across all the organizations, backing up identity systems, helping people recover very quickly. That's one. The rubric agent cloud is the view of like, it's essentially like the parallel product now. And the rubric agent cloud is all about making sure that organizations are resilient for now and AI driven future. And that's the product that has the three core pillars from observability, runtime security would sage, and then resilience and rewind.

48:43And what is the deployment process typically look like? This is the thing that I think I'm happiest about with the Rubrication Cloud. And one of the startup learnings, actually, I'll share. It's like, how do you optimize for time to value, right? But one of the challenges with, I would say, post-training as a process is that it's not immediate time to value. You need to collect your data set, and then you get to train a model, then you eval it. One of the great things with Rubrication Cloud is you can actually just plug it into one of your agent runtimes.

49:10very easily and you start to see things relatively immediately. So if you want to hook into something like co-pilot studio or chat to PT Enterprise, we just launched an integration with Anthropic and the compliance API. You can actually basically do an API level integration. If you have the right permissions, it can take a few minutes to set up. And then the traffic is flowing, you're seeing all this runtime server ability. And you can add on more integrations as time goes on.

49:37Right, like we hook into a number of different sources. We hook into the mobile device management suite or the MDM for an organization. We cannot get the gateway level. But to get started, what we usually recommend is like, let's just pick one, authorize it and API level integration is pretty fast. And then you can start to see what the value looks like in that agent runtime. And then you can cover more of your stack as you want. And so when you take that first step and you do the API level integration, is the result of that primarily an observability tool or Is that also all of the decision-making that we've been talking about? Like, do you have to do more to get to decisioning or is it just there? It's funny, the world moves quickly. When we first rolled out Rubik Asian Cloud, it was predominantly observability and then you would need to configure your policies and then we could show you what's violating a policy or not. And people love the demo because I would show them like, look, you connect into your system and now I can write a policy and I'd show them like, I'm just typing out a policy in natural language. So something like...

50:35Don't take sensitive data and put it in an email, and now I can go and like run it. But then when we're deploying to customers, we realize a lot of organizations don't know all the policies they want to enforce out of the box. Instead, what they want is like, give me 80 or 90% of the answer. Yeah, catalog or something like that. Yeah, give me 80 or 90% of the answer out of the box, and then let me go ahead and pick and choose and customize as I want. And so what we launched relatively recently is a component where you hook us into the runtime.

51:02And we are like, you're good. We are running immediately. And we are showing you, you know, we've just naively called it insights, but we're automatically showing you like what Sage is picking up as traffic, the runtime enforcement, what's dangerous. We're suggesting remediations for you, like create a policy to prevent this in the future. And then you can do the additional work to customize it further. But our goal has been like fast time to value and really solving most of the challenge out of the box. And so once you've connected it, That's really all you need to do to be able to get a large chunk of value upfront. So given that the space is moving very quickly, where do you see it all going? What's next? What's coming? It's funny because earlier in the show, I think you did ask about, hey, if I'm old-fashioned and I'm worried about this idea of just deferring all control to an AI system or a harness, how should I rationalize that? We kind of took this bet last year that we wanted to do more and more and the loop.

51:59And I actually think last year when we made the bet there was a part of me that was nervous and I used to hedge the answer a little bit more and say like we're really going to want both, we're going to want non-deterministic and deterministic. I do see the world increasingly now just moving to AI systems that are running these processes more hand to hand. And so I think where the world is headed is I think right now a lot of agents are still stuck in read mode. I think that's going to change.

52:23I think agents are going to graduate from read mode and go on to write, delete all the card applications because I think they're essentially going to be doing the types of work that humans are doing. I think token and inference spend is going to continue to scale up very, very quickly. I think we are going to see a series of large scale, I don't know how we want to call them, mistakes, incidences. I think agents are going to, like, I think that we're going to see a massive amount of productivity and I think we're also going to see some inadvertent incidences where things have gone wrong.

52:52And I think that what we're going to end up centralizing on is that there isn't another good way to be able to solve this problem. The rules in the human loop will still keep for certain applications because they also help us feel good and because they're necessary for some applications. But I think what we're really going to go center around is like we want these workflows to become increasingly AI first. And that we're going to use the same type of technology that's introducing the risk to also be a part of that solution.

53:17So the world's moving very quickly. I think agent harnesses have absolutely exploded earlier this year. I anticipate that to continue. I anticipate coding to be still a dominant use case through this year. But by the end of the year, I anticipate the same thing we're seeing for coding is going to be happening for many other types of sophisticated knowledge work. And I think we're going to start to see the same type of request that we need now for monitoring coding agents starting to extend across the agent stack. And I can't see another way to do it without being AI in the loop.

53:44Yeah, it's interesting when I think about that. I tend to agree. I wonder about, like, of the tools that I use, you know, for example, like the, you know, G Suite stuff. Does it have an ability to do snapshot? Like, can you hook in for that third pillar of yours? Like, do you have a way to say, hey, this agent just blasted this user's calendar? Can you restore it for me?

54:12So, uh, you know, I do think that a large part of like Google works, uh, workspace is something that rubric does have integrations to be able to back up. Yeah. So this is tying over to the like traditional core product of core business. Yeah. Exactly. Um, so, you know, to the instances, I think there must be some instances where we don't back up, but certainly like a large chunk of like important, like where important information is, is tends to be the areas that we like to be able to back up.

54:39And you because builds a large business on the backs of that And so I think that it's a good question How do I make sure the agent like look to feel comfortable giving your agent full right and delete access to your calendar? You'd probably feel a little bit better if you knew you could rewind that action in case And ultimately that's why I think that this is like the third pillar and also the pillar that I think like we're You know the third pillar and also the way we approach the second door I think those are the two unique things we're doing because frankly a lot of people are gonna be in the agent observability space

55:08And if that's all you need to solution, there's going to be multiple vendors on there. I'd have my point of view for maybe YS, but there's going to be other good vendors. But what we really think about differentiating us is Sage and Rewind. Well, Deb, thanks so much for jumping on and sharing a bit about the way you're thinking about agentic security. Of course. Thanks so much for having me, Sam. It was a lot of fun to have a conversation. A lot of fun. Thank you.