The_TWIML_AI_Podcast_formerly_This_Week_in_Machine_Learning_&_Artificial (3)

0:01I'd like to send a huge thanks to our friends at Eleven Labs, the company behind AI Voices that don't sound like AI Voices. The Eleven Labs platform empowers developers to deploy lifelike conversational voice agents for a wide variety of use cases, from customer support agents that resolve complex inquiries in real time, to AI phone agents that handle high volume calls with natural dialogue, and even personalized one-on-one tutors that adapt to each student's learning style.

0:29Join the thousands of developers already building with Eleven Labs' voice APIs. Experience what truly human-like AI voices can do for your applications. Visit elevenlabs.io slash dwimal to get started for free. In, you know, let's say 2023, 2024, the way most people are trying to build agents is a human designs a system and that system kind of breaks the problem down into multiple steps. It assigns each of those steps to an LLM. Maybe there's some rules built in, but...

0:59Oftentimes, the real world is messy, and that workflow that you built might be an oversimplification of the actual process. A real expert at this task would follow. And when models are able to learn how to do the process by being rewarded for succeeding at the process, they're able to figure out, in many cases, something that's better than you could easily sit down and design yourself. All right, everyone. Welcome to another episode of the Tuomo AI Podcast. I am your host, Sam Charrington.

1:38Today, I'm joined by Josh Tobin. Josh is a member of Technical Staff at OpenAI. Josh, it is great to have you on the show. We last spoke. Can't believe it's been over five years now. That's amazing. Yeah. It is amazing. In fact, yeah, I think it was. I think we spoke in Vancouver, like right on the tail end of Newreps or during.

2:04during the NURUPS there. I think that's right. I had a paper there that we were talking through. I actually haven't noted here, geometry-aware neural rendering. Yep, yep. Yeah, that was actually the last thing that I worked on and my last stint at OpenAI before rejoining again in September. Awesome, awesome. Well, you've done a few things since then, I think.

2:29Fullstack Deep Learning was one of the things that folks may have heard your name pop up on and then you were working on a startup. Tell us about what you've been up to over the past few years. Yeah, so I left OpenAI back in 2019 and co-founded Gantry. We were a machine learning infrastructure startup. And when we were kind of thinking about sort of where to take that business, at the middle of last year, ended up through a bunch of different circumstances, winding up, coming back to OpenAI. And here, I've been leading the agents research team. So our team builds the models that power our agentic products, like Operator and Deep Research and the Codex CLI, which we launched a few weeks ago. And we're going to spend...

3:21a bit of time talking about agents and those agents, agentic products. I'm curious in thinking about Gantry and what you were doing there and I think Gantry was in kind of a crop of ML infrastructure, ML ops companies that I think Ran into this like although the air getting sucked out of the room by Gen AI like I'm curious if you have any takes on that space and like how it relates to You know what we're all trying to do with Gen AI now. Yeah, absolutely. I mean, I think it was you know in the pre chat GPT era I think a lot of a lot of companies mental models was that Every company is gonna need to be training models You know before we had GPT 3 and

4:16later models that did this more effectively. The way to create actual value with AI is to train models that are fit for purpose for the thing that your business needs to do. So there's a whole category of infrastructure that was built under the assumption of every business out there more or less is going to be training its own models in the future and so what tools will they need. I think what turned out to be true is that general purpose models, GPT-3, GPT-4, other large language models are, you know, are like quite good at most tasks that you can specify. For any given domain, you might be able to build a better model than GPT-4, but it's going to be pretty hard and pretty expensive. And if there's an off-the-shelf model that can just do those things well for you, it's so much faster, cheaper, more efficient to build.

5:12to whatever business needs to build on top of that commercially provided model instead of building up all the infrastructure expertise and data and domain specific knowledge that you need to in order to build them all yourself. I think that didn't fully invalidate, but it made the business model that a lot of that crop of ML infrastructure startups was thinking about a lot less feasible because now I think My operating assumption is when I advise companies on how to think about ML, I generally tell them, don't even think about training your own models until you've exhausted what you can do with the models that OpenAI or any other foundation model provider will sell you. Yeah, that's absolutely a smart approach. I recently wrote an article that reflected on a paper that I wrote back in that 2019 time frame.

6:07I called out this idea that I thought that I got both right and wrong. The core idea was that in order to be competitive, businesses would need to become model-driven. This core idea of pulling patterns out of data and putting them into operational workflows in order to make decisions faster and more accurately than humans could was a key element of what ML was offering folks.

6:37And I think, you know, businesses being model driven is even more true. But the idea that like everyone was going to build those models themselves, as you just noted, turns out to be less true because of foundation models, right? Yeah. And I think, and you're starting to see kind of a crop of CEOs now, you know, publishing their thinking and the way that they're kind of urging their companies to use AI internally. And so I think that is absolutely true and is actually coming true now. There are important companies that are sort of walking the talk when it comes to adopting AI and making it a pretty core part of a lot of what they do. Which is kind of like what we all hoped or thought was going to happen back in the day. It just happened a little bit differently I think then.

7:35maybe some of us anticipated. Yeah, I think you're referring to TweetX that went viral by Shopify CEO, I forget his name. And he talked about kind of trying to drive urgency among their teams to adopt AI internally. And I think, you know, that maybe sets the stage for one of the things that we want to cover in this conversation, which is like, what it's going to take to make agents real more generally. I think the presupposition is that we're not fully there. They do a lot of interesting things, at least when bounded by whatever appropriate scope. But I think the general vision of my...

8:26psychic agent that you know sister you're waiting to or even anticipating my needs and then going out in the world and operating on my behalf Without constraints, you know, we're still a bit of ways from there. Yeah, how do you think about? What agents are really good at today? You know relative to what we might want them to do. Yeah, you know, I think it we're closer to that than it seems I think that the misconception that a lot of folks have about agents is Before I came back to OpenAI, had spent a little bit of time tinkering building agents on top of LLM APIs. And the problem that we ran into, which I think is the problem pretty much everyone runs into when they try to build their own agents using workflows on top of LLM API calls is that it's very tempting to think, oh, LLMs are good at making point predictions, making decisions.

9:25And so we build a workflow around this. Then we can sort of automate a process and take a bunch of decisions and turn it into something that just happens on its own. But I think what a lot of us found trying to do this is that the temptation is there because you can create a demo of this very, very quickly. That looks great. But then when you try to actually deploy this and get people to stop doing this work themselves and instead delegate it to an agent.

9:54You start to run into all kinds of edge cases and failure modes and getting things to work reliably is really hard. And so the root cause of the problem is that most LLMs historically have not been trained to do agentic work. And what that means is that at any given step of the process, maybe they're relatively accurate because they're pretty smart general purpose AI systems. But as you run a process that requires many steps, the small errors at one step compound as you take multiple steps. So even if you're, you know, 90% accurate on one step, if you have to take 10 steps, then your accuracy will fall off. And so that's, I think, kind of the core challenge of building agents on top of traditional LLMs. The missing ingredient has been that we need to directly train these agents end to end to do these

10:53workflow-like tasks. By doing that, you can train the agents in such a way that they see failures during their training and they learn to recover from those failures. So even if each step is only 90% accurate or 95% accurate, now the model has seen what it looks like to fail at that step and it's able to reroute itself. It's able to think, oh, this doesn't look right. Let me go back and try that again. Do you have a canonical example that kind of illustrates this problem?

11:24Yeah. So a good example of this would be if you are trying to build an agent that does research for you. So if you do a single web search, and maybe you get the search term wrong, like the user doesn't know exactly what to search for. So you try to search for terms that seem relevant. And you start to pull back a bunch of docs that don't have information that really gets to the heart of the problem.

11:54Then a naive agent might just get confused by that and it might think, well, okay, I searched for the term that I thought that they meant and all these docs are irrelevant. So maybe this is, maybe what the user asked for doesn't make sense or maybe I need to go down this other rabbit hole. Whereas an agent that's trained to do web research has been trained using reinforcement learning to sort of be good at this, multi-step process. Well, in its training, I've seen many instances where it searched for the wrong term. And the training has incentivized it to learn to recover from those instances, then instead go back and think, oh, you know, I searched for this term, but I got results that weren't relevant. Maybe that means that I had the wrong search term. So let me go, you know, try to try again and pick a different one. And so the key is really like, um, raising models, uh, power that are trained end to end to solve the kinds of tasks.

12:56that users need them to solve using reinforcement learning so that they're able to kind of like see these multi-step processes, see the kinds of failures that happen in training and learn to recover from them. It strikes me that the creation of models that are specifically designed to do this is one powerful difference that we're seeing in this new generation of models but also it's like a collapsing of capability.

13:24a bunch of stuff around the model that was trying to keep it in the right path to just making the model better. The deep research... I forget the name of the paper, but there was a paper that kind of proposed this deep research idea really early on, and it was like...

13:47a multi-agent system with one agent that figured out search terms and the other agent that reviewed the results and rated them. There were several agents in this loop and they would ultimately write whatever the report is. The impression that I get is that as we're training these models to more directly take on these tasks, it becomes less of a multi-agent type of...

14:14solution. The way I see it is like in, you know, let's say 2023, 2024, the way most people are trying to build agents is a human designs a system. And that system kind of breaks the problem down into multiple steps. It assigns each of those steps to an LLM. Maybe there's some rules built in. And there's two problems with that. The first is the compounding error problem that we talked about before. But the second is just that like, you know, I think I'm uh, borrowing this idea from, uh, an old idea from Andre Carpathia, but like, uh, good models are just much better at us than designing these types of systems. Um, and so, you know, you can, you can kind of imagine how you want to break the problem down and think through the steps that you might take to do it. Um, but oftentimes like the, the real world is messy and you're that model that you built, that model that you built, that sort of, that workflow that you built.

15:13might be an oversimplification of the actual process. A real expert at this task would follow. And models are able to learn the process, learn how to do the process by being rewarded for succeeding at the process. They're able to figure out, in many cases, something that's better than you could easily sit down and design yourself. When you think about applying that idea, you know, broadly among, you know, developer team fighting software and startups and enterprises like the ability as you alluded to earlier, like the ability for open AI to create a model that's designed to do these things that improves these things is very different from, you know, a startup or, you know, enterprise academic institution like

16:10I guess part of the question is like, do we just have to wait for like, what can we do now to build more robust agents? No, you can use, you can go use like real working actual useful agents now. I mean, deep research I think has been incredibly useful for a lot of people. It's, you know, people are using it for kind of like business and research workflows, but also scientific research.

16:39travel and shopping, programming. Sure. Not to take away at all from deep research or codex, but research and writing code are two of many, many problems that I have or might want to tackle with agents. How do we get to a more generalized, agentic model that...

17:07is easier for your users to build their own models with. That's the general question. A sub-question is like, is that GPTX or OX or whatever? Are all models going to get better at agentic capabilities?

17:34error correction instruction following kind of melee that you just referenced or Should we expect to see you like Agents agent specific types of models. Well, I do think it's very much like as the models get better Building these types of workflows will be a lot easier already. I think o3 is like quite good compared to older models at at being able to kind of understand the level of complexity of instructions that you often have to have to, you know, to automate a process or build a complicated workflow. And it's better at adhering to those instructions and using tools over multi-step trajectories than anything else that can be for it. I still think there's a ways to go before that is a generally useful agent. But I think that, yeah, as the models get better,

18:34building custom agents is going to get dramatically easier. Are there characteristics of the models? Like, is it always going to be the case that you need a certain size or complexity of model to do agentic things well? Or, you know, can these types of behaviors that we're talking about like be distilled into smaller models just as easy as kind of the core language?

19:03Fluency that you know, we've been able to distill in a smaller models. I don't think we've really exhausted our limit to of how good we can make small models But I think the advantages that large models have are that large models tend to be better at generalization So if you're and and you know most of the time if you're building something custom on top of models You know that that thing might not be something that the developers of the model anticipated so larger models will tend to work a little bit better. And I think the other feature of models for agentic use cases that's really valuable is reasoning because a lot of agentic tasks have this quality where there's a range in difficulty levels of solving the task. And doing the right thing at each step of the task is very important to make sure that the overall task succeeds.

20:02So, letting the model choose how much reasoning effort it wants to apply to figure out the current step of the workflow. To me, it feels like a pretty important component of what's going to make these systems work well. And is that particular aspect new in O3 relative to O1? I've noticed with O1, It was a solid reason for the same amount of time or on the order of the same amount of time all the time whereas 03 can give you a snappy-ish response but also take a bunch of time to think through things. It's not new but we're continuously getting better at making models that are smarter and knowing how much to think. So I cut you off when you were talking about deep research and some of the ways that folks were using it.

21:00I think probably everyone listening gets the core value proposition thesis behind deep research or their things that you find that surprise folks that it can do that folks might not know or are there examples of how folks have kind of pushed the edges with it? Yeah.

21:26I think when people think about deep research, they think about using it for market research and scientific literature review and kind of other tasks where the model just has to go out and think and read a lot of documents to come back with a synthesized answer. But some of the other things that people are using it for, which were initially surprising to us, are coding.

21:56So it turns out, even though we didn't really design the model for this use case, there's a lot of code publicly available on the internet. And because of the capabilities in the base model, the model is able to understand it very well and is able to search and kind of find code on GitHub and understand how different pieces of the code base work and come back to you with descriptions of the code or plans for how to implement a feature or things like that. And then I think the other kind of underrated use of deep research is for kind of finding very rare facts on the internet. So I think people, like probably the bulk of what I think most people use deep research for is going broad and then synthesizing. Like, you know, find a, go read about this topic and come back to me with a summary, like covering these points, where the model has to like go touch a bunch of different topics. But the model is also quite good at finding

22:54Information that is just kind of buried in the corner of the internet somewhere like if you have a question about You know if you really want to know to recall like the details of some television episode that you saw a long time ago Where you know Maybe it's it's something specific enough that it wouldn't just be on the Wikipedia page Often the model will be able to find kind of that like one fan page or something like that from ten years ago and Pointed on information to yeah, even four O's really good at that I'm trying to imagine how obscure you have to get in order to go to deep research for that kind of problem. And remind me, deep research is not accessible via an API currently. It's only via one of the surfaces. Is that anticipated? We found that mostly the kind of use case that we were imagining for deep research was

23:54kind of you're you're in chat to BT for something that you normally use chat to BT for and then You there's something where you just really want a much more thorough answer than you get out of the box with chat to BT That's kind of the use case that we designed it for But um you're right I mean there's a lot of other things that you could use it for where having an API would be useful we talked just a second ago about kind of the Spectrum of compute that you might want to throw at and a response and how, you know, 03, for example, can, you know, kind of start to manage that. Like, at some point, I might want the model to just know if my question calls for deep research and just do it, right? Yeah, that would be the dream, right? Like, I think the thing that I would love ChatGPT to become is a place where you can just go and it's like talking to

24:53Your friend and your co-worker and your personal assistant and your coach all in one To where you could just you can ask a question like you would to a person And just like a great co-worker would it knows You know when to come back to you with a really quick answer off the top of its head versus like from the context and from knowing you When it should actually go and do a bunch of research and come back to you with something more thorough Well, or both like yeah, so impression, but I can go research that yeah exactly, right? It's like yeah Here's here's the answer off the top of my head, but like I'll go dig into this a little bit more Or it could you know come back and like in deep research There's this phase in the beginning where it'll ask you a few follow-up questions, but I think eventually it should be able to Just come back to you in the middle of its research and say like hey, here's what I'm finding so far I'm gonna keep going, but is there any other is there any feedback?

25:51Do you have any feedback on this? Is this what you're imagining? Or are there any, you know, now that you've seen this, are there any other avenues that you want me to explore? So I think like evolving ChatchaBT into an entity that feels a lot more natural to co-work with is one of the main things that I hope we're able to achieve. You mentioned the follow-up questions that happened at that initial phase of deep research. Can you talk a little bit about the Well, how explicit was building that or tuning the model to do that? Yeah, what we found is that deep research is, since it's designed to go deep and collect a lot of information and come back to you with a very detailed report, that is really excellent at covering all the finer points or the nuances of the initial framing of the question that you wrote.

26:50the quality of the results that you get are very sensitive to, they're not very sensitive, but you get better results if you put time up front into thinking about what you really want to see at the end. Because the model is very good at adhering to those things and it's very good at like knowing how to incorporate the different parts of the question that you ask into its search. And so the model will also come back with reasonable answers if you kind of just fire off a quick question to it.

27:20But the upfront sort of back and forth is meant to help flesh out a more detailed thing to run by the model because we think it produces more compelling results. Okay. And was that an emergent, you know, phenomenon like you train the model with an angle of like producing good results and it figured out that it needed to ask these follow up questions or was there a degree of direction in there that I don't know how you might do that. I mean, yeah, any number of ways you might try to do that Yeah, the upfront questions were something that we we added kind of after the fact as like from observing the way that users interacted with the model and You know from experience you can suggest that you're not sure how to frame the question and that you might want some you know, you can prompt it to you know

28:17ask you questions and that works just fine as well. And one thing that we've seen a lot of people do is to use 01 or now 03 or 04 Mini to help craft the question. So if there's a topic where you don't even really know enough to frame a good question, a lot of times folks will have like a conversation with a smaller model and then they'll use that to sort of flesh out all the details of the question that they want to research and they'll put that into deep research and have a go, you know, spend 10 or 20 minutes coming up with a much more research response. So deep research was the first agentic offering? Well, we also have operator operator. You consider that an agentic system? I do. Yeah, I think of an agentic system as as any AI system that is able to go work on tasks for you that take like longer than a few seconds.

29:14And that has to interact with the real world in order to solve your problem. So yeah, operator deep research and Codex CLI I think are all good examples of agentic systems. Got it. So let's talk a little bit about operator. I tend to think of it primarily, I've not used operator actually, like I thought of it as like a computer use analog. But it sounds like it's a higher level.

29:42system that you interact with and have it do I mean I've seen demos and that's the way it was presented but I thought that those were demos of an underlying API as opposed to a Service that I might want to use yeah, no operator is it's also a service inside of chat GBT And so you can go to operator There's a link to it from chat GBT, but as a separate kind of web app That works a lot like chat GBT and so you can kind of you can type in a request of something that you want to do book a restaurant reservation, let's say. And then you can kind of watch in a virtual browser. The agent kind of navigate to the web page, kind of click around the web page and do the task that you asked it to do. So it's really, really cool. It's like super fun to watch because it's just you can kind of feel the intelligence and the thinking as you watch the model, like click through the web much like you would.

30:37Are you finding that there are folks that are using it productively versus education and entertainment? Yeah, absolutely. So operator, I would say is like the technology to make operator work really well is incredibly difficult thing to build. And so operator is not intended to be the thing that every single person in the world uses every day on day one.

31:05It was meant to be kind of like an early launch of this technology so that people could give value from it and could also just get a feel for where this is going. But despite that, there are a bunch of people who do use this a lot for power users. I think of it a lot like early GPT-3 API, if you remember that, where it's like, people use it and it was, I don't think OpenAI would have framed it this way at the time, but it a lot of ways is kind of a technology preview.

31:34It was something where most people tried it and they were like, oh, this is really amazing. It's amazing that humanity has created this thing. But they didn't immediately find it useful. But even on day one with GPT-3, there were a bunch of people where they were just really drawn to the technology and they tried it. They played around with it a lot and they got to the point where they figured out how to make it really useful for them. I think operators kind of more at that stage of...

32:04model development, where there are power users who love it, but it's not for everyone yet. Yeah, that is my impression as well. It's not surprising to hear you put it like that. I've seen some examples of, think the theme that I've seen the most is arbitrage. I'm going to have this operator find a bunch of stuff on eBay that I can sell for more on Craigslist or find Airbnb's and Message all the hosts to try to get a discount or something like that like I looked at them and I was like they were novelties But you know, maybe someone is actually doing that for or something along those lines for You know something that they think is useful. Yeah You get a lot of value from it If you if you sort of put in the work to figure out how it's useful for you Yeah, and then beyond figuring out how it might be useful for you are there

32:59kind of non-obvious ways to make it useful, meaning I've got my use case, like how do I need to think about the world or my problem or operator in order to, you know, make it work? Yeah, so one thing that I found to be really helpful for getting the most value from it is there's a way to add sort of site-specific instructions, add or customize site-specific instructions. And so, you know, if you've, if you're imagining operator visiting a website for the first time, it's kind of like you're, you know, you're seeing this website for the first time without any context or the very limited context of the world or who the user is. And so there's a lot, there's like a lot to figure out, but you can provide instructions that help the model understand how to use this site to solve their problem. And that tends to make it like a lot more repeatable and a lot faster. So like, don't just say click.

33:55upload in the menu, say, you know, the menus, the hamburger thing on the right, and upload, like, how granular do you need to... First of all, are we talking about, you know, location, localizing capabilities on, you know, the page, or are we talking about other types of context that is useful for the agent, and then how granular does that tend to need to be? Yeah, I think it depends a little bit on the site.

34:23And, you know, sometimes it takes a little bit of tinkering to get exactly right. But I find mostly just like helping to clarify the intent. Like, oh, if I want to, you know, if I want to book a flight, here's how you do that, that kind of thing. I guess there's Codex is the third agent in the portfolio. And with Codex, the Codex CLI, Codex has been around for Quite a while. I think when did I talk to Greg about the launch of that? That was August 2021 when we caught up about that. This is a different Codex. We're reusing the name. But it's not intended to be. There's no underlying model. I think the original Codex was the first code completion model. This is a throwback to that name, but a very different model.

35:17And yeah, Codex is our local code execution agent. And so it's just a package that's fully open source that you can install on your laptop. And then you can talk to it. And you can ask it questions about your code. Or you can ask it to implement things in your code base. And this agent is able to use your computer to navigate the file system, find the relevant files, understand how they work, write code on its own, apply those patches to your files, run tests, try things out, and come back to you when it has an answer to your question or, you know, it's written some unit tests or it's written some code or whatever you really want it to do. So I think it's a...

36:18really powerful demonstration of where I think AI systems for coding are going. And people are loving it so far. People at OpenAI use it a ton. There's more than 20,000 stars on GitHub, which is amazing to me. We've got around 100 people who have contributed to it. And so I think a lot of people are feeling the magic of it. In a world where you've got a cursor and...

36:45GitHub co-pilot and WinSurf and maybe hundreds more of coding agents slash IDEs with coding plugins. Like what differentiates Codex CLI, Cloud Code type of experience from those others? Yeah, I would say most people, most software engineers here use Codex CLI and potentially like some AI-powered functionality in their ID.

37:14They're kind of used for slightly different things, I think. The IDE functionality is great because when you're inside of the IDE, well, when you're writing code, you're most of the time inside of the IDE anyway. And so those tools are amazing for just having very quick access to the AI system in your flow state. So if you're in the middle of writing some code, you can get a suggestion or you can...

37:44you know, delegate something to a model and then just keep going with what you're doing. These these agentic tools are better for like, or at least I find myself and a lot of folks find themselves using them more for like DeNovo type work where you, you know, maybe you're just at the start of the project, or you're working in a codebase for the first time, or you have an idea for a feature that you want to build that is not like literally the thing that you're working on at the moment, but it's something that you might want to delegate to someone else. That's when this agentic paradigm becomes really powerful because today the way the Codex CLI works is you give it a task to do and it just does it on your laptop. One of the really cool things about it is that it's able to do it in a network sandboxed way.

38:44Meaning that it can just go and kind of work for you in a way that is it is safe because it's not able to you know It's not hitting the network and so it's not able to you know accidentally You know run some on safe command or something like that But I think over time the But it's still kind of like running on your laptop and so you still need to have your laptop open and Sort of watch it as it goes I think over time like where this is going is that interacting with these coding agents should feel more and more like delegating to someone where they're able to take larger and larger chunks of work and execute them more and more autonomously to where you can just, you know, if you have like 10 things on your to-do list in the morning, maybe you go work on one and then you come back at the end of the day and the other nine have been finished for you. So that's kind of like, I think the paradigm of agented coding and, you know, we're not there yet, but where we're trying to get to.

39:40When I think about the idea of it being best for DeNovo types of projects, I think of, okay, you know, we've got these three classes now, we've got the cursors and the like, we've got your V0 bolts, which I think of as like DeNovo web and now the Codex CLIs of the world or maybe like DeNovo back end or something like that. I guess when I say DeNovo, I don't necessarily mean the whole project is DeNovo. Actually, one of the amazing things... Or it could be a feature, right? Yeah, a feature or... What you're saying?

40:10Yeah, just something where it's distinct enough from what you're literally the code that you're typing right now that it's worth starting a new thread for yourself to start working on this independently. With that in mind, I was going to raise a question around context. One of the really interesting things that I've you know, seen recently that helped me understand things that I've experienced is this idea that like when you're interacting with AI-assisted code generation tools and IDEs, like one of the things that they're, you know, limited by and doing is like managing the context of, you know, that has ultimately passed to the LLM and that's why you get

41:05you know, different results between like just typing in a prompt in cursor, for example, and, you know, versus copying your whole file and, you know, putting it into chat GPT or whatever, you know, chat model use. And, yeah, it's been suggested by some that, like, one of the things that these kind of CLI-based agents, you know, do better somehow is context management, like.

41:32Is that true? How do you think about context in this context? And the role that it plays in in development tools. Yeah, yeah, I think that like the Codec CLI is you can almost think of it as like a contextless tool where my mental model for for giving a task to Codec CLI is it's kind of like you're giving a task to a like a superhuman intern who has never seen your code base before. That's only a little oxymoronic. Yeah. An intern because you can't really trust it yet with like really, really complicated stuff that where you need a lot of state or a lot of experience working on.

42:26this particular type of project where there's a lot of different pieces that need to be broken down. I think you kind of want to delegate like intern-sized chunks of work to it. Superhuman because it's able to read code, write code, understand code, understand coding patterns much faster than any human can. And seeing your code base for the first time because it is contextless. And so when you start a new task in the Codex CLI, the model has to explore the code base on its own before it can start working on the task. And the way it does it is just, it's pretty amazing. It's not using any kind of bespoke context management tools or ideas. It's just using the same command line tools that people have been using for decades. Yeah, like you see it said files and stuff like that. Exactly, yeah, to explore the file system.

43:25And it turns out, which is, I think, really surprising in some ways that the model is just able to build an understanding of how to navigate the code base and how to find the information that it needs and where to write code and what the patterns in the code base are extremely quickly just by doing that. And the coming back to what we were talking about before of the magic of training models using reinforcement learning to achieve an outcome, I think this is kind of like the type of magic that emerges when you do that type of training is models are able to learn how to use tools really effectively to achieve their goals. And so, you know, my impression is that these models are much more efficient than humans are at like how much code they need to read before they're able to actually build something. So that's that's that's kind of how I think about context and in for these models, but

44:25That being said, I do think that we could improve on this by giving the models more context or having richer customization. So right now there's a way to customize the behavior of the models by sort of placing a file in the repository. But you can imagine all kinds of other types of customization too, like you can imagine giving APIs or MCPs to the model.

44:54you can imagine giving the model memory so it can remember things between rollouts. These are lots of ideas that we're kind of exploring for how to make the models kind of even better at understanding the specifics of your code base and get smarter as they interact with it more. And how do you think of Codex, particularly with it being open source, like in the spectrum of research preview, here to get you thinking, first steps to something that reflects what we ultimately want to deliver? Yeah, I think the open source piece is really critical to how we're thinking about building Codex. We're very much trying to do this the right way as an open source project. We're kind of building with the community. We're accepting tons of contributions from folks. I think there's around 100 contributors now.

45:52We have sort of opinions and a vision for where we want to go with it, but we're also letting the community kind of help guide the direction that we take with it. But in terms of readiness, Codex CLI is really useful now. Yeah, people at opening, I use it all the time. I think a lot of folks, kind of a lot of the early adopters have found a lot of success with it. I think especially with O3, it's able to just kind of solve a lot of...

46:21Sort of yeah smaller medium pieces of coding work that might have taken you a lot of time to do otherwise Or might have just been really hard because you don't understand that part of the code base or you know It's some front-end work and you're a back-end engineer or vice versa So yeah people people who are using it are getting a lot of value from it one of the Complaints if you will is about like cost and cost opacity with using these tools You know everyone's miles gonna vary in terms of value, but like how do you think about that? How do you see is that an in the area or Is that something that you? Anticipate evolving over time. Yeah, I think like if you look at the history of models the cost has been falling pretty dramatically and also I think we're in the early days of you know the

47:17The Codex CLI is brand new because the models have only recently gotten good enough for this to be a really good experience. We're also in the early days of model capability. As the models get better and more consistent at producing high quality results, the cost is not going to matter as much because I think already in many cases, it's saving people a lot of time.

47:48Like the you know paying a few dollars to save you hours is you know depending on what you're doing is often very easy trade to make But but yeah, I think also like if history is any indication You know, I think it would be reasonable to predict that the cost will also come down. Yeah. Yeah, I was trying to get a sense for whether the predominant like trend or answer to that question is, you know, the cost is just going to go down naturally like we've seen or yeah, we hear that and you know, we're going to be creating more visibility like, oh, that request is going to cost you 42 cents. That's a cool idea too. Have you seen integrations with Codex CLI and other systems like, you know, Slack like or Devon like Slack?

48:43interface or something like that. Like does the open source aspect lend itself to folks doing kind of interesting things and connecting it to other systems? Yeah. I mean, that's one of the things that we really hope is going to happen. And that's part of why we decided to open source it is, you know, I think that there's a lot of places where you can imagine just programmatically triggering a coding task to start happening.

49:13So one of the obvious ones would be in CI. Like if you have a build that's failing or some tests that aren't passing for whatever reason, we've seen people starting to build Codex CLI into their CI workflows to try to take a first pass at automatically fixing those issues. But there's all kinds of other things where it's like, oh, you have a automated. Or even an issue posted. Yeah. Give it the issue test.

49:41you know, what might be causing this? Yeah, where you have like some kind of like feed of potential coding problems that are coming through. And, you know, and kind of like with deep research as well, like the, I think that primarily what people use these tools for initially is to do work that they just otherwise would not have done. Like if you're, you know, if you're kind of a busy software engineer and you have 10 things on your to-do list, Maybe you're only gonna do two or three of them and that's not that the other seven aren't valuable It's just that they're kind of like below the cut line of what you can really realistically prioritize And so I think that's that's one of that's the thing that I'm most optimistic about using these agentic tools for in the near term is it's just kind of like Doing all the stuff that you want to do and that would be good to do but you just can't you can't do as a as a person with limited time

50:38Yeah, the intern analogy I think is the most popular, but I always like the sous chef analogy. Like, you know, there's this thing that I'm really good at. I have my recipes, whatever. But like, don't make me cut the carrots. Just give me the chopped carrots and, you know, let me do my thing. That is another place where we've seen people use the Codex CLI a lot is like, yeah, doing the kinds of engineering work that you just don't like doing.

51:07Again, yeah, oftentimes I see this with like, you know, back end engineers who don't like writing JavaScript or, you know, very like product oriented engineers who don't like doing a bunch of data plumbing stuff are very happy to just, you know, sometimes you have to do that stuff for your work. And so makes people happy to be able to have the model take a first pass at it. Yeah. What do you think are the biggest things on the horizon that are going to change the capability of these coding models. I think folks who have used them, maybe let me ask this question. Kind of the same question, but let me ask in a very different way. What's your take on the whole vibe coding thing? I think that we are in the early phases of a dramatic shift to the way that software is built, where, you know, look, I don't think writing code is ever going to go away.

52:09But I think that the process of manually writing code, or you have to write down kind of every single line of code yourself and manually test it and think through the architecture and kind of do each of these pieces by hand, it's going to become rarer and rarer. I think that most code, I think the vast majority of code is going to be written by AI systems.

52:41much sooner than people think. I think the job of developing software is going to become a lot more like thinking about the functionality that you want the system to have, thinking about the trade-offs, thinking about the edge cases, guiding the AI system, providing feedback to it, finding ways to validate its work. So I think that just the nature of the work is going to change. And people who write code are going to become much more productive.

53:13Some of those tasks that you outlined sound like engineering tasks, but some of them sound like product tasks or product manager tasks. Do you think that this new world precipitates some kind of shift in the relationship between those roles? I think that it is part of the shift that's already happening.

53:44As programming has moved up in abstraction over time, the ability of one person to kind of bridge the entire span of programming has gotten bigger. And so we're already seeing the proliferation, especially in the startup world of design engineers, like folks who are great designers and can also build the things they design, or technical product managers.

54:16Or, you know, full stack engineers, engineers who can like write front end code and back end code. So I think this kind of accelerates that where it's going to be less of your engineering skillset I think is still really valuable, but less of your mental energy will be devoted to kind of like learning the nuances or the ins and outs of a particular framework or language and more of your mental energy will be devoted to thinking about how things work, what they do, why we do things a certain way, how do we know it's working well. So that gives people space to think about a bigger chunk of the problem than they did before. Do you see that the way people acquire those new era skills, like is different than how they've acquired them in the past, schooling on the job experience? Like, are there...

55:19That's a good question. I mean, I think that by the far the best way to learn any subject now is using ChatGBT and R3 in deep research. It's amazing because it's like you have access to effectively a world expert in many domains that can spend infinite time with you and can get to know you really well and there's no question that's too basic to ask it.

55:45So I kind of, I see all, I see how people learn these skills changing dramatically, regardless of this phenomenon. Like I think, you know, it's, I think, um, it'd be much easier to learn programming now. With chat to BT, then it was to learn, you know, programming where, you know, when you, when you didn't have access to a tool like this, but though we're pointing to like learning programming, isn't necessarily learning programming. It's learning architecture and patterns and all these other things. I don't think that changes your point that.

56:14JetDPT can be a great resource for learning these things, but there's also this element of like what it is to learn. This is like a controversial opinion, but I still think it's important for people to learn programming, even if it's a smaller percentage of what they're going to spend their time doing if you want to build software. Because it's kind of like, I don't know, like in grad school for machine learning, there's like kind of, you know, some of these rights of passage type activities, like building a computer from scratch.

56:45or building a back propagation library from scratch from first principles that like, you know, okay, is this the single thing that you can do that will make you, you know, that will like maximize your ability to perform your machine learning research or your machine learning job the best next week? No. Like you don't really, 99% of the time when you're doing machine learning, you don't need to understand the way that back propagation works at a very fundamental level. However, like when things go wrong, being able to kind of spelunk all the way down the stack and think through like, okay, well, this isn't working. Why isn't it working? Having like fundamental understanding of pieces lower in the stack is really important to being able to do that efficiently. And so, you know, I think like a lot of people who have gone through that exercise will tell you like, oh yeah, every once in a while,

57:44Actually, I do have to kind of like think about this. And I do realize like this is where the bug came from is something about the way that back propagation is working. And I think that, you know, the same will be true for a code for a long time, where like if you if you aspire to be really good at this, there will be a lot of cases where, you know, being able to look at the code that the model wrote, even though you'll do it less and less, is still very much worth your time. And the people who are able to do that well will be better at, you know, this like more vibe coding, style coding than the people who can't. Makes me think of me spending years in grad school learning how to model MMKQs and never really using that, but I am great at picking lines in grocery stores and airports. Yeah, these weird skills that we pick up along the way, right? We've talked a little bit about, or at least you mentioned it, MCP and tools and the role of tools and

58:42You know working with and building agents like what's your take on that landscape? mcp is you know, maybe six months old and like Really don't want paid attention to it for you know three or four of those six months and now like Everyone's building an mcp server and like it's become a really hot topic. Yeah, it seems to be really taking off Yeah, I think exposing tools to model like you know, if you think about the formula for useful agents in my mind, it's like you have to have a really smart sort of general purpose or reasoning model. And then you have to expose it to the tools it needs to do the thing it needs to do. And then hopefully you should provide a relatively small amount of high quality task specific training to teach the model how to do the thing that you want it to do. And so

59:43being able to expose the tools to the model in a way that's that it's able to use them both for training and also for Doing the task in the real world is like a really critical piece of how the ecosystem fits together. So I think yeah, I mean everyone was thinking about this now for good reason and are there obvious gaps to you that you know MCP or some other protocol or even just practice best practice needs to fill in order to kind of bridge this gap between tools and agents. One thing that feels not fully developed yet is the ability to specify a level of trust for the agent using certain tools. So for example, like the, you know, I think the future that everyone is picturing is they'll just ask,

1:00:47chat to BT to go book a vacation and it'll know you really well and it'll go do a bunch of research and it'll come back and it'll discharge your credit card and it'll send you the itinerary. Hopefully asking you a bunch of questions and stuff along the way if it needs to. But the act of doing something like using your credit card is a very high trust act. And just exposing your credit card to an agent that you don't trust.

1:01:16or exposing your credit card to an agent without specifying how you wanted to use that and when you wanted to use it is a very scary idea and potentially super risky. So one thing I feel like the field doesn't have a great handle on yet is how do we kind of like build in that kind of like ways of specifying and enforcing certain levels of trust between the human, the agent, the tool and the task so that you can kind of like go just ask the agent to do something and it knows kind of when it needs to come ask you for permission to use certain tools based on your interactions with it so far. Do you have a sense for what that looks like? And I'm asking because like, you know, we want to, I think, build these models, you know, based on

1:02:22you know, data and training and not, you know, heuristic and, you know, systems and like, you know, the carpathian algae used earlier. But it seems like, you know, we've seen the research that says that, you know, the models will kind of misrepresent the, you know, their thinking and thought traces and like disregard instructions.

1:02:48It seems like there might need to be some extra verbal way of enforcing these kinds of constraints. Yeah, I think at a fundamental level, what it comes down to is having a set of guidelines that you can provide to the agent about when it needs to come get your permission to use certain tools. And so the most strict version of that would be like...

1:03:22So it's providing the guidelines and also having some mechanism to enforce the guidelines or like some level of trust that the guidelines will be followed So like that the simplest example of this would be like if you just if you have a guideline that says You have access to a credit card tool, but every time you call the credit card tool you need to come ask me to approve then Then you know that you don't need to have a lot of trust in the model to use that tool because you always will have the opportunity to review what it did. Now, there's still problems of deception because the model might try to represent what it did or what its goals of using this will are. But at least you have the opportunity to say no. You still have to have a high degree of trust in the agent, even if you specify some set of parameters.

1:04:24And I think it's an interesting question whether the current crop of models or model architecture or the way we train models or the state of alignment warrants that trust. And it depends on the task. It depends on the model. And so I think building that trust to do high risk actions will need to be something that model developers think really carefully about product developers think really carefully about and Users will need to build that trust even after all that users will need to build that trust iteratively through interacting with the model and so, you know, I think there's I Think there's still a lot of thought that we need to do as an industry to figure out how to make that work Well, it's funny that you use that example because like

1:05:24Maybe an hour ago, Harrison Chase posted about CEO of LineChain posted about some news with Visa. They are partnering with Visa on Visa Intelligent Commerce, which is enabling AI agents to shop for you. In their little, I think it's a mock-up, but there's a slider and you can see what your max spend is.

1:05:55But my response to that tweet was is visa gonna allow me to like do a charge back when I tell the agent that I want X and buys Y Yeah, I think You know that's both kind of pointing out the the timeliness of this conversation, but also the fact that there are like technical and political solutions to the problem like actually a visa is gonna be like you know, take the role of providing some degree of trust by allowing me to, you know, beat them up when the agent does stupid things. That could be interesting and useful as well. I don't think they will, but... Yeah, I mean, there will be residual risk there. And so I do think there is an interesting question of, like, to what degree should different parties bear that residual risk? Like, is it a risk that I just take on as a user?

1:06:51Is it something where the model provider is sort of saying that it's okay? Maybe the payment provider? It'll be certainly interesting to see how it plays out. Well, Josh, it has been wonderful catching up and chatting about all things genetic in your world. Yeah, great to catch up. Appreciate you taking the time. Thanks. Thanks so much.