The_TWIML_AI_Podcast_formerly_This_Week_in_Machine_Learning_&_Artificial (5)

0:00At some point, the agents were looking for some information. They were supposed to conduct a Google search, and they failed. And you can imagine what they did. They wrote some code to send an email to request an FOIA for them of information. And email that organization to request that data. It's like, hey, we're conducting this research. We need this information. We can't find it. And by law, we're supposed to have access to it.

0:25And they crafted this email, and they were going to use an email API to sort of send it. All right, everyone. Welcome to another episode of the Twimble AI podcast. I am your host, Sam Sherrington. Today, I'm joined by Victor Dibia. Victor is principal research software engineer at Microsoft Research. And we've got a great conversation lined up for you today. We'll be reviewing Victor's take on the most important AI agent innovations in 2024 and what we should expect to see in the year to come.

1:08And of course, we'll also discuss Victor's work on multi-agent frameworks in general and AutoGen in particular. Victor, welcome to the podcast. Thank you, Sam. It's great to be here. I'm super excited for the conversation. I know in your world, as in mine, I guess more so in your world even than in mine, agents is a hot topic that comes up.

1:30all the time, and I know you've got a lot of interesting takes on that topic. Let's get started by having you share a little bit about your background. I'm a research software engineer. I work at Microsoft Research. I work specifically with a group called the Human AI Experiences Group. And essentially by design, we are interested in scenarios where a human works in tandem with an AI model to solve tasks.

1:57In terms of background, my training is mostly software engineering, some work in HCI. So I have a master's in computer science information that working from Carnegie Mellon University. And I did a PhD in information systems at City University of Hong Kong. And my PhD is mostly focused in human-computer interaction, user behavior psychology.

2:22and how we can conduct a set of experiments that help us understand how people make decisions as they use technology tools and interfaces. And the whole idea is that we take all of that knowledge and we sort of apply it in designing better interfaces. Your group at Microsoft sounds like a traditional HCI research group, but you ended up kind of building auto-gen.

2:49I don't know if you own that as a product. I guess my point is it feels very productized as a software infrastructure product coming out of this HCI group. So how did that all come about? Yeah. So I could talk about my own personal path to agent, and then I could talk a little bit about the history of some early stories around artigen.

3:14So I started out, right after my PhD, I started out as a postdoc at IBM Research over at New York, Yorktown Heights, and then I stayed on as a research staff member. And one of the things I really, I worked on there was, I did, I was with a HCI group, and we worked very closely with a core machine learning group. And at the time, IBM had just come up with the Cognitive Service APIs, and we're building all this complex, multi-modal demos around like speech attacks, text-to-speech image recognition.

3:43using all of that together in like physical room-scale experiences. One of the things I did back then was that I trained perhaps the first model for automated data visualization. So I didn't actually remember the sequence-to-sequence models, and so they were typically used for language translation, and they were like the state of the art back then. And we showed that if you could represent visualizations in JSON, Vigalite, and you could represent like data in the same JSON specification, then you could learn translations across the two. And so at runtime, we gave this system this model some text. We sampled a couple of rows from the JSON data set directly. And it will generate a bunch of visualizations that were grounded on that. So that was really interesting. And if you think about it, there's a bit of action there, right? And so we generate, like, bigger specification. We compile it, and we give it visualization. And so after that,

4:40I spent some time at Cloudera, traditionally big data warehousing company, but it had like a machine learning group and we built a lot of prototypes to some customer consulting. Then after that, I joined Microsoft Research. And there I did some work on a tool called LiDAR. And so LiDAR is, again, an automated visualization tool, but had like a bit of a pipeline. So first of all, we got some data. We got like an LLM to generate a summary.

5:10and reach summary of this data. Based on that, we got an LLM to generate a sort of hypothesis that made sense for this data. And then for each of these hypotheses, we could get an LLM to write code on the background. We did like the processing, post-processing. We got code, we executed it. Now we gave folks a bunch of visualizations. And so, and this was in 2000, just pretty early, I think, 2020 before chat GPT. And so it was an entire interface that did all this pipeline work on the back end. And it used the codec set of models. And for the most part, you can see some agentic behavior there. And so it's not just LLM generating stuff. We're compiling code within preprocessing and post-processing. And so once you create a lot of these pipelines, you start to see some broader patterns. And so the next question is, can we go from that?

6:07to assist them without having to manually build out the exact steps in the pipeline. Instead, we defined the set of agents and we gave them a task and collaborated autonomously to solve this problem. And one instantiation of work like that was Autogen. And so for the most part, Autogen started out exploring this theory that maybe we could explore a new way to develop applications.

6:38building specific pipelines that express the problem to express the solution to a problem. We could instead define a set of agents with fairly broad capabilities. We'd give them a task and they could sort of collaborate to solve a problem. And there was a really, there were a few really clever people within our broader group, so to explore this, did a bunch of experiments, grew the paper. So I wasn't an original author on that paper, but we worked very closely, I worked very closely with that.

7:07with our group, and essentially that's sort of like what led to Autogen as a framework. And it's been about a year and some months, a lot of things have happened. We've got a bunch to dig into here, and I don't necessarily want to belabor things by talking about defining agents, but you mentioned that you heard me talking with Chip on that topic.

7:35recent interview and you had your own take on how agents are defined. I'd love to have you share that. Yeah, so I think a simple definition works. I think a lot of people are converging on the idea that like if we take an LLM and we give it access to tools that let it take some action so essentially this LLM can now act, then we have an agent. And from the software engine point of view, that's like the basic instantiation. So you take an LLM, you give it some tool calling capabilities, you give it the ability to execute the results of those tool calls, and then you have an agent. And I feel like I'm happy with that base definition. In practice, it can be, if you want it to be a bit more precise, I think there might be a few other things. So you want something that has the ability to reason, has the ability to act, know, via tools, has some adaptation capabilities, in this case, perhaps memory.

8:34And then finally, some abilities communicate. And so it can send messages to other agents or to humans. And so it's reason, act, communicate, and adapt. And so this is, I think, like the four built-in blocks I would say sort of make up an agent. It seems like that reasoning ability is a key differentiator between a traditional software system that uses large language models and an agentic system in my mind.

9:03and that it is what unlocks the ability for it to be dynamic as opposed to like statically defined workflow. Yeah. So reasoning, yes, I do agree. I think one way to think of it is from the perspective of let's say planning. And so you get a task.

9:28you decompose it into a set of steps and the idea is that if you succeed at executing each of the steps, you go from a state where the problem is unsolved, the task is unsolved, then you arrive at a step where the task is now solved. And I think you touch on the idea of dynamic. I think the interesting bit here is like, as you take each of these actions, you might Now, the problem might exist in a dynamic space or a dynamic environment. And each time you take an action, it changes the environment. And in some cases, this changes my lead to errors or failure conditions. And I think the key part is a good autonomous or good multi-agent system should have the ability to recover from that and either abandon the current plan or make adjustments and keep making progress.

10:25And a really simple example is that, let's say you're trying to solve a problem, your agent writes some code, executes it. There's some errors there. It looks at the code, based on the error, it sort of modifies the code, executes it again. It might be missing libraries, incorrect arguments. And if you do have this sort of behavior where every action has some results, some outcome, and you can respond to that and then keep making progress, then I feel that sort of speaks to the dynamic aspect of a multi-agent system. So we're going to dig into that, I think, in a lot more detail. But I think for this first part of the conversation, you put some thought into what you, from your perspective, the most important developments in agents over the past year or so.

11:19and how those set the stage for the upcoming year. And I wanted to start by digging into some of those. So I think the first thing on your list is about adoption. Yeah, let's start right there. So over the last five months, I figured it would be great to just sort of keep track of what's changing. So what I did was that each time I saw a research paper, or a new product or a new tool. I kept a bunch of notes. And at the end of the year, December last year, I sort of figured, no, what a high level category is here. And in terms of adoption, I feel like a lot of enterprises and teams adopted the 10 agents, but they did that with some caveats. And so for the most part, what most people deployed last year or in the last year was mostly

12:14and LLM as a theme wrapper around existing APIs and tools. And so as opposed to fully autonomous behavior where at runtime the agents can explore unknown or unscripted paths, essentially they just took the existing APIs, very, very tight structured action space, and all the LLM can do for the most part is to...

12:41sort of make calls to these APIs. And this is a really good game plan because you get a lot of reliability out of that. So I think the second thing on that little list was the rise of agent-native foundation models. And so about a year ago, we mostly had models like GP3.5 and the equivalent and Google from Google and Anthropic.

13:08And most of these models mostly focused on language modeling. And so they were writing text. They were writing code. And for the most part, they were mostly text-in, or in some cases, multimodal, text-in, image-in, but only text-out. And one of the things we saw in the last year was that these models were increasingly integrating multi.

13:34agentic capability is just big right into the model. So some of the capabilities like the ability to reflect things. So if you remember the react pattern where the idea is like you get the element to come up with a thought, get it to reflect in that and then take more actions. And so we're seeing that like with things like the O1 model family, the ability to just think and reflect is sort of just lifted up into the model itself.

14:00You know, you give the model a task, it does all this internal introspection and reasoning before you get like a result out. And also we saw things like natively multi-model in and out models. So I think the Gemini 2.0 models, so these things can take in text, image, video, audio, and the same model can sort of spit out results across all three modalities. And so I think that was the second interesting thing we saw in 2024.

14:29The third thing had to do with interface agents. And some other people have referred to them as computer use agents. And the idea is that as opposed to agents just calling tools, your API is a code. We now see a shift towards agents that act by simulating what humans do with interfaces. And so examples of that are agents that sort of solve tasks using a browser.

14:57And so I think about two days ago, we saw OpenAI release the operator agent. And the whole idea is that you could tell things like, you know, book a flight for me, and I'll go to, let's say, flights.google.com, I'll click around, put in all the information, dates, source and destination locations, all that stuff, and then probably come back at some point, get some feedback, get some confirmation, and get things done. Fun fact.

15:26The autogen land we've built out systems of tools like this and think one of the Excitements of the last two days was that like you know once operator came out We say that you know he has like 40 lines of code and you could implement your operator using the autogen Did it did the browser control framework already exists? In the autogen world yes, that's an excellent question So in autogen world we have a bunch of presets and so we have like a preset assistant agent that like It's a classic. It just has an LLM model set of tools. But we also have this preset called a web surfer agent. And underneath this agent drives a Chromium web browser. And it uses a multi-model, any multi-model LLM model. And so essentially, it has an action space about how to get work done on the browser, text input, click and run, navigation, all of that.

16:26And essentially, it's pretty much just acts by sort of driving it in its browser. And it's a really nice, well-designed agent. It was done by one of my colleagues, a really brilliant fellow, Adam Funi. And essentially, all you have to do is plug in this preset into your multi-agent team, and you get all of that capabilities. Nice. So interface agents? Yes, yes. So interface agents, so we have that with Audigen, the web server agent in Audigen.

16:55We also have tools from Anthropic. They have a computer-use implementation. And there are a bunch of other tools that sort of exist in that space. And so we saw a few of those sort of advancements in 2024. The third thing had to do with complex tasks and frameworks. And so I did see that as a community, as a field.

17:24Lang chain got us very, very far. So Lang chain showed how you could sort of get a set of determine six steps, put them together and chain execute them. But no, there was a bit of appetite for more complex workflows. And we essentially more autonomous kind of workflows where like the task, you want a system that can address any task. And in fact, it reminds me of an article that like Bill Gates wrote about I think a year and a half ago.

17:53talking about how today, back then, if you wanted to sort of, let's say write an email, you went to an email processing app at Luke. If you wanted to do CRM stuff, you went to a CRM app. And if you wanted to do music stuff, you went to the music app. And he talked about the idea of an everything app, a unified interface where you just expressed your task in natural language. And the system just, if it needed to manipulate or reach out to all the systems, it did that.

18:20And so I feel there's a lot of value, a lot of time-saving, effort-saving value proposition there. And I think the community sort of started to resonate around that. And I think organically that has led to the design of frameworks like OtherGen, LandGraph, Korea AI, Llama Index, because we want to figure out ways to provide good presets that help people build this sort of like...

18:50generalist systems. And I think, again, I might touch on what I mean by complex tasks, but I think that's one of the shifts that we saw in 2024. So beyond scripted deterministic pipelines to more autonomous systems that could address multiple, disparate tasks. And then the final update had to do with moving beyond just benchmarking.

19:21models independently, but essentially extending to just end-to-end evaluation of agentic systems and tasks that require action across multiple domains in the real world. And I think one of my favorite, one of my favorite benchmarks there is the Gaia benchmark. And essentially, I think if I recall correctly, it's about 300 problems that as at the time of release, it looks really simple.

19:50These problems are really simple, easy for humans to accomplish. But the best models at the time, I think it was GPT-4, just failed really bad. I think we had like a 10% pass rate there, if I recall correctly. What are some examples of the guy attacks? Yeah, so it might be things like, how long would it take Aliyud Kipchage to run across the earth, let's say 50 times. Now to do something like that, you need to figure out, oh, who is Aliyud Kipchage?

20:20what's his maximum uh he's a marathon record holder so you need to figure out what's his speed and if you know what's the circumference of the earth um they need to do that little math that sort of puts everything together um and it might be things like you know like what did sam charlton say in the 70th minute of his 2025 i don't know January 1st podcast now to do that you need to go to youtube Find like who's some charity and find the exact YouTube video that's being referenced extract extract the transcript and go to the 78th myth and figure it out now as a human this is really straightforward Frankly, but how does the machine go about stuff like that if you really think about it? they're all kinds of ways where this machine might fail and I remember a group again led by one of my colleagues developed like

21:19a general release agent system called Magentic One. And essentially for a long time it held the state-of-the-art performance on tasks like that. And so I think, and all of that process was extremely instructive. We learned a lot about how these things could feel, how the differences between how humans think about problems versus when machines, even the ones driven by sophisticated algorithms, tried to address the same tasks.

21:49And so I think that was like the third and more interesting, the third interesting update for 2024 or the fifth story. So let's dig into these. I have a bunch of questions across this list. Maybe let's start with these kind of agent native foundation models you call them. Talk a little bit more about kind of the way you think of them as agents. I guess my personal experience is that I originally thought of them in a very agentic way, but then as we've seen with DeepSeq showing you the thought tokens, it seems less agentic in a sense. Does that make sense? I guess it's like...

22:50It seems more like a straightforward but slightly more complex application of traditional LLMs in some way. Yeah. So I guess what you hinted at is if you didn't see the thought tokens, then it looked like it was doing something more clever. But when you saw the thought tokens, it was just an autoregressive model, so it's just predicting.

23:20predicting the very next token. So I guess the interesting thing is what is different when the model is primed to explore the iterative thinking process as opposed to just generating the next likely token. I think from the human behavioural psychology perspective, and I say this with caution, LLMs and the humans, um they're not like humans in any form but if you if you think about it um you know there's the whole concept of system one thinking system to think in thinking fast and slow and there's a whole idea of like you know for things that are simple as humans we've adapted to use heuristics right so if i did ask you Sam what's your birthday um you don't think about it you tell me or like you know like what time is it or is it morning or evening you know the answer to that so

24:20You can rely on heuristics. So these things are like right there at the top of your mind But if I did ask you like, you know, like the question earlier how long will it take a marathon runner to run around the earth like 50 times now this requires a bit more more investment a bit more computation and effort and a Lot of people did complain earlier that like they say about a year and a half ago if you ask the model these two questions, you'll take exactly the same amount of time to give a response and there's just something not right about it that like some two problems So different so complex We are signed about the same effort and I think a lot of that has informed some of the work in test time compute and The whole idea here with the own reasoning and district family models is we want a way to communicate to the model or the system that like some problems perhaps require a bit more

25:19more investment, more computing investment than others. And it turns out that it does work when you design the system that way you just get better results. Underneath is still an autoregressive model doing autoregressive stuff. But it just turns out that the setup, the problem setup sort of results in better results for thinking and reasoning style problems.

25:47advantages of those types of models for complex information gathering and presentation report generation, those kinds of tasks is pretty clear. Are you seeing those reasoning advantages play out in terms of the planning style of reasoning that's important in making complex agentic systems work? Yeah, yes.

26:17One of the good things about like let's say a tool like Autogen is, let's say you could decompose your problem and express them as agents. So you could have an agent that's explicitly just focused on planning. So for example, I mentioned earlier the magentic one paper. So the way that that setup was done was that like we had no, we argued for the design of a generalist system that can address multiple different types of tasks and we tested them across multiple agent take benchmarks, the exact same system. So nothing was fine tuned for a specific system. And they were composed of four agents. So the first was an orchestrator or planner. So all they did was it took a task and it would decompose into a plan and assign steps in the plan to other agents. And there were, I think, four other agents, something called a coder. All they did was write code.

27:07There was one that was a computer terminal or did with the execute code. There was a web server agent. Essentially, the task needed interaction with the websites. And then there was a file server agent. So you needed to open things like video files and image files or PowerPoint presentations, that sort of thing. And the cool idea is that for, let's say, the orchestrator, for each of these things, you could design them different models. And so for the orchestrator, I think my theory is that something like that that's meant to like reason through the problem, do some sort of task decomposition, assign steps to different like agents. And agents like that really would benefit a lot from like what some of these sort of like test time computer reasoning models. Now the other is something like, let's say file software, all it does is just has a bunch of tools that lets it interact with files.

28:10probably not a lot of benefit there. And is that intuition, or have you seen benchmarks that, you know, take a system like a Magentic One and insert a reasoning agent for that orchestrator step? We haven't released any results, let's say. Let me use the word release. We haven't released any results yet, but early experiments and some of...

28:36I think I don't remember other papers off the top of my head, but I think I have seen a few where just dropping in the reasoning model did give a significant boost. I guess I wanted to poke at like there's maybe a nuance difference between seeing a significant boost and like unlocking a whole new area of capability. Do you see these types of models doing the latter?

29:04I wouldn't see exactly a whole new type of capability. I think it's just maybe performance improvement. I think a lot of the types of problems we're thinking of solving with this system is still the same class of problems. Maybe I think the more interesting thing here is if we look at this system from the perspective of like failure modes, Bad plans or the ability to come up with good plans on the first try is like a significant like Performance issue for like this sort of like autonomous systems and from that perspective You should get something that like reasons well comes up with the plan or good plan on the first try Then you get some benefit there, but the type of problem hasn't changed. It's not like we are suddenly I don't know

30:05but suddenly doing new types of things. It's just that we're getting reliability or performance, maybe even safety improvements where possible. With the magentaic one work you mentioned that one of those magentaic types was a coder, was that primarily used in the context of coding problems, or was it code that was generated in...

30:33the process of solving other types of problems. I've come across several different papers that use code as this intermediary for planning and other things, and I find that a really interesting and compelling way to use code. Yeah, it's a really, really, really good question. So it brings me to how I think about tools. I think there are two types of tools. So they're task-specific on other domain tools, and then there are...

31:04general purpose tools. And I feel like a code interpreter is a type of general purpose tool. Some problems, a lot of problems can be, the solutions to a lot of problems can be expressed as code. And getting, the key point here is getting the orchestrator to figure out a separate problem could be solved really well when expressed as code.

31:31and then getting the coder to sort of write that code and then getting the code interpreter to execute it is like an emerging pattern. So to answer your question, it wasn't just solving like software engine and task type problems. It was mostly like, you know, he has a task that we expressed. There's more code interpreter than code generation. Yeah, yeah. That would be like a good focus. And of course, there are caveats there, so if you If you have a system that has this very wide action space, then it can do a lot of interesting, maybe even unusual things. So an example that we, a funny example that we like to talk about like in the margin, and we talk about in the margin to go on paper. At some point, the agents were looking for some information. They were supposed to conduct the Google search and they failed to find that information. And you can imagine what they did. They wrote some code to send an email.

32:32to request an FOIA, essentially to send like a freedom of information. And email that organization to request that data is like, hey, we're conducting this research. We need this information. We can't find it. And by law, we're supposed to have access to it. And they crafted this email, and they were going to use like an email API to sort of send it.

32:56I'm imagining the agent sending it to like trying to send it to Google as opposed to like a government organization or something. Yeah, so it's a fun fact but it is true if you don't constrain like the action space of what these models can do because code is just this really expressive thing they can take any kind of action expresses as code and then it could lead to like things that like And the way you solve this is that the orchestrator can make some high-level decisions as to, is the task being installed? Is it going the wrong direction? Sort of metacognition, right? As these agents act, the orchestrator is sort of inspecting progress. And it's saying things like, are we stalled? Was our maximum stall count? And should we reset, modify the plan, abandon this route, and take a separate route? So, yeah.

33:49that caveats to use in general purpose tools. And the way you talked about those aspects of the orchestrator is maybe a segue into talking about these complex tasks and frameworks, which was one of your items. Specifically for those types of parameters you were describing, are those things that the user of a framework like Autogen Are they thinking about them? Are they setting parameters? Are they coding them? Like, how do you manage the level of abstraction that someone working, you know, trying to build an agentic system to tackle complex tasks has to deal with? So a question like this, you know, ties into, like, slightly how do you design frameworks? How do developers think?

34:44And I could tell you a little bit about how... As an HCI guy, I feel like I've opened the box. Oh, yeah. And I could be a bit more practical to tell you about how we are pushing with Autogen. And so in Autogen, they're currently, they're two API levels. So there's a core API, and the idea is that it mostly just provides you with the bare bones.

35:11capabilities for things like just message delivery. So anything could be an agent. You could define anything as an agent inherited from base class. And the only thing you're required to do is to modify a method that says the agent has received the message. What does it do? And so it could be as simple as it receives a message. It does nothing. Or it sends back the exact same message. And that's all. No opinion to whatever the agent does when it receives the message.

35:41the developers responsible for that. And essentially all we guarantee is that there's a concept of a runtime. When you define your agents, the runtime spins up, creates instances of this agent, enables message delivery, and this agents might leave across multiple machines, they might be on the same machine, and that's all. But for most developers, this is still too low level. And so we have another API called Agent Chat.

36:11And Sam, you're from the old world, you probably remember Keras, where Keras was like this high-level abstraction, very intuitive, but beneath, it could run on a TensorFlow backend or PyTorch backend or a Jax backend. So think of agent chat as the Keras of this world. And the kind of presets we have there is things like a basic assistant agent. And so this thing is what I think is the...

36:39the fundamental representation of a basic agent. And so it can take an LLM, it can take a list of tools, it can take a list of memory banks, and essentially that's the standard interface or the standard definition there. And so model client, a list of tools, which could be functions, it could be anything. And a set of like memory banks, so just enable a rag or like just in time retrieval of information.

37:08And we have another preset, which is like the web server agent, which essentially is just all the things you need to drive a web browser and sort of accomplish tasks using that. And then we have, I think, one or two other presets not very important. So that's at the agent level. Then how do these things sort of collaborate? So we have the concept of teams. And so think of them as containers that you put these agents into, and it mostly governs the order in which.

37:37messages flows across these agents. And so we have a preset, something called a round rubbing team. And what it does is that once a task comes in, it just sort of sends messages across each of the agents until some termination condition is met. And then the final abstraction we have at the team level is a termination condition, which can be really tricky. It's like, this guy said, exploring a task, how do they know when it's done?

38:06And so we have abstractions like text message termination. So means if any of the agents you could define in their prompt their behavior, they might constantly sort of inspect the state of the task. And if the task is done, they might respond with a word like terminate. And so let's say you scan for that in the messages, you decide that things are done. It could be budget based. So timeout based or a maximum number of tokens used or a maximum number of steps.

38:35It could be some external signals or something external just monitoring the state of the task and then it sounds like a termination condition. And you can compose all of these things in all and combinations. And if you took all these presets, agents with all the stuff inside, teams, round robin, selector group chat, graph base like selections, all of that and termination conditions and you put all of that together.

39:03We are seeing that that has been a powerful way to express autonomous multi-agent systems. Popping up a level, you started talking about the way developers think about problems and things like that. And it sounds like at its core, the way Autogen is organized is message-based. And there are other approaches that are graph-based, other approaches that are...

39:32you know, I don't know of specific other ones, but there seems to be like, you know, message and graph is like one big, you know, difference in paradigm, you know, are there others and, you know, why do you think message is better, you know, historically, like, or traditional software, it's like loose coupling is an advantage of message as opposed to other things, like talk us through that whole abstraction, you know, thinking. I'll talk about two things. So first of all, Right now, the current version of Autogen is based on the message passing architecture and how we got there. And the second thing I'll talk about is some of the emergent patterns. I alluded to some of that, but I'd like to structure it a bit more. Emergent patterns we're seeing for building autonomous multi-agent systems. So about a year and a half ago, when we released the first version of Autogen, the interesting thing is,

40:30It was built on this idea of conversational programming. And so the idea is that to solve a task, we just get these agents to each act. So each time they act, they have a shared conversation, history, or list. And LLMs were actually being fine-tuned, and they're still a paradigm. The chat completion paradigm, where every time you ask a question, you append it to a long chat, and the model just gets No, it's been trained to use all the context. That's where it's context is coming from, yeah. Exactly, all of that context. So essentially, solving tasks is just all about building context. It's just like extends. But the problem was that this thing was a list that lived in memory. And so all the agents had like this list literally, like there's this list and they all like appended stuff to it. So each time they executed code or they responded to a message, you're all appended, sort of appended into that list of memory.

41:28Now, if you do that, you can build systems where the agents live on multiple machines. And this is a really common production requirement. You want the agents to live in a separate machine. They have security boundaries, have access to information that nobody else should be able to have access to. But if your design is that everything leads in memory, you can do that. In addition to that, you also want an asynchronous stack because each time the agents act, an action can take an arbitrary amount of time. And you want a scenario where these things can run in the background while the rest of the system sort of continues. And so quickly, we ran into all of those sort of issues. And a solution to that is the actor model. And so you sort of reference that where you treat every element in the system as an actor, and they're loosely decoupled from everything else.

42:27and the only communicate via messages. And you send this asynchronous messages. The messages can be delivered in a arbitrary order. And if you do that, you can compose this message-sending behavior into all kinds of complex patterns. You can have agents or systems that leave on multiple machines as long as they can count.

42:53Connect to the same like communication layer or message pass in a message deliver layer and Even from the application development point of view You can build this async applications websites UIs communicating with things like teams Slack where like no messages are just sent and received asynchronously so things like you know enabling true like distributed agents is one of the driving principles there, and also just application, just better integration with external applications and one of several benefits that you get from a message-driven, same actor model kind of paradigm. So hopefully that sort of provides some background and why message passing is a good idea. Of course, it has its own complexities, message delivery, ordering.

43:51It's really hard to think about and debug async code, but that's what a framework is there for. It's meant to help with a lot of these issues. So that's one thing. The second part I wanted to talk about is, you know, I mentioned the idea of graphs and chains and all of that. So I feel like there are two high level patterns that we are sort of observing in the multi-agent space. So the first is control flow patterns. And so if you have multiple agents, how do you determine the other?

44:20in which the act or the other message delivery across each of these agents. And the simplest version of it is a deterministic chain where you say, you know, I want something that generates, I don't know, finds me some news articles every day. So you might just keep it simple. It has three steps. You have an agent that takes in my query. It takes in my request, generates a web search query.

44:50You have another one that makes a request to being a Google. And you have a third one that downloads all of the data and all the results and summarizes it into some simple outcome. Extremely simple. A simple chain like that is also a graph, but just a very simple version of a graph. And the core idea here is the developer already has a clay idea of exactly what they want the system to do when you run this graph.

45:17Now the graph could get a little bit more complex, which is what tools like line graphs support. So you enable conditional edges, enable loops. By the end of the day, you can still make very clear deterministic predictions about what would happen before you execute the graph. So you know that this graph is directed, is a safe link, and it will always come to some end goal or one of several end goals, which are all good things.

45:46And so that's like the first two patterns, so simple chains, graphs. And then the third piece is more implicit planning or group chat kind of thing, where we model the solution to the problem, not as graphs, but just by a conversation history, which is where Audigen started. And this agent sort of, there's no predefined path.

46:17But there are some structures around like, you know, we might have some round-robin communication flow or we might get an LLM to decide just in time which agent speaks next or takes another turn. And I'll say that's the second plan. And this implicit plan, group chat, shared context kind of thing, it can still work across multiple distributed agents. It just is a bit more...

46:45non-deterministic and we still see a lot of failures and I think as a research group that's the part that's really really interesting because like we want to get a point where we understand this thing really really well and make it work for everyone because this is how you get like truly autonomous like new step function like increases in system behavior at least in my opinion and then there's the opportunity to implement things like metacognition, to just thinking about thinking. And so I mentioned magentaic ones, an early version of that. There's a concept of an inner loop and an outer loop. And so every time an agent takes a step, just the orchestra sort of takes a pause, has a task making progress. If yes, OK, let's keep on going. If no, we increase some stall counter, like K we know we're stalled. And after some threshold,

47:42We just reset all of the agents. We agreed that we are on a bad trajectory. We might be summarized what went well. We keep it. We use that formula to new plan with some notes, and then we sort of explore new trajectory. So a bit of medical cognition, reflection, that kind of thing. And then the final set of patterns are around task management, around first human delegation. And so not all actions are equal.

48:13if an agent came in and said things like, hey, I'm going to download this file to disk. No big deal, right? It downloads the file. But if it said something like, hey, I'm going to send, I don't know, I'm going to transfer some money to Sam Chianten's account using a CH. Now I want to know about that, right? Right, right, right. I want to know about that. So there's the idea of like, I want to know about that too. Yeah, you want Sam to know about it, right? So he doesn't get worried like, hey, you know, where did all this money come from? So there's the idea of like, the inherent risk associated with action. So we need to be able to quantify that and maybe figure out if we want to delegate. And also there's the idea of how do we know when the task ends? And I've hinted on that earlier. So just like the two high level patterns, control, flow, and task management, our pattern is the same. Kind of putting it on the hat of someone who is thinking very pragmatically is maybe at a startup and is building.

49:12something or wants to build something. Is it too reductive to translate what you said or a little bit of what you said into like Autogen is a research project and you know they're doing things the way to make it you know interesting and complex for research and open up new avenues of research and you know I might want to go a different direction if I am really just trying to solve a problem. So maybe six months ago that would have been correct but I think two weeks ago, we released a new version of Audigen. If you search for it, I know this is a little bit confusing for some people who have used Audigen, but there's a new version called Audigen 0.4. And it's based on this new asynchronous message delivery behavior I mentioned earlier. And we've put in a lot of effort there to make it production ready. So like I mentioned, there's a core API, right? And so I'd say my suggestion

50:12is that if you're starting to build a production level application, just take a look at the core API. There's a chance that your business logic is so niche that maybe some of the presets we have in the higher level API might not be directly or immediately applicable. But take a look at the core API. Use it as a core building block, because at the end of the day, you still need Message delivery capabilities. You need some of the utilities around tools around like model clients that sort of thing So I'll take the core API is where you want to sort of invest time in and also another fun Another fun fact that I probably will tell you is that when we build other gen you know again as a research group we are more interested in autonomous behavior and

51:08This is a good thing. Everything starts out as research. Somebody has to explore it before we solve all the bugs and make it work. And so, yeah. And so we started out with the autonomous thing. But then we saw that this is just my own assessment, but I think about 70% of the people who come here to use Autogen, they already know what they want to do. They know my problem has six steps. And essentially what they're trying to do is they're trying to shoehorn.

51:38other gen to just do that six steps that they're there to do. And we're working very hard to make things like that possible. And these are all great ideas. I think a lot of the reality is that a lot of the use cases, so for a startup, right, let's say you're in finance, you know exactly what the problem domain is like, it's pretty structured, you're not trying to build the everything app, right? And so from that perspective, what you want to build is some type of workflow or pipeline or chain or graph, maybe with some dynamic behavior here and there as opposed to something fully autonomous. And so go ahead try out the core API, use it to express your business problem. And I think that's where you get the most benefit from. And as the space grows and we figure out really how to get this autonomous thing to work really, really well then maybe at that future time.

52:34the higher level API or the autonomous exploration kind of agent might be a benefit. I think just being pragmatic, this would be my thought process here. Going back to our review of 2024, you mentioned interface agents and computer use. I think that is...

52:57You know really starting to capture a lot of kind of maybe energy or imagination is maybe even a better word because like you can like really visually see the computer like Doing tasks that I don't want to do book my flights. You know do my grocery shopping Yeah, I'm thinking a little bit of another recent interview with Dan Jeffries where he talked about actually how hard it is to do those kinds of of tasks. You know, he used an example of like trying to get an agent to book a flight on Google flights and like pulling up the calendar and you've got to click the numbers and like getting the agent to get, you know, localize on those numbers is just really hard. Like, I think, you know, to maybe kind of open up this further conversation, I'm just curious like your take on, you know, where we are with regard to computer use and

53:57What needs to happen to evolve it for it to be? How practically useful is it in your opinion? What's your rundown of the state of play with regard to agents controlling browsers? So if you've ever done a user study where you get people to use this agent's accomplished task, one very funny thing that tends to happen is that There's something really unsatisfying about a human just sitting and watching an agent struggle. It's like, okay, I'm going to open this browser. Then I'm going to click this. I'm going to click that. The views are on some parts. Some people are like, hey, that's cool. That's really fun. But some of the people are like, hey, I could do that in a quarter of the time. That sort of thing. And then sometimes, just like you mentioned, it feels that some funny.

54:55like step and then it has to restart and all that. So I think this is all green pains. As a researcher, I have very high tolerance for stuff like this. Of course, as a product designer, this is not the case. But I think there are two things, right? So we are seeing specialized models just for UI interactions. So some of my colleagues, Elsa and Microsoft Research, released the model.

55:26couple months ago, something called OmniParser. And essentially, it's a multimodal model, and it's just fine tuned to predict the bounding boxes with fairly high quality of interactable elements on any screen and UI. And again, if we treat this as an engineering problem, as we gather more data about how humans interact.

55:55think of it as like all the good work that OpenEI encoded on supervised fine tuning. So you just invest a ton of time in just figuring out like, you know, getting a bunch of experts to interact with the GPT models, assembling this SFC data or for reinforcement learning from human feedback. I feel like we are going in that direction where like these models Develop intuition similar to how human beings sort of act and so some things you'll do if you see an ad you close it immediately before you do anything So like you don't you don't try to like click other things while an ad is open You know, oh, that's an ad. I gotta click it. It's a cookie pop up banner. I gotta get out of the way And so we need more we need more of those types of examples Some icons are really small There's a models optimized for that. I think also about a week ago

56:50does a new model from byte and something called UI TARS. And also just significant, I see a lot of progress. So the benchmarks are just getting better. And I think we'll see more of that. And models like that, they have to specialize like traditional, just generic multimodal models into that well there. So just having specialized UI interaction models is becoming a thing. So I think that's how we get better there.

57:20I think there's also a lot of work around like memory and adaptation. And so we need to figure out ways to get the model to do things like remember preferences automatically. Say anytime we give feedback or anytime like it explores a path that seems like it works well. We need ways to sort of serialize and recover and retrieve that sort of successful trajectory in the future.

57:50So, short answer, I think that, you know, accuracy will increase, latency will still be a problem. And from that perspective, I doubt that the right model to use these things is to have a human sit and watch. I think you really want this thing doing back office stuff, like, you know, I am a, I don't know, I am a Cosmo Sales Consultant.

58:21a new ticket comes in. Just without me being there, I want this thing to open up Salesforce, open up our custom, I don't know, ticket management software, do all the data transfer, open up an Excel sheet, write all these things down, and then just let me know when it's done. I'm not sure that the right model is that the human actually actively supervises it. Maybe there could be one training or one...

58:49learn by demonstration fees, and then every other thing should be served in the background. And the truth is, if an agent takes a minute to do something that I can do in 30 seconds, I think this is fine because 30 seconds of my time is worth 100 times of 30 seconds or one minute of the agent's time. I think people get hung up on that 30 seconds of your time.

59:18It's really, really worth a lot more than all the computer that an agent needs to use, even if it uses just slightly more time. Yeah, I used to kind of think that, and maybe it was more true than when we were further from seeing computer use and things like this in a while, but for any problem that was sufficiently valuable enough to point an agent at it, You know you may as well just like build the API integration or something like that your Salesforce and ticketing example is a great example But you know is is clear that as you know the cost and capability and complexity of like deploying an agent that works through a web browser You know it goes down and they become more reliable like there are way more integration problems out there than we have the ability to tackle all of them so

1:00:19There's certainly a place for this back office type of work that you're describing, even though it's, you know, as an engineer, it's painfully less efficient than just like hitting an API. Yeah, I completely agree with you. This whole thing is an anti-pattern. There's something not right there. The least efficient way to solve this problem. Yeah.

1:00:50I agree with you. I think maybe reduction in costs and maybe increase reliability and the added flexibility, right? Just not enough. The surface of all the things you need to build APIs for is just pretty large. Yeah. Yeah. I guess another take that I've had is that historically, A lot of integration problems aren't like technical problems. They're like political problems. The example I always use is like, you know, forever you couldn't get Southwest flights on Google flights and that's not because they couldn't integrate them. It's because Southwest didn't want their flights there. Do you see Do you see pushback starting to happen, you know, on agent systems, like, you know, in the traditional, like, in the search engine where there's robots.txt, like, you know, keep your bots away from my content. You know, do you see a thing like that starting to, or do you see a future in which people are trying to prevent robots from, well, I guess they're already trying to prevent robots from accessing their sites, but like, how does it...

1:02:06I guess how does it change given the kind of technology that, you know, genetic bots are based on? Yeah, you bring up a really important point, and I think it's something that we all should... Well, I mean, I guess from the technology standpoint, we'll all adapt to it in some way. But I feel, I agree with you at some point, we'll have something like agents.txt, just like have robots.txt.

1:02:34And it might be things like, you know, we don't want robots or interface agents like sort of interacting with any of our interfaces, or it might be just a specification for what we expect the agents to do or the right way to sort of interact with our interface. And for that matter, beyond agents.txt, like, you know, we'll have Cloudflare and you know all the other services that people use to try to prevent You know non-human access to their to their things. Oh, we already seen stuff like that We already seen a lot of websites that just try to other detect if you're using a playwright instance or no if it's actually human and Essentially the website is just just blocked completely and there's a chance that like a big chunk of the internet we just

1:03:28which is because in some ways it's like costs for the folks running the websites. And I think at some point we need like some clear language just standard around like web interfaces or any interface in general that's designed for human versus an agent. And it brings me to something that I've been thinking about, the concept of like agentic noise. And agentic noise is just thinking through like you know and at some point we'll have like a lot of agents acting in the digital world on behalf of humans and if these things are not implemented well they could sort of compete in unusual ways for the human bandwidth and so as a human you know we all have finite bandwidth and as we navigate the digital world sort of everything is sort of vying or struggling to sort of get a chunk of that bandwidth and I think as humans we want to optimize for like

1:04:29on regretted interactions or in some way that might also translate to interactions with other humans or interactions with like I don't know like just high quality like high quality like artifacts and if we just have like agents sort of loose on the internet then in some ways they might sort of capture like bandwidth or I don't know like they might like capture attention that like is that different or in what ways is that different from Like the spam problem, right? It's like as the cost of sending emails and spending sending text messages, you know has dropped to essentially zero now like there's this kind of You know attention tax. Well It is this theoretical at least attention tax and our inboxes and our text message boxes

1:05:26you know, they kind of vise for our attention, you know, and, you know, per or in line with our previous conversation, you know, that, you know, technology has been created to fight that, right? And so now we don't even think about spam anymore because it's in this, you know, hinterland in our inbox that we never even bother to check anymore. I think the point is we would need to sort of, as we design the agents, we need to figure out the technology that like curates our interaction with these agents.

1:05:56in following the email example, we have filters that figure out what a good email is and what a bad email is. And so we would need technology that does the same type of filtering as to what is a good, what are good, agentic interactions and what are bad ones. And sometimes these things can have all these other secondary effects, right? So imagine that if you've...

1:06:22If you've interacted with some government websites, there's a notion of Whitley saying, you know, to get an appointment, you go to some calendar, you sort of like click around, you get an appointment. And I imagine like there were a couple of ambitious agents that just went and took up all the appointments. Now, that's a real problem, you know, like it's sort of interfering with the social contract that's like we have when we interact with all the systems. And we need to figure out who is to evolve.

1:06:51evolve and sort of it might be like some new types of capture just like you mentioned some new types of filtering to new types of humanness Confirmation that's the same but either way we would need to evolve and get better at that you mentioned the rise of kind of these end-to-end agentic benchmarks Are there and you mentioned I forget the name Gaia was that the name of the benchmark that you refer to?

1:07:22Are there frameworks or methodologies that you're seeing people using to kind of benchmark their own tasks with agent performance on their own tasks as opposed to benchmark tasks? So evaluation is a whole, it's a whole field. It's a whole thing and yeah, it's a whole kind of what I'm trying to.

1:07:47What I'm trying to ask is like what you know, how does all of the kind of energy and work that's going into eval like apply to Agents and in particular like yeah end-to-end real-world agentic performance Yeah, um So I think Like it whatever we like it or not. I think a lot of devaluation here is still so follows a lot of them other George kind of thing so no you could benchmark things and Some problems have like objective like results of one on the Gaia benchmark You might have things like you know, how long does it take so the answer is a number but then The result that agent I don't even know that Good good I was just gonna say like one thought that I had earlier

1:08:46really questioned the objectivity of the result like the marathon runner Yes, like I can divide the speed of the marathon runner by the circumference of the earth but like That gives me one answer But do I expect the agent to take into account wakeful, you know time versus sleep time? Do I expect the agent to take into account routing? Do I expect it to take into account transportation? Like I don't know that that objective number is actually objective Yeah, yeah, exactly. So, so you could have one run where the agent says, you know, I'm gonna assume that we run exactly around the equator, right? And we use that exact distance. You might have another agent that's a bit more clever and say, hey, no, if we look at the map and make estimations around like from this exact point to this other exact point, this is the only land travel route. Complete different answer.

1:09:45Just like you mentioned, we might have another agent that sees things like, okay, I'm going to take it into consideration that how this person is going to take an app. A completely different answer. And so it becomes very hard to do stuff like that. And so the key point here is you shouldn't just benchmark the final answer. You should benchmark the entire trajectory. And the best that you can do is to have an LLM as a judge where you define the criteria for evaluation.

1:10:15It might be something like, does this result? Is it based on solid sound or reasonable assumptions? Is the calculation correct? Is all this fuzzy logic evaluation criteria that you probably might adopt to benchmark how the system behaves? Yeah, what you're really trying to do is you're trying to benchmark its reasoning ability.

1:10:45in a way that I think is different from reasoning benchmarks and that those are all benchmarking, those are all comparing against an outcome and answer. And what you want to benchmark is like a thought process. It's like an interview where you're given a really hard problem and you're expected to talk through how you get it. That's what you want to do. And the point is not the final thing, you come up with the point is whether...

1:11:12you're thinking in a reasonable logical manner. I mean, you could have a couple of rubrics, right? So all interviews go into rubrics. So essentially what you're doing as a judge is that you're defining the rubrics. And more importantly, right? How do you interpret the result of these things? Every number itself is not very meaningful, but it's relative numbers. So if you start up with a B's version of the system and you get your first set of numbers.

1:11:41Now, what you want is that as you tweak the system, the number increases, right? You don't care about the absolute number. You just care about the fact that, like, you know, this number is indicative of progress in some direction. And essentially, it's not the number itself that matters. It's mostly, like, as I make changes to the system, do I see changes in a direction that, like, I care about? So I think this is a common thing in this space.

1:12:10um and designing the right like structure and as we get like cheaper open source like reasoning models like deep seeking co- we get a chance to be a bit more creative in how we sort of evaluate like you do like LLM as a judge um and I think a lot of people are beginning to sort of integrate like this sort of approaches to their own like like business problems you know invest a bunch of time come up with those rubrics, structure it well, and then optimize. I think Jason Liu, who's also been on the podcast, has talked about being a bit creative in coming up with low-level metrics that are not as expensive to compute. Also, again, these can be good relative numbers that you can use to make sense of the direction of the impact of changes to your systems as you iterate.

1:13:11We talked a bit, quite a bit about multi-agent systems and some of the, you know, we talked about architectural considerations and abstractions. And I'm wondering if there are other aspects of that that are worth digging into. I feel like we kind of dug into specifics, but we didn't really talk about kind of broad motivation of multi-agent.

1:13:42And when the complexity of multi-agent systems is warranted from a use case perspective? Yeah. So the idea of multi-agent or autonomous systems is really attractive. And one of the downsides is that you might see teams just hurry, just rush to try to apply these things, even when it might not be the best tool for the task. Of course, Choosing what to use should always be a careful scientific process, you know, like what is your business problem? And does it fit the parameters of the tool? So at the end of the day that the focus should always be solving the business problem or use the problem now in in determining when to use like an autonomous like multi-gen system I have found that like no a good framework to use is something called like the complex task framework and so

1:14:38And my thesis is that autonomous multi-agent systems are sort of good if your task is complex. And what does that mean? I think there are four high level areas that I sort of ask people to sort of think through. The first is planning. Will your task benefit from some sort of just careful planning stage? Can you take the task? Can you decompose it into a bunch of steps such that successfully completing each step in whatever order?

1:15:06will take you from a state of unsolved to solved. Now, it's your problem. If it doesn't have that, maybe you don't need a multi-autonomous multi-agent system. The second is for each of these steps, does it make sense to sort of, are these steps sort of distinct enough that like they benefit from like multiple expertise or tools or specialized knowledge? And the idea is that if they do, then you can represent each of them as like agents, so kind of like domain-driven design.

1:15:36Well, let's say you're writing some pieces of software, you need like, I don't know, someone that translates the user requirements into a set of like product requirements and then you need something, some UI engineer that designs the user interface and then you need like some backend API engineer that like creates the backend and then they need some software engineer that like builds out the front end. You need some integration work and then final. And you can then decompose each of these things into steps.

1:16:06These are independent expertise. These guys can do all their work and you can map them to individual agents. Another property here is does the task require consuming extensive context. Again, if we look at the software engineering example to write code, sometimes you might need to read a bunch of documentation. You might need to figure out like API references and argument. Now putting all of that across multiple domains in the same model, the same agents.

1:16:37might be challenging because, you know, we all know about, like, you know, as context just gets long, like, oh, limits might lose context. So it just, it does make sense to sort of isolate some of that context and all that extensive context processing within individual agents. And I know that, like, there's all these arguments around, like, long context and all of that. But, again, there's your problem have this parameter. And then the final piece is...

1:17:03Does your problem exist in a dynamic environment? So dynamic here means that let's say you take a step on action and the environment changes. And those changes could lead to errors that you need to then recover from. And so in that case, you need something that can adapt, something that can explore, like retries, branching, and adaptation logic. And so across these four areas, planning, diverse expertise, processing a ton of context. And then the task exists in a dynamic environment to require an adaptation. I think if your task just fit like four of these things, then maybe like you're probably, you've landed with a problem that would really benefit from autonomous multi-agent system. And are there, you know, beyond kind of use cases as kind of

1:17:59fitting into those patterns, are there specific use cases where you found folks getting the most bang for their buck with multi-agent? There are four high-level areas that I've seen a lot of people explore multi-agent systems. And then these things might not always be fully autonomous, but on the spectrum between, let's say, something with some deterministic chain, with some complex retry logic. It's a little bit of what I was getting at because...

1:18:29Like for example, the canonical example of multi-agent system is like a researcher. Something's going to grab some context on the web. Something's going to write something. Something's going to evaluate and edit that. And it's going to be a loop. But I don't know if that's because that's the best way to build that system or because that's the easiest way to demonstrate that system. And there's a difference. Yeah. So it's on a spectrum. So what I'm saying is like some complex graph or loop.

1:19:00And then on the far right, it's more autonomous behavior. But software engineering, so things like Devin, magic code, back office tasks, process automation kind of tasks, legal and finance, customer service and sales agent. So these are like four high level areas. And yesterday, just before the school, I sort of pulled the numbers.

1:19:25I pulled data from Y Combinator. I mean, it's not a perfect representation of everything, but it's a good sample. And I looked for all the companies that mentioned like AI agents explicitly in their task description. So in 2022, there were 17 companies that sort of mentioned AI agents. And in 2024, about 92 companies, a 441 increase. And across all of these companies, you know, most of the value proposition that they had was around like...

1:19:53the automation of tasks that were previously reliant on human labor, things that required a lot of repetitive processing, things like data analysis, things that required communication across multiple systems. So the core idea is if your task has repetitive processes, we can automate it using LOMs or like agentic systems. If your task requires...

1:20:19Say individuals actually humans are having to coordinate across multiple systems and we can automate some of that and that's how we sort of provide provide value so I Think it's also instructive to sort of look at that list and see what like you know, those companies are sort of doing but I think this is like we can also categorize all of them into these four high-level errors like software engineering Back office tasks in some cases it's like given health or dentistry and management systems In some cases it's just legal like hey no we'll help gather all documents required for your case preparation We'll generate briefs will save your lawyers a lot of money or a lot of time And in some cases just customer service like we're triage all the information around the customer We'll like come up with automated like resolutions as we're seeing we'll have a human in the loop

1:21:14And so this is kind of like what I am seeing. Yeah. One aspect that comes up all the time is like, do I need a framework to build an agentic system? You talked already a lot about what the framework is providing. And a lot of that sounds complex, especially when you're talking about dealing with the...

1:21:41you know, challenges of message passing systems at scale and distributed computing in general. But, you know, talk us through, like, you know, use a framework versus, you know, build it yourself, and that whole thinking. Yeah. Yeah, that's a great question. And I like to think back to the early deep learning days, you know, say five, six, seven years ago, frameworks like TensorFlow and PySource were just coming up.

1:22:11And the truth is, at the time, a sufficiently skilled machine learning engineer could take a model and represent it using non-pi matrices. And they'd write down the sort of represent their weights using matrices. They could write their own automatic differentiation library to implement gradient descent. They could put all of that into a loop, write a training loop.

1:22:39But the problem is that like half the time you make just a single mistake and all the numbers are wrong and it takes weeks or months to sort of debug that stuff. And so as a community, the machine learning community sort of correlates to organizing to good abstractions. So for example, we want some good abstractions for automatic differentiation done. We want some good abstractions for a forward pass, a backward pass done. We want some good up.

1:23:06abstractions for an optimizer done. And it turns out that if you can compose all of these abstractions, then you can represent almost any neural network architecture, any type of training loop and that sort of thing. And I feel the same will apply to multi-agents or autonomous agent systems. If you're sufficiently skilled, you probably can write things from scratch.

1:23:38If your setup is relatively simple, you just have a simple set of chains. You probably don't need a framework. However, if you want to build something that's autonomous and you wanted to think through what is the right control flow? How do we define when the task is completed? You want to define how do we figure out when to delegate to humans?

1:24:07How do we express what patterns do I use? It gets pretty involved. The configuration space for the systems, they sort of interact and they can get sort of combinatorial in some sense. And at that point, it's helpful to have framework. So the whole idea is like, so for example, like two days ago, OpenEI released the operator agent that sort of exposed tasks by driving web browsers. And with the Autogen API, they had a little API, you could implement about the same functionality in about 40 lines of code. And so this sort of being able to take building blocks, sort of put them together, enables like accelerated development. And then a lot of...

1:25:04known or stabilized patterns that just get baked into the library. And so these are good reasons to use a framework. And of course, one caveat is that if you use a framework, there's some level of indirection. And so frameworks have defaults. They are default system messages. There are some default transformations to the, I don't know, as message flows through the network. And so maybe what hits OpenAI is really different from, let's say, the input that the user provided. And there might be some behaviors that, you know, some assumptions that are made underneath. And so from that sense, you know, as a used framework, it's always a great idea to sort of...

1:25:56sort of get familiar with exactly what happens underneath so that like your debugging process and just making sense of what your end system does is just better. And does the, do you see the framework moving towards giving the user more visibility and control over some of those underlying assumptions and transformations that you mentioned? Yeah. So, I think the right way to go about this is to have two levels of APIs. And so have a low level API where if people are comfortable expressing just anything they'd like to, it's possible. Is your core versus chat in the case of Autogen? Yes, yes, core versus chat in the case of Autogen. And in situations where you really, really need to be in control of everything the system does.

1:26:57definitely go with the low level API. And with the high level API, the abstractions, I think a good framework should have a strong observability story. So first baked into the developer experience. So for example, in Autogen, there's the idea of, as the agents sort of interact, they sort of yield these asynchronous messages.

1:27:24tell exactly what each of the agents are doing at any specific time. And you could take those messages, display that in the UI, write it to some login system. In addition to that, we also emit open telemetry events down the stack. And you could have your own open telemetry endpoint sync, data sync, and just store all of that. And it's just great as a way to debug and review exactly what went to the API, exactly what came back from the API.

1:27:54Just down the stacks. I think observability is one way to sort of like open the box and the second has to do with just the flexibility to either use a low-level API or a high-level API. Got it. Got it. So not necessarily a world in which the developers overriding the system prompt assumptions that the framework is making or those kinds of things or Well, by design everything is like you can Everything is parameterized. So for example, even in the agent chat high-level API, to define an agent, you can supply your system message directly. It's an argument. You can supply the list of tools. Again, an argument. The memory interfaces, you want this thing to use. Again, an argument. And there are a bunch of other stuff. So they are good defaults, but they all can be over-reading. And again, you can also

1:28:46override or overload this class is just classic software engineering and then implement your own core behaviors. So everything is extensible. I was just I was speaking in terms of like the developer that really doesn't want to do anything at all. You make zero choices about that. And then you have to live with those assumptions and the observability gives you some visibility into that without you having to necessarily specify everything. Yeah, correct.

1:29:16So let's jump into your thoughts for 2025. So I think there are a few things that I think will happen in 2025. A lot of this is informed by my perspective on work done with other gen. So in 2025, I expect to see continued improvement in models. I talked about how agents, especially in autonomous mode, might explore trajectories that might be suboptimal.

1:29:46That's the thing. I feel like this is a rewrite reinforcement learning trajectory. And so let's say you spin up, you ask an agent to book a flight. There's a chance that we figure out ways to collect enough data, do some sort of fine tuning that gets the agent to just go through the most efficient trajectory to get the work done every single time in the first try.

1:30:14I also think the same kind of thing could apply to memory and adaptation. And for the most part, it's figuring out what to remember, what type of previous experience to retrieve in order to have a better chance of solving the current problem. Is this dynamic optimization of the context? Is that the way to think about this? Yes.

1:30:46Yeah. Okay. The right pattern, the current right pattern, you know, it sort of makes all these assumptions that like you have the right thing in the database and you can sort of retrieve it and use it just in time. But there's also the other part of like, how do you know when to put things into the memory bank or into the database to your effects of database? Does this always have to be explicitly by the user? Is this something that you can do in some automatic dynamic manner and I think it should be a combination of both and making this like a reinforcement learning problem. I feel this will make us like help us make progress. Another thing that I think will happen in 2025 is the consolidation of agentic patterns and I talked about like control for patterns and task management patterns.

1:31:43And I'm hoping that like we all as a field or a community sort of align well on what works. And the goal is that like it gives us some shared understanding, some shared language as to like, you know, for this class of problems, then here's the right pattern for multi-agent system that works best. I've also been working on the idea of declarative agents or declarative multi-agent systems. So imagine that we could 100% specified entire multi-gint system as a JSON file or something like that. And then if we do that, then we can rapidly get to a point where the construction of these systems could be dynamic. So as opposed to a developer having to say, here are like a magic one. Here is like an orchestrator and four agents, and this thing is going to be what solves the task. Maybe we can even just pop up a level more.

1:32:42And whenever we get a task, we construct this declarative representation of the entire agent system that might work best for this task. And then we sort of instantiate and run this thing, and maybe even optimize along the way. And then finally, I think we'll make progress on the UX for agentic interaction. And so a lot of people have made this strongest parallel between an agent and, let's say, a junior developer or an intern.

1:33:12An intern should be proactive. And we need that interface to do the same. So go get some work done, come back and notify the user. It should be interoptable. And so just like an intern, if you get them to, if you see they're going down the wrong path, you should be able to sort of interrupt them, provide feedback, and then get them to keep going. We also talked about how the big chunk of the digital world needs to adapt to just agents becoming a part of this interaction. It might be agents or TXT. It might be new types of filters and captures, that kind of thing. And then there's a whole idea of figuring out how to ensure that human bandwidth is not completely overrun by just agents sort of interacting. A lot of those point to control of agents in the wild and not necessarily control on the part of the people who are publishing the agents, but

1:34:06other actors in the world. Yeah, yep, correct. And then there's a final piece that might not be technology or agent-focused, but there's also the consideration of how the workforce will change as we have more just agents out there. And a lot of people talk about the concept of hiring agents instead of software engineers.

1:34:34Would we see these sort of parallels companies advertising that now? Yeah. And what does it mean? Of course, you know, as technology sort of emerges, like a lot of things change. But I feel like, you know, at least on the minimum, we should be having like a lot of conversation around like how like agents will sort of impact like the software engineering field and all that. So yeah. So these are like the things I'm thinking of when I think of the agents and in 2025. Do you have a personal take with regard to software engineers as an example? Yeah, so I actually wrote an article about how AI might impact the software engineering career. So the summary of that take is mostly around it probably will not replace software engineers one-to-one because there's a lot of other things software engineers do.

1:35:36Especially senior or both software engineers, there's just a lot of things that these engineers do that are beyond writing code. Yes, communication and context and translating like human requirements that are usually severely under specified into like software technical systems. Those things are iterative, require a lot of effort back and forth with a human, actual human that like, yeah, maybe do well. However, Junior engineering rules, like the kind of like, hey, build a web page in React that shows a company's logo or something like that. I think jobs like that are gone forever. Or things like write a script that like, I don't know. I mean, we used to have interns that would just write one script that like did one thing. Think jobs like that, jobs like that are gone forever.

1:36:37Another thing I started to see is like I've been in teams, where in meetings where there are some engineers that appear more productive and probably more capable than other set of engineers. But essentially what's happening is that these engineers, the first set of engineers, they're just using AI, they're using AI really heavily. They have their IDE setup, they have their workflow setup.

1:37:07They figured out how to take problems, write design documents, give that to AI, get nice modularized implementations. They've learned to write tests. They've learned to be really vigilant about the kind of mistakes that these models will still make. And they've learned to integrate the entire thing into PRs that are error-free or bug-free that the rest of the team sees. Now, this is a skill.

1:37:36Internet literacy, digital literacy was a skill that everybody needed to cultivate, how to use Google search, how to navigate the web. I feel like software engineers need to invest in that skill that lets them effectively integrate AI into your workflow. And the difference is really stark. So these two groups of engineers, I know for a fact that they're pretty capable about the same individual capability, but just one has invested in...

1:38:03just going through that integration process and ensuring they can come out with correct high quality code while the others just haven't done that yet. So that's sort of like my high level take. There won't be like one-to-one replacement, but some jobs below some level are probably gone. And then second, like investing in just AI software engineering literacy really creates like significant...

1:38:32differences in productivity across engineers. So what do you think, Sam? Yeah, you know, I find myself frequently struggling with, you know, the classic difficulty seeing exponential change, right? It's like, I use AI coding agents, you know, not professionally because I'm not building any software system, you know, of any significant scale, but I, you know, I've got you know, cursor set up and I use it, you know, fairly heavily. And, you know, there are times when I think it's like magic and incredibly productive and I can get so much further, so much faster. And then there are times when like, I get into the loop of like banging my head against this thing and it's not making progress. And it's like, you know,

1:39:27reorganizing the chairs on the deck of the Titanic, you're asking me to change things around that have no consequence. And I think in some ways I think it's probably like self-driving cars in that it's going to take a lot longer than people think because it's easy to see the 80% progress.

1:39:55But the 20% progress is going to take years and years and years and years and years. But I think what you mentioned, I think there's something in what you mentioned in that there's a skill to use any systems. And there's not just a skill, but I think there's a level of investment in building structures around these systems that that I may not see in my personal use, but if I'm doing this at the scale of an organization that has thousands of software engineers, I'm able to invest in the degree of customization that takes some of the frustration I see out of it. If you know anyone I can talk to to dig into how folks are using this stuff at scale, or if anyone has a recommendation for me, that's an interview I'd love to do.

1:40:53Because it's weird because on social media, it's like, yeah, it's all bullshit. The stuff doesn't work. And then it's like, there are doomsdayers and cheerleaders. And I think the reality is in the middle somewhere. And certainly what you're saying about skill is an important piece of that. Yeah.

1:41:23Half of the people that like complain that it doesn't work Some I think half of that is a skill issue. Did you just did you just haven't? Come up with like a structure for using these things if you if you're pretty efficient is like hey There's a class of problems where it don't even bother like don't even bother And you build intuitions as to like if I get if I ask oh Want to do this stuff. It's just gonna get confused and reorganize the deck and like oh If I ask you to do this without mentioning this really important context, you'll completely make a mistake. So there's all the stuff that like, I feel some type of tacit knowledge, you know, there's tacit and explicit knowledge. So tacit knowledge is like the kind of thing that like, it's very, you know it, but it's really hard to express in words. It's like, if someone asks you, how do I swim? You can't really describe it to them and then they jump in the water and swim.

1:42:23And I feel similarly like just building the right intuition as to when and when not to use it just comes from practice. So, yeah, I think it is when to use it when not to use it. And I've also found that there's an intuition around when to go to fundamentals, meaning, you know, so I get a lot of value out of cogent when I'm working with, you know, libraries or APIs that I've not used before. And, you know, I just want to do something quick and dirty. I don't necessarily want to go to the docs and read them top to bottom, you know, but at some point, you know, you get a sense and my, my ability to sense this has refined over time. And so now I'm much quicker to recognize that I'm at this point where I just need to go read the docs and understand what's happening around this.

1:43:22this thing that I'm trying to do because the models, I'm not maybe asking the question the right way or the model hasn't seen enough in the training data about this and I need to help it get over the hub. So that is also a skill or a feeling that is really useful in this. Yeah. Yeah. Absolutely. Awesome. Well, Victor, it has been...

1:43:50Wonderful. Both catching up with you personally has been a long time and chatting with you about all this stuff. You know, great conversation. I really appreciate the time you've taken to go through this with us. Yeah, absolutely. Thanks for having me, Sam. Pleasure. Thanks so much.