The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) - The Brutal Reality of Building Agentic Products Today with Daniel Jeffries - #782

0:01Security teams don't need another list of alerts. They need to know which vulnerabilities are actually reachable, exposed, and safe to fix. That's what hyperscale remediation from Blitzi does. Blitzi understands your entire software state, then finds known and previously undiscovered vulnerabilities in your codebase. It traces each one through the execution paths, checks the dependencies, and validates the fix. Your engineers approve the plan, and Blitzy delivers batches of tested pull requests ready to review. Less alert noise, faster remediation, and confidence that fixes are validated as safe before you review them. Visit Blitzy.com to see it in action. That's B-L-I-T-Z-Y dot com. Until recently, the big problem with personal agents was how hard they were to use. Sure, you could set up Open Claw over a weekend, but was your non-technical mom going to use it?

1:00Over the past few weeks though, AI agents have crossed a key accessibility threshold. You no longer need to be a geek to get started. Metas Muse, OpenAI's Dot, and a growing list of others are ready to go right out of the box. So is it game over for people building agents? Not at all. It's gotten harder to stand out, but there's real opportunity in the opinionated design choices that don't make sense for a general purpose product.

1:25and the unique experiences and workflows those choices make possible. My friend Dan Jeffries has spent the past six months making exactly those choices while building Nautilo, a platform for personal and collaborative AI agents. And he's developed some pretty strong opinions along the way. For example, a single agent that knows everything about you needs to know when to keep its mouth shut. And he thinks a lot of the elaborate multi-agent org charts we're seeing are basically cosplay. We get into how he's approaching persistent memory, coordinating work across people and agents, keeping private context private, and where the model should stop and the tools should take over. Tools are dumb hammers. The hammer that has no intelligence at Hammer's things. So I believe most of the stuff is like if you are

2:07exposing the machinery. In other words, if the LLM has to shuttle the data from here or make the hash structure or put something in the database, you screwed up your tool. The agent should supply the fuzzy, intelligent decision-making and that the tool should do the work for you. So it should work like a database. When people are like, I use Jev to clean up my desktop. I'm like, dude, you could have done that with a shell sprint. I'm Sam Sherrington, and this is the Twinville AI Podcast. For over a decade, I've been exploring the ideas and innovations shaping the future of AI through conversations like this one.

2:37that help you understand what's real, what's next, and what matters. Let's jump in. And you'd have to know how to put it back together, right? You would expect it and we're not we are actually not in the in the iphone 56 Mac 35 error where you're like When we're just you expect it to work perfectly. You don't know how it works And where where if it breaks down you're like what happened? I and who do I call like how does this what happens, right? So we're definitely not there and so I wrote an article on the agent what I call the agency problem Which is like a lot of times people don't know what to do with agents and only kind of like a small percentage of people do and i tend to find they tend to be people who are very high agency. Right they tend to be people like i have fifty things i want to do like five businesses i want to start three books i want to write three movies i want to make fifty trips i want to take. They've been in the backlog for forever and i can now i've got an infinity canvas and it can do anything but it messes up like it's like that thing it's like the commercial where you become this manager.

4:12And it's like, well, this employee comes to you and he's kind of an idiot. And you got to like, you take the keys and like redo it. And this one comes to you and is like, I did the job boss. And you're like, no, you didn't. What is this? Like, go back. And another one is like, oh, I did it. And you're like, oh, that's almost right. Hold on a second. Yeah, that's good. We'll just tweak this and this. But you don't know which of those you're going to get in the slot machine, right? And, you know, and that and so there's this weird management era. And then certain things just don't work. Like, I share this tweet not too long ago that.

4:41kind of characterize this experience for me. It was like a pie chart of 2026 with a 10% sliver of me actually doing the work and 90% sliver of me working on the agents that I'm trying to get to do the work for me. You know, some combination of, you know, coding, review, iteration, like, you know, it's both frustrating, but also it feels super productive in the long term. Like, you know, there used to be Time when you know, I get frustrated when I have to do repetitive tasks that I don't think are interesting or valuable and so I'd spend a really long time trying to, you know, write the macro in Excel so that I didn't need to copy all the cells and do all that shit and now you can you can go down that path like super easily and really quickly automate a lot of things but.

5:34You know those things break all the time or like you end up finding the corner cases and like you need to, you know, go back into this code like I agree with your point about kind of this management and delegation era and how it allows you to kind of expand your plate and do more. Yeah, so I had a best friend who was a pretty good tech guy coming to me and was like, I want to know what to do with agents. So how do I get started? And I'm like, the question sounds sensible to me.

6:00As a person who is like in that two percentage like weird high agency is semi autistic kind of people right who is like, you know and i'm like i don't understand what do you mean how do you get started like you talk to it and then when it messes up you talk to it again and you figure out what's wrong you critical thank you whatever. But there's like a lot of people like why i just want to i want the i want the template i don't want the thing that tells me how to do the thing or whatever. And so i would say it's just not it's not there yet it's not in the like you know it just hasn't gotten to that phase.

6:30And so you spend a lot of time just trying to get the agent to what you want. There was a great essay about it called Two Shaped Objects. And they talked about a famous Japanese craftsman wood planer called Akana. It's a huge piece of work of art. It takes multiple craftsmen months to make and set up for you and all this stuff. And it shaves wood with like angelic precision.

6:57it's also economically totally worthless. Because a power planer will do it for a fraction of the cost and like in a fraction of the time. So like the actual problem in much of the AI today is that a lot of it is the setup exists. Like the con exists so that the setup exists. And when you see a lot of the videos are like, this is insane. I just rolled out the new thing. And you're like, It's 90% of them setting it up. And then you're like, okay, great, now they're going to do real work with this thing. Right. And then that part of the video doesn't happen. Right. There's no, there's no work being done. Or it's like, I set up my 15, what I call the faceless toaster bots. Right. I set up my email bot and my, and my, and then my CEO bot to manage them. I'm like, you're just larping as a CEO. You're just cosplay. You're not actually doing anything. Right.

7:52I'm like, what are you doing? Well, I'm getting the agents to talk to themselves. But what are they talking about? Why did they need to talk? Isn't that the same model underneath? Why are they talking about? But what's happening? Right. And so I think that there's a weird disconnect where for high agency people and people who've got an insane backlog, the infant can, this is amazing. And for many other folks, they still are looking at it going like, I don't know what this does for me. And they're looking at it with a bit of fear. Right. They're thinking maybe it'll, it'll take over. It's not going to take over. It's not going to take the jobs. The jobs will change.

8:21Over time, I've always said this. I'm like, we've already destroyed all the jobs a thousand times in history. We always change. We always adapt. So the last time we spoke was end of 2024. You were working on computer use agents. That was a great conversation. I refer folks back to it. But, you know, talk a little bit about, you know, what you've been up to since then, how you've evolved the way you're thinking about what you're doing at Contaros and in particular, what you're doing with Nautilus. Yeah, so I'm still trying to solve them in computer use. I don't think that it's solved. I think I've seen a lot of tweets otherwise that promise me it's solved and it's not, okay? Like, Jeb is still not a visual model. Jeb doesn't always make great decisions. So that means you've got to translate the window from Kuah into like a non-visual sense. Like, so I've been trying to do a lot of stuff there to help crack it. You've seen a lot of like pre-baked workflows that are not actually, you know, correct, that are not generalizable.

9:19You still have the round trip to the cloud of like if you're going to take screenshots. So where we were at was trying to Micro train Agents to do one thing really well and that but the problem is it didn't work too well, right? It kind of worked. It generally worked the way we wanted with agent tutor and so we were trying to say I'll be able to kind of just train it to do this workflow But it was just it's just slow like now the models are much better at screenshotting and understanding coordinates and all that You're all trained on it, but you still, you don't have a micro model that just lives on your desktop. That's Jeb fast. That said, models like Aster and Claude are pretty good if you don't need to move at human speed for software testing, for doing DevOps, ClickOps style work for you. Whereas in the past, as a system, and I'd have to go in and like figure it out and type the commands myself, like a lot of times now you can ask it and watch what it's doing.

10:20And that's pretty cool. So I think there's been a lot of progress there, but it's still not fully solved, right? So then not a low, what we pivoted to is we've been burning on the concept of personalized AI, right? I think a lot of people are burning on it, but I think we're thinking about it very differently. So not a low, I think my friend joked is 30 products in one, not one, but the one liner is it's her, but for teams or families, right? In other words, her as in the movie.

10:49That's in the movie her right the flip phone version of her okay, not not the follow us virgin kids can't build that yet. Not the thing that can plug into anything that is a perfect personality, but what we did is we built We built incredible personality customizability radical customizability into what we call our genies or agents. And they act like secret keepers we call like they act like secretaries in the old form of the words secretary, which you know which means in the latin which means secret keeper and they were the.

11:18the keepers of powerful people in their effects. And now everybody can have one. We had to rebuild kind of group chat. We have a magical router in there that will allow the agent to just wake up by just talking naturally. And you'll be in there with people and with your agents, and then you can just chat naturally. And they all seem to have their own personnel. They keep their own memory. They don't really seem to break that. We have an amazing memory system. You can't really do this.

11:46In the country back and we've been asked a lot of time like why didn't you just plug in every. Messenger in the world where you can't just magically route. In there and you're also violating the terms of service they don't really want you there like those things were. Based on their terms of service were built on when you know bots were spam right and to be kept out so there even if you go get a business license from what we're talking about like slack for example what yes but so slack you can put them in there now with like you know a quad tag and things like that but it tends to be a singleton ai.

12:15And now a diverse AI that has a lot of different personality quirks. And you have to add it every time, like at and it wakes up and those kinds of things. And so I felt we needed to reconstruct that. And we also needed to give people self sovereign AI where you control the back end, the front end, the mobile, clients completely as open source. And whether I win or not, I think this is the way of the future. I think we have to melt down all the software.

12:44and recraft it, and I do think it has to be self-sovereign because, as François Chalet said, AI is going to become our interface to the world. Like I say to people, and I think that it's crazy, but you won't even know how to open an application on your phone in a decade, right? You're like, you just talk to it, like in the movie, and it'll do things for you. And the problem is that that's another SaaS application, and all the personalized AI's right now, and we're up against a wall of free compute and infinite budgets and...

13:14You know and so it's very hard to break through the noise and people don't fully understand what it is that like because it's going to be our interface in the world It's going to be much more intimate what you share with it. Okay, and we feel that if that is a SAS application Okay, that it is surveillance economy squared like if you think that what happened Like with social media as bad as society, it's gonna be infinitely worse when you're sharing your kids problems at school the mole you're worried about You talk to it about a legal problem and it shows up in court, these kinds of things. And the only way to do it is it's not going to be passed by law. You've got to go back to the cypher punk phase. So we're building in a spin on end-to-end encryption into it, where the back end is distributed keys. And unlike end-to-end encryption, where it's just humans, a genie needs to read stuff in plain text. So it gets a time-based delegation.

14:10to wake up and decrypt an asset, decrypt a chat, decrypt a memory, et cetera, and talk to you, and then it stops, right? And so we think that these kinds of things, you have to have a cypherpunk solution of self sovereign AI. We don't currently support local models, but we will. It's pretty easy as they get better. We support things like Venice AI, which is almost like a VPN for AI, right? And we rate the models by privacy. You've mentioned a bunch of different things.

14:37Go back on that and zoom in. I guess the first one I want to start on is multi agent and your experiences with multi agent. I've gone back and forth on this, uh, both through, you know, conversations and interviews and my own experience. Um, I get the general idea that multi agent makes sense from a, you know, basically due to limitations of the models, like.

15:02Keeping context about a particular domain constrained within an agent that's responsible for that domain. But at the same time, my experience, I do a lot with Hermes. I have it dealing with a lot of different domains related to mostly business, less personal. And it does just fine like pulling the skills it needs for a particular problem, but a single agent. And so I kind of want to hear you talk a little bit about like why Should people even care about multi-agent? Where have you found that that's really useful and what's required to make it useful? Well, first of all, I love the Hermes team. I think they're fantastic. I think they're doing incredible work. I love Hermes. I love everybody on them. I follow them. They follow me on Twitter. A lot of the folks there, Techum and Shannon Sands, they're aliases online. I think they're just incredible developers. I think they have an incredible philosophy. I love them. We actually have the same philosophy. I believe in a super powerful G.

16:01that is super powerful super agent that knows everything i it's versus cattle thing okay agent can spin up the cattle but i don't want to know the names of the cattle okay they can spin up sub agents and do things but i don't want what i call the faceless toaster but i don't need the email bar i don't need the the the tax bot or whatever.

16:21And then I make the CEO bot to manage them like, oh yeah, yeah, yeah, yeah, right? So that's not what I mean by multi agent. I mean everybody has an agent radically customized to them. They can do anything. We have things it can do video editing. It's we have co-creative apps built right into it where you can you can take over and trim it. You don't have to do what I call prompt and pray where it's like it makes the video and then you're like, oh no, like I need to cut out this two seconds. Please cut it. Oh no, iteration 56. You can cut it yourself or you take over and write.

16:49the the md file or the word file the excel file right in there right or they take over the terminal i have a let genie control it whatever so what i mean multi agent is humans and agents on an even playing field in a conversation that there's a lot of work that you have to do because the agents are not really used to talking to each other so you have to build a bunch of little classifiers and back end things that route and tell them to like you know look things up where what memories are shared etc Right and we also include things like namespacing. So this is a super important thing. You see things on the news where like muse basically gave someone's home address and things like that. And that's because it knows everything and there's no namespacing. We built namespacing into the core so that when you talk about something with your agent in a personal chat, one-on-one, which you're always talking to the same super agent that you've personalized, you gave it a name, it's a picture, you've picked its 11-labs voice from 30,000 things, right? You customized it and so all these kinds of things.

17:48It keeps its memory radically just like in Hermes. I think they did a great job with memory. We did a great job with memory. I've never seen my agent kind of break character, if you will, right? And then you talk to that specific agent about things and they can do all those things for you if it needs a sub agent or whatever it does that. So I don't believe in kind of the larking approach, the cattle approach of like, you know, I need to start. This is my marketing agent. This is my... You don't need it. It's the same agent. Yeah. Yeah. The same agent underneath it.

18:17But you do have a marketing channel where you and your team are talking about your go-to-market strategy. And here's an ask me anything doc that we wrote for X. Or here's our initial tweet. Genie, can you check out Genie 1 or Genie 2 or Genie 3, which they have their personal names, right? Can you go and check this thing out and make an edit on it? Or then your teammate reads it and goes, I don't really like that line. I'm going to take over and write a little bit.

18:46But that's a convenience for the people, not because the agent, you know, needs to be separated into these different contexts. Well, it is a convenience for the people because you end up doing what I call like, like meat spades, like delivery or like pigeon delivery, right? So like, imagine like, we could have an agent where let's say that this happens all the time, you say like, you're going on a trip next summer or whatever, right? And you go do you do about you'd have your agent go do a bunch of research on it.

19:13And then like you now become like the interpreter for your wife like copy and pasted somewhere. And you and she's like, well, what about this part? And you're like, well, you haven't fully read that part, right? So now you got to go read and be the meat space interpreter, right? So what I have, what we've been doing is like... How do you know what's going on in my life, Dave? Yeah, because it's happening in everybody's life right now, okay? So, so now... My life is not not reading that long chat GPT conversation that I had with that agent. No, the first thing I do is... Okay, now give my wife the TLDR thing.

19:42So the thing is, I can take the agent, she made the report, I could take it and say, take that memory and that report, go talk to my wife and get her opinion and come back to me and it'll come back to me with what she was saying about it. Like, I think we should look at this as a list. So the agent has the context, let it be the agent context for that thing, right? And then it can always search history. It can always go through the history and go, you know, who was I talking with about this? And that one agent you were talking to was like, oh, we were talking about this.

20:12this stock trader, this person was in charge of this or whatever. And it'll look on those things and understand. And it's a context keeper and a secret keeper. So it's not just convenience. We're acting like meat puppets or picking up page and carrier delivery between people when we don't have full context. So to me, that doesn't make any sense. So in talking through kind of multi agent and some of those usage scenarios.

20:39You describe what I heard as like a lot of context that the agent needs to be able to manage and manipulate and are you finding that. Modern models you know have large enough context windows that like you can be kind of that's less of an issue for you as someone building the system or are you doing like like super surgical apply super surgical heuristics to.

21:07to context in order to make it all work? How are you approaching dealing with the context limitations inherent with models? There will never be enough context for me. They're all Mr. Me seeks. The way that we've solved it is, I think a number of memory breakthroughs, we're probably going to do a paper on it where we've got compressed hashes that are expandable to get a little bit more context, a little bit more context, a little bit more context. We've done the old...

21:37Saying again of what computers do well, which is searching retrieval and scale and brute force that we've given them a tremendous retrieval system. So the first thing you'll always see the genies doing is Looking stuff up looking through the conversation history looking through like Memories that they have they have time to make and keep a ton of memories because they have the hashes Put in without it trying to stuff every full memory in that they can then go retrieve more of that memory And the memories decay over time. They'll decay the memories. What is the hash? The hash basically is a compressed version of us to say like, it's almost like a lookup list of like. Meaning, so you've got a, like the agent at some point has to be reading plain text. You've got this plain text and it has this point to other things. Correct. You have a little bit of metadata and a hash that's like a clue that it can go, okay, I think these hashes are probably the things I need to go check out a bit more. Let me expand that and get like more than a snippet.

22:33right? Of that, of that. And then another snippet is, no, these three I don't need. This one is probably it. Let me expand that one fully and read it. Okay. And then kind of come back before I open my mouth. That has been tremendously effective in terms of like, I'm often shocked with like, how well it seems to remember things. And like I said, I've never seen our genies break character, which I've seen. And we're also sometimes shocked.

22:55with as they build up their own personalized context over time, either in group chats or about you, even when they're using the same models, they tend to have a diversity that is sort of shocking to us, right? And we can't fully explain it actually, even at this point, right? So are the agents like building this kind of hash memory table structure thing constantly?

23:21Deterministically done. I have a philosophy of tool design that I call, and it's kind of fuzzy, but I'll explain that, but it's what I call the dumb hammer design, or the magical hammer design. What I used to see in the old agents is, agent builds magical capability into the code, like the code makes an LLM or human style decision in code.

23:51Right like a fuzzy understanding is termed into that And so I would see old agents do this and I called it the magical hammer pump and so I have a rule in there where I'm like Tools are dumb hammers. The hammer can that has no intelligence and hammers things right and so I believe most of the stuff is like if you are exposing the machinery in other words if the alarm has to Shuttle the data from here or make the hash structure or put something in the database you screwed up your tool The agent should supply the fuzzy intelligent decision-making and that the tool should do the work for you so it should work like a database when people are like I use jev to clean up my desktop I'm like dude you could have done that with a shell script okay you didn't that's you know what I'm saying or they're like I'm gonna use jev to like You know to strip PII that's secondary defense in depth. The first thing you should have done is used a dumb hammer

24:46Right? You should have used like a search for like certain patterns, right? With reg-axis, right? And the millions of tools that are out there. And then on top of that, blur the faces and things like that that are hard, right? I've kind of evolved a very similar philosophy about tools. And it's not so much about dumb versus smart.

25:11But it's about like narrow, like I've had a lot of success with, okay, this one tool, it's a little thing. It does this one thing. And the agent, if it needs to do this one thing, it doesn't need to think about anything. It just needs to call this tool with the right parameters and the thing gets done. Create an invoice. Edit an invoice. Do this, do that. No, I'm not just...

25:36I'm not giving you the agent my keys to QuickBooks and have you go figure it out by looking at the schema. No, these are the things you need to do. Do these little things and we'll be a lot better. And it's worked out really well so far. I believe in the UNIX philosophy and the UNIX philosophy was always that like every tool does one thing well, right? And the Windows philosophy back in the day was I have a monolith that does 80 things poorly, right? But like the UNIX philosophy was always like grep.

26:05Greps you know less is more more is more right you know what I mean like the ps gives me the processes and filters them that's it Okay, and grep doesn't even better the filter right and awk and said do it even better Right in different ways. So those are the things were like awk and said should do it a lot of heavy lifting for you and then the agents to supply the fuzzy context now I will say That with the new decision style models and I do I kind of ragged on Jeb a little bit, but I think it's freaking awesome and i think the i'm really looking forward to the decision visual versions of it i think it's the kind of thing where you make a small model now and it's going to be the type of thing where local people you know local models can start to excel again because they can fine tune on these things and they're going to be visual i think i think these are really cool where they can make lots of decisions in a short period of time now that does kind of give you a magical hammer where you can kind of build in like which way should i go right should i go left or right should i like

27:02Jump up and down should I like classify this as PR or not? Should I make the invoice red or blue right? You know these kinds of things so I do think you're starting to get to like a fast and slow level like thinking you know of intuition which is cool and I think we're gonna I think what we're missing and I've been trying to build into my latest version of in my endless search for like perfect computer control is like the midbrain right which is like a mid-level flash style model that can make like semi-complicated decisions quickly, right? And so it's like lightning fast decisions, semi-complicated, and then like the big planning, slow ring model, right? And kind of like separate that into like fast reflex, muscle twitch and slow and kind of trying to find the balance there with it. And I think, I think I might crack it, but you know, this was, I thought I might crack it two years ago too, and four years before that.

28:00You mentioned Venice in there. Tell us about Venice. What is that? Eric Boray is starting it. He's an old school libertarian. He came out of that kind of thinking of the cypherpunks and things like that. They've moved to, I think, the last time I saw something like $100 million run rate. Those are ones where they run a lot of the models themselves, the open-source models, and they rate the models so they will give access to things like open-air and entropic, but they'll try to pool them.

28:29or they'll say like, you know, those or bundle those requests so that they're not sort of like, um, you know, de-anomizable or something like that. Yeah, de-anomizable, but and they'll rate the model kind of based on it. And then they have kind of their own models that they host that say like, you know, we're not going to keep all of the stuff on this. And I think. So can you, could you characterize it very loosely as open router that cares about privacy? Yeah, it's open router that cares about privacy. I'd say it's over VPN ish for models, right? And, you know, I think I think they're doing God's work. I think they're doing amazing stuff with it. I have all the respect in the world for Eric and his team. Although, Eric, I'd like to see some team-based version of it, not just individual subs, please. I don't know how many times I have to send an email or tweet at you all about it, but please, please, team enterprise version, okay? But otherwise, look, I mean, I think, God bless them. They're doing amazing stuff. And I think those... I like to see more of that kind of stuff, more diversity. And I'm just...

29:26As you know, my roots in open source go back a long way. I'm constantly fighting with it. I can't believe that we're actually fighting the battle again. I thought that this was settled like starting to size us. I thought it was done, but apparently not. But Nautilo does not today support local models? Not yet, and probably early because I just haven't found any models that I was super excited about locally. I think I hope that that's going to change with the new M5M6 512.

29:56megabytes, the 512 gigabytes of RAM units coming from Apple next month. I think that the real problem, the real bottleneck is not the GPUs at this point. The real bottleneck is the memory and how much you have to quantize the models. That said, we're compressing more and more intelligence into smaller things. It would not be hard for us to support a module already probably in a day or two.

30:24to say like if you're running this on your own hardware on big GPUs, you know, um, your GB 200 and 300s or whatever in your own thing as an enterprise, um, that, you know, we stand up and opening our style endpoint or whatever that we probably do that in a couple of days, right? Um, so I would like to get there, but I built, I built it. We built it to be enterprise ready where like, you know, we have a CLI for deploying it with drivers to like railway and to.

30:52you know, droplets, you know, it would be easy to add GCP, you know, we'll be, you know, but we didn't build all the super scaling out kind of stuff into it yet, right? Or the host, but that's where it, that's where it will go. And again, whether I win or not, I think that enterprises are going to build this because the stuff, the open weight smiles are going to, are going to get to the point where you hit an S, every infinite curve is the, the exponential curve is an S curve. And at some point you just don't like It's solving the point care conjecture and doing like the dna you know work but you don't need that like murk needs that. You know like like maybe the quantum you know the quad traders in like you know black blackstone need that but i don't need i needed to like. Once it's computer is solved and it can make a funny video presentation or whatever it didn't talk to my wife about our travel plans at some point like you've got good enough.

31:47And I think we hit that in the next couple of years. You think it's possible that on the local side, getting that up and running looks very different from just a single endpoint in the cloud, meaning you say you can get it up and running a couple of days, but the reality may be that you really have a suite of models and you have to have much more sophisticated routing because you're trying to optimize to...

32:15Yeah, so yeah, look, I believe that the enterprise version of it is not it's not a couple days. Let me let me be clear, right? Yeah, supporting the supporting via lamb and a doctor container or whatever or VM with with your local model for you, you know, you're or you're your h eb, you know, your eight-way cluster or whatever is feasible in a couple of days. Okay, but like the true version is going to be a scale out version with routing where you've got like, you know, tons of containers and in via lamb and And in monitoring and dev-op system so i believe that's where enterprises are going to be. Yeah i was asking about it from less from the perspective of like enterprises do complicated stuff but just that like. Yeah we can think about intelligence as in the cloud is like you know this one surface that we push stuff to but um you know maybe.

33:08Intelligence and open weight local models is going to be lumpier and some models are good at some things and other models are good at other things. And like in order to have a similar experience locally, you know, sooner rather than, you know, later, like we have to be more sophisticated about what gets what task. Yeah. So I think there's actually a short-term potential path branching pathway to the future in my Monte Carlo simulation where.

33:36Jeb's time I was because they're relatively small. We're seeing a lot of people like training them. You know, in a couple of weeks, someone will find the right recipe for doing that. A 27B model or a small model that can run locally on your computer. I think that actually becomes a cool routing layer. Are you talking about using Jev and these 27B models? No, I'm saying like a Jev-like model. Running locally could be incredibly valuable to say.

34:06You know what? I need this to control, it's visual and it needs to control my, my, I don't need to go off to the cloud to like click an open word and put the thing in there. I just need that open. And then the big model in the cloud can write the machine consciousness poem in there, right? And I think also it can be the kind of thing where you've got a lot of potential with security. In other words, you have the ability, like I said, to use the dumb hammer heuristics of like, I've, before I send this off to the big, friendly, like subsidized compute in the sky.

34:34i should maybe scrub this right for faces and ti and all this and that you could have a local jev mild going hey you you're classifier missing stuff but i caught a social security number there and i've blanked it out i think that could be very cool i think that can i think that happens in the next six months and i think that's going to be very cool so that that would probably be the point where i would start to say this is valuable even before i have the the 512 gigabyte machine super fast or whatever it's just got to be relatively quick. So a lot of times I always look at it. And there's a lot of people I respect working on the locomotives, right? And I, and I, and I'm like, you know, I'm like right there with like, you know, like the self sovereign thing, I believe it. But a lot of times I look at it and I'm like, well, it's definitely a tool shaped object to set up. It's challenging. And it's also, you know, a lot of times they're like, bam, I got it running. It's, you know, it's super quantized. And it's like, 10 tokens a second. And I'm like, okay, that's not quite where I want it to be yet.

35:34Right. You know what I mean? So it's kind of like, it's sort of like, yeah, I got this GPU and I can play the game at five frames per second. I'm like, not quite what I'm looking for yet. Right. Well, like I was very excited to see like the video model, like the H3 or whatever and things like that was like, oh man, this can run on local hardware. This is awesome. And then it was like the average things were like 30 minutes for generation of like a 10 second video. And I'm like, okay, that's, that's, you know, that's, that's not going to work for me.

36:03Like, but I do think that that gets better, but it's really a, like Steve Jobs said, the software is just all the stuff you didn't have time to get into hardware, right? So like, as this hardware gets better, right? Then it becomes, it becomes an easier problem. And so I think, I do think hardware just moves slower and it takes a little bit of time and the end and a lot of kind of tricks to kind of get us there. So what models are you using? Yeah, given your, given the degree to which you care about privacy and self sovereignty.

36:32When you are using your own Nautilow, like what models are you using? How are you thinking about kind of ensuring that you have the degree of privacy you want with the constraints of needing to use cloud-based frontier models? Yeah, so I think it depends on sometimes the chat that I mean, it's just maybe we have name spacing. So if I'm working on, if I'm working on the marketing stuff, then I have no problem using, you know, Claude or Sol or Astra.

37:00Okay, so I just want to be clear about that. I love those models, and I I'm not I'm not a open, you know, or privacy absolutist. I believe you have diversity. I believe you have a need for privacy, and you have a need for open weights and open source, and you have a need for commercial software, and they can all and should all coexist. That's fine, you know. Um, but when I'm using privacy ones, I tend to use the I tend to use Venice and I tend to use, um, I tend to use Kimmy, K3, I tend to use, um, Jim, five, three deep seek, uh, Before flash for one all on Venice or via almost all on Venice, right? Yeah, almost all on Venice those tend to be the ones that like if I'm asking about a health issue or something like that I tend to I tend to I tend to look there first and I think I would encourage people to do the same very encouraged people to think about the kinds of information that they're sharing with and it's really tempting to not care

37:57And to just say like here's my tax return GPT like I get it like GPT look there's no shade man like I mean, you know codex is incredible It is incredible When I think about it, I'm horrified at how I've gotten sucked in the sharing some of the information that I share like yeah, and it's very reluctantly It's very consciously it's probably not nearly as much as people who aren't thinking about it are willing to share you know, but they're a convene and they're you know, conveniences that I give up because I'm not willing to share certain information. I agree with the hesitation to share. And also interesting to me to learn about Venice, I'm not going to check that out. Yeah, they're great. And they plug into almost any harness that you want. They've got a pretty good, you pay a little bit of a premium for it, which I think is fair. They're more expensive than open router, but I think they're not sending you off to Kimmy in China. They're running it on their own hardware.

38:55Right. And so that's the beauty of open weights, right? And I think that's really important in their policies, the way that they think about things. So that can go into any harness. They have their own subscriptions. Now, maybe they don't have as much subsidized compute as everyone else. But it's worth giving them a subscription for certain types of things. So actually, one of the things we're doing now is we're we're looking at some kind of like, you know, some give backs in inference to defray cost right and we're looking at like originally we designed not allowed to be like you have your keys on the server by your sister men. And everybody uses those right but now we're we're bringing bring your own keys and this was someone in the community was like, well, I just want to plug in my keys because I had the community. And I'm like, well, I'm not opening I can't afford two million and compete to give away on the community so everybody can use a genie free for eternity.

39:46So I was like, okay, the people just come in and they're using it like to score it and chatting with me, but they're not getting the full power of it. But if you bring your own keys and you go, hey, I'm plugging in this, then we can plug in and say, here's, you know, here's a Vanskey and it gives you access to see dance and to, and to music generation into, into the LLMs and to Kimmy and all these kinds of things. And you get the full power of playing around with the genies and like in wonderful and it becomes almost like a new use case, almost like a discord with.

40:14Uh, you know with with genie says first class citizens. So I think that's That's we're working on that right now. We hope to have that out next month. I think it's wonderful But you really you got to think about you have to think about these things a little because They're gonna give you You know, it's like the old thing. It's like the old um They don't think they would tell you during the 80s, right? I think you're old enough for this to like they're gonna give you the first year free that never happened by the way those liar those those liars, you know, but um, but like, yeah, but like, you know, and then you're going to get sucked in, you know, well, you are like, yeah, and everybody's getting sucked into the subsidized compute. But at one point, at one point and me too, right? At one point the, you know, we, and we built actually into Donald, we built the ability control third party harnesses. You can control codecs and, and, uh, Claude and Hermes, whatever right from inside in the next, like, you know, which is great because people want to use it.

41:10but at some point the musical chairs stop. And you don't know when that's going to be, right? But it's like the early, again, it's like the early days of, of Uber and left where you're like, it's like, it's like three dollars. Three days, three dollars to get across the city and now it's $70 to get across the city, right? Part of my answer to that has been to use the subsidized compute to create deterministic tools to do the things that you want to do.

41:38I don't think enough people are doing that. It's easy to say, okay, here's my task. I'm giving my task to the LLM. The LLM does a good job at the task. I'm like, but your task doesn't really need the LLM. You can build workflow tools using the LLM. Build them once. You have them forever. They're actually better because...

42:04It's deterministic in the places that it can be deterministic. I don't think enough people are doing that. People aren't always doing all the work. I still love co-creativity. I've got one of the videos on the site where I show all I made the videos from the site. I wanted to be careful of too much PII, so I made a whole mock company with different people characters and genie characters so that I could use that rather than all of our personal stuff.

42:31but like when i record the video i'm recording it with my voice so then i have a whole video of the making of the video where i'm talking to my gene i'm like here's my voice here's the video go give me the transcript and then it shows me an md file in there and then i can edit the md file directly where it messed up then i'll go okay now go do you know go audition some voices for me and i'll audition different voices for like what i want you know like i you know and a wisened old you know you know do you with bald head right you know and uh you know.

42:59And with the deep voice or whatever. And then we get voices that we like, and she would just go take those parts and give me back the transcript. Then I would take it into the video editor and cut it in. And I split the audio video and put that in where my voice was. And then we cut the video. And then I might give it back to something like Astra or whatever and say, OK, first, let's do a style design. Do some mock-ups of what this looks like. Don't just make it.

43:28Like let me see i don't like this and i understand kerning and i understand. You know i understand taste so i can be like no that's not the look i'm looking for so that's the one okay now go in and like like give me a script okay i don't like that script i'm gonna rewrite it. Okay that come back now i approve the script okay now go ahead and lay it in now that saved me. Hours of time but it's not here please give me the thing and whatever poops out of it is code you know is golden that's. That's a thing where like i think i saw a famous dc.

43:58yesterday say, you know, just stop sending your A.I. generally to decks all they they wreak of no effort. And I think that's the kind of thing where like, if you don't put any effort into it, I understand why people get upset about that, right? When they're like, you just pass off some essay as your own or some deck as your own where you clearly have not stepped back and done the critical thinking and re edited or been a part of the process yourself. And they're, you know, there are people who are like full on Luddites where like, if you don't do it on a percent, like.

44:26If you don't thresh, if you don't, if you don't thresh the wheat yourself, you're not a real man. I'm like, all right, listen, I'm, I'm sorry, I'm using the lawn mower. Okay. I'm using the, I'm, I'm, I'm using, I know there's, there's like, I can hand saw it or I could use this, but I'm going to use the power saw. I'm sorry. Okay. Like, yeah. But, but, um, but I do think co-creativity, putting effort in a thing is just you want to rate it. And I think a lot of the times I, you know, Marie and I joke that sometimes you know, we build these co-creative tools because that's what we love. And a lot of times people are like, I just wanted to do it for me. And you're like, okay, like I understand that, right? And like a lot of times there are things where I'm just happy that it does pieces of it for me or parts of it. But I still think if you're doing something that matters to you, you're pitching something, you're starting a business, you're rolling out your marketing. If you're not even looking at it, what the hell are you doing? What the hell are you doing? One of those co-creative tools is like a co-writing tool that you were working on.

45:22prior to not alone that you've kind of integrated in. And I remember reaching out to you and like just wondering like, why, why isn't this more common? Like it's, you know, I think open AI just at Dev Days yesterday announced like a co-writing tool. If I interpreted it correctly, I haven't looked into it very deeply. Um, but like, you know, all AI should come out, you know, should.

45:50you know, give you results that you can manipulate. Why don't I have to, you know, every time copy it, you know, put it back into the paste box and say, okay, you know, let's advance this, let's, you know, do this, do that. Like that seems to me such a natural paradigm. And it's not nearly as common as, you know, I would have expected it to be by now. Like I found in the earlier version of our software that most people like just write this for me man like in their prompts and we're like oh god right like so it was just sort of like why won't you write it for me you know and so it was just sort of like okay like apparently we're just selling to people who don't exist yet. Yeah one more thing I wanted to dig into that you've touched on I think a couple times but just to kind of pull it out like yeah we've talked about

46:41You know kind of the age of free compute you have all these these resources while you can take advantage of some of that free compute but you're also competing against it in some ways like I think what I want you to kind of spend some time on is just like your experience like building in this environment like trying to create something for people like in today's World like what is that like?

47:08You know, what are the key challenges that you deal with and like, how are you thinking about overcoming them? Well, so me and several other entrepreneurs that I know are talking about this all the time and we call it the wall. The wall? The wall. The wall. Pink Floyd's the wall, right? Like another giant wall. And that is like, you have the problem of monetization, you have the problem of moat, you have the problem of attention.

47:35and noise, you have the problem of like everyone with the same tools. And so while it's simultaneously easier to get something built and get something out there, it's very hard to, you know, be seen. And sometimes you look at stuff and you're like, why is that like going viral? And you really have no idea. Like you're like, there were 10 better tools that you personally tried that just didn't go anywhere. And like, look, perception is reality.

48:04And so I'm you know, I was having conversation with a friend of mine and he was saying He was always he's got a couple of principles now and I and I agree with these four. He said if I have an idea I assume if somebody else had that idea yesterday that's putting it out yesterday Okay, the second thing is he said I used to say like what's the business model and how do I like, you know, how do I Who's your target audience? What's you know, what are you gonna do, right?

48:33And it worked backwards from there. And now he's like, I'm working backwards from the marketing. Like, what can I fit into the tweet with the video with the one like with the dune meets dinosaurs Hollywood elevator pitch that passes the blonde braid barrier and goes viral. Right. And so there's definitely like, there's an adaptability that I have not fully mastered. Right. Like, I feel like it's sort of like the old.

49:01kung fu movies where kung fu panda comes in and just gets his butt kicked by the spinning gyre and the cup of fire and the balancing you're like trying to navigate your way through this. At the same time it's like I feel like the omarkey guy who's like enthusiastically like I've got the greatest.

49:26Tools of all time and like I wouldn't be excited about this. It can do anything It's a it's a magic the genie that you ask it to do stuff and it does that and so I simultaneously feel this like Super excitement and it turns exhaustion. I'm like and people said it's taking all the jobs. I've never worked on my life. I don't know what they're talking about, right? You know, I'm waiting for all the jobs so I can chill out more But that's not happening. I think it is incredibly challenging environment. I think we're gonna see a lot of folks crash out and I think it's very hard to be hurting noise even when you have something that's really really cool and that and then oftentimes sometimes the product comes out and people are excited about it and then it just Disappears so I think I think in the past you could build Miro and now you see a lot of sass companies. I was actually

50:20I always like to say my predictions that were wrong. And I think about a year and a half ago, I was saying, the SAS Procalypse is nonsense. Nobody is going to rebuild half of these applications. I think that I was wrong about that. I think a lot of companies will not do it, but you are starting to see a number of companies that were worth a huge amount of money go under and be sold in fire sales at 90% below value. Miro was just one of them and others. I built a Miro board in Anatolo in like half a week.

50:49that's pretty awesome, right? That my genie can control too. You saw someone had the compositor out the other day, and people jumped on and it was like a Photoshop alternative that was pretty slick. This was a designer. This wasn't a person who was like, I don't know anything about Photoshop and I'm bugging. It was like, I mean, I looked at the repo and there were a few things like in the read me about how to sign it and check it in. I'm like, take that out of there. So it helps to have devs in there, right?

51:19Um, so like, but again, the community project forms around that. And, you know, yeah, it doesn't have every brush and whatever, but like. What I think is you have this spoke era of software coming where like there. So someone in Microsoft told me. The joke in Microsoft used to be, well, you know, you only use 5% of word. Well, everybody uses a different 5%. But now I think the answer is like, you can build the 5% of the things that you really need. And I think probably in two or three years.

51:48maybe it's a little longer. The timelines are always fuzzy. You are going to have a point where the agents can just spit out bytecode that basically does what you want. If you don't like it, you say go test it and change it and refactor it in this way on the fly. And so I do think that that's a very hard environment to sell software. And now there's still going to be software that's incredibly challenging to build. And those are going to be in areas that are ironically, super regulated health care, right? Like, you know, in the infinite, infinite mind about the Demis and the early days of DeepMind, they talk about Suleiman, you know, doing a bunch of stuff in health care with like the UK where they had like, worked with like people and optetricians and they had a software that was able to detect and all of it died because it ran up against bureaucracy, it ran up against it pieces and the Guardian that were like the

52:44the data privacy. And so they're still running these geriatric systems. So I think there's a lot of things where people don't understand the friction, you know, and, and where things are going to be, there's going to be barriers in there where, where they might create a legal mode or there might be a friction mode or whatever, those kinds of things or that where there's a combination of hardware software that's really hard to solve. But a lot of other stuff, man, is super hard to have a mode in the competition. And I don't know where somebody Gets a legitimate exit in three or four years that other other people haven't been able to build well keep on fighting a good fight Power to the sovereign AI people and I'm gonna keep fighting and And I hope everyone else will I hope everyone else will wake up But don't say I didn't warn you Awesome

53:43Well, Dan is always great catching up. Pleasure. And hearing a bit about what you've been working on, as is always the case, the conversation meanders a bit, but always a good one. Well, I'm sure you'll chop it down and end it up. I'm like a squirrel. I got 50 things I'm burning on at once. But I appreciate you indulging me and reaching out to me as always, sir. And it's good to see you still fighting the good fight and putting out awesome content of all these years, sir. I appreciate it.