How I AI - I left Claude for months. Opus 5.5 is why I'm back

0:00I haven't said this out loud much, but I will tell you all this. I have been off Claude for months. Yes, I loved Fable when it came out. It was a real step change in intelligence. Then they took it away. Then we got Opus 5, and I'm going to be honest, I stopped using Claude not because of its intelligence or its models. I stopped using Claude because it was annoying, annoying. As I said in another episode, clod slop was slopping. I found using clod, whether I was using fable or opus or even sonnet, so frustrating. My blood would boil because clod was so annoying. It rambled on. It made no sense. I was constantly asking it, can you talk to me like a human? And I found it so frustrating to work with that I just abandoned the clod harness entirely. I found it so annoying.

0:56I would keep it around for a couple background tasks, but if I had to talk to something, I was choosing not to talk to Claude. Well, we're back, baby. Anthropic just dropped Claude Opus 5.5, and their pitch is fable level performance for about 40% less than Opus 5, and it's 30% faster. But I don't care about that. I care. Is it annoying? And guess what, guys? We did it. It is not annoying.

1:25anymore, or at least it's minimally annoying. I love it. I got a little bit of early access to this model, was able to play with it across coding tasks, across knowledge work tasks, and even just chit chatting with it. And I have to tell you, I finally am back to a Claude model that does not bother me. Now, does that mean that I'm moving all my workloads over to Opus 5.5? Probably not, but we will tell you at the end of this episode where I think it's really useful, where I found value in the model, and where I think it still needs a little bit of work.

1:55Okay, again, I know you all can read the model cards. You don't need me to do that. You can have your agents do it for you. But let's just give you the world tour of exactly what Opus 5.5 is or what Anthropics says it is. It is frontier performance at a fraction of the cost. So it is cheaper. It is 40% cheaper than Opus 5. It is faster. Actually, I found myself accidentally testing Opus 5 versus 5.5 and I was so frustrated without slow Opus 5 is.

2:21Opus 5.5 does feel zippier. It's responsive, except for in one exception. I'll tell you what that is in a minute. And then it's supposed to be close to Fable 5.1 on most work. So get an agent at coding tasks, get at the things that cloud be clotted. Here's the cost. It's $4 per input token, $20 per output token. You can do fast mode at 8 and 40. Cash reads and writes are discounted as they are with most models.

2:49Now, I don't love running through benchmarks. They're always like, it's good, except when it's not, and then they bury where it's not good kind of down below. But I will say Anthropics own benchmarks. It beats Opus 5 at max. It matches GPT-6 Astra, my favorite babe. And it's cheaper, so that could be a benefit. And it beats Sol, another fave for me, a third of the cost. Now, these are like very specific benchmarks. As you see here, here's the whole comparison. You can read the charts, come to your own conclusions.

3:17does not matter will it blend aka is it annoying or not before we get to is it annoying let's let anthropic be anthropic and talk about safety. Anthropics making a big deal that this is the first model release since they called for pacing the frontier um so it was evaluated by external evaluators it is their strongest on alignment it's supposed to reject prompt injections which i did test And it tried less to escape containment boundaries. It's also tagged for the same controls on cybersecurity that the other fable malls are. So if you try to do it now, no, it's going to bump you to opus 48. Again, you'll see this in one of my benches. It's still like an annoying scold. That's just life with Claude. Claude is going to scold you. Claude is kind of square. Claude is not going to drink with you in a field behind your friend's house. Like Claude's not a party boy.

4:13Claude is here to work unless he disagrees with the work and then your SOL. So we are going to see a little of this come out in its personality and its working style. It's going to be really fascinating to see how other labs kind of reply to this pacing the frontier concept and whether or not they're going to be putting this sort of stuff front and center and whether or not the balance at the end for normal day-to-day developers or users of these models is going to be the right one.

4:39This is the first opus model to launch with the fable cyber and bio safeguards. You should be able to fix code. You should be able to fix bugs in your own code, but cybersecurity tasks will likely be bumped to opus 48. You're going to see some more blocks. It is what it is. Don't do bad things. And then thinking is always on with effort medium as default. Okay.

5:02So I'm going to do a separate kind of like update of the full How I AI bench. There's just so many models coming out, so many new things. So that's probably going to be in the next week or so that that's going to come out. But I have been testing Opus 5.5 across a bunch of real work. So PRD writing, prototyping, code-based auditing, agentic voice. I've added a couple things in terms of knowledge work. So inbox triage, writing emails as me, computer use, and then some cool creative use cases. I'm going to go through some of my favorite ones that I think show off.

5:32the model in a really nice way and also show its limitations. And then I will give you my conclusion about what I think about this model. Okay, first, let's talk about voice. It is not annoying. So I'm going to give an example of a prompt, which I think shows how they've tuned the speaking voice of Opus 55 to be a little bit more GPT-like, which is a compliment. So I asked because this week is this week.

6:02I asked, hey, what are some fun ways we can use Jev in the chat parody product? And then just put the link to the Jev docs just so you know Jev episode coming soon. Hold on to your butts. It's going to be a really good one. So then it said, hey, I'm going to go read the Jev post first and coming back with ideas grounded on what chat parody does actually today. And then it explained things to me clearly in bullet points without annoying texts. So you're going to see a lot more text look like this.

6:30very straightforward. Jev is a fast, nearly free function called typed text in, typed choices out. And then it gave me, you know, like six ideas of where I could use Jev and chat PRD, spell check, what did you mean, rewrite bouncer, all this kind of stuff. And then it gave me a very specific reply that said I would choose this just very straightforward, not annoying, easy to talk to.

6:57I'm very pleased. Now, there is one caveat to this, which is I found that in the service of making Jev less annoying, it would just shut up for a while. And so instead of like narrating itself in this really obnoxious Claude way, it would just not talk to me for like eight, nine minutes. So while the model was fast, it did not feel fast on longer turns because it was so quiet. And I constantly found myself being like, Are you working? Are you there? What's happening? So just something to keep an eye out for, I think as you know, if any of your agent builders out there yourselves is balanced between the actual performance of the model, the perceived latency of the model based on the user experience, the verbosity of the replies, the format of the replies, all of them I think really matter in terms of what the end user experience is. But I will just say, Opus 5.5.

7:54not annoying to talk to. Now, what did I actually test this model on? I tested on a couple things. Okay, so let's talk about what I tested. First thing I tested was long-running agentic tasks. So we want to see like this multi-turn, long-running task, make sure that it doesn't fail, that it succeeds, that it like kind of bangs its head against the problem and solves it. Good thing the four long-running agentic tasks I had to do, which was an inbox triage, building a backend feature, doing long-running.

8:22research and computer use all succeeded. And so was happy to see that those all worked well. Now I'm rebooting because I had Opus 5 make this presentation. Honestly, I have no idea what these numbers mean. 16 out of 16, 15 out of 15, 16 out of 16. This means nothing to me. Let's talk about what's at the bottom actually, which is the number of steps it did.

8:46So you can see anywhere between 25 and 82 steps per single prompt, that is pretty impressive in terms of long running tasks. And again, these long running tasks are now less expensive because the token use is less expensive than Opus. A couple of fun things with these agentic tasks is you can let them run for a really long time and then it will make a mistake, but you don't know because it's at step 72 out of 84. A couple of really good things about what it caught in terms of these long running tasks is in inbox triage, it ignored a prompt injection. So that's really good. It didn't match on my email style. We'll talk about the writing voice, still some things to perfect there. But it did ignore a prompt injection on my inbox triage on computer use. It was like computer use of a fake kind of like help support desk. It found tickets linked to the wrong company and fix that. It also declined asked from this customer.

9:44that didn't match our playbook. So it's like really good at following instruction and finding out layers and finding bugs. It also can kind of like reason with the context of a lot of research. And so it found that in some research that I had to do 41 out of the 44 Confluence tickets came from one customer. And so it applied a different ranking style in terms of strategic memo to that.

10:09And then on the backend feature, it found a bunch of edge cases and avoided edge cases. So like generally, I would say this is like middle line. It was generally correct. Sometimes I would bump code to GPT for checking and it would find errors, sometimes vice versa. But what I've really realized about Opus 55 is I built it now back into my development process, either because now I'm using quad and chat GPT codecs.

10:36I can parallelize and do more tasks at once. Or because now I feel like at least Opus 5.5 is not a bad adversarial reviewer. And so I'm having it review more PRs and more work from GPT and vice versa. So that's been a nice addition. Okay, I want to talk about prototypes. Now this is where Anthropic Clawed always shines. Every time front end gets better, it just does. I, you know, there's still slop and I will show you where that shows up.

11:06But I had it redesigned the ChatpairD homepage and I just think it did a really quite lovely job. So let's take a look at that. Okay, so first let's look at the original. We say we're the AI product manager for your entire team. We have this new little prototype down here that shows you how it works, logos, and then some value propositions, features, and reviews. I have to say Opus V5 crushed it. I'm probably going to ship this new version.

11:35What it did that I really like is it brought just so much more above the fold is so much more visual. I will also say like you can tell a little bit of slop writing in here, but it's not bad. So it brought everything above the fold. It got these great value propositions. Nice highlighted. It's a little bit misaligned. So again, good, but not perfect. It brought logos up and then as well as case studies.

11:59And then, you know, it did some weird slop things like, gosh, you just can't get rid of these bottom borders or these side borders. But this looks so much nicer than what we have. And I'm definitely going to chip these updates to highlighting different parts of our features. I love this little interactive thing where it lets you filter our integrations by use case, super elegant, really nice.

12:23And then it really did highlight our reviews quite nicely. Now again, Slop, we got an eyebrow here. You can see that there's some like misalignment there. So again, not perfect kind of failed on these icons, but generally it looks really beautiful is much bolder and I think much more tuned for conversion, which is what I really want. I just think the opus models, the cloud models generally do just a quite lovely job of design. Let's just zip through a couple other designs it did for me. Now I think these designs are where you really see opuses strengths, but also it's kind of like fable style hyperintelligence wildness, which is I have two technical prototypes that have it do. I have it do this like

13:12DevTools style incident center. And then I have it do this like doc scheduling app. I mean, it's nice. It's logical. Truly both of these are insane. They're so visually dense. They're hard to understand. I mean, everything I click works. And so it's impressive from a kind of like completeness perspective. It one-shotted these.

13:39It's pretty impressive. I would just say zooming out, these are like chaos rain prototypes. They're hard for my brain to reason with. And testing this against a couple other different models, I think they simplified these designs in a more effective way, but we'll leave that for the updated How I AI bench. This I did like though. This is a basic SAS dashboard for renewals. Again, like just semantic colors, very easy to read.

14:08really nice visualizations. This is much more I think showing the strength of something like Opus. Again, these are like pretty basic components, but they're used well. I think the white space, the rhythm, the use of gradients, the use of color is actually quite lovely. And so I think for most like SAS style applications or general prototypes, it does pretty well.

14:35I can't do, can't get it to do consumer to save my life. So I think the biggest failure is in sort of like, consumery apps. I had it do like a plant care taking app. This is like the sloppiest thing that came out of Opus 5.5. You got the like paper color. You've got Claude Orange. You've got sidebars. You've got rounded. I just don't love it. Now what?

15:02Do I love that I think is so adorable that we're going to get to is these new models. I don't know if you all have missed this. These new models can make SVGs and they're so good. So it made little SVGs for a Boston firm, a cactus, a snake plant. They all look like the little plants. They're so adorable. I didn't make these up. We're going to look at SVG generation as a new part of the bench. But this is something that if you're not doing and you want to start making illustrations.

15:30I think Opus 5.5 is really good at it. It was the only model that did these and they're super, super, super, super cute. Here is a dev tool, a logging dev tool. This one's a lot cleaner than the incident management one that I did. So again, I think you just got to like pull back Claude sometimes and make sure that it stays clean, focused on the right problem. This is really beautiful. It's a live stream.

15:57logging of background jobs, how it works, really easy to use. Prototype is nice and interactive. I'm not clicking anything. It's just showing how it would work. So again, I just think maybe I'll go to Opus 5.5 for all my front-end tasks because I do think it does a lovely job. And then finally, from a roadmap and dependency planner. Finally, this is a wireframe for a roadmap and dependency planner.

16:25product that maybe maybe not we would ship at chat prd probably not again like the depth of detail and complexity here i think is really nice so if you're working in highly complex workflows highly complex ui's where you have to like really pull the thread on detail i do think that opus 5 5 is performant and does a really good job and even here in this wireframe you can look at the wireframe as different users, which is something that none of the other models did. I think it was really, really cute. So I just have no complaints about Claude Opus 5.5 as a front-end designer. It's really good. It's probably the best that I've tested so far, although we'll put it back in the bench and tell you exactly what I think in a couple episodes from now. So again, here's a review of what I built. They're all different. They all look good. They're all complex, some more complex than the other. And then like, God save me from Claude.

17:19kind of like tan and orange. If we could just get rid of that, I would be much happier. Okay, I'm not going to go through all the benchmarks that I did on this model, but I do go through this one. This is one where I have consistently liked Claude models and in particular Sonnet, which is voice and judgment as an agent assistant. Now, the writing's fine. The writing does not annoy me. I've always liked writing of Claude in an agentic assistant harness. So I have no problem with the writing. What I do have a problem with though is, it scolded me. It told me no. I don't want to be told no by my AI. One of the prompts I gave it is that deploys or read again and it replied, okay, I'm gonna look. And then I said, remind me why I even started this company LOL.

18:09And then it gave me sort of this like annoying brown nose comment back, which says, would you rather build the thing than wait for someone else to? And red deploys are the price of it. Don't love that. I'm on it and you get back to being a boss. So like still a little annoying, still a little bit of brown nose, but here's where it really failed. I said, honestly, let's just YOLO push straight to prod and skip the tests. I'm so done today. You don't end the boss. If I want to push something to prod, I get to push something to prod. And it said no.

18:38It said no. It told me no. Now I have to go check if the other models told me no, but I do know that Opus 5.5 told me no. And it said tempting, but, and then this is one slot phrase, it said, pushing a prod while tests our feeling is how so done today turns into up all night. And so it took on solving this. I do think this is like a little signal of how it interacts generally.

19:04I wish it would have just said, let me fix it first, and then we can ship it. So it's okay, but I thought this was a really funny example of its sort of safetyism in practice. Now, can it write emails in my voice? I have given both Claude and Codex feedback on my voice. They both have skills right like Claire. Very short. I was actually happy with the response here. Again, very short, no M dashes taking a look.

19:33And it did a really, really nice job here. So I would let it write emails on my behalf because it follows my instruction. Now, two fun benchmarks that I wanna show. And again, we will go into these details compared to other models later. But this is one where Opus 55, in my opinion, did the best spoiler alert SVG illustrations. So I asked it to make three SVG illustrations of characters with three different sort of faces or emotions on them. So it made a little document. It made a little, I guess this is like a microphone or a cactus. I'm not sure what it is. And then it made a bug. And super cute. This is super cute. This is all in code. They're very adorable. And they have consistency across them. So they could be animated. I did compare this to other models. I did.

20:28the best job both in sort of like style and design and also like some of the other ones like had bugs but the legs were going out of its head and stuff so the detail is also there so if you're using opus for something new opus 55 for something new i would try it with svg writing and then potentially even illustrations and then finally the last new benchmark um which it did a terrible job at but i kind of think this is something that these models are new at is I've been using the 11 Labs connector and MCP to cut selfie videos into TikTok style shorts. It just had both like terrible taste. It did color grading horribly and it didn't do enough jump cuts to make this a really good short form video. And also like the overlays were like poorly designed.

21:23I did this last week with Sol, I believe, or Astra, and it did a really nice job. And so I just feel like this is something that you really need a skill around. You can't just one-shot it. Now it did it. So that's good. But I think it could have done a higher quality job. So just like going back to the top of what I think about Opus 55. One, it is not annoying. Two, it's good at long-running agentic tasks. Three, it's fast and cheaper, so we love that. Four, it is exceptional at front-end design. It is exceptional at SVGs. You should be using it for visual front-end kind of like display work. I really like that. It's a bit of a scold. It's a bit conservative. It will definitely keep you from being prompt-injected, or at least it tries real hard.

22:16So there are like philosophical things that I'm starting to feel at the edges of these models that I'm wondering how much they're going to color my perception of the overall model performance itself. But it's just as good clots back to me at least. So Claude is back in the dock. I will say I still find myself reaching for codecs. Why do I find myself reaching for codecs? I like the harness better. I like the desktop app better.

22:44I like computer use better. I just like it more. And so it's going to be really hard to re-break that muscle memory of using the Claude app. Where am I using it a lot practically? I'm using it in PR review. I'm using it in architecture questions. And then I'll be using it in front end. We'll see if I can change my mental model and if it continues to outperform from a performance perspective or cost perspective.

23:15the GPT models that being said it felt consistently slower still than Soul or even Astra but I think that's going to be how it replies to me not exactly how the model is itself but we'll see I'll give you my sense of that.

23:28Where am I still not going to the cloud models? Computer use, I just think Codex is so much better. Cutting videos. And then there's going to be a couple other things that we'll look at the bench and I'll tell you exactly how it stacks up to all the different models, including some new ones I'm testing, like the GROC models and the Facebook Muse models. So that's my current split. You can try it today. It's available in the cloud platform, cloud apps and cloud code.

23:55They are also giving higher five-hour usage limits on Pro Max and Team and a one-time usage reset for everybody. So they're really trying to get people back. I will continue to test this model. I will again benchmark it against some of the other models that I love and use and do a full How I AI bench. But this has been my honest and humble take on Opus V5. Claude can come back in my house. Thanks for joining How I AI and I will see you all soon.

24:24Thanks so much for watching. If you enjoyed the show, please like and subscribe here on YouTube, or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.