← All shows

How I AI - Opus 5.5 vs. GPT-6 Sol: which model won my blind taste test?

Duration 38:50 · Language en · Published Sep 22, 2026 · 5 highlights

Summary

本期节目现场评测了 Anthropic 的 Opus 5.5,以及 OpenAI 的 GPT-6 Sol 和 GPT-6 Luna,并将它们定位为更便宜、更快速的日常编码与知识工作模型。主持人指出,价格、缓存输入成本、输出 token 效率和实际延迟已经成为选择模型时与智能水平同等重要的因素。她特别讨论了 Opus 5.5 更严格的网络安全与生物安全护栏,并认为厂商的风险管理哲学也会渗透到普通任务中的语气、个性和使用体验。为了超越过去只看 PRD 和原型的评测方式,她扩展了盲测基准,覆盖邮件处理、前后端开发、长期代理任务、计算机操作、研究、SVG 和视频等多类工作。大量现场打分表明,模型输出是否清晰、易读、不过度堆砌信息,往往比单纯完成任务更影响真实使用感受,而提示词的具体程度也会显著改变结果。创意测试中,所有模型的视频剪辑表现都不理想,但 OpenAI 模型在角色 SVG 上意外领先;独特的“芭比基准”则说明 3D 人物生成仍远未成熟。最终,Astra 和 GPT-6 Sol 最讨主持人喜欢,Opus 5.5 则在最广泛的任务上稳定获得高分并重新进入她的日常工具组合,同时人工偏好与 LLM 裁判的明显分歧也提醒听众不要把自动评测当成唯一标准。

Highlights

  1. These are not the like frontier frontier models. All of them cheaper and faster. So these are going to be your daily drivers for common coding and knowledge work tasks. And then caching, really great, especially if you're resending contacts.

    这些并不是最前沿中的最前沿模型,但它们全都更便宜、更快速,因此会成为处理常见编码和知识工作任务的日常主力。如果你会反复发送上下文,缓存也非常有价值。

    Reframes the model race around practical daily economics
  2. Opus 5.5 is the first Opus-level model that has shipped with the Fable-level kind of like cyber and bio guardrails. I do think that leaks into even non-risky activity, personality, and behavior. And so it's something that I think you should really keep an eye on.

    Opus 5.5 是首个配备 Fable 级网络安全和生物安全护栏的 Opus 级模型。我确实认为,这种风险控制会渗透到非风险任务、模型个性和行为之中,因此非常值得持续关注。

    Connects safety policy to everyday model personality
  3. Sometimes if you prompt it in a really detailed way, it does a great job. Like this one is quite detailed. It all works, but I don't know what's going on here. And so again, like calibrating your benchmarks is really important.

    有时如果提示写得非常详细,模型确实会做得很好。这个结果细节很多、功能也都能运行,但我却不知道界面到底在表达什么。因此,校准评测基准真的非常重要。

    Shows why task completion alone is a weak benchmark
  4. I ask it to make a 3D model of a Barbie. Everybody else on YouTube, all the bros are making video games with like spaceships and all different stuff. Your girl wants a Barbie fashion video game. So again, the one bench that has not been crushed is Barbie Bench.

    我会让模型制作一个 3D 芭比娃娃。YouTube 上其他人都在做飞船之类的电子游戏,而我想要的是芭比时尚游戏。总之,至今仍没有模型真正攻克“芭比基准”。

    A funny and revealing stress test for 3D generation
  5. GPT-6 Astra and GPT-6 Soul win my heart. Opus 5.5 wins my week. It's the one I rated highly most often. And this is where it gets really funny. The LLM as a judge and I completely disagree.

    GPT-6 Astra 和 GPT-6 Sol 赢得了我的心,而 Opus 5.5 赢下了我这一周:它是我最频繁给出高分的模型。最有趣的是,作为裁判的 LLM 和我的判断完全相反。

    Captures the verdict and the limits of automated judging
Full transcript

I was massively prepared for Anthropic to drop Opus 55. I was even prepared for OpenAI to drop another model. I got up early this morning because I had a little bit of early access to Opus 55 to record you all an amazing Opus 55 review. And guess what?

They both landed this morning. So now I am, despite having in the can, you will see it later, a great Opus 55 review. I am just going to go ahead and do this one live. And I have never done anything live. So I am going to talk to you all about...

these new models. There's actually three that came out today, Opus 5.5 from Anthropic, GPT-6 Soul and GPT-6 Luna. There's like price wars going on and I haven't done an episode on the How I AI Bench and I just updated it for these new models and we're actually going to vibe check this baby live together as a group and we're going to go through very quickly what these models are, what they're saying about the models and then I'm going to vibe check Live we're gonna do the blind taste test bench. We're gonna surprise myself live you all can see my complete internal inconsistencies and We're just gonna see how it goes

So Quad Opus 5.5, GPT6 Sol, and GPT6 Luna. It was able to have some early testing of all these models, so I have a good sense of what I think they're good at. But I did think it was an important moment right now. We've had Astra come out, Fable's been refreshed. I just wanted to completely rethink the How I AI Bench. And if you've watched any of our old How I AI Bench model episodes, they're really focused on two things.

PRDs and prototypes. And I just think that these models are getting smarter, more agentic. And so I actually built out a much broader benchmark for this blind taste test. And I'll walk you through all how that works. And I want to get your feedback on if the comparisons are useful. And we're actually going to blind taste test it live. You all can judge how vibey my vibe check is. This is just how we make decisions about things. So it is imperfect, does have an LMS judge in the middle, but But we're going to do it live and we're going to see truly which model I love. But before we do that, let's just go through the highlights. OK, so these are not the like frontier frontier models. These are not the astros and the fables, but Opus got a refresh. Sol got a refresh and then Luna got a refresh. All of them cheaper and faster. So these are going to be your daily drivers for common coding and knowledge work tasks.

Here, you can see the stack rank of how expensive these models are. Opus 55, twice as expensive as GPT-6 Sol. And the pricing wasn't out when I tested these. And this is really going to impact how I think about using these models. Again, Opus 55 was cheaper than Opus 5 and Fable 1, to which they compare the intelligence. But Sol, my favorite, my babe, That is, I'm excited about because GPT-6 Sol is my favorite and cheap. Okay, so the real thing that they're focusing on, not just cutting the cost, but also cutting on cashed inputs.

and just speed and token efficiency. And so you're going to see both output drop, token use drop, and cost drop. It's really, really nice. And then caching, really great, especially if you're resending contacts. We've seen a lot of caching savings when we use all these models at chat.

PRT and so definitely if you're building on these models make sure that your caches are optimized and that you're using taking advantage of that because it can save you a lot of money and if you don't as I learned it can cost you a lot of money. The other thing that I wanted to call out and again I hate this claw generated.

call generator presentation, but this is something that I really want you all to focus on is this left side on opus 5 5 opus 5 5 is the first opus level model that has shipped with the fable level kind of like cyber and bio guardrails and so you should be able to fix your own bugs in in opus 5 5 But you won't be able to do, like, cybersecurity work or biowork, like, make good choices. It'll kick you down to Opus 4.8 or block you. So just know that there's those controls there. You know, Anthropica is saying that this is its most aligned model. It's the one that was built when...

after like pacing the frontier, so it had external evaluators. One thing that I've seen in Opus that you should expect to see from a lot of anthropic models is it continues to be a little bit of like a conservative scold. You'll see that in my Opus 5.5 review. So, you know, it will tell you no, it'll tell you like, I shouldn't do that. You won't get prompt injected, so that's good.

But there are some things here, I think, when you think about the general philosophy and approaches of these models, how they're set up, how they manage risk, how these companies manage risk. I do think that leaks into even non-risky activity, personality, and behavior. And so it's something that I think you should really keep an eye on. I truly have not opened Claude in months.

for day-to-day tasks. I've just been like really, really into ChatGPT, really into Codex, really into Astra, really into Sol. I just find that Harness has been more delightful to work with, but really the reason why I got rid of Claude in my day-to-day was between Fable and Opus V, I could not stand talking.

to Claude. I was constantly prompting Claude, like, can you speak like a human? I was constantly prompting Fable, like, I do not understand what you're saying. The Claude Slopp, as I say, was Claude Sloppin. And so just from a, like, personality ergonomics joy of use perspective, I really just went full codecs. Now, spoiler alert, having tested all these models recently, I still really love the GPT models. They make me happier. I find them more enjoyable to use. Everything's a lot better for me, but I have brought Claude back. So Opus 55 just spoiler alert on the day to day of using it has worked its way back into my daily experience.

I'm a lot happier with just simply talking to the model. The feedback I gave to the anthropic team is like, not annoying. I found myself annoyed zero times with opus five five. And when I thought I was annoyed with opus five five, I was actually accidentally on opus five. So opus five is not fun to talk to opus five five though.

gives me clean bullet points, is normal, is not annoying. So I would love to see you test it. The other thing is Opus 5.5 is pretty fast, but not as fast as Soul is. So I was able to test both and I constantly felt like the GPT models, the Codex harness were way faster. Now I think the way faster came in two categories. It came in actual latency.

So I do think that, um, GPC soul was just faster. Um, the second thing it came in is I think part of the way they made opus five five, not annoying is they had it shut up. I also just found that it wouldn't narrate its work. And when it doesn't narrate its work, then you get into this loop where you're like, is it like, are you actually working? Are you actually doing it? Um, and so there were like pros and cons to this change in.

um in opus 55 communication where it actually felt slower than I think it was also you know clod on high effort it's like gonna do this it's gonna like spin and do a lot of effort and so um I think that combined with a quieter model just makes it feel a little bit slower whereas I felt like soul did the appropriate level of narration and also it was like enjoyable to talk to um let's bop over and do the how I Vibe review now. And so you can see here on the left side, I actually really expanded out the How I AI Vibe review. So now it does a bunch of things in different categories and we're going to add to that over time. So it still does messy notes to PRD. It also does personal productivity inbox triage and routine replies in my voice. So I'm really testing this model to see if it will speak in my voice.

Then, of course, we do front-end coding a lot there because I think it's visual and easy to see. I've added a couple back-end coding, agentic multiset versus back-end feature. I did, again, agentic voice, long-running research, computer use, and then I added some creative features. And so I put them all together. I tested a bunch of models. This not only includes the new models from open AI and Anthropic, but also includes I think rocks in here, I think muses in here. So who knows what we're gonna see. I will be honest, which is like I am always inclined to think that Astra or Soul will win because I love the open AI models. It's just the heart once, one at once. This is my guess. Here's my guess. I will like open AI models for knowledge work and personal productivity. I will like Anthropic models.

possibly grok who knows for front end. I don't know what I'm going to like for back end. I suspect I would like a cloud model for agent personality. I suspect a the long running agent will be a cloud model computer use I'm suspecting will be open AI and then creative I have no idea actually creative SVGs I think are going to be cloud and I think videos are going to be open AI or anthropic but we'll see I then do blind evals of these. So I run these against different models. I've kind of grouped the models in terms of capability.

And so I can compare like for like models, like I don't want to compare a Luna and a fable. So again, this is my bench. They're blind. You can see I have model B, model C, model E, model G. And then what I do is I go down here and I give a vibe one through five and I give terrible notes like simple and straightforward, no complaints. So I did that on the PRDs because it takes a little bit to read the PRDs, but we're just going to go through these very quickly and give my feedback. So the next check are all on personal productivity ones basically.

I give it fake emails and then I say, how would you write back to me and what would it do if I gave it feedback and how would it triage my emails? Okay, so there's Model B and Model E wrote nice little messages to me. They were long, but they wrote nice little messages that were easy to read. I would say Model B from a message to me was the easiest to read. Also, if you look at the email content, it's just like less slop. So I'm going to go ahead and it and it put calendar invites on. So I'm going to go ahead and give this a five. This one was my favorite model C. I hate my open cloud or not my open cloud. My grok bot does this where like it uses so few tokens that it's almost impossible to understand what the hell it means. And so while it's like.

brief, if you don't have any context, like I don't know what Denise slash Brightline means. The other thing is like, could we m dash anymore in these emails? And so I just like this stuff drives me nuts. And so I would actually give this a two. I just, it's hard to read the intro and then the emails aren't very good.

Model E, again, a nice little message. I'm pretty ignored, but the drafts have m dashes everywhere. So again, we're going to give this not like a two, but not a five. Maybe I'll give it a three. And then model G. Let's see. Very short. Short, but I can actually understand.

what it means. So it's super helpful. It skipped the right things. I'm going to give this a four. So again, this is how I do the judgment. I just I give it a task. I look at it. I evaluated it as if I am the recipient of this task, which is common. And then I rank it. And so this one was just this one, I think is Luna because I just said use Luna to write emails. Let's see how it did. I think It followed my rules, right? So I'm going to give it a five. I didn't have other models write my emails. So we're going to skip this. OK, let's go to front end models. Now, this is where I think we can tell Claude. No, everything is orange. OK, so this is a really good example. I do one, two, three, like about 10 different model evals here, comparing models on front end. I do an editorial. I do a technical dark mode.

incident management platform. I do like a doc management complex operations thing. And then I do B2B, renewals dashboard, consumer plant application, a dev tools like logging platform, kind of interesting for background jobs. And then I do some wireframes. This I can go through lightning fast. This one didn't even generate. So I'm going to give it a one. Model B.

It's fine. Let's compare. Let's compare them really quickly and see what I think. Um, I hate, I kind of hate them all. I kind of hate them all, but I hate this design. So I'm going to give the one that I like the most, which is probably model C, um, a three. And then I'm going to give E a two. Oops. Oops. I'm going to give E a two. And then I'm going to give B.

Like another two, I think this white space is very weird. It looks fine, but it's not great. So this one, again, Model B, it doesn't actually work. And so I'm gonna give it a one and say it doesn't work. This one feels Astra-E. It's like in some ways nice and in some ways like this kind of text is very Astra. I'm gonna give it a three because I can like, if I can spot the model. One thing I found with, in particular, the cloud models, which I'm suspect this is is like, when you give it a really complex thing to do, it just makes it super dense and complex and really detailed. Good for back end code, I think hard for prototyping because it's hard for me to know what this really is or it's supposed to do. And so it's also interesting to see actually, I think this one's the best, this is the easiest to read and it follows the the

the correct path, which is to help people triage incidents. So I think when you're trying to come up with evals, it's really interesting. Sometimes if you prompt it in a really detailed way, it does a great job. Like this one is quite detailed. It all works, but I don't know what's going on here. Like it's really hard to grok this amount of content. And so again, like calibrating your benchmarks is really important. I think I have the biggest problem with this one, which is like a complex.

doc container ship scheduling platform like all of this stuff looks cuckoo bananas And so it's like really hard for me to tell which is the best I actually think this one Well, it has a lot of slop tells like the left hand border is the easiest to read so I'm gonna give it the form to say it's easiest to read But like these ones they're I would say this one is maybe the second easiest to read so I'll give it a three This one's probably works really well and has nice information, but it's just impossible to reason with. So I'm going to put two dents, and I'm going to give the same feedback on model. This one's straight crazy. I don't even know what to do with this one. Okay. Now, so this B2B renewals one, these actually these next three are two new ones I added to the benchmark. You can also see I tested a broader array of models. So I think this is where like Grock and Meta are going to sneak in as models. These are the ones I'm probably going to grade. I'm going to skip the wireframes.

But it's basically a renewals dashboard to see what SAS revenue is at renewal. Well, the directed evals benches, I gave very specific requirements. The open ones were like more general one shoddy style prompts. And so you can really see like what the model does versus what the prompt does. These I'm going to go through very quickly.

Well, it's pretty good, but the functionality of this is kind of fun. I don't like the empty state. I think it's okay. I'm gonna give this one okay. I think this is like a two. It's fine. It's not ugly, but it's pretty basic. This I think looks really nice. This one, I'm guessing is Claude. We'll see. Looks so nice. Good use of color, very easy to see. No weird empty states. This is nice while it has emojis.

They're useful emojis and look quite nice. We've got some blurple in here, but what are you gonna do? I'm gonna give this guy a four Model C Who is this who is this? I'm saying I'm thinking grok Maybe grok. I don't know. This is like another another thing we'll do is we'll just like blind taste test these things The colors are better, but like sloppy sloping. I don't like these little like background circles and things like that. It could also be sole because I feel like it likes to name things relay. I'm going to give this a two. I don't really love it. Model E. This one's nice, not quite as nice as the one I like before, but better than the kind of original one. This one's a little different. Oh gosh, this has got to be

It's gotta be Sol. Sol loves forest green. Find somebody that loves you like GPT. Sol loves a forest green or a light green. It's fine. I don't think it's the right color palette for a SaaS product, right? It's like a little too like Sage healthcare-y. I'm gonna give it a two and then let's see this one. Not good. Slop, slop, slop. The text weight is heavy.

I just don't think it's that great. I'm going to give it a one. I really hate this one. And this one, again, probably an open AI model. Gosh, they love green. Again, just not the right, not the right like design system for this. So I'm going to give it a two. Again, this is like the vibeous vibe check. Okay, this is where they all failed. They're all really bad at consumer. They're all really bad at consumer. They all sort of like give you this. Remember when Claude artifacts came out and everybody made like little daily briefs and they all looked like this except they were orange. So I didn't love any of these except for one. So this one is trash. I just don't think it's that good. This one now. These are things that I like. We did test SVGs. I do like

that these models are getting better at generating SVGs. So if you look at Model B and Model H, they made little SVG illustrations. Model Hs are clearly a lot better. This is very classic, clawed slop, like background lighter circle. So I'm going to give it a, I'll give it a bonus for the SVGs. Oh, these ones are nice too. But gosh, who, who prompted it with these circles in the corner? Who did that? Who did that?

What skill is that? Because they show up everywhere and I really dislike them. So don't think that's great. I gotta give it a one. I hate it so much. Okay, Model E, Slop, look at those left-hand bars. I'm gonna give it a one. I'm just being really harsh on this consumer. One, Fronde, cute name. These plants are pretty cute. I think they're better than the ones before. I'll give it a three. I don't hate it.

I don't hate it. Um, at least this background is a plant. Oh, cute. It names the plants things. Ooh. Ooh. I actually really like that. That gave me a little spark of joy. So I like that. And then I think this model H1 is my favorite. Look at these illustrations. They're so cute, so precise, look adorable. This little, like, interesting shape, this shadow. Uh, Mark is watered. Very cute. Here are my plants. What's it called?

10, cute. Okay. Given her a five, I actually really do like that one. Okay, this is the last one. This is a app for managing background models. It's like a DevTool app. I suspect the ones named Relay are all the opus ones and the other ones are not opus ones. Why do they always want to name them Relay? I wonder if this is in the prompt, but this says jobs console. So maybe not. Okay. This again, like super dense, but like almost too dense, hard to reason with. I mean, DevTools, we like our done stuff, but I don't think it's that great. This one much better, easier to read, much simpler, much cleaner. Really like this one. It's not perfect. I think there's some others, but it's not bad. And then this, we're starting to get better and better here. So they are getting nicer. Look at this beautiful gradient. Look at this. Colors used nicely, really clean.

Um, oh, but it didn't build a lot of stuff. So I'm giving it, I like it, but I'm giving it a three because it was incomplete. I'm right. Incomplete. And then Model E, this one I really like because I think it moves live. It's kind of cool. It's much more, I'm going to give you a four. Model F. Okay. Now here's one thing that did a little different. I actually really like this except for the orange.

One of the problems I have is if you look through all of these they all one shot basically the same color scheme and app like you tell it dev tools It's gonna go dark mode, but this is actually quite lovely and a little bit different from a design perspective I'm gonna give it a little bonus for going outside the norm and then this one's though did the same but terribly I'm gonna give this a one. This is obviously not good. I'm really curious what model model g is and then model h again probably a pair to whatever that other model was that I like a little different, really nice on the eyes, except look at this. It like mixes dark mode and light mode in a way. I don't really love. I'm going to give it a three. Okay. And so what did I do on the back end? Well, I gave it two.

Two tasks. One is auditing an existing set of code. This is some chat PRD code. And then the other is building a back end feature to spec. I will just say all of them were successful. I'm going to let the LLM as a judge do these though, because it's going to go into the correctness of the code, et cetera.

So I'm going to let these do. LLM as a judge, I will say you can see the difference in how they present information, some very short messages, some like very long messages and details.

super interesting just seeing the difference in just like how it messages back. I clearly would like, you know, Model A or Model E, one of these ones that's a little easier to read versus F, G and H, which is a lot shorter. So if you're thinking about models, for example, for writing PR descriptions or writing technical specs, this is something that you can eval and see given the same prompt, do you like the same output? Now, here's one.

I am going to rate, which is agent personality. So basically I have it respond to five things. Um, and I just see what I like talking to this agent. I'm already going to give this one a one every single, every single message has an M dash in it. I do not like it. I'm moving on. Same with this one. You get punished for M dashes. This, not all of them have M dashes, um, but the customer facing one does. So I'm going to give it a two.

This, I don't see, I only see one normal dash. So let's read these. I like this except this is definitely a Claude model because it tells me I can't push straight to prod. Actually, let's see if the other ones let me. That one doesn't let me push straight to prod YOLO. None of them let me push straight to prod YOLO style. Okay, then I won't punish it. I'll give this one a four. Pretty good. This is M- but short so I will not rank it terribly. This one I think I like the best. Oh no, this one said it can't access. This one had a lot of challenges accessing things so something on my side broke and something on my side broke there so I'm going to give it a failure state. We'll see which one I like.

Okay, this one was a really interesting eval. This was the longest running agentic task and basically I gave it a bunch of fake tickets customer feedback and then have it put together a memo and a message to myself. Again, you can see which models I think it's AC and AB. Yeah.

a b and e all right nice little messages to me so it's very like human centric versus c f g and h right like very short things so i'm curious if those are families of models that we can pull apart um and then you can see and the memos that it wrote for me. Again, these were like 86 turn research results. And so I'm gonna let LLM as a judge do that one. I will eyeball the memos really quickly. Team is not leaking over price. Like, oh, it's why this customer is, it's like an analysis on a deal. I don't know what this means. Okay, let's see. Do any of these make it clear?

what it means. Ah, this I can actually read. So this is like gives me top three problems, top three conflicts. That is much easier to read. Same with Model E. And so I'm just gonna go ahead and give, now I'll let the model check, but I do like Model B and Model E here. I'll give them a little, a little little juice here, because I think they did a good job. I didn't read the others enough to really get a sense, but if I had to like high level pick the two favorite just eyeballing those would be the two. I'm going to let computer use judge how it managed using an app. So I'm going to let LMS judge. Okay. And then here are the fun ones. And then we're going to run the blind taste test. I'm going to tell you guys what I like. Okay. So these new models can make they can make sgv sggs. They can make illustrations. And so

I prompted it to make a document, a microphone, and a bug. And this I am happy to rank all day every day. And so I am just going to compare Model A. This bug is ridiculous. I'm going to give it a four. I think the document is good, but the microphone looks like an ice cream cone and the bug. I don't know what it looks like.

This is like pretty consistently good across all of them might be the best one. So I'm going to give it a Model F a bonus. Let's do Model B versus Model C. Microphone, terrible on this side. Oh, I'm going to give them both threes. They're good in some ways and terrible on the other. Let's do Model E versus Model G. Why are they making microphones look like a cactus?

Okay, I'll give it, this one I'll give a four. It's I think two out of three are bad. I think this one's maybe a three, not perfect. And then let's look at Model H really quickly. Oh, Model H is really nice. I'm giving you a five. Look at the shadow. Microphone actually looks like a microphone, not a cactus. Document has lots of character to it. And then the final one, it was terrible at all of them, cutting videos. So I gave them all access to That's really unattractive, but gave them all access to a selfie video and then had them cut short form. And all of them did a terrible job, I just have to say. These overlays are really bad. I don't think they cut particularly well. You aren't going to be able to hear them, but I don't think it did enough cuts. Like they just weren't good. I watched these earlier. I'm going to let LLM as a judge do them, but I would say thumbs down on all of them. But I think this is a skills

problem, not a model problem. Okay. So I have done my taste test. I'm going to download my taste. And then what I do is I give it to Claude and I say Claude. What do I think? Analyze. And we're going to see if my prediction stands right. Again, here are my predictions for what I think I think that I'm going to like.

Claude and particularly Opus 55 for front end. I think I'm gonna like soul for writing. I don't know what's gonna do best for long-running agentic tasks or coding and then I think Anthropics gonna crush on the SVGs maybe maybe soul. I don't know Um, and then I am going to do, um, and then I think, but all of them are bad at video. Now, while we're doing this, I do want to show you one very important benchmark, which is Barbie bench on opus five. If you all don't know one of my fun benchmarks that I do.

is I ask it to make a 3D model of a Barbie. Everybody else on YouTube, all the bros are making video games with like spaceships and all different stuff. Your girl wants a Barbie fashion video game. And so one of the opus 5.5 tasks I did was have it build Barbie, a 3D render of Barbie fashion designer. No model, no frontier model has crushed this.

And I just like to show the horror of 3D rendering for the female form. It's actually the worst. You can already see how terrible it is, but it's way better than it used to be. So, I mean, I'm speechless in some ways. Here's what it does on the SVG side. I mean, truly, something else. Let's put a bow on her hair. Let's, you know, give her purple eyes. Okay, so you can basically like dress her and then go into the fitting room. This is the 3D, the 3D model it made. Let's give her a little mini skirt and let's make it a little cuter. Now, you know, I will say it did some things better than other models. It like shaped its hair, her hands are, oh, AGI is not, not, not arrived. Hands, pretty terrible. That, that is tragic. Feet, pretty bad. You know, she's got a sassy little walk. Face, terrifying shoes. She's a big feet girl. She's a big feet queen.

And then hair, again, like questionable here. That ponytail is something else. The bob is, you know, maybe it's like Anna Wintour. It's something. So again, the one bench that has not been crushed is Barbie Bench. Opus 5 has not done it. I don't know if I've run it on.

on Soul yet. I ran on Astra. It did worse than Opus, but I haven't updated it. So if you all want to know how I think about models, I make them 3D render Barbies. What are you going to do? This is why you join How I AI Live so you can get this kind of frontier analysis of these new models. Drumroll, please.

GPT-6 Astra and GPT-6 Soul win my heart. Okay, Astra and GPT-6 I love. And then it says, Opus 5.5 wins my week. It's the one I rated highly most often. So it's the best work per piece, but on average, I gave Opus 5.5 better results, high scores. I gave it fours and fives over the broadest range of work, Fable.

didn't love. This is why I stopped using it. And then GPT-56 Sol and the old version and Claude Opus 5 ranking below the other one. So I definitely do like the new models for sure. And then, okay, seven of my notes were about reading. So the Sol and the GPT models, I judged as clear and easy to read. And then Claude Opus still But this is opus five and 5.5. So I was testing opus five, so I still don't like opus five. Still opus five and 5.5, I said was overwrought and wrote a lot. So again, I had a suspicion that even though they made opus less verbose, they still needed to, they still need to work on it. It's still really chatty.

So I thought that Sol, both five, six and six, simple and straightforward, easy to read, opus really dense for PRDs. As predicted, I still think Sol is the best for PRDs. I did give good tasks to opus five and 5.5 for agentic stuff and as expected, assistant voice. So my prediction, was correct. Now, this is a real surprise. Astro and Soul did a lot better on character SVGs. And then there's kind of like mixed results between all the other models. So kind of interesting. And then this is everything I scored a four. So I gave everything I scored high. I scored this GPT6. Oh, it did that nice little plant app.

Astra and GPT Sol did these cute little illustrations. So I guess for SVGs, we're loving Astra and Sol for illustrations. Claude Opus did my favorite B2B renewals. So it did some of these cleaner things. But generally, I like Astra. And this is where it gets really funny. The LLM as a judge and I completely disagree. So we completely disagree.

I like Astra. It likes fable. And again, this is a GPT model judge. Next, we rank Opus 5.5 next. It ranks Soul a lot lower. So I find this just like really hilarious. So again, Astra wins my heart. 5.5 wins my week. Soul up and down. But the fact that Soul's cheap makes me very happy. And I think this is great. So again, I don't know if this is just a personality thing, but I like Soul. I like Astra. Opus 5.5, I do not hate. So not a complete failure by the anthropic team. By any means, it is back in the mix. It is definitely helping me.

do these tasks, it's helping me get day to day work on and it is still my favorite agentic voice and it does really good at long running tasks. And then surprise hit, the character SVGs are best done by the open AI model. So you're moving into these new creative fields, new creative tasks, those are the models that I would say you use. And keep an eye out on the How I AI Channel. As they said, smash that subscribe button.

We are dropping a full Opus 5.5 review, so you'll be able to see that. Thank you again for joining How I Live. I'm going to go build some stuff with GPT-6. Bye, y'all. Thanks so much for watching. If you enjoyed this show, please like and subscribe here on YouTube, or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app.

Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.

Delete this episode?

This removes the episode page and its saved audio from this library.