The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) - Canva Is Tackling AI Design’s Most Frustrating Problem with Stefano Corazza- #784

0:00As good as modern image generation models have become, designing with them can still feel like playing games. Put the right magic words in the prompt and you might win at the slot machine. Try to iterate on the result and you're playing whack-a-mole. That's fine for memes, but when design is your job, the game gets old. Canva's AI research team wants to make that process more predictable. With greater control over layouts, easier revisions, and models fast enough to keep up with the creative process. To get there, they've trained their own family of models using Canva's design data and integrated those models right into their tools. Here's Stefano Caraza, head of AI Research at Canva on what his team is trying to change. You have to build around iteration. AI will get 70%, 80% there. I will never get to 100%. And so you have to factor in that for the last mile, you need a human in the loop and be able to iterate in a way that is

0:54Fast and also fun. I'm Sam Charington and this is the Twimble AI podcast. For over a decade, I've been exploring the ideas and innovations shaping the future of AI through conversations like this one that help you understand what's real, what's next and what matters. Let's jump in. This has been a pretty busy week for you and your team. Well, first of all, the team was announced. Was this like the debut of AI research at Canva?

1:31Yeah, that was the view last week, right? I think 12 hours before we actually announced CanvaCraft, the model, and we also announced Canva Research. We had a blog post, a website with a lot of information about who we are and what we do. So it was really fun. And then this month has been the busiest month probably of my life, where every week we are announcing a new model. And so exactly 40 minutes ago, we had this announcement about...

1:59Canva brush. So very exciting. This visual language model for design, creation, and editing. Excellent. And so have there been more models than just those two over the course of the month? Yeah. So we have a total of 100 AI models in production roughly right now. And some are smaller and may not be a huge, large language model. Anything like that.

2:26We started injecting and fusing AI in all parts of the product and to really help this design workflow in every possible way. But now we are taking bigger steps with the Craft Day foundational model for image generation. And then with Brush, we are completely revolutionizing the way people can create content. And also, we are revolutionizing how fast and affordable that is because those two parameters Like, you know, pioneers using AI are okay waiting a minute for the magic moment where something magical gets generated. But like, if you want to really be using the day to day, like that's not an option. You know, in game development, there's a rule anything that takes more than 10 seconds is completely out of the question. And we wanted to get to like instantaneous AI. So yeah, I mean, particularly with the

3:23way that you need to iterate when you're creating like those delays really stack up. Yeah, absolutely. And people, I think there's a little bit of Silicon Valley bias that I've seen myself being part of that bias that you think like the new tech comes and it's going to be single shot, perfect. And then people will just adopt him. But the reality is never the case, especially if the task has some creativity in it. You know, it's not just a purely technical task.

3:53And so you have to build around iteration AI will get 70% 80% there will never get to 100%. And so you have to factor in that for the last mile, you need a human in the loop and be able to iterate in a way that is fast and also fun. So that's a key part I think of what we do until the moment when we will have telepathic, you know, helmets that you can wear and you can transfer the full amount of your intent.

4:21and your vision, but that is a little bit far out, I think. We're going to dig into the models and what you've created there. Talk a little bit about how, but I'm trying to decide how interesting it is that you even did this. On the one hand, there are lots of, their frontier labs have models and they A big part of their businesses are allowing companies like Canva to use those models as part of broader processes. There are open weights models that you can build on. And so there's a lot that you could have, you Canva could have chosen to build on. On the other hand, it's like, maybe it's obvious that a Canva with all of the data and user interactions that you have access to would build a model. Do you think it's interesting or do you think it's obvious that you did it?

5:16No, I think it's very interesting because I think it also applies to other companies maybe of similar size when basically there's two factors. So first of all, you have unique data that nobody else has. And then the second part is deep vertical integration of AI models in your stack is going to provide incredible performance and cost reduction. Those two factors are basically the driving input. And I'm sure there's a lot of companies that are like in the you know, thousands of employees or even smaller that will do that and where it makes sense. And so for us, it was kind of the discovery of the fact that you can have, you can fine tune or train a model that is two orders of magnitude fewer parameters if you have the data and you can beat the bigger model and then because it's smaller, you're going to be like another model faster and cheaper, right? So that is the belief that we had.

6:15And right now, we are in the 40x cheaper than the most expensive frontier models. They can do the same design task, but they are using others among two more parameters to do it. Instead, we have smaller models that we can fine tune, we can do the first learning with unique data that we have. And we can basically squeeze a lot more, same level of quality.

6:40sometimes better, but most importantly, incredible speed and cost reduction. That's one part. The second part is that the product itself with AI is evolving dramatically. The way AI is helping humans generate design is really changing. There has been a lot of siloed innovation. We're like, okay, I have an image model. The image model creates an image.

7:07Then I have a next step. An LLM looks at it and thinks, oh, we need to change something. Goes back to the image model and says, hey, can you make something different? And like in that, back and forth, you lose control and you're also throwing away information. And so we have with craft, we have created like an image model that most of the work actually is creating a scene graph of what the design should look like. And then the conversion to pixel is just the last step that we train from scratch.

7:36But at the layout, let's call it the design layout stage, the LLM can already intervene and make changes before the pixels are generated. And so we are seeing that there's incredible product differentiation by integrating the LLM world and the diffusion model world, if you want, in a more cohesive way where the iterations can happen earlier.

8:02And you have all non-destructive modifications with humans in the loop, instead of having to bake the result, show it, iterate, and then to basically throw away in that process a lot of information every single time. So those were, I think, the two main factors that drove our decisions. And also, if you look, the top models are no open source or open weights on the image generation space. I think video is a little bit of a different ecosystem.

8:32it's quite, this will be like probably an episode on its own, how the ecosystem for like video model and image generation model is completely different sometimes upside down, it's kind of fascinating. But in what sense without making it a full episode, in what sense? Something happened like about I think four months ago where like some of the uh uh companies developing uh video uh video models started like uh releasing open weights versions and that had a fantastic domino effect where like it forced all the competitors to do it and so now you have an ecosystem where a lot of the image models are are not open weights but the video models are and then if because of that i think also because of that you see the cost per uh um second

9:23of AI video generated, like really plummeting, because there's a whole community that is trying to figure out how to crack that problem. And because there's a lot of models that are open weights, more people can apply to that problem. And so it's a fascinating different dynamics on the two ecosystems, if you want. With Kraft, did you start from an open weights model and train it, tune it?

9:50Or is it a from scratch train so for craft we started is like train from scratch Be because we realized you know, then there were some several breakthroughs that we believe we're gonna unlock a much more powerful Model and those were not just they are in the existing model. So that when we train from scratch for canvas brush, which is our vision language model We started from an open source and open weights model and then we built on top of that. So you're using whichever of those makes the most sense given the time or given the particular use case. And it sounds like with Kraft, I have covered this idea of a kind of two stage generation with layout and then rendering. It was in a conversation with Qualcomm who has some papers in that area.

10:44I don't believe, to my knowledge, there are no popular open weights versions of that. Are the two parts of a model like that, are they trained independently or end-to-end? Or what's the system approach to training a model like that? Yeah, that's a great question. The answer is both. We train them individually, but then there's always a time where we're trying to look at the end-to-end.

11:12But I would say that the steps are actually three now. First, we take all the input and the prompt over the context, maybe, and we try to amplify and make it as powerful as possible. Then there's a process of taking that and converting into basically a structured form of data, more like a scene graph, if you want. Then the last step is generation.

11:40of the pixels where our modeling from scratch has shown really, really good performance. By breaking down into these steps, you're also allowing for a better integration with agents, with LLMs, because there could be some components of the pipeline that could be useful for some tasks. If I say, hey, can you just make me 10 different layouts of this single page? I don't need.

12:08the land to generate pixels. It can use the craft design planning, can create different plans, can have a human in the loop to confirm and then can do all the generations, for example. We are seeing new workflows coming out from breaking down into these steps. It seems like that would also help with editability. Historically, You have a scene and you say, oh, make the person a little bit bigger. And the thing you get back is totally different. We're a lot better now. The GPT 2.5 image models are really, really good at consistency between revisions. But it does strike me that if you could change the scene graph and move the thing over or make the thing bigger and not have to re-render, that

13:01that would help with consistency as you're trying to move things around. Like 100%. We believe on this so strongly that we release magic layers at the beginning of the year, which is basically taking a flat image, decomposing in layers and elements, so that people can really have that fine control. I mean, how many times have you tried to say to any image model generation, oh, can you move that thing a little bit to the right?

13:30Two feet, but in that image, there is no information to know what is two feet or two inches or centimeters. And so it's a lot easier for a designer to just move that element to the right and then regenerate. And so our belief is with craft, basically, the humans and the designers can edit elements, can make them bigger, smaller, move them to the left or the right, and then regenerate the pixels later.

13:59And we also are extending that capability to images that are generated outside Canva using magic layers. So actually the majority of images that we process with magic layers come from outside Canva. And people generate them somewhere else and then they come into Canva and they want to edit. And so there we can decompose into layers and elements and people can start moving things around and have that fine level of control. I mean people want consistency.

14:28and control, right? Those are the two things that any creative has asked AI for the last, I would say two years and we came quite far along, but like it's still, you know, we don't want to provide slot machines, you know, we're like, hey, if you generate 20 times, you might get lucky, right? We want to actually make sure the time investment that people are putting into it is really controlled and the result is predictable.

14:54You kind of referenced earlier, two dimensions that were important, quality and efficiency. Talk a little bit about each of those. We can start with quality. The model performs well, like on design arena is one of the number four now. Is that right? Yeah. What you brought to bear to create a high quality model? What went into that result? I think we underestimated the day.

15:23the power of like a decade of design generated and design data that we had. I think we were even ourselves a little bit surprised to rank that high because it was also the first submission that we did. So we have, of course, internal evals and we do a lot of also human evaluation internally, but you never know until you actually get into the real world arena. But yeah, it was basically A lot of work over the last year really collecting data that makes sense for designers and really harvesting data that the camera has been creating for over a decade now, including also data from users with their permission or data that we license from the outside. And I think we put together like a pretty good body of a pretty good training set and I think deliver a lot of value and also

16:21We had like several breakthroughs about how to use the data and how to train the models. So I think that was another, you know, the ability to run, you know, hundreds of experiments and really figure out what worked and what provided the quality. I think that was a great, you know, several breakthroughs, one on top of the other that I think made the difference. Can you give us some examples of the kinds of things you learned that really moved the needle for you? Without getting too much into the weeds.

16:50We underestimated how detailed of a scene description you can give to a diffusion model. Actually, if the diffusion model is above some number of parameters, can handle like incredible detail of the scene. So you can, the size of our scene description really grew in the last six months significantly as we realized, oh, actually you can be a lot more accurate, a lot more precise.

17:20And so we expanded basically not only the initial prompt, of course, but the description of the scene is now to a level of detail that I think no human would want to actually go and look through and edit, unless it's visually editing. And so there's so much detail. And by increasing a little bit the size of our diffusion model we trained from scratch, we were actually able to capture all that detail, all those nuances.

17:49and really squeeze the maximum quality out of it. And is that a unique property of your particular diffusion model that you trained or is that just a general observation about diffusion models? I think it can be generalized for more cases. In our case, I think because we understand so well the coupling between the generation of the scene and the diffusion model, we really work to make sure we didn't over-generate on the scene description or under-train on the diffusion model. The two are personally calibrated to be the input and the ideal input and output model of each other. Are the scene descriptions only unstructured text or they're also some degree of structure, coordinates, bounding boxes, sizes, things like that?

18:46they are absolutely structured. It's a fully structured, it's still text-based, but it's fully structured with all the details that you just said, bounding box, relative position, also position like all that. We have found that the more detail we can provide to a composition really the better and we have the ability to basically render all that detail in the distribution model. Also one thing that I want to mention is which is great for the creative industry overall, I think. There is this race to build the most capable LLM out there, right? And now we are already like in the, you know, tens or hundreds of trillions of parameters. But for creative tasks, I think we have found that, let's say, you know, 100 billion, it's already a pretty large number that can, if properly tuned and post-trained and

19:44you know, and fed the right data, a model of that size can already achieve so much. And we are seeing also a lot of other models, also video models that leverage LLM as an input to create that structured data, like the minimax H3 is another model that has a similar approach, even though it's a video model. But we are seeing the size of the basically vision language model or LLM is the input.

20:13not growing 10x every three months anymore, but being kind of stable. In an industry where everything grows, 10x every month, that number, I've shown that you don't need more than 100 billion per meter to do most of the creative tasks, I would say. Probably video understanding maybe needs just a little bit more because it's probably the most complex type of task, but for everything else, You know below 100 billion you can parameters you can already achieve Wonders and so to be clear on what you're describing there in this two-stage model where you have an LLM like thing or an LLM period Generating a scene description. You're saying this kind of diminishing returns beyond a hundred billion and

21:06Smaller models do just as well as larger models and kind of nailing that scene description and providing the right input to the diffusion model. That's correct. And while being, you know, a lot faster and a lot cheaper, right? And so, just the economics, they don't work if you use like the latest frontier model to do that task, then, you know, it will be like a dollar per image generated or something like that. And so, and so this, this I think, Together with the ability of people to train those smaller models in very efficient ways, I think really unlocked a lot of value and a lot of usefulness of AI in the creative workflows. To what degree did you use synthetic data in the training process? Is that a big part of your recipe? Yeah, it is. I think like in every data set of this size, we also want to make it as diverse as possible.

22:01all kinds of data coming in from stock images and that kind of stuff to synthetic data, to user-generated data, to designs that we created. We have millions of designs that Canva created or licensed over the last decade, and so that was also, of course, a big part of it. It kind of helps also with the taste, because we have seen AI slope and AI generation where everything looks very similar.

22:28And so I think we put so much effort on diversity, which also aligns with our company value. And so we are a company that wants to empower everybody to design. We are global. We serve every market, every language. And so we wanted to really push the diversity to make sure that the output also of the model is as diverse as possible.

22:54Talk a little bit about the way you think about benchmarks for image generation. You know, there are well-established benchmarks, but I always wonder, you know, how well they capture kind of the human perspective and human judgment and taste as you just referenced. And I'm wondering, you know, someone who works on this constantly, like, how do you think about, you know, the applicability of the popular benchmarks, how you supplement them, that kind of thing? That's a great question.

23:21I would say the benchmarks overall are lagging a little bit behind to where the power of AI in design is going to be, because they're mostly focused on single shot, the first generation. You have benchmarks for image editing, which is great, but design editing is different than just image editing. As of today, there is no established benchmark.

23:50for how good a model is at editing a design, for example. There's some public benchmarks for generating, let's say a presentation like this, a present bench, being one has like 200 examples of presentation related slides, related tasks. You can test it, but again, it's new generation. There's not a lot of benchmarks that really can assess the effectiveness of editing a design.

24:18the speed and the cost also to go with that. So I hope there's going to be more. Of course, we have internally, we have a lot of those, but yeah, we haven't found an industry-established benchmark for the ability to actually edit a design because the industry has focused on, oh, AI is so magic. It's going to single-shot everything, which we know is not the case. And so I think now the understanding is finally landing.

24:44I hope we're gonna see more benchmarks that really start from a given set of pre-existing designs and defines rubrics for how those designs can be modified and then does the comparison of the benchmarking on that. That kind of implies that you can use an LLM or VLM as a judge to.

25:08look at an image and assess it visually from any number of perspectives. Are we there yet? Or how good are VLMs at articulating the qualities that you might want to optimize an image for? I think they're pretty amazing. And we have also to keep in mind that if you structure the rubrics really well, they are really good at defining, I don't know, design readable. What's the score of readability? What's the score of layout and composition? We have very well articulated rubrics about design quality, and then we can use VLMs to judge that. We get quite far along. Of course, we always do also human evaluation, and some of the arenas are strongly leveraging that because it's the ultimate test.

26:04A long way, it forced us also to really define what a good design means. And so if humans are now able to articulate that, then VLMs are not going to be able to score it. And so I think we had to really go back and really understand what makes a design really good to be able to leverage VLMs to do it. But I think now we're getting to a place where it's It's quite reliable. When you think about this two-model approach in craft, what of the qualities of an image come from having the two separate systems or models and what just come from having better training data? Yeah, that's an interesting question. We have seen just different behavior depending on different tasks, for example, which we are always trying to understand.

26:58Is the limitation the diffusion model or is it the, you know, the length side of things? And sometimes it's hard to know. We have seen, for example, the model performed really well on like portraits and people and also nature landscapes. We have seen the model doing not so great on the cartoon type of content, for example. Is it not so great?

27:26Yeah, I mean it's not ranking in the top three for that and so that's an area that so that like all that is together that is counterintuitive to me because. You know a portrait for example seems like it would kind of know the value of like the planning and the scene component like it's a portrait you know and a cartoon for example like you know you got all these things in a cartoon you know that that would.

27:55lean a lot on a scene generating component. But the two results that you're seeing suggest that that model shouldn't be as strong as it is or that that's not the source of the strength of the model, right? Am I thinking about that correctly? Yeah, it's interesting. So the number one thing for people in portraits that you want is basically to be as photorealistic as possible, right? Like you don't want to have that AI skin effect and all that stuff.

28:24And we put so much effort in this. Because we were like, we got an error, it should not look like AI generated. I think that effort may have bled into the cartoon side where instead you want very stylized. And so it turns out that the images, the cartoon images generated with craft initially were a little bit too realistic and too 3D looking. But again, this is something that we...

28:52we discover when we put it on the arena because probably they had a different population of designers that would test it. But at the same time, it's something that is more about the taste of the model. And that is a lot simpler, a lot easier, a lot faster to tune and correct. And so that we are in the process of basically doing exactly that. And tuning the taste of the model and the style of the model is more about fine tuning and RL type of a task, you don't have to pre-train. In the pre-training, we of course had a lot of cartoon content. Just now that capability didn't surface to the full extent. Now we are teaching the model, okay, this is a style that resonates more with humans. If this cartoon needs to be flat shaded and these lines and so on.

29:47I mean, it's fascinating, honestly. It's a journey of learning as much as the model is learning. We are learning how to train the model and how the model behaves. Sometimes you find this really weird, but sometimes I have two small kids, and sometimes I learn things about learning as a high-level concept.

30:12that i learned by training the models i'm like oh this i'm sure applies also to humans you know and so it is kind of a fascinating especially on pre-training if you see pre-training is basically like teaching a human as many words as possible of a vocabulary right but has nothing to do with the style of talking performance or like if that human becomes i don't know stand up comedian how many words it knows has nothing to do with the style of the delivery. And so I think the concept is so true for pre-training and post-training and AI model and is also true for humans. In the early years, you want as much variety and as much exposure to different type of activities, sensory information, experiences, languages and so on.

31:07And so the pilot actually is fascinating. I think I'm sure this books are rewritten about it. Let's talk a little bit about brush. Is brush, what's the best way to think about it? Is it an editing model? Is it an agent, a design agent, a little bit of both of those things? Yeah, we call it like a vision language model, but it's basically an agent that can create design and can also edit design. So it has a deep understanding.

31:35What's going on in the layout of your of your design and can really take very fine grain Directives from humans and then operate on the design itself. So what is the design in this context? It's not an image. It's think of it like a slide or a canva document multiple layered composition of things. Yeah, could be multiple layers could be multiple elements could also different type of elements I can have some vector graphics and some image mixed up together and then some text as font. So it's basically this heterogeneous composition of different elements. They have particular features and particular type of details that humans can tweak and also the agent can tweak. Then they all come together to create that unique design that is the final output. So it's a little bit like,

32:33Warcraft generates in the first step, right? It's basically all the underlying structure of a design. Brush really understands that very well. It can really act on a design and then relay out, can move parts, can make it more readable, more professional, can take high-level inputs like, hey, make this thing pop or like make it cool. I'm like a, you know, make it Gen Z, like this.

33:00you know, directives will be extremely hard for a designer that is not maybe a GMC designer to really understand, you know, so those are high level directives instead can really be applied by brush in a quite successful way. I'm kind of thinking about it like so with Kraft you've got this scene creator and you've got this generator and the generator, you know, generates one kind of one image and Brush you have the same scene layout thing but maybe it has different tools and that and one of those tools is generating images and it's got you know layout tools and the output is like a Composition of sorts is that a reasonable way to think about or where does that way of thinking about a breakdown from what's actually happening? Yeah, it's not that far like I would say

33:56Right now, Brush can do more general things than just operating a layout description of the scene. But basically, it operates a set of tools, like a large set of tools that are all design tools. It can make decisions on design. I think it goes beyond what the image generation and editing, right? It can also create like 30 slides deck, can make sure that deck is cohesive with branding.

34:25I think it operates at a higher level of abstraction than Kraft, which is really about image, and soon will be also doing more of image editing. Brush is basically a full agent that you can ask for more complex work and more abstract type of instructions. And then we are constantly iterating on... I mean, a lot is like having basic NLLM in the loop.

34:53Using a bunch of tools and be able to see what it generates and that that loop is really war has changed everything because until I don't know a year ago LLM's would just do like we're blind and we'll just do a single shot and then be blind to the result right and instead now Not only that single shot becomes an iterative process with human in the loop But LLM's now can see what they generating can iterate and the faster you make the iterative loop the more effective everything is because now an alarm can do maybe 10 iterations before even shows the result to a human. And before every iteration will take a minute and so that was not feasible. But now that's why speed is not just a nice to have. Speed is like a complete game changer because it can really increase the quality, it can increase the user experience. It can really make it like an iterative process. Sounds like there's a simpler.

35:52would think about a craft as a model and brushes it kind of an agent. And so in creating brush, like I'm imagining like part of the task here is like a custom harness. And I'm wondering what all that entailed. How did you approach that harness development? This is such a great question because I think people are like usually they are all focused on the on the model and not the harness. But I think the The importance of the harness cannot be overstated. For us, this probably was the hardest part and the part where we had the most iterations. In fact, we already had our third generation of harness for Brush. The high-level goal is basically to give as much freedom to the LLM to really run free of constraint.

36:48and really be able to tap into the tools and have as much insight on what is able to generate. So we have really focused on making the harness thinner and smaller and more agile and simple. Then at the end, a brass can write code to operate tools and make design operations. So we were at the same time trying to leverage how good these LLMs are writing code.

37:16And at the same time, simplifying the harness in the most basic possible way to really unleash the full power of the LLM. And so that's where we did a lot of iteration. And actually, it was more about removing the human knowledge and just letting the LLM run free to some extent. And when you talk about the code generation, what kind of code are we talking about?

37:42I can imagine a lot of what the brush might want to do is running a Python code, but I could also imagine you creating like some design DSL that's like more natively represents the things that you care about. Yeah, it can be that or it can be just writing code also to operate API calls and like layout operations.

38:04Yeah, and that kind of stuff. So is it primarily code or a little bit of both or like, it's definitely both. Yeah, like the capabilities are quite broad. What's an example of something that user would do in brush that uses, you know, either code or this kind of some kind of DSL or abstraction. I mean, a lot of the editing operation that you can do on on an editor, we they can be expressed as like editing to an editing API, right? And and the.

38:33Problem is there is there's like infinite ways of describing that design operation. Let's say I take an object, I move it to the left and I rotate and I make it bigger. These three operations can be encoded as a format change. You can have a format to describe the whole state or you can describe them as diffs.

38:58and they can be achieved through like an editing API. And so that is the complexity that we had to navigate, trying to understand what was more natural for an LLM, which actually has to do with what is the most available data out there that the LLMs are trained on, right? So this way, you kind of leverage a groove that has been already created by an enormous body of data instead of trying to teach something new or like there is underrepresented in the training set or is out of distribution in the training set of another lab. And so I think that's where, you know, you try to align with the wind, right? And the capabilities as opposed to trying to like find it. Earlier, I mentioned the kind of these two north stars you had quality and efficiency. And I think we double clicked on quality.

39:50but didn't really double click on efficiency. Recap your kind of vision for efficiency and talk through in the case of craft and brush, like what were the levers that you manipulated to drive that efficiency? Yeah, efficiency in inference specifically really enables affordability of AI.

40:15The majority of our users are actually free users of Canva, and we love that, like students, non-profits can use Canva for free. If we can bring the cost of AI inference low, we can also allow them to use more of AI. And so we really wanted to make sure AI was something that was within reach for everybody. And so the optimizations we have learned is we are overall relatively early.

40:42in the journey of optimizing AI inference. If you can squeeze 10x performance improvement in a month, means you're very early, right? We are so far from the plateau right now. And that's very exciting because it's also very satisfying for our AI engineers to be able to work on a problem for like two weeks and be like, oh, okay, it's five times faster than it was two weeks ago. And so we have put a lot of effort.

41:11And you can only make it really efficient if you own the full stack, right? Because there are optimizations you do at the GPU level, at the hardware level. There are optimizations you do at the kernel level. These optimizations are reducing the number of steps in the model. That's the distillation. And so going from basically high level to low level hardware.

41:39And how you survey how you batch it like are you cash it there's like a pile of things that you can really do and then if you tackle all of them you're going to squeeze like significant gains out of each each step and that's how we have seen you know for example the cost of generation of a second of 720p video in a little bit over a year go down two orders of magnitude right now you can generate some sense.

42:08where a year ago it was like half a dollar per second of generation. And so there's still a lot to do there. I'm not going to forecast that in France it's going to be as cheap as storage. But we have seen what happened with storage at the beginning. We were paying by the gigabytes, and it was so expensive, and then you had to do your math. And now I don't even know how many terabytes I have of storage on my Dropbox account.

42:37I don't know. I don't think I would ever have the problem to think about it. And so I don't think, you know, I first would get to that level, but it's getting so much better than it was, you know, a year ago. And I think given the first derivative is definitely still very healthy. I think we're going to see probably another order of magnitude going down. And when you get to like 0.1 cent, generate an image or generate a second video. I think it's another breakthrough and all of a sudden the democratization and affordability go beyond what we see today. In building that full stack and operating that full stack, are you operating the full stack? Do you have hardware in a data center or do you delegate part of that to the cloud but you're running everything on top of cloud providers, clusters? How far down do you go?

43:32Yeah, we are as far down as we can go other than owning the actual hardware ourselves So we yeah, we run those models on our cloud. We optimize them We manage, you know, how we optimize the inference how we serve it all that so the raw thing that you're getting from someone else's GPU instances and You're clustering them you're loading them with whatever your inference stack is your tuning, LLVM and all the bits and pieces, batch sizes and all the stuff you talked about. Yeah, exactly. Talk about how is your knowledge about that space evolve? Were there things that you found surprising once you started to work on these new models or did it kind of roll out as expected?

44:25Yeah, it's interesting because it was an industry that was completely dominated by AI researchers, if you want, where you had like graphics engineers or like cloud engineers coming in and being like, oh my god, there's so much more juice to squeeze here. And so it's been really like a phenomenal to see how, and here in San Francisco, I saw like a whole wave of startups actually focusing on uh, inference optimization. I mean, because there is so much money left on the table right now. And so there's been this like massive wave. I think I can probably name like a dozen of companies that do that because, you know, people that have a different background realize, okay, these models are really not run efficiently. We should rethink about how we feed all this data and these jobs into, into the GPU. So there's been like really like a flurry of interest.

45:22Because it's also a very simple value proposition to explain, you know, this thing costs you now like a billion dollar. And if I do this optimization, it's gonna cost you your one tenth of that, right? Everybody gets it. It's very easy to explain. And it's a very, very easy ROI to put in place. And we are seeing now, as AI adoption is still skyrocketing, we are seeing a lot of pressure on reduction of cost, right? And making it scalable. And so it's been like fascinating to see A lot of those startups and also in big companies bring in knowledge that is more about graphics. And graphics has been optimizing things for graphic cards for decades. And now that knowledge is brought to combine with cloud scalability knowledge and AI models. All those three components together are really delivering wonders. It's very, very exciting. And are there prerequisites to...

46:21being at the steep part of the optimization curve, meaning do you have to have a certain degree of scale so that you can manipulate batch sizes or do you have to build your own models such that you can take advantage of deep levels of control. Where do you think it makes sense to own that full stack? It was really like an interdisciplinary type of endeavor where we actually put together different teams that different skillset right and when we saw the combination of those come together that's where like the best games came. And yeah, absolutely like if you you have to host, you know, you have to self host your model and then you have to have access to the weights of the model. You know, if you have, you know, if you're using like an open weights model, that's, you know, that's good, but in some cases.

47:16you don't have access to that. So in that case, you build your own model. But when you're self-host and when you have access to the weights, there's a lot more that you can do and a lot more juice that you can squeeze out of those models. And again, we found that by bringing together a very interdisciplinary team with different backgrounds, we were able to really tackle the problem with the right, with a different perspective. And again, it goes back to diversity.

47:45In this case, it was a very diverse mindset from the eye researcher that we brought in and I think it really delivered. Going back to Kraft and that kind of two-part model, does having a unique model like that get in the way of some types of optimizations like distillation? No, distillation is something that we do pretty regularly on every model that we develop.

48:15And so for craft specifically, we have announced that there's going to be a quick follow on craft fast and craft ultra fast. And those are basically models where we are doing also distillation among many other optimizations. And again, by having all the model in house, we can also make sure that the distillation that we do on the LLM side and the one that we do on the diffusion model side are also tuned.

48:43to work optimally together. So again, it goes back to the concept of making sure that the input and output are really tuned for the best quality. And so we're going to have more distilled versions of Kraft coming out in the next few weeks. Because again, we want to make it even more affordable. And we want also to leave the choice to the user whether it is a developer using an API or a designer actually using it to the UI or to MCP. We launched it. I think now it's also available on MCP as of yesterday, Kraft. That's another place where if I'm building a prototype app, I don't want to run on the most expensive model, of course. The ultra-fast and fast version of the model I was to design.

49:36For that they may become, you know, kind of the default, but then when you go, when you want quality, then you switch to the higher speedality model. Sorry, and so the MCP is Canva MCP or Kraft MCP? Through the Canva MCP, you will be able to also access Kraft. Okay. Yeah, there's basically a bunch of like AI capabilities that we are exposing.

50:00uh two mcp and the craft is being added and then as we add more variants of craft we're gonna keep adding to mcp which by the way we are seeing like the usage of mcp on camera like a real skyrocket i think people are finding a lot of value so we want to give more capabilities uh you know access to more capabilities to mcp going forward do i understand correctly that it beyond the mcp access to craft.

50:30You are also making craft and some of the other models directly available to folks. I don't know the specifics, but I'm envisioning via an open router or via Canva directly, but just per use access to the model. Yeah, absolutely. So we released it through our API called Canva Connect.

50:52And so now it's available. You can access through the API as well as MCP. So for developers that want to do that. That feels like a very different business, like running a super low latency, very narrow model inference service as opposed to this is an API for our product. Is this the first model that you're hosting in this way?

51:17We have exposed APIs for various capabilities. I think also you can access magic layers and some of the image generation models that we had before, like Lucid Origin. But I think now the level of interest we are getting is a lot higher. And yes, you're right. It is a type of different business if you want, right? But we are getting so much demand coming from mostly like two categories. So enterprise customer, our enterprise.

51:45Business is the fastest growing business that we have in the company. Really triple digit growth year-to-year. We are getting a lot of demand also from those enterprise customers to access the capabilities to an API because they may build their own custom workflow. Now that people can buy the code, anything, there's going to be more demand for that.

52:11And then we are getting also a lot of demand from people using, I don't know, cloud or charge EPT through MCP. They now want to access more capabilities, right? So the agents can go there and say, oh, you want to generate an image? I can use Canva and then they can always then open in Canva for further editing. And so we are seeing these two type of use cases really growing. And sometimes those users are already using Canva and they just want to use it in a different way or they just want to build, you know, scalable.

52:40applications using some of the Camber technology. So we have seen a lot of demand and yes, it's a new and exciting business and we're still learning a lot about how to best serve it. But I think this we can say is not unique to Camber, it's across the industry, right? There's a lot of companies that all of a sudden are like, oh, this is like, we are basically building not only for humans, but also for agents, right? Even though, you know, humans is our focus even for for features, but it's gonna be in the future just more agents. Using some of those capabilities, I think that's unavoidable. What do you think those agents need that's different from what humans need? I think they need more structure. They need documentation in a certain, define a certain way so they can digest more things. They need the ability to try to preview things.

53:35without having to subscribe. And so a lot of things that basically are designed to remove friction, finding information, more easily streamline workflows, because ultimately many times it will be the agent that will decide what solution to use as opposed to the human. And so we have to basically make sure, for example, the agent is able to like spin up, we love friction and not a lot of burden on like initial credentials or credit or purchase.

54:03And it can spin up something that the agent can test, verify it's working, and then go back to the human and be like, hey, I created this prototype app using this API. I tested it, it works. You are not giving it a shot. Now it's approved. Now we can go for the real scalable thing. So that's the kind of approach. Just to kind of wrap things up, where do you see Canva evolving with in terms of research in particular? It's a world where You know, typically you plan like a year ahead and now it's completely impossible. It's like, it's like just a month ahead is already like a pretty, you know, pretty foggy kind of thing. But at the high level, you know, we have a very clear mission and vision. That one is not changing. So we want to make AI more accessible. We want to make it more part of the iterative creative process.

55:03and we want to remove any barrier, and we want to reimagine all those workforce. So that's the direction that we've been pursuing. And what we're actually doing, what models we work on specifically at Canva, this is going to change every few months. And so I told my team basically, don't fight this ever-changing, constantly-changing ecosystem, but just embrace it. And so I think It's really like a mind shift of like everything is going to change, which means also like, what is important today may not be important tomorrow, which opens up also for opportunities, right? It's a double-edged sword to some extent, but at the same time we see that it's a big opportunity. And so we are constantly analyzing our business, see where there are opportunities, where there's bottlenecks, where there are

56:00technical part of a workflow that we can automate and then we prioritize those and when we work on them. A year ago, none of the models that we ship now was underway. It's logic to think that in one year, we will probably release models that we haven't even started working on today. It could be terrifying or it could be very exciting. I think researchers are probably the most exposed to the speed at which AI is changing and so it takes really some major adaptation and mindset shift, but at the same time it's quite exciting. Well, Seth, thanks so much for taking the time to share a bit about what you and your team are up to. It's very cool to hear more about it. Thank you, Sam. It was a true pleasure and an honor to be on your podcast. Thank you so much. Thank you.