The_TWIML_AI_Podcast_formerly_This_Week_in_Machine_Learning_&_Artificial (4)

0:01I keep hearing the same frustration from AI builders. It's fairly easy to create impressive demos in controlled settings, but super challenging to build a system robust enough to handle real-world inputs. So what's the secret to AI apps that consistently delight users and deliver real value? A solid evals program. Great news. Podcast favorites Hamel Hussein and Shreya Shankar have created a four-week course starting May 19th on consistently improving AI applications with evals.

0:29With experience from GitHub, Airbnb, and Google, they'll show you how to evaluate subjective outputs, incorporate synthetic test data, automate evaluation workflows, and develop those magical data flywheels that keep your AI improving over time. Plus $1,000 in modal credits is included. Even better, Twiml listeners get $100 off. Visit twimlai.com slash go slash evals and use the code SAMTWIML when enrolling.

1:01unique properties of the problems we're solving. For example, our stack relies on really powerful special awareness of everything going on around you, but they need to understand the world in 3D even better. Another thing you want to incorporate is very long memory over a scene. You need to reason often based on the history of several seconds or more. That can be a lot of frames. You need to think how to prevent hallucinations. So all of this, you know, are still areas that are very much ripe for good work, even though we have decision language models. All right, everyone, welcome to another episode of the Tuomo AI Podcast. I am, of course, your host, Sam Charrington. Today, I'm joined by Dragomir Angulov. Drago is VP and Head of Research at Waymo. Before we get going, be sure to take a moment to hit that subscribe button wherever you're listening to today's show. Drago, welcome back to the podcast. Thank you. Pleasure to be back.

2:08I'm really looking forward to digging into our conversation. I went back and checked. It's been four years and a month since we last spoke. It was February 2021, and I have got to imagine a ton has been happening on your end. So I'm really looking forward to catching up on all that. We'll be especially digging into some of the work that you've been doing to incorporate foundation models and that whole idea into...

2:36the self-driving experience at Waymo. But before we get going, I'd love to have you jump in and share a little bit about your background for folks who weren't listening four years ago. So, well, a lot has happened in the last four years. I'm Drago. I've been at Waymo for almost seven years now. I have been leading the Waymo research team since summer of 2018. And we focus on pushing the state of the art in autonomous driving systems with machine learning or AI or both. And I have been a machine learning researcher for over 20 years. I spent eight years at Alphabet working on image understanding with deep neural networks. We published some of the early architectures for image

3:33classification or object detection, and we won the ImageNet challenges that was 11 years ago, 2014. And I used to work on Street View as well on 3D pose estimation and 3D vision for Street View. As the vehicles used to drive around, we would reconstruct the world in 3D, align all the different trajectories of all the platforms. We even not just have cars, we had snowmobiles, strikes, bicycles.

4:01Sometimes we would put a trike on the boat. We've done all kinds of things at the time. And so you had to have accurate posts so that when you navigate for all these photos and the pancake in 3D needs to look good. Essentially, I led a small team that was enabling this for a few years. And then around 2015, I got into autonomous driving.

4:26Initially as head of 3D perception at Zooks, another reasonably prominent autonomous driving company. And I did this for two and a half years and then I had the opportunity to start and form this way more research team that have been leading and evolving together with the team since then. And I think soon it will be seven years.

4:55That's amazing. That's amazing. I think when we spoke, you guys were early days in the Phoenix market trying to get some on the ground test experience for the vehicles. Talk a little bit about how far you've come since then. So I think 21, we had the Waymo 1 service in Phoenix. I think it probably was still with the older Pacific vehicles.

5:21And since then, a lot has happened. Just recently, maybe two weeks ago, we announced that we are now giving over 200,000 trips to customers in four major markets and 200,000 trips a week. So this spans... Fully autonomous. Fully autonomous to paying customers.

5:48Right at reasonable scale at this point it is a significant part of people's lives in the cities in an option that they can clearly Use anytime they would like you can get the up You can just hail away more in San Francisco Phoenix and LA directly And now in Austin, in a partnership with Uber, they need to go to the Uber app to hail a vehicle, and often, or reasonably often, it would be Waymo. But in these four cities, we operate at a reasonable scale, in large territories too. So San Francisco, we have the full city and even a little bit of, I believe, Daily City, so around 55 square miles in Phoenix. That's our oldest area where we provided the service.

6:43and is the largest area, I think, generally in the Western world. I'm not sure in China how it goes, but Phoenix is over 300 square miles, and there we even provide rides to the airport directly. And then we have the other two cities, they're around, again, LA is maybe 90 square miles, and Austin, where we're relatively recently around.

7:0937 I believe when we were building it. We were not sure how enthusiastically it would be received and how much people would love being in an autonomous vehicle I can tell you that at least for myself and I'm a biased user of course because I contributed a lot of well systems of features to That you know either part of the vehicle or help support the vehicle I think it's an amazing experience. I think a lot of people also really appreciate the safe driving, the comfort, the fact that, you know, there's significant amount of privacy there by yourself and safety. As part of the experience, you can play your own music. I think generally people that try it, a lot of them are hooked pretty quickly. And so it's validated the vision that autonomous vehicles can be compelling, they can be safe.

8:02The company recently published some updated safety numbers. Can you talk a little bit about that? Yeah, so we share safety studies and also safety numbers of our service as we operate in all these cities and we update them regularly. There's a hub that the safety dashboard on the Waymo website where one can keep checking in. And so the most recent update was at 50 million miles.

8:29And there we share statistics about our performance as we drove people for first 50 million autonomous driven miles. Actually, every week we drive more than 1 million miles using our fleet. We drive customers. And so in this 50 million miles, there's different groups of incidents ranked in terms of severity. And so the most severe...

8:59we track is, for example, incidents in which airbag was deployed. These are fairly serious high-impact incidents, and we are over 80% safer than normally human drivers would perform. In our estimate, right in the cities where we're driving the conditions we drive, when we're dressed for all of this, we find that We are, I think, 83% fewer crashes that result in airbag deployment than if humans drove these miles. And that's a non-trivial amount. So if you're given the miles, that's over 60 crashes that we could potentially eliminate it by having Waymos drive this. And the second category of incidents is ones where there was some kind of injury, right?

9:57And that one, we're over, again, 80, 81% safer than an estimate of what humans drivers would given our domain where we drive. And the third class is ones that are actually police reported. So more minor incident, but still often you can report to police. That one, we're about, I think, 68% or so, 64% more.

10:26how to say fewer than our estimate of what humans would do. So these are significant improvements. And also, separately, we've had studies done by insurance companies, third party, that also confirmed that in terms of like property damage or, you know, injury claims, we are about 80% less than what humans would do for comparable amount of driving. That Swiss Ray is a company. So this is another...

10:56separate benchmark, not just ourselves benchmarking us, but we will continue releasing this, but it's, I mean, that's what a lot of us are in the space, not just for the experience. I mean, I think we always, many, many people are the joint autonomous vehicle domain, I'm motivated by the great safety benefits and many other benefits that these vehicles can bring to the communities that they serve.

11:25And so this one's the main one. There's tons of people that, I mean, safety comes first. Our mission at Waymo is be the world's most trusted driver, right? And they tend to pin everything we do. Is it still the case that the vehicles are to some degree remotely operated either on an intervention basis or otherwise? And do you publish stats around that and the number of incidents per mile or something along those lines?

11:53I think in terms of a Canon operator, you know, inspect the vehicle. I think that is possible, right? But it happens rarely. And I think the fact that we've scaled to such amounts of vehicles and such scope, you're not going to do it if there's people watching over every vehicle, right, all the time. That's just not a good product.

12:18You know, what did you eliminate? You moved the driver to sit behind the workstation. That doesn't work. And so this is mostly in very rare cases. And also, the vehicles, just to be clear, they're not teleoperated. So it's not like someone drives them remotely in these cases. At most occasionally, a person can confirm something the vehicle wants to do, and usually the vehicle already needs to be stopped for this to happen.

12:45It seemed like that having a real-time loop for that kind of confirmation would be very difficult Yes, and you do not want it but then that means that you actually need to understand and deal with these issues At least to understand to stop for real right which is You know Again very rare. That's also not even a good experience. You don't want to do it. So while it's possible It's very rare. So tell us a little bit about how From a research perspective, you've been tracking and thinking about incorporating all of the ideas around foundation models, LLMs, VLMs that have sprouted up since the last time we spoke. So, yes, generally, we look at the state of the art in machine learning in AI, our team, and if it's tried to adapt the advances to our domain. And usually,

13:41it has involved some creative adaptations. So there is some unique properties of the problems we're solving. And traditionally, we had to do adjustments or fairly creative adjustments of the techniques we see. Now, every two years or so, my experience in this space, and I'm in it since 2015, actually in December, maybe early 2016, is there's big technological...

14:10And a lot of them have to do with machine learning or AI. And I think generally in this space, you want to keep reinventing your stack. So as these advances occur, you digest them, you want to build the forefront and take the benefits. And I think what's happened in the last two years, if anything, there is even bigger jump than before with this.

14:37multimodal large language models with generally generative AI technology that you know also spans multimedia obviously images video understanding audio Handling all of these things it's been a bit of a revolution and You know, we are keenly aware of the advances and we have been exploring them for the benefit of our driver, right?

15:08And so we've engaged in understanding and leveraging the technologies such that vision language models, such as diffusion prediction for generative outputs that can include videos, but they can also include like generating driving scenes for testing we have done.

15:32we leverage things like 3D reconstructive advances. There's a lot of very interesting new technologies, relatively. Like Gaussian splatting and that kind of thing. Gaussian splatting, before that nerve, right? These kinds of technologies are also very interesting. And we've been working on them. But generally, right, recently there has been this trend to scale the models.

15:59you know, train them on a lot of data, scaled architectures, overtime, other enforcement learning, right, to target even better reasoning capabilities. All of these things, I think that's the general trend in the space, and you can see it with all the large models. We have been scaling our models for a while, and we've had actually transform architectures for a while. If you see our publications, we had a few even externally published around 22, 21 with transformers even before that for certain applications in perception. And so we have been on this trend, but I think what has changed more is just the understanding how much more scale we can leverage. And so now we have gone and

16:56scale the models and evolve them beyond what normally we used to do before. And we have seen clear benefits from doing it. Now, just one thing I want to describe is why is our domain different than the vanilla vision language model? And by the way, speaking of vision language model, we have, for example, experimented with them. We have even published a paper called Emma last year where we took a state of the art multimodal, large language model, what Gemini people like to call theirs and tried to adapt it for driving tasks. And adapted it reasonably successfully. We got some very nice results on the driving tasks with the... And this is the idea of converting trajectories into tokens and passing them through the VLM.

17:52So the way it works mostly is, okay, so what does Gemini bring to you? It brings to you world knowledge. It's pre-trained on the internet, and it's a large model, so it understands a lot of concepts already that you don't have to teach it directly with your own labels or data. And then you fine-tune it on your tasks. So now you bring your own data and your own tasks, and you adapt it to do well given all this knowledge it brought to the tasks that you...

18:17fine-tune it to do, and of course you need to adjust a bunch of recipes, you need to make sure that the tasks are well represented in the gem and the inputs, and that all the sensors are properly scaled and the right assets are created and so on. But then when you fine-tune it, it does a reasonable job actually on these tasks. But there are still limitations. For example, our stack relies or likes to rely on...

18:45really powerful spatial awareness of everything going on around you. It's helpful for safety, but then you need to understand the world in 3D even better, right? And we have additional sensors like LiDAR and radar that are really superb for contributing to 3D spatial awareness that you want to incorporate, which vanilla Gemini model does not do. Another thing you want to incorporate is very long memory over a scene. You need to reason often, you know, based on the history of several seconds or more.

19:14that can be a lot of frames. You need to incorporate this memory, somehow maintain it, somehow structure it. You need to think how to prevent hallucinations, right? So the model can predict things, but every once in a while, if you go to domains where it has not seen things, it can be wrong. So you need to think, okay, what are the way we have actually a lot of experiments, experience, you know, launching machine learning, powerful machine learning in every part of our stack, but doing it in a way with the stack design.

19:43that is built to also mitigate all these issues of falling off the data manifold hallucinating and so on. So all of this, you know, are still areas that are very much ripe for further work, even though we have this vision language models. But we want that power. We want to bring external knowledge, right? Internet has a lot of knowledge when you're pre-training. There's a lot of value in having these kinds of pre-trained components.

20:11And we also can benefit from some of the architectures and scales. What are some of the specific tasks that you've explored using VLMs for? I mean, for example, they're obviously can be explored for understanding all kinds of perception outputs. So they understand semantics very, very well. They're trained such that you show it an image you can ask a variety of questions.

20:39A lot of them are really powerful in all kinds of even fairly obscure details. That traditionally is a property that you want to benefit from because otherwise in the old days, maybe five to ten years ago, you would use humans to label all these concepts. Now that model, for example, would understand a lot of them out of the box. That's very powerful.

21:06All right, so the idea being the model sees the scene and as opposed to the old world of bounding boxes around humans balls Scooters that kind of thing the model just knows what those things are and that they're important Yeah, and without us having to label all these concepts Right so you can directly bootstrap yourself the other power of all these models is the more scale they are the larger they are The more the better they generalize So if you see some new variant of the concepts you for some reason didn't label that you need to handle Before you would label have to label a lot of examples for the small model to really capture what those things like and then every new particularly different looking thing you'd need to label more examples Here for perception mostly I'm thinking here as I'm describing now the the large models are just like I mean you can take Gemini or a lot of these other external vision language models

22:06They mostly wouldn't understand what it is already, and it's a big model and it generalizes again. When it sees a trench, it can see a trench in a whole bunch of conditions and know it's a trench. For the small models, this is Harvard. It requires more bespoke engineering to ensure. So that's a nice benefit. Now, the scaling part is appealing, but there's also this interesting limit. There's this question, how far can you push scaling?

22:35And how much of this knowledge can you bring in our models such that now we can get data from new cities, from new countries, from different software platforms so you can move the cameras, you can change the lighters on the vehicle. Over time, right, which we do, we have a six-generation driver that keeps evolving the sensor stack, which is then something to think about.

22:58So when you have all these properties, the question is, can we have a model that just generalizes across all of these things? We want to scale to dozens of cities. We want to do that much bespoke work. And the best way to scale a model is, let's try to scale it in the data center first. So remove constraints of real time response, which needed to be for safe driving. Remove the constraints even potentially to only see events as they unfold. You can see potentially how in the future.

23:28remove constraints on how much you can scale the model. Now, how much understanding can you get out of this model? How much can you generalize? And so we have been pushing to understand this question. And I think we have been aiming to build what we call this Waymo Foundation model, which is not just a vanilla VLM adaptation, but we want to bring in all these capabilities I recently talked about that expand on the standard vision language model.

23:56and build it large, build it in the data center and see how much can you just understand all the data that we see and we collect. And the better this does, the more straightforward it is then. This model then acts as a teacher to the models in the car. You can distill it down. And of course, you still need to do this with the thoughtful design, ensuring that the onboard stack fits all the safety constraints, all the latency constraints, and so on. But then this thing is just the source of knowledge that you can always ask to mind data, to understand data in your mind. And there's a very interesting question, how far can you can push it? But the more you push it, the more you accelerate the scaling or seamless scaling of the service. So we want to explore that. I think that's a very exciting.

24:51direction and it's in tune with the time today, right? I think it's the age of these types of models. And it tackles one of the historically, you know, challenging aspects of autonomous vehicles. And that is that generalizability like the, you know, the vehicle, you've got lots of data captured about a particular road during the day and clear weather, but the vehicle needs to operate in that same road at night in bad weather. Historically, you either need to capture that data manually or use some kind of data augmentation scheme to train the model so that it can adapt to those variations. But if you're able to achieve sufficient generalization with a scaled up model that eliminates, I would think a lot of the challenges of

25:50you know, getting coverage in the market you're in and getting into new markets. So the reality is we have good generalization and coverage in the existing markets already, right? And we operate on them clearly and have passed a certain bar. But new market and the more different the new market is or the more different the new market and sensor stack is the more it pushes your ability. And then, you know, you can You can do it a lot faster if you have this kind of capability than the process today, which would, of course, involve you would go explore the market, collect data, label some data, understand anything that may be different, make sure you're confident that you're safe, right? Like there is a standard procedure how we ensure safety in the market, I think, which is important, but we could accelerate it. How long do you typically need to be?

26:47testing and collecting data in a new market in order to launch in that market? I think it's been a very much evolving process, meaning the first time, say, we did Phoenix, we did East Chandler first, which is a bit of a suburban area of Phoenix.

27:07And suburban areas have one set of challenges, for example, unprotected left turns on the freeway, well, not on freeway, but reasonably high speed road, 45 miles an hour, and then people speed, so it can be a lot higher. And it's really important how you can go in front of traffic going at such speed. And there's a few other challenges there, but it took a while to sort out, right, and make sure our system is robust.

27:31Then you go to San Francisco and you see, oh, it's totally different. There's all these crowds and totally different behavior and the streets, big part of the city are a lot narrower, right? And it's just different dense urban scenario. So then it took a while, too. And then we go to Los Angeles, which somewhat combines the properties of the suburban Phoenix.

27:55And by then we had moved more to downtown Phoenix and so on. So we had a bit more of a span. Everything from a bit more suburban to urban to dense urban. And you have San Francisco. So when you combine those two, LA actually was fairly straightforward. Right? So as we cover, it's about a bit of covering the design domain. The more you cover it, right? True. And in our path, we've been doing this.

28:21the easier it is for the next city. They generalize. So every new part you collect that helps you with the next part. So it's an accelerating process, too. Now, separately, what do we work on a lot today? Freeways. Freeways offers a whole set of new challenges compared to the ones we've seen in, say, the suburban areas or dense urban. There's different.

28:49Things to be concerned about right and you need to validate for example on freeway You need to be robust at 70 miles per hour or 65 miles per hour to any kinds of failures that can occur and also you need to worry about You know stopping at any position on the freeway. That's inherently unsafe Right and sometimes there's not even shoulder for certain parts of freeway. So you need to think what do you do there and that's a different bespoke work you need to think about. And another one, of course, will be snow ice. That brings yet another dimension. There's a few of these. So I would say I can't tell you an exact estimate because it really varies. But we have a robust process to ensure that we gain sufficient confidence in the market before we launch it. And the markets that you're in now, do the vehicles operate on freeways or are they only on surface roads?

29:44So for most customers today, we do not provide freeway, fully driverless and freeway. But we are testing on freeway for a while, actually, without driver in some capacity. So it's still an area which we are scaling and working on. And so we, of course, would very much like to provide this to our customers, but it's still an ongoing process.

30:13Right? For example, take Phoenix. We're clearly motivated to the freeway because Phoenix, for example, is 300 square miles. Going from one end of that area to the other, you really want to take the freeway. Otherwise, your service would take a while compared to what is possible. You want to provide good service. So it's clearly an area of high importance.

30:36So kind of going back to the foundation model work, when you think about this challenge of scaling up the foundation model in the data center, one of the challenges that you mentioned is incorporating all of the different sensor data that you might want to incorporate that aren't necessarily native to VLMs and to images. How do you approach that problem?

31:04So it's a very interesting problem actually, it's a bit open-ended how to do it best. And I think maybe one way to think about it is also audio is an example of this too. Initially when you do a model, people try images first and then later audio became a domain too. So you want to build a multi-model model.

31:31how to actually build it and how to insert data from a new sensor into a model that was trained potentially or pieces of the model that were trained without it is it's actually an interesting research problem and there are many approaches so I can't tell you how exactly we do it we're experimenting with several and learning along the way but we find that it's one of the key questions in our domain how to do that.

31:53Building a foundation model on top of existing VLMs, like the part of the motivation for that is taking advantage of the world knowledge that is embedded in that. But the alternative might be incorporating, I'm sorry, training from the ground up a foundation model that understands these different modalities and is able to do the things that you want to do.

32:22So you can train? It's not clear what exactly that would look like either. Well, exactly. There's this interesting question of, okay, so let's say you train something fully from scratch yourself. But then you need to worry, well, how do I bring all this knowledge that people already usually put into this from the internet, right? You still want to somehow then maybe need to get that. There's still some kind of fusion of, you know, modalities that needs to come together. It's just a different way of looking at the problem.

32:53Right, and so there is another thing though we can do which is We It's also expensive of course to train a model fully from scratch, right? So you need to think that's one aspect of things the other aspect is okay, how can we learn and Build the model from our domain right using data from our domain directly Separate from the fact okay way one is you leverage hopefully world knowledge from models one way or the other right? You need to think how maybe you can even use them to label data for you or you can take pieces of them. There's many things possible. Now the question is how do I create, how do I train model in our domain to understand our sensors, right? And there's this interesting question you can learn in our domain to predict the future also, right? So how do you pre-train large language models? You predict the next token in language. What can you do in our domain, right? You can do things like you can predict

33:51what the sensors give you in the future. So imagine if you're going to tokenize your sensors, you can predict the next sensor token and the next sensor token. And if a model is able to predict accurately how sensors look in the future, whether it's lighter or camera or radar, they need to learn a lot about the environment because it will actually capture all the knowledge in here and to predict how things evolve. So you can imagine how you will see them.

34:20And that's very powerful, pre-training. This is equivalent to this kind of pre-training, next-word prediction. The issue is only, of course, a word can be tokenized in maybe 100,000 tokens or something to that effect, and then you're predicting a distribution over 100,000 discrete tokens. That's fairly straightforward.

34:41Now our sensors they're higher resolution they can see that different multimodal now the question is well for those And there is that opens a whole bunch of interesting questions, right? And this is a question. Do you need autoregressive models? Do you need diffusion type predictors? How do you how do you set it up or a mixture of those? What is effective? There's work in the space already in academia trying to build these and in the industry already trying to build what it's called world model. You're dreaming the futures in the world that you are modeling, right, from your own sensors. And that can be useful to also train large models, right? But how to best build it is also a very fascinating question. And of course, the more the richer futures you dream, the larger models you need. Because now you need to capture a lot of the nuance.

35:37beyond the certain level of, oh, can I scale some model to just predict what I should do and maybe more some high level properties of that is versus predict everything in super high pixel lighter scan level fidelity, right? You need potentially larger models to do that well. And so there's trade-offs like how much of the compute now you're going to spend to pretend these very large models, how much of that actually results in driving improvements, say, for you in the end. And how does the math work out? Is it compelling enough? This is also a very interesting question. But for good or bad, some of these world models are needed for the simulator. So you need to think how to build them regardless, right? We want to... So in our domain, it's not just about building the driver. There's two main problems. One is build a driver, right? And I talked to this also about this in my GTC talk.

36:32The other problem which is validated and ensure that you are confident this model or driver perform well in the vast majority of cases we need to handle, right? Like the environments in which vehicles drive, span all kinds of diversity and season as we already talked about different operational domains. There is all kinds of human behaviors, some of them very rarely seen that you need to deal with.

37:01in front of you, you need to interact with humans. So all of this, of course, is great to test in a simulator and is a great tool to test at scale. But now the question is, how do you build the simulator? And now you want to use machine learning to build it, right? And so before we dive into validation, I want to dig into this idea of like predicting future sensor values. You know, when I think about that, I think that there's like a causal relationship between the vehicle trajectory or the vehicle behavior and then what the sensors, you know, pick up as the vehicle progresses through, you know, a scene. And I'm trying to kind of wrap my head around like what it means to predict future sensor values as opposed to predicting the thing that is like the root cause of that, the vehicle trajectory. How do you think about the relationship between future sensor prediction and like

37:59the vehicle itself and what you, using that as a control input or something. So I just contrast that to a little bit and see if we're thinking of the same setup. So in our Emma paper, for example, right, you have a vision language model. And one of the tasks we can teach it to do, even though we tried several, like you can predict the road graph that's in front of you, you can predict the 3D boxes that you see.

38:23But one of the interesting properties, the main one, maybe, you predict is, what's the driving trajectory you're likely to follow? And you can, of course, sample several. And the start of that trajectory, actually, then you can turn in the controls. The rest is more speculative, kind of showing you what it's likely to be like over time, right? And, of course, the longer you predict the trajectory, by the way, in our domain, the more uncertain it gets whether you're actually taking it, because It also depends on certain assumption of the world. I can think I'm driving this trajectory for eight seconds, but if the vehicle in front of me swaps, I'll adjust it. So it's all relative to what the others do a little bit as well. That said, it's a very good output and the start of it you always would drive. So it's related to planning quite strongly. But the trajectory, how do you train this model in our domain? You would actually observe.

39:19Either are drivers or potentially how others drive. And you saw what they did in the future, and you collected it. And now you're teaching the model to predict just those tokens of how a driving tokens for a few steps in the future. That's a signal, right? That's the most topical signal for how you should drive. You watch how people drive, you see their tokens, and you learn to predict them. Now when you think of sensors, it's it's essentially the environment that goes with how you draw. Instead of predicting just how you draw, now, for example, maybe a next level, imagine is observe others and predict how they drive next to you. A lot of the models do this already, right? So not just predicting yourself, we have a model called MotionLM published maybe two years ago and it's in even all the work showing how to turn motion into conversation. And you can train the models with that level of...

40:19abstractions you say oh i'm not just teaching a model to predict how i drove i'm going to teach it also to predict the other people's motion tokens around us and so you predict jointly how a whole group of them now behave so that's that's a richer signal you're teaching the model uh more things from everyone example and they're highly topical now you're teaching it all not only how i behaved but how everyone else reacts to me and then maybe how i would react to them your You're modeling what we call the joint distribution of at least some group of agents around us. So that's a richer signal. Now, imagine you go to sensors. That's maybe very high bandwidth, detailed variant of everything that happened before. You not only see the environment, not only changes because you, say, did certain driving paths.

41:17It also changes because others reacted to you and changed their behavior relative to you. That's also reflected in the sensors. And beyond that, you need to think how everything actually changes appearance as you and how objects may look from the other side if you surround them and how reflections may work and how certain lighter intensity relates to the RGB pixels because now you're dreaming the future of the environment.

41:47the full environment in its full fidelity. So there's an argument that it's like a more grounded prediction and that it's reflecting the actual environment that the sensors are intended to see. It's the most high bandwidth prediction you have. This is all the information you actually had about the future, which is your full sentence. And that's the most you could try to predict. But then the issue is, are you overdoing it? Like, you know, so for example, humans, I don't spend my time thinking how the tree looks from the other side.

42:16when they drive, right? Right, right, right. Or like, you know, as they go, how would the branches occlude each other from a different viewpoint? Or how the shadow will evolve on that pedestrian? I don't necessarily have to think about it. So while there's a lot of knowledge inherent that the model can capture, not all of it is relevant to driving, some of it is. And so now there's this interesting trade-off. Do you really want to go all the way there? How much benefit you're going to get?

42:47or do you do something? Clearly some version of fidelity is relevant. So like predicting what people did a few steps in the future is the minimum you can do in the most topical. And then you broaden and it becomes increasingly less relevant. Like, you know, you want to model how the clouds move in the sky. Maybe it's relevant to it raining in an hour, but I'm not sure it will help your driving suit. So there's this trade-off. There's a ton of signal in these sensors.

43:17But you could choose to model it. It's helpful in some cases for a simulator to model it. But it doesn't always help drive. So we're discovering what is the right risk. Remind me or maybe talk a little bit about the way your approach to end-to-end training versus subsystems that maybe map to traditional approaches to robotics or vehicles, like, are there separate control, perception, et cetera, systems, or are you moving towards or away from end to end modeling? I vaguely remember us talking about this a few years ago as one of the differentiators of your approach, but I don't remember which side of that.

44:12divide if we are on the practical side, we will take the thing that works best. That's what I would say. Now there is end-to-end in general is a bit of an overloaded topic. I know it's been made into a big issue by a lot of people. It's compelling in some embodiment. But let's untangle what end-to-end means. So end-to-end is if you think, there's kind of mild definition and a strong definition. So end-to-end means typically, right? The strong definition is, hey, you're gonna pass some gradients from, you know, sensors, from controls, what you want something to do through a whole system, maybe to the sensors. And along the way, learn features.

45:07that you may not necessarily be able to inspect yourself or that have semantic meaning, they just emerge from the training process that may help you drive. And to end, in general, is a training strategy. It's orthogonal from whether you have modules in the stack. You can still have modules and train things end to end. If the models are connected properly and you can pass gradients between them, You can, right? And you can still train many tasks end-to-end. So it doesn't have to be sensors to control, right? You can even potentially consolidate large pieces of the stack. So every piece of your stack became more end-to-end because initially, say, had bespoke 10 different models handling certain tasks, and now you've consolidated them, that part became more end-to-end, right? So it's a complex topic. Now I would say that...

46:08What we've learned over time is that you want few large components if possible. It simplifies the development, right? It allows you to essentially scale those components more and optimize them as opposed to have a bunch of little custom things each doing different stuff with different data generation. So you want few consolidated components. And now the question is how few?

46:37And of course, the other part, you know, is if you have a few, you still want to ideally back propagate through them. But what's the, how few and can you just make do with sensors to controls the extreme? And the sensors to controls I think is what people are really excited about in academia. And it's very powerful in an academic setting. And it's a compelling idea just collecting enough data. You've got some black box in the middle that's sufficiently, you know, has the sufficient, you know, scale and number of parameters and you've got some desired output.

47:07You let the middle part figure it out. And our Emma paper does this too, right? So we do train it end-to-end clearly, right? And predicts nice trajectories. And generally does a good job with a fairly straightforward recipe. So that's appealing. But the big questions there is, can you simulate and prove that this system does not hallucinate, does not... Proofability, controllability.

47:36introspection, testability. Now I need a simulator that's, you know, always from sensors to controls. And in our domain, there's quite a lot of sensors, and they're not that easy to simulate. The technology for this has been evolving. And then you're really tied to this. You need to scale it. You want to imagine driving over a million miles a day in the simulator and now having to simulate everything you could possibly see in these sensors, and usually most companies have at least a couple sensors, different ones. And you need to, at a million miles with over a dozen cameras, with a few LiRAR, for example, or LiRAR, right? And it needs to be really realistic, otherwise it doesn't count. How do you do it, right? That is the challenge. How do you prove to yourself now that all these cases can be validated just with this setup?

48:35That's hard, right? And so there's push and pull also the other big questions. Let's say you train a huge model, and now you want to change it. Well, what do you do? The only thing you can do is you change the data to return the model. Now you fixed it, but maybe it broke something else. It's really hard to fix something really quickly if all you have is a model that goes into it. So there are all these considerations. So I would say that there is consensus, reasonable consensus in our space that you need few large components, but there is no consensus yet if it should be one component. And I think a lot of it comes from testability. And so if you want to say, in our case, when we're doing actually full autonomous driving, it's top of mind. You design your stack in a way that you're sure you can test. So that's a constraint. So anything we do, we're doing with the mindset that, hey, we actually know how to test this at scale to actually deploy a driver.

49:35When you're doing driver assist, it's less demanding. You don't have to test it to this extent. So maybe there you can get by with a fully end-to-end model, as some people have. But in autonomous driving, when you have to release it there to drive in hundreds of thousands of trips and millions of miles a week, that's a high bar.

50:01Between that and ability to fix things that arise, these are top of mind when you build a stack, when it's fully end-to-end. So you need to have an answer. So these are the trade-offs and we are doing our best thing. I think it's appealing to do end-to-end generally. It learns good features when you can do it. Just prove to yourself you can manage it. Does the more modular approach that you've taken give you...

50:28the controllability, testability, introspection, you know, quote-unquote, for free, or are those all active research topics as well? Like how to ensure a degree of, I guess, explainability was the word that was briefly escaping me. Explainability and controllability. You want, if you have a model and...

50:58It's some big VLM, even or whatever, like we did with Emma. You want to have a way to detect and steer it when it's doing wrong things and it's hallucinating. So you need a harness around it. There's two kinds of harness you can have. You can build a harness around it that operates in the system you deployed that makes sure with, you know, there's ways to do this. We have experience. You can make sure to catch this model when it hallucinates things that it should not.

51:27do and control it on top. The other part is you can catch it in the simulator. The other part we discussed, right? But then you need to build a fully realistic simulator. And in both cases, you need to understand the world a lot deeper and more symbolically than just having sensors to control. So I would say that. And you need to somehow get the confidence that you've run sufficient simulation samples in order to get the coverage you need to oh that's it's called science actually it's extremely hard and i think there's this other interesting aspect of the simulator is so for example as you collect more data and make your model larger you can have it to predict better trajectories for you so it's very efficient in some way data efficient too if you just want to improve over time and you get more data you can keep adding data and the model will improve in certain ways at least over time in general

52:25When you want to test it, you cannot do that. You actually need to think of all the cases that could go wrong. You need to create some ways to validate all of those. They can require all kinds of hardware failures, sensors going wrong, rare cases, you name it. You need to somehow handle all of this in your release cycle. That does not work as well. You just take data and you just feed it in the simulator and the simulator tests you better. It's also true, but it's not sufficient. So you need very thoughtful on the testing side. It's not purely a data-driven exercise. You need experts thinking what could go wrong and working on mitigating that. And it scales less straightforward than something where you can just put data in and driver would improve with more and better data suitably picked. That is true.

53:20But your ability to test it is not similarly scalable. How do you characterize the test coverage or the testedness of a given model? Do you need to get to approvably some standard of proved valid or is there more of a statistical determination of testedness?

53:48I think you need to be very comprehensive and that's why it's hard to do just it's not just some data driven model that you just do throw data in it somehow tests you but you're also not trying to get to like NASA levels of mathematical proof of you know complete controllability or are you I guess I mean there's also this question how do you even prove it right because if people do very unreasonable things some of them you cannot even mitigate Like you're stopping and someone tailgates you at full speed. What are you gonna do? You can't even get out, right? So it's all relative to what's reasonable to expect. But I guess if we have published our safety framework, we are one of the fewer companies that have done it. And that details the whole set of tests we subject our driver to when we want to release it. And it's a very broad set.

54:46And it includes both kinds of hardware testing and failure of the injection and all kinds of standard analysis techniques. And it includes simulator, and it includes driving in closed courses and testing with drivers and all kinds of things, right? This is actually difficult to develop. It's very comprehensive. It takes a lot of effort, but we must show that we can iterate over time.

55:14and release improvements to our stack while going through this validation cycle. And I think that's one of our great things that we've been able to accomplish is to have that, right? That's at its bespoke. It's diverse. Many things are combined. But if you ask me, what are we testing for? You have to test for a lot of things. One obvious one is you should feel you can be safer than the human driver.

55:44And right, that's one. Second one is you should be confident. You don't get stuck all over the place either. The third one is okay. You don't want to, you know, you want to be thoughtful and mindful around say ambulances and police vehicles. It goes on and on and on. The amount of things you need to be able to do is broad. You need to in construction, for example, or if directed by an official with some gesture, you need to try to understand it.

56:14It's a broad set of cases. You need to check the truth. And, you know, going back to the question about data only and end-to-end versus, you know, we didn't explicitly talk about like a rules-based, like how do you incorporate that long list of scenarios into the model in a, you know, Just throwing it into throwing more examples into a data set isn't good enough like what how do you do it? And there's different ways, right? I think generally we have a lot of experts right as well into what is good driving motion planning experts. That's one second is there's generally this question of how do you define a reward function for driving? Right, so what's good driving even how do you define it? How do you ensure your stack?

57:13satisfies it. It's a very interesting question. I mean, so far we discussed maybe imitating some drivers. That's one way to define it, but that's not the only way. You can say you should not do a whole bunch of things, so you can read this rule book and ensure that certain things are true and optimize for it, right? But I think it's a difficult endeavor to define it and then make sure the model take that into account.

57:43It's an evolving space. We have reasonable approaches. I think generally, as you can see, some things easier to define than others. For example, don't hit things is somewhat straightforward. I'm thinking of like I was just yesterday, I was on a road and there was some work happening to a transformer on the side of the road. So they combined the two lane highway into one lane and they had signal people in each side.

58:10with either the slow sign or the stop, you know, the sign that spins from slow to stop. Like, and I'm thinking about in the context of this conversation, like, is that something that you just get enough data about those situations and the car figures out what to do, or do you have some rule set somewhere that, you know, kind of defines for the car, like, when this is happening, like, this is what's going on, and this is how you're to behave, or does it follow more readily from what the car in front of you is doing than I might think, like.

58:39It's complicated because a lot of those things as well normally shouldn't cross, cross, say the lane, lane boundaries. But when they put the cones, you actually shoot, but then separate from the cones and the sign, the human actually gestures you to do something, right? These are very, that's what makes it very complicated. Yeah. Yeah. So we combine different approaches to make sure we can handle this as well as we can. It seems like an example of where of at least a failing of a fully end-to-end process. You need to collect a lot of data about a lot of what are probably individually unique situations that have this pattern in order for the vehicle to on its own do the right thing all the time. And it's limiting. One thing that, yeah, so if you did not see some situation, the model may just nail it or it may not, right?

59:39And that's a challenge. Now we need to know, does it nail it often enough where you say it's good enough or does it fail often enough where now we need to think about other things to do? That's part of the, you know, convincing yourself, validating the driver's dog. You mentioned earlier, we talked about nerves and 3D Gaussian splatting and diffusion models. You mentioned those as technologies that you are looking into. And a lot of that comes up in the the validation part of the equation. Can you talk about some of the ways you're using those technologies? So I can say the following. I mean, so far I was trying to tell you, hey, validation is difficult to scale. It's complicated. You need to, you know, handle all kinds of individualities and some of them need experts. It's true. But the best way or one of the best ways to scale validation is still to use machine learning to build the simulator better, right?

1:00:41because simulator is one big part of your validation story. And now you need realistic scenarios and interesting scenarios to be played in this simulator. And traditionally, maybe 10 years ago when I started, the state of the art in simulation was computer graphics methods. So you collect a whole bunch of assets, houses and trees and, you know, whatever, and bollards and...

1:01:07And then when you drove it, you understand those assets where they are and how they look. And then you just replace it with the assets. You place them there, ideally automatically. And then you have a world. You have a 3D world. Now you can drive in that 3D world and you can use computer graphic technology to make it look realistic. But even then, there was a sim to real issue where the models wouldn't react to the graphical versions of the things in the same way that they'd react to the real world versions of the things.

1:01:35it can easily be. And this is still true in a lot of robotics. Now, it's true that they're correlated, right? They look similar-ish, but they're not the same. And so, why? Because, I mean, honestly, computer graphics, again, is certain approximation and who said the properties of the, you know, what should the sun be like? And what is the diffusion?

1:02:00aspects of the environment and is there fog like you need to set a whole bunch of knobs properly what is the camera you know detailed is it grimy or not and you need to somehow needs to model of this it's really hard in a computer graphics environment there's a lot of knobs even though it's nominally powerful right and so then there is this gap and also how good are you at taking what you drove and place all these assets. Did you have all the assets? Did you not have them? If you place different assets, if you set the conditions differently, is that the gap is there? So you're not guaranteed that you will be able to well reconstruct anything you drove. You can drive at night in the fog, in the rain. Things may look different. Now, the dream from scalable simulator is to be able to just reconstruct it from your own sensors.

1:02:56We have enough sensors, they're extremely rich, they're, you know, cameras and 3D sensors. You should be able to go a long way into just building yourself the simulator from what you drove. And so the dream is right, and that's something we work to enable, is any situation I drove in my car, I can build the simulation environment from. I may have to populate it with additional traffic, suitably, that's where General TBI comes in. I may need to evolve it in response to the things I do because My decisions impact the decisions of others. They respond, I need to model that correctly. Otherwise, you're not going to have realistic outcomes. If the agent drove like what they drove, when you weren't in front of it, it will plow into you just because you did it natively. You can't just replay what the agents did natively and get all the signal you wanted. So you need intelligent agents too. So between reconstructing the world and being able to drive, say,

1:03:52this intersection in San Francisco, I want to be confident I can drive. And being able to populate it with agents that behave like it is reasonable for pedestrians and vehicles to behave, that's a really interesting, challenging machine learning problem. And so I'm inspired by it. It's fascinating. I think some of these technologies are 3D Gaussian splats, diffusion modeling on the 3D Gaussian splats and nerf for the reconstructive side of things. So you can reconstruct it really well.

1:04:21for representation in the environment you draw, but you can't, those technologies don't necessarily allow you to augment it or dream it or turn day into night or place new agents so easily as it would be now, for example, something like a diffusion model. So that's more of a generative type of model. We can dream new parts of the environment or agent behaviors, a complex one where the appearance matters or futures, but, you know, again, Do you dream freeform or do you want to condition yourself and land yourself at some specific intersection in San Francisco? How do you do the two together? There's some interesting questions here that I think the field is now making progress on so it's exciting and we are alongside the field as well making some I think exciting progress.

1:05:09One of the ways that you make some of this progress is by offering challenges to the broader community to help you solve some of the things you're focusing on in a given year. And you recently announced those challenges for 2025. Can you talk a little bit about those? Thank you for mentioning this. We've been engaged with the academic community since 2018. We started working on this when I joined Waymo.

1:05:36In 2019, we released that set at the time, the largest, most comprehensive we had in terms of actually having high quality Waymo sensors and quite a few scenarios compared to what was reasonable for the time. And we kept upgrading it every year, adding more and more, either tasks or potentially additional examples of interesting driving or things that need to be understood.

1:06:05In parallel with data set work, we organize challenges on tasks, which will be really exciting. And there's headroom to make more progress in terms of interesting approaches and solutions. So this is our sixth tier. We actually have not announced them yet. We probably will announce them by the time this podcast airs. They're launching, as opposed to launch in end of March.

1:06:36Every year we do four challenges. So we pick certain tasks. We've done over 15 so far over six years. This is our sixth year. We have some very exciting ones. So one of the interesting ones that we're aiming to launch that relates to our conversation today a bit is end-to-end driving from just camera inputs. We are sharing some very interesting scenarios among the ones Waymo has seen.

1:07:05And we want to see how people can take some of these large models and get them to generalize on reasonably rare conditions. So that's exciting. We also have a simulated agents challenge where unique in running such a challenge. It's a very interesting problem where you're building agent models that populate the simulator and have ways to validate whether they're good models.

1:07:33We have improved the metrics a bit this year. We have one on traffic generation against something we discussed. So let's say I give you the road graph of an intersection. I want you to potentially populate it with realistic traffic that, you know, the relevant positions, velocities, behaviors are reasonable and not.

1:07:59not ad hoc, but we have ways to measure how realistic a traffic scenario is and we want to see what people can do. And there is interaction modeling as well. That's a challenge we're bringing back with a bit improved metrics from, I think maybe four years ago is when we were on it, when we last talked. So we're bringing that one back. And here I'm back on your podcast also four years later. That's an exciting one. Modeling.

1:08:26how agents react vis-a-vis one another well is an interesting challenge and I think a lot has changed so I'm excited to see how the new crop of methods that people try would would do on it. Well we will include a link to those in the show notes as well as a link to the past conversation if anyone wants to go back to that one but thanks for taking some time to chat with us about what you are up to there and how you're approaching, you know, this new world of foundation models, very cool stuff. Thank you so much for having me on. It's always a pleasure. And yeah, well, I look forward to trying us and telling me how it goes as well, when you're in the city with a way more driver. For sure, for sure. Thanks so much, Drago. Thank you. Bye.