Gradient_Dissent_Conversations_on_AI_Uber,_Nissan,_and_Mercedes

0:00Autonomous driving is all about looking at the AV problem with an AI approach. There is an opportunity there to drive accidents down to near zero. So what we did is we raised one and a half million dollars, got some friends together, we rented a house, and we put our car in the garage and started hacking away. Taking it from expensive retrofit vehicles, which relied on compute HD maps, infrastructure, to mass market vehicles that you can buy or manufacture for 30, 40, $50,000 each. It had in-built hardware that's in global supply chains. Doesn't need an HD map, so it can drive anywhere. We're now the first company to have driven zero

0:29shot in over 500 cities. Throughout Europe, Asia and North America, it's also driven in over 10 different cars, from electric vehicles, advanced SUVs. We went north of the Arctic Circle, tested driving in 22-hour darkness and snow. We were even in Tokyo during a typhoon, where local trains and buses shut down, no disengagement. It's perfect with only the whole day, and it demonstrated that we truly can generalize to any vehicle anywhere. I think now the question is, how do you bring it to global scale? And that's the shift that we want to take the industry through, where every vehicle is capable of driverless operation, which is clearly

0:59the city state of where we're going to go to. You're listening to Gradient Descent, a show about making machine learning work in the real world, and I'm your host, Lucas B. Weld. Alright, here we are at the Corrie House at Nvidia GTC. I just got done interviewing Alex Kendall, who's the CEO of Wave, a company that I've admired for a long time and has done phenomenally well in self-driving. And I think you actually seem behind me messing with the Corrie F1 car. So, this is fun interview. I hope you enjoy it.

1:31Well, honored to be here with the Alex at GTC. This is our first podcast that recording anywhere except my basement or the office. So this is pretty exciting. I have to start by asking you to tell me the story of Wave because I think it's an especially interesting one. Yeah, actually. It's a fun. I just called from giving a speech here at the events and they put the full my face up on the big screen in front of a 50,000 person stadium before.

1:59by five minutes before Jensen came on and it reminded me because I've only spoken in front of like 50,000 people twice in my life. And the last time was at a conference called Web Summit back in 2018 where we had this seed company pitch competition where I got to the final and had to give it, I think it was like a three minute pitch in front of the crowd. And long story short, there was a crowd vote that I ended up coming last and yet...

2:26The judges gave the competition to me because they liked what I said, but just gives you the idea of 10 years ago when I started the company, what we were doing was completely contrarian. The idea that, so what Wave started with a thesis that autonomous driving is all about looking at the AV problem with an AI approach. So it's a decision-making, complex reasoning challenge, and we took the approach of the best way to go tackle that is with the Intwen Deep Learning.

2:53And a decade ago, we started working on this building a single model that could reason and strive and generalize in the largest scale possible without the AV1.0, the first generation approach, which relied on retrofit sensors, compute HD maps, infrastructure, and really limited scaling. And so we started that a decade ago when this was really laughed at and dismissed by the industry and we sort of quietly building it away. And we're at the point now where Not only is the market industry excited about EVs again, but we have an AI model that we're now the first company to have driven zero shot in over 500 cities. So this means this model is driven throughout Europe, Asia and North America. It's also driven in over 10 different cars from from electric vehicles to vans to SUVs. And so it's now emerging as an AI model capable of driving any vehicle anywhere. And we've got a sprint ahead of us over the next year to get this deployed in both robot taxis and consumer vehicles you can buy.

3:53I remember having a really amazing experience driving with you in London where you let me decide where I wanted the model to go. And I hadn't spent much time in London, and I was astonished by how challenging it seems compared to San Francisco, where people I think obey the traffic laws, as I understand it much more thoroughly than in London. And there's just much more surprising things coming at you. And I couldn't believe how well your model works at that time. And I think this was maybe two or three.

4:21years ago. But before we get to the quality of your model, I kind of want to go back in time a little bit and ask you about your history, right? You grew up in a farm, I think, in New Zealand, is that right? Yeah, in Christchurch, in the South Island. And how do you think that's kind of affected your leadership or your thinking about autonomous vehicles? I think there were a few things that were really important to me growing up on reflection. I mean, I spent half my life.

4:48building things, whether it was video games or different Lego or robotics or other contraptions. It's been a lot of time messing around with that kind of stuff. And the other half was in the mountains, climbing mountains, surfing, mountain biking, all this kind of stuff. And I called Wave Wave because I want riding a car to feel as good as surfing a wave. But I really think when it comes to pushing forward frontier technology, it's all about an adventure, it's all about doing something where you have to plan, you have to risk manage, you have to be ambitious, push boundaries. And I was only a bit of that that I got from those experiences and those values that we can take forward to today because what we've been doing, it's one of those ones where when we started, this wasn't possible, but I thought and hoped and it's turned out that the future, building this technology would become possible.

5:44And I think we've been lucky that so many things have come together that make that a reality, but it really did seem like optimizing for the Vex's adventure. I was fortunate enough to get a scholarship to take me from New Zealand to Cambridge University. Turning up to that place was like a castle from the middle ages full of the most amazing brilliant minds to just learn and explore. Again, optimizing for adventures has somewhat led me to where we are today. Okay, you got to tell me about building the first end to end prototype because I would imagine that we're particularly hard.

6:14with the way that you're approaching the problem. You can't really cheat, right? If you're trying to do end-to-end machine learning. Tell me about the experience of that and how you made it work. So in 2017, I just finished my PhD thesis and I'd be able to build the first deep learning model for different perception tasks, localization, stereo vision, semantic segmentation, uncertainty estimation, all of these kind of systems that could take high-dimensional, multi-million-dimensional images, and turn them into compressed representations of the world, learning into end. At the same time, friends down the road at DeepMind had just produced AlphaGo. It has amazing. That's a system that, but there's more states of the game of Go, than there are as in the universe. Extraordinary complex. And they showed through self-play, you could learn, and simulation, you could learn an agent that could beat the world champion. And so I saw these things, and I thought, okay, we can now understand the complexity of the real world through computer vision.

7:09If you have unlimited data through a simulator in a very low dimensional state, you can solve the hardest game there is, if go. Now it's time to bring these together into the physical world, whichever has been excited about about physical mobility and adventure and robotics. And so I thought we could go try it. So what we did is we raised one and a half million dollars, got some friends together, we rented a house and we put our car in the garage and started hacking away. And the first thing we tried was on policy reinforcement learning.

7:38We put the car on a road and say, OK, go drive with the reward of drive as far as you can without any human intervention. And your reward is distance traveled. And so the car starts driving randomly. And essentially, over a period of months, we develop an algorithm that acue learning algorithm essentially lets you intervene and grab the wheel every time it tries to drive the road. And the big breakthrough was the moment where with just 10 bits of intervention, we got it to learn how to lane follow.

8:05And I got that breakthrough, I remember it was on like a Sunday or et cetera. It was on a weekend where I was like pushing really hard. Nothing was working and all of a sudden I tried it and it worked. It might have been the random seed. So these kind of algorithms tended to work one out of 10 times. Well, it was like trying to understand. So you only intervened. It had no prior information about the world. It's a link in a camera. Yeah. And after 10 interventions, it understands that you wanted to follow lanes and it can follow lanes. That's right. So the input was a front camera. There was a GPU on board doing on policy optimization. So on board optimization.

8:34and it was essentially, yeah, trial and error, a few different 10 trials, and then it was able to learn a policy. There's a video on YouTube that's got this that were muted the audio because the audio is me whooping and cheering as it starts to drive, but it was essentially just able to name-follow quite robustly after just 10 bits of feedback with no prior knowledge of the world. Were you afraid when you got in the car the first time and it had no prior information, like we actually on a road? This is going slowly, so you had time to intervene before it drives off the road.

9:03But that brings up a good question is that that method obviously doesn't scale. You can't do on-policy tests at scale around the world. That's pretty unsafe. And so we quickly moved to an invitation and offline reinforcement and paradigm and started scaling up world models so we could learn through dreaming and understanding through their experience. And so later in 2018, that's where we started to take things and growth of there has brought us to this point today where we can drive any vehicle anywhere all around the world.

9:31So what happened after you got the link following? What was kind of the next step? Mixed it was starting to scale it up. So we tried so many different things. We tried injecting priors around computer vision. We tried building world models and doing learning and dreaming and learning in imagination states. We tried imitation learning. And we tried some to reel. Learning and simulation, chance for the real world. We just sort of tried everything. We wrote all these blog posts. They ended up being industry first and all those different areas. And then settled on what a recipe that seemed to work seemed to scale. And then started really pushing, had a couple years of pushing infrastructure, fleet build out, and actually bringing out this technology to a point where it was hard to drive in London. And retrospectively, learning in London was great because it forced us to build something that could deal with the chaos and complexity of your average London street. You have like 100 pedestrians and cyclists around you at any one time.

10:28And so it forced us to build something that didn't rely on mapping infrastructure or anything like that, but it could actually drive in the city that's got 2000 years old. I remember 10 years ago when you were getting started. The autonomous vehicle space was super crowded with lots of different approaches, lots of VC money flowing in. It seems like it's consolidated at this point, but also it seems like it's really working. I use autopilot in my test all the time. I read Waymo's Every Day. I feel like it's kind of quietly snuck up and become a technology that at least there in San Francisco, we use all the time. I'm curious how you think about the competition at this point. Is it becoming a commodity? Is there something different about what wave can do or how you approach things? Yeah, that's a good question, because I think broadly speaking, when we started in 2017, that was probably the peak of the hype cycle, where raising one and a half million dollars to go tackle.

11:24was like 10 companies that just announced billion dollar ambitions and deals was a bit lopsided. But it's fair to say we saw a, you know, trough disillusionment. And I think largely that was from the share expense, the 100 billion plus dollars that's required to build that AV 1.0 paradigm of mapping rules, infrastructure, and to get that to maturity. And there was a lot of consolidation there where really only, you know, one or two companies had the appetites of capital to push through and make that work. At the same time, Tesla FSD or ourselves matured an AV 2.0 approach, an extension generation stack that's more affordable, scalable, and is now going to the point from a demo, which we had five years ago, to a point where it's actually ready for global automotive product. And I think the most exciting thing today is we're also seeing huge validation of the product.

12:20You and many others are now buying FSD and paying for it, and it's actually loved by a lot of customers. We go to Shanghai and San Francisco, places like this, and people will pay, even pay more than a human taxi experience for a robot taxi, even though it has longer wait times, it's limited by a geofence, and it's a product that people really love. And so now the question is, How do you bring this to any vehicle anywhere? And I think that's the step change that we've built and we can offer. Taking it from expensive retrofit vehicles to mass market vehicles that you can buy or manufacture for 30, 40, $50,000 each, that have inbuilt hardware that's in global supply chains. It doesn't need an HD map, so it can drive anywhere. And it's got the flexibility to be in a fleet that you ride how or in a car that you own. And so I think...

13:14And by the way, this ambition, this business model is only available because we've built this generalizable AI driver. But I think now the question is how do you bring it to global scale? And that's the shift that we want to take the industry through. But for me, I'm thrilled that we're now seeing, what are the conditions in the market? So I mentioned that people are now paying, and there's real commercial validation of this as a product. Regulation is in place, not just in the US, but We work really closely with the UK, Europe, and Japan. We co-chair the UN Committee on Decast Regulation for Autonomy Systems that later this year is now going to come through and actually legalize these products. And then of course, manufacturers are now building vehicles at millions of scale volume that have centralized compute, surround sensors, and the ability to over their update and get data off them, which makes a fleet learning product possible. So all of these factors together give me a lot of optimism for

14:12going through a massive commercial inflection point in the coming year. It's kind of interesting you didn't mention in that list improving the algorithm. And it's very hard to do the quality of an algorithm where the safety of a self-driving car when you're in it. But I have to say when I was inside of your car, and this is now, I think, two years ago, it really felt ready for prime time. I think it felt safer than the cruise cars that were on the street in San Francisco, which I don't know if I should trust my own judgment or not. But...

14:41Are there still things to do with the algorithm to get it to the point where you feel good about using it globally? Or is that no longer the challenge and you've kind of moved on to logistics and regulation and things like that? Absolutely, there's still things to do. We don't yet have a driverless service today, but this is no longer the critical path. We are seeing that the quality of innovation coming from our frontier and body day eye science team, plus the data and compute growth that we have.

15:08is just driving compounding scale. And then you layer on top of deeper integration into vehicles with our partners, get a sensing and compute on the vehicles. These things compound that performance is no longer the critical path, but we see what work to do there. And so we'll keep pushing that and pushing it into a generalized state. I think a large challenge is evaluation and validation of it. How do you prove the level of performance is sufficient to yourself, to regulate as consumers?

15:38And then of course commercially deploying and setting up. I mean, the last couple of years for us has been a big year of growth of the company to build strength of team to partner with regulators and automotive and fleets in major markets like Germany, Japan and the US. It's to get the integration going. It's the multi-year sales cycle of automotive. And now it's going to be the deployment and bring up of these systems. And so I think that's where the critical path is, but we're not taking the foot off the accelerator in terms of performance, which...

16:07you know, clearly, clearly has autism magnitudes still to go in terms of opportunity ahead. One of my favorite demos I've ever seen was someone on your team who showed me plugging your perception model into an LM so that you could actually see what the car was thinking as it was driving, which seemed surprising and delightful. Was that a toy thing that you realized you didn't need or is that something that you've continued to invest in?

16:37Yeah, you know what? In 2021 or 22, like way before CHGBT, one of my team, Vijay, came to me and said, hey, Alex, I want to start working on language for tripping. And my first reaction was, no way, man, this is a good stay focused. Whenever a product ships, come on, stay focused on driving. And he laid out the argument that actually the knowledge you can get from text data is enormous.

17:01And it might produce new interesting products and interaction modalities. And actually by bringing in language, we can improve the representation, the performance, and the product experience. And so, we actually said, okay, let's start exploring this. And so, very quickly, we got up a prototype that it actually wasn't plugging together in aluminum because aluminum didn't really exist in that large scale yet. It was more, we trained a vision language action model. We trained a single model that could see the world driver car and understand language.

17:30And so it started off as a gimmick. You know, you could drive along and have it explain its driving. You know, I'm going around a double park bus. I'm stopping for a red light and we just explain it. And you know, you could cherry pick some really delightful things. But then again, there were also some things where it was just hallucinating and wrong. But from that point, we started to train it up. We actually got it to a point where it would become very robust. We saw that there's actually one interesting thing I saw, which is if you drive and you have an oncoming car and you want to make a turn across that oncoming car to go into a side street. If that oncoming car stops and flashes its lights at you, it's a signal for you to go in front, right, even though you don't have right of way. Array, I wouldn't do that. It would stop and yield and just wait for that car because it had right of way. When we started bringing language pre-training into the VLA model, it actually demonstrated that behavior when it started flashing its lights, it would now turn. I saw that for the first time.

18:27And so that was an example of we could actually improve the representation and reasoning capabilities by bringing in language into the system. And so today, it's a core part of our training. It gives us boost in performance. It's opening up new interpretability capabilities. And of course, opens up product experiences of personalization and interaction with the vehicle. You can now get in our car and prompt it to drive in different driving styles. And I think that'll be a neat experience because when you get into a...

18:54A-dass or aerobotaxi, driver assistance or aerobotaxi car, if that car is driving too conservatively, you're going to get ready frustrated. If it's driving too fast for you, you're going to get freaked out. And different people have quite wide variety in different expectations there, or different brands. You can go out some brands that want to be sporty, or some brands that want to be safe and reliable, and not say that sporty won't be safe and reliable, but you know what I mean. And so providing a personalization is quite a nice outcome of this work. It's a funny moment.

19:24for self-driving at least in San Francisco where we suddenly have way more everywhere. And I love it. I think it's an amazing experience. But we're also witnessing all kinds of different crazy traffic issues that are being caused. And also people maybe for fun or maybe maliciously kind of messing with them in ways that you wouldn't have expected. So it does seem like that capability might actually be completely critical to making us help every car assisting people trust. Yes, although a couple of things. Firstly, our cars.

19:53You know, over time you won't notice anything different because they're just normal cars or built-in sensors. So I hope that people don't behave differently around them compared to other traffic. And then secondly our system is designed to drive in a very human-like way. The way you see it picking up and dropping off is a robot-experience in busy cities or the way you see it nudging through crowds of pedestrians and doing things that are very human-like and not just seeing stuck. I think these kind of things are really going to improve.

20:21public acceptance, which is on the system to operate in the messy cities we live in, and not require infrastructural behavior change. I think that's really important for adoption of embodied AI. You need to use mass market hardware, you need to be able to leverage availability of off-policy data, and you need to be able to operate as a drop-in to existing infrastructure, not requiring massive cap exchanges. I think if you get these things right, then embodied AI is a delightful deployment experience. How did you make the decision?

20:51to not build your own car, and to license your technology to other car companies? That was very clear to us from the beginning. I think the mindset I've always, and we've always had as a company, is itself driving as an AI problem. We want to work on the hardest problem first. So we started working on AI, and we started aggressively partnering on cloud, on car, on infrastructure, on insurance, and working with the best companies around us, so that we can focus on the critical path problem.

21:20And by always focusing on the hardest problem first, it's kind of not that a stumble into some glass ceiling that we had down the road. So I think it was that mindset. But then retrospectively, it's become important because I think it's a mistake to focus on one vehicle form factor because it's really unclear what will have product market fit. Is it a normal SUV like today? Is it a bi-directional vehicle we used to facing each other? Is it a two-seater electric vehicle?

21:50Are there other applications that really take off? And I think that's unclear. But at the end of the day, if you focus on one form factor, you're also going to have a small part of the market. And so we've tried to focus on how do we build a generalizable driver that can address everything? And then follow where the market is most advanced at any one stage. We started off with retrofit when there wasn't really anything for us to integrate into. Then during COVID, grocery and fleets started really skyrocketing in profits and so we started partnering with that sector then things changed but then automotive who would never speak to us to say this is unsafe it would never work all of a sudden started talking to us and building software to find vehicles so we get into great with them and now today we've got the excitement of robot taxi so we've kind of followed the market but been open to integrate into anything and the end state of course is consumer vehicles delivery robot taxi trucking

22:46non-automotive forms of robotics, we want to license everybody AI and make all of them intelligent, safe and possible. Where you sit right now, let's see, do you have a stack ranking of those different modalities in which one's most top of mind for you? Yeah, today it's all about consumer vehicles and robot taxis. It's about point-to-point driving in urban and highway situations with mass market consumer vehicle hardware. So this is an aligned product that we can now commercialize with great partners.

23:14So this is the real focus for us today. They can generate really extraordinary business. I mean, you look at the scale of whether it's data, revenue, number of vehicles, exposure, about 100 million cars produced each year. And people forget that. Often, they only think of robotaxies, but there's less than 10,000 robotaxies in the world. And so getting this natively integrated into consumer vehicles opens up tens of millions of vehicle opportunity.

23:42That's going to be far our scale rubber taxis in the short and medium term. Before long term, of course, every vehicle is capable of driverless operation, which is clearly the steady state of where we're going to go to given the safety and convenience and product benefits it can bring. What geographies do you plan to launch in first? I mean, we're really looking to make sure the system is global, but we're focused today on Europe, including the UK, as well as Japan and North America.

24:10So we have fleets and offices in London, Stuttgart, Tokyo and the Bay Area. We're driven around most cities across Europe and North America and growing in Japan. And those are the markets we're starting with given the customers we work with. Do you ever sense the way we'll go mass market press? It's those markets. I think in general, we're seeing faster movement in North America and Japan, closely followed by Europe.

24:37But of course, China is, unfortunately, it's very hard for us to work in China and vice versa. I won't get into geopolitics, but it's a bit of a shame that's where the world's at. But in China, this technology is moving so quickly, and the rate of innovation is very impressive. And so I think it's worth looking at if you haven't, from the automotive technology perspective. Another proof point that when this technology is deployed, consumers really love it, and it's now alongside...

25:04Price and safety, it's now a top three reason why people buy a car and try to know. Well, there's autonomy. So you had amazing results the AI500 Roadshow last year. I don't know if you want to talk about it. But you took one single AI model deployed all over the place with I think many of those geographies actually not having any prior training data on it. Re surprise that it worked so well. No, because that was a core thesis by building a general purpose AI model we could drive anywhere, but to actually see it.

25:33was huge validation. So yeah, we drove around the world. Some very cool anecdotes. When we drove in places like around the Arctic Triumph in Paris, have you driven around there? I have, yeah. Well, not driven, but I've gone there. It's chaos, right? There we went north of the Arctic Circle in Finland and Sweden, and tested driving in 22-hour darkness, snow, and things like that. We were even in Tokyo during a typhoon, where local trains and buses shut down, and actually that, we were giving a bunch of media drives.

26:02We had a day long of journalist drives, no disengagement, perfect or tiny the whole day and literally the heaviest rain I've ever seen. And our cameras are situated just behind the windshield, the front camera. The windshield's just constantly wiping water off it. But it drove reliably around Central Tokyo, which was quite remarkable. Less pedestrians on the road that day, but still busy traffic. But yeah, we've seen all of these things in that. And it demonstrated the first company to do something of that scale.

26:30and it demonstrated that we truly can generalize to any vehicle anywhere. Given that over half of these cities, we had no prior training data in and so it was true zero-shot generalization test at scale. That almost sounds like superhuman level performance. Yeah, I'd be careful with that. I mean, you and I, if we jump in a new city, you can get a rental car and go drive there, but...

26:57to claim superhuman. I mean, there's some great reports that I read one from Swiss Re, for example, with the insurer, where they measured the impact of an autonomy system of Waymo operating of their operational history and found that they had much fewer accidents and safety events than the equivalent human drivers. So, I think there's some robust evidence that's really showing autonomy can be superhuman. We have worked to do to demonstrate that, but we're making good progress there.

27:26But then you also look at, sadly, over 99% of accidents are caused to human error. And so even just making a self-driving experience that can never be distracted, drunk, impaired, this can see 360 degrees or at once make decisions over 10 times a second. And there is an opportunity there to drive accidents down to near zero. And that's what we want to see. But I think you've got to see scale deployment to really show that. And we want to go do that in the coming years.

27:54Are there situations that still make you nervous at this moment? What would be the kind of configuration that would put you on the edge of what the model can do? I think it's very hard to give general statements to that because where the models today tend to struggle is when you have a culmination of many failure modes. If it's, you know, dark, you have high speed traffic. You have adversarial behavior for someone else driving on the wrong side of the road or not yielding. Maybe it's bad weather.

28:25You know, it's when you have multiple confounding things, there may be ever sense of failure. When you get multiple confounding factors, that's when you tend to find challenges and failure modes. And so, we need to keep addressing that with better system resiliency or redundancy. We're building that with some next-generation vehicle architectures. We need more data to improve the generalization of risk assessment. And, yeah, making sure we deploy responsibly in areas that we're confident and meet.

28:55meet the required bar for safety. But we'll do that responsibly with our partners as we grow. And I think the other nice thing about being deployed in consumer vehicles in robot taxi, starting with supervised trials, is that we can incrementally grow this with communities and regulators to the instate where every vehicle is autonomous at a global scale. I remember there was a moment in sort of ethics in AI, maybe when AI was like, self-driving was hard, that people were asking questions.

29:24Maybe toy questions about should a car prioritize the safety of the passenger or the pedestrian? And you're kind of sidesteping those questions and maybe you're not inserting yourself. Is that true or do you need to push the data in certain ways to get certain types of ethical behavior that you want? So this is the trolley problem. We have to, we've got two bad decisions and you have to choose bad decision. I owe bad decision B. There's many ways of framing it.

29:51The practical reality is as an industry, and I think I speak on behalf of most AV companies for this, is that we need to design a system so there's always a third good option. It's called a minimal risk maneuver, which is if you're in a bad scenario, you try to, and you see risk, there's always a third option you can take, which is to minimize risk. And that typically involves reducing kinetic energy, maintaining a predictable course of action, and stopping or pulling over the vehicle or something like this.

30:20And we always need prepare to do that. If any systems fail, there needs to be a fail operational ability to do that. And so we design these systems always have a third good choice, not to be stuck between two bad choices. The second point is that to get into that two bad choices states, so many things have to go wrong to get to that point. And the probability of those kind of scenarios is so small. And even if you are in that position, the probability that you can actually accurately sense in those scenarios that those two things makes this more of a theoretical thought exercise that actual practical decision the industry needs to take. So we focus on, and the industry focuses on building these minimal risk maneuvers that take us away from those situations. Having said that, if you do get into that unlikely situation where you're in there, then of course the behavior will depend on how you bias it from your training data and the system that you design. And so there, of course, we need to be thoughtful with the principles

31:17and the knowledge that we distill into the system and the behavior that we generate. I mean, there are some good principles there around managing risk, making the system predictable and, of course, maximizing safety. But I would imagine that in your end-sent training, you're not seeing a lot of crashes. So if there are crashes, you're probably generating them in the training data somehow. So I think that the types of crashes that you're feeding into your system would give it a certain amount of...

31:48I'm really proud of our SAP Record actually. We've had no incidents since we started operating in 2018. So really fantastic SAP Record around the world. And essentially, there's a few things that are important here. Firstly, we don't just learn from the data of our vehicles driving. We have enormous data partnerships that give us quite general purpose knowledge.

32:13Unfortunately, there are still a lot of road accidents today, and we partner with dashcam providers, car companies that get data from their consumer vehicles, and sadly, there are still a lot of quite frightening scenarios that they experience. The good thing is, as we can use that data to learn from it, then secondly, we have our wall model, Gaia, that can simulate these scenarios that are too rare or unsafe to see in the real world, and make sure that we're robust to them through synthetic data and simulation too.

32:39And so, whether it's those strategies, ultimately, we need to build robustness to those events. And we still see some pretty bizarre stuff on the roads around the world. Unfortunately, some pretty unsafe behavior. And we can re-simulate or augment or modify these scenes to, of course, to build resiliency to them. And it's a constant game of flea learning and data iteration to make sure we're grinding up performance over time.

33:08So I remember when Tesla launched, I believe what they said was end-to-end train full set of driving. You really felt it, I think, when that version came out, at least in my Tesla, and one of the things that was really notable about it was, it really stopped at every stop time in San Francisco, which actually, San Francisco drivers don't really do. So it's kind of annoying if you're really following that rule to a T. And it made me think, actually, the average of everybody's driving is probably not what you want.

33:39How do you deal with that in your intent trading? It's a great question. I was just in Japan last week and there everyone drives about 20% of the speed limit and it opens up an interesting question around policy, regulation, liability, you know, if you're eyes off for a driverless system where the system's taking liability for driving, it should meet the road rules. But if you're a consumer, a lot of products today in a driver assistance setting allow you to set the set speed above the speed limit. And so it's a really interesting question. I hope that self-driving improves road safety, the point where you can actually increase speed limits because it's safe enough to do so. But we're probably some distance away from being able to have those kind of arguments. So today, the first thing to realize is when you are training a world model is you want to see as diverse data as possible. You want to see good driving, bad driving, you want to really understand the full spectrum of dynamic events in the world. Then when it comes to actual

34:40policy learning of how you drive, of course there, a data distribution that matches whichever customer of ours preferences, the OEM driving style behavior, the rules and policy of the country they're operating in becomes really important. And so we can separate those two data distributions. The first one you want to be as diverse as possible is largest possible. This is going to be a highly curated small.

35:06whether it's RLHF style feedback, whether it's expert demonstrations, there's many ways to do it to make sure that you drive in a very safe and considered way for that application. So you can see probably these data distributions. Actually, one thing I wanted to ask is someone is a kind of fellow entrepreneur is you started to think of CTO and another co-founder. We then left and he became CEO. I wondered what that experience was like. I guess it was earlier in the journey, but you wanted to share about that. Yeah, it was early on where kind of stage, I'm sure you remember where everyone does a bit of everything. And so as we were getting started, there was a bit of that sense. But yeah, for a number of reasons, it unfortunately made sense for my co-founder to leave the company and pursue other things. And we're still in good touch today. But that transition was my first real experience of change management communications with the company I think we're about.

36:0525 or 30 people at the time, just Ray's series A. But also, I guess, matched my experience over the last decade where every year has been a completely new challenge. OK, we started. I started the code repo and was coding a lot of the first demos. Then I started to work on hiring the team, finding commercial partners, growing investment.

36:33you know, now managing customers and managing through an executive team, where I'm so thrilled to be working with some people that are truly at the top of their game, their industry legends, you know, whether it's in science, engineering, product, you know, people and finance, commercial across the board with a truly well-class leaders. And so this will be a few things is the importance of building complementary teams, work with people that you can learn from, and also getting used to as soon as I get good at a skill, pretty quickly. No, growing it and having a new challenge as soon as I get good at something, it's like no longer the relevant skill for me to be working on because it's usually me plugging a gap for the organization and then growing the organization to absorb it and then there's a new challenge. I enjoy that though, I think that's part of the original theme, it's part of the adventure of building a company. And I think the next year for me is now I'm like,

37:32This phase is a new one where I'm now starting to live on a plane, work with global customers. It's going to be our first 10 years in. It's going to be our first product launch. We're going to start to have major public and regulatory exposure through that while keeping up strong and consistent execution velocity. So this is a completely new challenge that I'm really quite excited for and let's see what comes after that. If you can send a 30 second message of advice back to yourself when you're just starting out.

38:02What would you put in there? Oh man, I think. I feel like if I build wave again, I don't know. If you build things again, how much faster do you think you could build second time over? Oh, totally. It's like two or three X, right? Yeah. There's so many shortcuts or things that I was hesitant to doing or mistakes that we made that I think I could just give the tactical playbook.

38:29And if you had to do this still at a higher level because the only of the 32nd window to not just give the loss function recipe and Relationship I always thought the interesting question is I feel like wouldn't be a bit You know useless if if if we went back in time like 200 years with today's knowledge what would you do because I don't know what You've learned through weights and biases, but you've built on top of stuff that just were standing on the shoulders of giants that just did not exist two years ago. And so there's a bit about timing. In many ways, our product, if I've developed three times as fast, we may have been ahead of the industry where automotive do not have the right infrastructure or products to actually accept our product. So I think timing is everything. And in many ways self-driving has been a question of like staying alive and keeping the company capitalized while making progress on our

39:23the ex-generation approach, but yeah, if I was to distill it down, I've always had conviction in this approach that's not like, I'd say, you know, keep going, Alex, because I've been kind of resilient towards that one. So I think, yeah, I think I've been fine with how bold and brave I've been in building this. Honestly, the biggest thing has been, I think, yeah, I think, I think two things actually genuinely come to mind. One is about talent growth at Wave and I think I've seen so many great things about great people leading to fantastic people leading to brilliant people and growth of the expertise we have in the company that I think moving faster and harder and starting out of a PhD growing from London compared to the scale of the amazing people we can work with today.

40:27I think there's a growth journey there that I could have been more into than the second one is to try to get the experts earlier. Yeah, I think so. We started off with a generalist mindset really trying to figure things out and had to build it a lot ourselves, but I think maybe there's a mixed feeling on that one, but I think the better, the more...

40:56A bigger thing that comes to mind for me is I think I overindexed on algorithmian innovation compared to infrastructure. And like the big learning I have with deep learning is that it's 1% algorithms, 99% infra, whether it's cleanliness of your data, reliability and iteration speed of your learning loop. We're very happy adopters of weights and biases over the years, the acceleration you get with it, whether it's...

41:24introspection, measurement, evaluation tools. I think we've been on a journey there and under-invested on it. I think going harder and faster there, rather than the shiny, sexy algorithmic innovation is something that I wish we'd done earlier. The thing is, we were held back by a robot platform for many years, and as soon as we had a robust robot platform, it's like giving our AI, being blind and giving it glasses. All of a sudden, I could see and drive in a remarkable way. So, focusing on that.

41:53earlier and harder on building that maturity. I think it was quite key to bringing up performance, and we were too late in that. In simulation and evaluation, I think both large language models and body-day eye is a real challenge in evaluating the performance of models. It's an open-ended problem. It's not like you could have a clear metric around that today. And so developing these techniques would be on a journey with...

42:18Consistently tried different things from generative models to procedural game engines to nerf and gush and splitting to now generative world models. And throughout those times we haven't been afraid to forego some costs. And we spent millions of dollars, months of time building something and then just delete it, move on because we found something better. And so that culture in that journey has been important. And today I think we have something on the cusp of a major breakthrough there.

42:45that will really transform the industry, but that's another example where getting that right earlier, it's chicken and egg. If you figure out how to measure it, then you can just optimize against that target, but getting that right is something that we should have moved faster on. Okay, some quick questions to end with. When you took Bill Gates' Refishing Chips and yourself having a car, were you nervous or something would happen? And what did you talk about?

43:09Yeah, that was one of the first times we've got the system reliable enough. And actually, there were a few interventions on that drive. So the bill was still excited at the time. We talked about, it's interesting how when he was commercializing Microsoft and Windows, he had to get a bunch of design ones with PC OEMs. And in a similar way, we've had to get design ones with the biggest car companies in the world. So some interesting similar dynamics there that he was interested in.

43:39He was in London for a day, wanted to get fish and chips. One of his team went in to get the fish and chips handed it through the window to us in the car, and he just put down and sat there and wasn't going to eat it. I said, come on, but we've got to eat it. So we've got to, we can't not eat the fish and chips. So he goes, all right then, grabs a piece of fish and his hand, brings it up, breaks it in half, and just starts eating this battered piece of fish in the back side of the car with me, which I love. He clearly has a soft spot for some greasy fish and chips. Yeah, nice, nice.

44:06All right, what about when you built the first prototype in your rented house and you said you threw out the biggest parties of your life? What was that like? We invited everyone when you were in Cambridge to the house. We, you know, the living room was where we worked with a bunch of computers. The small bedroom was our board room with another bedroom full of servers and was like constantly 40 degrees Celsius because of that.

44:2760 GPUs that were running there. And we just did the house into a bit of a party house. We got a few musicians in the team. We were jamming. And that was a really, really fun vibe. Yeah, those days are very special. And, you know, I really enjoy the lasting memories as a team that were really in the trenches together during that time. But that was quite a big one. I don't know about you, but I feel like I'm most chasing that feeling of the early days and everyone's cranking together. Such a good feeling.

44:57I just visited our Japanese office last week, which is about, you know, 2030 people really feels like that. That's super exciting. And then, I think, yeah, I think there are pros and cons to operating a different level of scale. You know, today we're about 1,000 people globally, but the...

45:15Culture you've set at that scale really drives through where we are today. And I think it does require different skills to operate at this level of scale, but the impact you can generate is just tremendous. But holding on to disruptive innovation, not falling for the innovators dilemma, these kind of principles are still really, things we really fight for, and I think still hold true today away. Well, about some of these success, it's been a joy watching you guys grow. Thanks Lucas. We said it. Chris, yesterday.

45:42Thanks so much for listening to this episode of Great In Descent. Please stay tuned for future episodes.