Podcast Summary
Autonomous vehicles industry progress: Waymo, a Google spin-off, leads the autonomous vehicle industry with over 7 million driverless miles, offering public rides in SF and safer/more reliable vehicles than human drivers. Dimitri Dolgov discusses industry potential, embodied AI, simulations, and training data.
The autonomous vehicle industry has made significant strides in recent years, with companies like Waymo leading the way. Waymo, which was founded as Google's self-driving car project, has clocked in over 7 million driverless miles by the end of 2023. The company began offering autonomous rides in San Francisco in 2022, following its expansion to the public in Phoenix in 2020. Waymo's vehicles have been shown to be safer and more reliable than human drivers. Dimitri Dolgov, co-CEO of Waymo, shares his unique perspective on the industry, having been an early pioneer in self-driving cars and having worked on the DARPA Grand Challenge at Stanford. Dolgov discusses the potential of embodied AI, the value of simulations and building training data, and his approach to leading a company focused on solving some of the world's hardest problems. The conversation between Dolgov and A16Z General Partner David George provides insights into the current state and future direction of the autonomous vehicle industry.
AI progression in self-driving technology: AI has evolved in self-driving technology through various stages including traditional machine learning, CNNs, and Transformers, each building upon the previous advancements to improve object detection, language understanding, and overall autonomous capabilities
The development of self-driving technology can be understood as a layered progression of advances in Artificial Intelligence (AI), with each new breakthrough building upon the foundational work of its predecessors. AI has been a crucial component of self-driving cars since their inception, with early systems relying on traditional machine learning techniques and hand-engineered features. A pivotal moment came in the early 2010s with the emergence of Convolutional Neural Networks (CNNs), which revolutionized computer vision and had a significant impact on the field of autonomous vehicles. CNNs enabled more sophisticated object detection and classification from camera, LiDAR, and radar data, making it possible to interpret the environment around the car more effectively. Another major breakthrough occurred around 2017 with the advent of Transformers, which had a profound impact on language understanding and natural language processing. For self-driving cars, this meant the ability to process and understand language in new and more nuanced ways, allowing for more advanced decision-making and planning capabilities. In essence, the evolution of AI in self-driving technology can be seen as a series of building blocks, with each new advance building upon the foundational work of its predecessors. From early machine learning techniques to the more recent developments in generative AI, each step has brought us closer to the goal of creating truly autonomous vehicles.
Simulation Technology in Autonomous Driving: Simulation technology provides a realistic environment for developing and evaluating advanced AI systems, enabling the exploration of various scenarios, generation of synthetic data, and examination of long-tail events, with accurate sensor perception and realistic behaviors of dynamic actors.
Simulation technology plays a crucial role in the development and evaluation of advanced AI systems, particularly in the field of autonomous driving. The use of simulation allows for the exploration of various scenarios, the evaluation of new systems, and the generation of synthetic data. The importance of simulation lies in its ability to provide a realistic environment that mirrors the real world, with accurate sensor perception and the realistic behaviors of dynamic actors. This realism is essential to ensure that the AI system's performance in simulation aligns with its performance in the physical world. Furthermore, simulation enables the examination of long-tail events and the modification of scenarios to create thousands or even tens of thousands of variations. It also enables the evaluation and training on events that have never occurred in the physical world. The realistic and quantifiable nature of simulation technology is vital to the advancement of AI systems, particularly in the domain of autonomous driving.
Autonomous driving simulation: Realistic simulation software is crucial for building high-quality autonomous driving systems, requiring accurate modeling of various road users and real-world conditions. Waymo's success is due to both real-world and simulation experience, with large models and end-to-end AI playing a significant role but continuous advancement necessary.
Building a high-quality autonomous driving system requires a realistic simulator, which in turn necessitates modeling of various road users like pedestrians, cyclists, and drivers. The simulation software should accurately mimic real-world conditions and be usable for creating variable scenarios. Waymo, for instance, has driven tens of millions of miles in the physical world and more than 15 million miles in full autonomy, but they've also accumulated tens of billions of miles of simulation experience. Scaling laws, such as model size and data quality, play a crucial role in autonomous driving. Large models with ample data help generalize better, but they need to be distilled for onboard systems. The DARPA and recent AI-based approaches both have merits, with the former focusing on rules-based learning and the latter on end-to-end AI. Waymo combines these approaches with big models, end-to-end models, generative AI, and VLMs. However, these techniques provide a significant boost but are not enough on their own. Continuous application and pushing forward of these state-of-the-art techniques in the autonomous driving domain is essential.
Scaling up and improving end-to-end models: End-to-end models and foundation models are necessary for autonomous driving systems, but they require scaling up, real-world understanding, and a seamless customer experience. Industry focus is on improving simulation tech, adding capabilities, collecting more data, and prioritizing user feedback.
End-to-end models and foundation models are essential building blocks for autonomous driving systems, but they only form the foundation. The real challenge lies in scaling up these models, improving their understanding of the real world, and ensuring a seamless customer experience. The speaker emphasized that while end-to-end models and foundation models are crucial for driving from sensor input to actuation, they are not sufficient for achieving full autonomy. There is a significant amount of work required to train, evaluate, and architect these models. Additionally, these models have limitations, such as issues with understanding complex goal-oriented planning, hallucinations, and weaknesses in 3D spatial understanding. To address these challenges, the industry is focusing on improving simulation technology, adding additional capabilities to models, and collecting more data to enable better understanding of the real world. Feedback from users is also essential in creating a magical and seamless experience, with a focus on improving pickup and drop-off locations. The speaker also mentioned that Waymo is currently operating in various conditions, including different weather types, and is continuously driving in multiple locations. However, to make the service even better, there is a need for improvements in scale and quality, with a focus on user feedback and creating a delightful experience from start to finish.
Autonomous vehicle optimization: Autonomous vehicles have reduced accidents by 3.5x compared to human drivers, but optimizing pick-up and drop-off locations in complex urban environments remains a challenge due to factors like garage doors, other vehicles, and ethical considerations. Regulators hold autonomous vehicles to a higher safety standard.
While autonomous driving technology has made significant strides with 15 million miles driven and a 3.5 times reduction in accidents compared to human drivers, the challenge of optimizing pick-up and drop-off locations in complex urban environments is an intricate problem. This issue involves considering factors such as garage doors opening, other vehicles, and safe and convenient locations for users. Regulators and ethical considerations also come into play regarding the acceptable level of safety for autonomous vehicles. The statistics and studies show that autonomous vehicles have a strong safety record, reducing property damage collisions by 76% and eliminating claims related to bodily injury. However, there are still instances where autonomous vehicles may be involved in collisions, and the technology is held to a higher standard. Overall, the goal is to continuously improve safety and provide a seamless, pleasant customer experience.
Waymo's sensors: Waymo uses multiple sensors for redundancy and enhanced capabilities, aiming for a generalizable driver in ride-hailing, deliveries, and trucking, with safety and accessibility as top priorities.
Waymo's approach to autonomous driving involves using a combination of sensors, including cameras, lidars, and radars, to ensure redundancy and enhance the capabilities of the system. Waymo's ultimate goal is to build a generalizable driver that can be deployed in various commercial applications, such as ride-hailing, deliveries, and trucking. The company is exploring different commercial structures, including partnerships and its own app, to make the technology accessible to as many people as possible while ensuring safety and gradual deployment. Waymo's high standard for autonomous driving is to compare the system's performance to that of a human driver and continually evaluate and improve it through various validation methodologies. The company's long-term vision is to make transportation safer and more accessible by deploying its autonomous technology in various applications and commercial structures.
Autonomous vehicle challenges: Achieving full autonomy for autonomous vehicles is more complex than anticipated due to the physical world's messiness, uncertainty, high safety requirements, and real-time decision making.
While there have been significant advancements and cost reductions in LiDAR, camera, and radar sensors for autonomous vehicles, achieving full autonomy is proving to be much more challenging than anticipated. The physical world's messiness, uncertainty, and the high safety requirements make the bar for autonomous driving systems extremely high. Additionally, real-time decision making is crucial, and there's no room for errors due to the potential serious consequences. Despite the challenges, progress is being made, and the reward is a safer driving experience compared to a human driver. Early on, Waymo encountered humbling experiences, such as a specific route from Mountain View to San Francisco, but these setbacks have not deterred them from their goal of fully autonomous vehicles. The journey may be long and costly, but the potential benefits make it worthwhile.
Finding meaningful problems: To start a successful venture or project in AI, find a problem that truly matters and is challenging. Keep building and don't look back, every small step brings progress.
Starting a new venture or project, especially in the field of AI, requires finding a problem that truly matters and is challenging. The speaker shares his experience of navigating through complexities while self-driving from San Francisco to Lombard Street in 2009. He encountered various obstacles, but with determination and passion, they managed to overcome them. For those seeking their first job, the speaker advises finding a problem that makes a difference in the world and is personally meaningful. It may be difficult, but the journey is worth it. Keep building and don't look back. Remember, every small step brings progress. As a listener, if you enjoyed this podcast, please consider sharing your favorite episode at ratethispodcast.com/A16Z. Your feedback is appreciated.