Podcast Summary
Real estate manager's 360-degree perspective delivers local insights and global expertise: Principal Asset Management uses a comprehensive approach to identify compelling real estate investment opportunities by combining local insights and global perspectives. NVIDIA's dominance in the AI chip market and the high demand for advanced chips make investing in this sector an intriguing prospect.
Principal Asset Management, as a real estate manager, leverages a 360-degree perspective to deliver local insights and global expertise across various investment types. Their teams identify compelling opportunities by applying local insights and global perspectives. NVIDIA's stock has been a hot topic recently due to its dominance in the AI chip market. The demand for these advanced chips is high, and understanding how to acquire and utilize them effectively is crucial. Companies like NVIDIA are not just providing hardware but also services and software to create a holistic approach for customers. The market for these chips is seeing excitement from various sectors, including AI and video games, with crypto miners also pivoting into the space. Our upcoming guest, Brandon McBee, will provide further insights into this topic. Investing always comes with risk, and it's essential to stay informed and understand the market dynamics.
CoreWeave: Specializing in Highly Parallelizable Workloads for AI, Media, and Chemistry: CoreWeave, a cloud services provider, focuses on AI, media, and chemistry sectors with highly parallelizable workloads, raising over $400M, and operating the world's most performant GPU infrastructure at scale. Demand for this infrastructure is high due to AI software adoption, making infrastructure modernization a critical and ongoing challenge.
CoreWeave is a specialized cloud services provider focusing on highly parallelizable workloads for artificial intelligence (AI), media and entertainment, and computational chemistry sectors. They have raised over $400 million and operate the world's most performant GPU infrastructure at scale. The infrastructure they build is designed to be intelligent, like NVIDIA's smart data centers, with a focus on expanding throughput and communication between pieces of infrastructure. They use DGX reference specs and co-locate within tier 3 or tier 4 data centers. The demand for this infrastructure is high due to the rapid pace of AI software adoption, creating a significant supply-demand imbalance. This infrastructure modernization is in its first year of a decade-long process, making it a critical and ongoing challenge.
Data center capacity crunch due to shift to GPU-focused compute: The shift to GPU-focused compute in data centers has resulted in a capacity crunch due to increased power density, requiring more power and cooling, and underinvestment in colocation spaces.
The shift towards GPU-focused compute in data centers is causing challenges in efficiently utilizing existing colocation space due to the increased power density. This has led to a crunch in data center capacity, as the same amount of power can now only support a quarter of the previous data center footprint. The main issues stem from the need for more power and cooling to support the denser compute setup. This situation has arisen due to the hyperscalers' past underinvestment in colocation spaces, and the infrastructure change towards GPU compute, which is 4 times more power-dense than CPU compute. This rapid change has put a strain on the industry, leading to a scramble for suitable colocation spaces that can accommodate the power and cooling requirements of the new compute setup.
Specialized infrastructure needed for advanced computing technologies: To meet the demands of advanced computing technologies like GPUs, data centers require specialized infrastructure including high-speed connectivity solutions and vast amounts of fiber optic cabling.
The data center industry is facing a new challenge due to the increased power density requirements for advanced computing technologies like GPUs, which are used in training next-generation AI models. This has led to a need for specialized infrastructure, including high-speed connectivity solutions like NVIDIA's InfiniBand technology and vast amounts of fiber optic cabling. The traditional Ethernet connectivity used in legacy compute data centers is no longer sufficient for these high-performance computing needs. This issue has arisen quickly and is expected to persist for several quarters. Companies that focus on building and providing this type of infrastructure, like CoreWeave, are in a unique position to help clients access the necessary resources to develop innovative AI companies.
Partnership between American Express Business Gold Card and NVIDIA's CoreWeave: American Express Business Gold Card provides flexible spending for businesses, while NVIDIA's CoreWeave offers quick access to performant chip configurations for AI infrastructure demand, which is primarily for inference and growing rapidly
American Express Business Gold Card offers flexible spending capacity and annual statement credits for businesses, backed by the powerful backing of American Express. Meanwhile, CoreWeave, a service provided by NVIDIA, aims to empower end users to access compute in its most performant variant and at scale, quickly. This relationship between NVIDIA and CoreWeave is crucial as it allows for faster access to new generations of chips and their most performant configurations. The market for AI infrastructure is growing rapidly, with most funding going towards training new models. However, the demand for inference, where models spit out results, is even greater, leading to a chip access crunch in the training phase. The scale required to support AI infrastructure is mind-blowing, and the demand for inference is where the true market potential lies. NVIDIA's ability to quickly bring online new generations of chips in their most performant configurations has given them confidence in allocating infrastructure to CoreWeave, making it a valuable partner for NVIDIA.
The AI industry's demand for GPUs outpaces current infrastructure: The AI industry's demand for GPUs is growing rapidly, potentially requiring over a million GPUs for one company within two years. Current infrastructure, including hyperscalers' combined 500,000 GPUs, is not enough to meet this demand, leading to construction of new data centers and supply chain challenges.
The demand for GPUs in the AI industry is growing exponentially, with one company potentially requiring over a million GPUs within the first two years of a new generation's launch to meet inference demands. Currently, the hyperscalers combined have around 500,000 GPUs available globally, suggesting that one AI company with one model could consume the entire global footprint. This demand is not limited to just one company, and the current infrastructure is not enough to support it. The construction of new data centers and the supply of chips are expected to be major challenges for years to come. The hyperscalers, such as Amazon, Google, and Microsoft, are working to ramp up their infrastructure but are facing delays due to the different requirements of the new compute. The supply-demand imbalance is expected to last for a while, and the slower ability to scale infrastructure than what's being dictated by the adoption rate of AI software will contribute to it. While the H100 is currently the focus, other chips and companies, such as AMD, are also developing new technologies that could impact the market. Overall, the AI industry is facing a significant infrastructure challenge, and it will take time to address it.
NVIDIA's dominance in AI infrastructure market due to long-term ecosystem investment: NVIDIA's ecosystem of developers, engineers, and hardware-software integration makes it difficult for competitors to challenge its position in AI infrastructure market
The dominance of NVIDIA in the AI infrastructure market is largely due to the long-term investment in building an ecosystem around its CUDA software and GPUs. This ecosystem, which includes a large community of developers and engineers, has made it difficult for competitors like AMD to challenge NVIDIA's position. Moreover, the life cycle of chips used in AI infrastructure is such that the first few years are spent training models, while the next few years are spent serving those models. This means that the chips used for training are often the same as those used for serving, making it essential for companies to continue using NVIDIA chips to keep their models up to date. Despite the significant size and resources of hyperscale companies like Amazon, Google, and Microsoft, NVIDIA's advantage in both hardware and software components makes it challenging for them to displace NVIDIA from the market. However, it's important to note that the cost structures of these companies are vastly different, and it would take some time to evaluate how the market might shift in the future.
CoreWeave's Efficient Cloud Infrastructure: CoreWeave offers 40-60% more efficiency on a workload-adjusted basis compared to other hyperscalers due to unique hardware and software configurations.
While there may not be a single "silver bullet" solution, CoreWeave offers a significantly more efficient cloud infrastructure through a combination of hardware and software differentiators. This efficiency translates to about 40 to 60% more efficiency on a workload-adjusted basis compared to other hyperscalers. This advantage comes from the unique way CoreWeave configures its infrastructure and software. The analogy given is that just as it's challenging for a large company like Ford to fundamentally change its production process to create a new product like the Model Y, it's similarly difficult for hyperscalers to change their existing infrastructure and software to match CoreWeave's efficiency. CoreWeave has leveraged this advantage to establish a strong market presence and continue differentiating itself. The company's origins in Ethereum mining demonstrate its adaptability and ability to evolve in response to market trends.
From cryptocurrency mining to AI and other high-growth markets: The company pivoted from cryptocurrency mining to AI and other industries due to the lack of a sustainable competitive advantage in mining and the recognition that enterprise-grade GPUs were better suited for powering large-scale AI workloads.
The company pivoted from cryptocurrency mining to AI and other high-growth markets due to the lack of a sustainable competitive advantage in the cryptocurrency space. The company's founders, who had a background in commodity trading, saw an opportunity in the arbitrage of cryptocurrency mining but realized that the only way to gain an edge was by producing their own chips, which they were unwilling to do. Instead, they focused on the potential of GPU compute to power other industries, such as AI, media and entertainment, and computational chemistry. The complexity of running a cloud service provider and the limitations of retail-grade GPUs used for cryptocurrency mining led the company to shift its focus to enterprise-grade GPU chipsets. The pivot from cryptocurrency mining to AI and other industries was driven by the need to find a sustainable competitive advantage and the recognition that enterprise-grade GPUs were better suited for powering large-scale AI workloads.
Crypto Mining vs Enterprise AI Infrastructure: The infrastructure for cryptocurrency mining and enterprise AI workloads differ greatly, requiring specialized knowledge and planning for conversion and scaling.
The infrastructure used for cryptocurrency mining and enterprise AI workloads are fundamentally different, and the reuse of components and repurposing of data centers from one to the other is a significant challenge. The pricing and technology gap between retail and enterprise-grade chips is vast, with enterprise workloads requiring near-perfect uptime and low failure rates. Crypto mining infrastructure, on the other hand, is often housed in less stable data centers that can be highly interruptible, making them unsuitable for enterprise AI workloads. Moreover, planning for building and scaling infrastructure for AI workloads is a complex and rapidly evolving process. The market demand for AI capabilities is growing, and companies must anticipate future needs and secure the necessary resources to meet those demands. Building relationships with chip manufacturers and securing a consistent supply of high-performance chips is essential. The process of converting crypto mining data centers into enterprise-grade facilities is a daunting task that requires specialized knowledge and expertise. In summary, the infrastructure requirements for cryptocurrency mining and enterprise AI workloads are fundamentally different, and the challenges of repurposing components and data centers between the two are significant. Planning for the build and scale of infrastructure for AI workloads is a complex and evolving process that requires careful consideration and anticipation of future demands.
Impact of semiconductor market and supply chain disruptions on large-scale computing infrastructure: High demand and supply shortages for GPUs have increased lead times for building and deploying new clusters, with some projects facing delays into Q2 2023. Companies must navigate these challenges amidst other global supply chain disruptions and inflation, with power optimization becoming increasingly important.
The current state of the semiconductor market and global supply chain disruptions have significantly impacted the ability to quickly acquire and deploy large-scale computing infrastructure, particularly GPUs. Traditionally, organizations would purchase components from OEMs, who in turn would source GPUs from manufacturers like NVIDIA. However, due to high demand and supply shortages, it's no longer possible to simply call the OEM and request more compute chips. Instead, lead times for building and deploying new clusters have increased dramatically, with some projects now facing delays into Q2 2023. Companies are having to navigate these challenges amidst other global supply chain disruptions and inflation. The situation is particularly challenging for those in the software development and startup communities, who are accustomed to being able to quickly access cloud infrastructure as needed. Additionally, the need to optimize power usage in data centers to maximize energy efficiency and reduce costs is becoming increasingly important. Overall, the current state of the market presents a complex and evolving landscape for organizations seeking to acquire and deploy large-scale computing infrastructure.
NVIDIA's hardware ecosystem creates a significant moat in AI computing: NVIDIA's open-source software CUDA and high-touch partnerships make it difficult for competitors to switch due to increased computing power required for inference, uncertainty of monetization, and expertise needed to use NVIDIA's hardware effectively.
NVIDIA's hardware ecosystem, including their open-source software CUDA and high-touch partnerships with companies like CoreWeave, is creating a significant moat around their chip technology in the field of AI computing. This stickiness is due to the increased computing power required for inference, which makes it challenging for competitors to switch easily. However, the success of these companies in making money from AI products remains uncertain, and if monetization proves to be trickier than expected, it could impact the timeline for widespread adoption. Additionally, the knowledge and expertise required to use NVIDIA's hardware effectively adds to the moat. The question remains how quickly other hyperscalers can adapt to this ecosystem and challenge NVIDIA's dominance.
Matt Levine and Katie Greifeld Collaborate on New Podcast and Video Series: Popular financial newsletter writer Matt Levine teams up with Bloomberg TV host Katie Greifeld to create a new podcast and video series called 'Money Stuff.' Listeners can tune in every Friday on Apple Podcasts, Spotify, or other podcast platforms.
Popular financial newsletter writer Matt Levine is joining forces with Bloomberg TV host Katie Greifeld to create a new podcast and video series called "Money Stuff." Every Friday, they will explore the world of Wall Street finance and other intriguing topics that have made Levine's newsletter a must-read. Listeners can tune in to "Money Stuff" on Apple Podcasts, Spotify, or any other podcast platform. This collaboration promises to bring Levine's unique insights and analysis to a wider audience, making it an exciting development for finance enthusiasts and curious minds alike.