moore's law

Beneath the race to train and release more powerful AI models lies another race: a race by companies and nation-states to secure the hardware to make sure they win AI supremacy. 

Correction: The latest available Nvidia chip is the Hopper H100 GPU, which has 80 billion transistors. Since the first commercially available chip had four transistors, the Hopper actually has 20 billion times that number. Nvidia recently announced the Blackwell, which boasts 208 billion transistors - but it won’t ship until later this year.

RECOMMENDED MEDIA 

Chip War: The Fight For the World’s Most Critical Technology by Chris Miller

To make sense of the current state of politics, economics, and technology, we must first understand the vital role played by chips

Gordon Moore Biography & Facts

Gordon Moore, the Intel co-founder behind Moore's Law, passed away in March of 2023

AI’s most popular chipmaker Nvidia is trying to use AI to design chips faster

Nvidia's GPUs are in high demand - and the company is using AI to accelerate chip production

RECOMMENDED YUA EPISODES

Future-proofing Democracy In the Age of AI with Audrey Tang

How Will AI Affect the 2024 Elections? with Renee DiResta and Carl Miller

The AI ‘Race’: China vs. the US with Jeffrey Ding and Karen Hao

Protecting Our Freedom of Thought with Nita Farahany

Your Undivided Attention is produced by the Center for Humane Technology. Follow us on Twitter: @HumaneTech_

In 2011, Marc Andreessen said, “software is eating the world.” And in the last year, we’ve seen a new wave of generative AI, with some apps becoming some of the most swiftly adopted software products of all time.

So if software is becoming more important than ever, hardware is following suit. In this episode – the first in our three-part series – we explore the terminology and technology that is now the backbone of the AI models taking the world by storm. We’ll explore what GPUs are, how they work, the key players like Nvidia competing for chip dominance, and also… whether Moore’s Law is dead?

Look out for the rest of our series, where we dive even deeper; covering supply and demand mechanics, including why we can’t just “print” our way out of a shortage, how founders get access to inventory, whether they should own or rent, where open source plays a role, and of course… how much all of this truly costs!

Topics Covered:

00:00 – AI terminology and technology

03:44 - Chips, semiconductors, servers, and compute

04:48 - CPUs and GPUs

06:07 - Future architecture and performance

07:01 - The hardware ecosystem

09:05 - Software optimizations

12:23 - What do we expect for the future?

14:35 - Upcoming episodes on market  and cost

Resources: 

• Find Guido on LinkedIn: https://www.linkedin.com/in/appenz/
• Find Guido on Twitter: https://twitter.com/appenz

Stay Updated: 

Find a16z on Twitter: https://twitter.com/a16z

Find a16z on LinkedIn: https://www.linkedin.com/company/a16z

Subscribe on your favorite podcast app: https://a16z.simplecast.com/

Follow our host: https://twitter.com/stephsmithio

Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.