Logo

    Patrick McKenzie - How a Discord Server Saved Thousands of Lives

    enJuly 24, 2024
    What was the purpose of the Vaccinate CA website?
    How did political bickering affect vaccine distribution?
    Why was there a reluctance to invest in software solutions?
    What lesson was learned for future emergencies?
    How can the private sector and government collaborate more effectively?

    Podcast Summary

    • Software responsibility during crisesThe federal government's reluctance to invest in software solutions during crises can lead to missed opportunities to effectively address logistical challenges and potentially save lives.

      During the COVID-19 pandemic, a group of tech-savvy individuals came together on a Discord server to create Vaccinate CA, a website that centralized vaccine location information in California. The initiative grew into the public-private partnership that became the clearinghouse for vaccine location information for the United States. However, the project came about due to a lack of government initiative and institutional responsibility in addressing the logistical challenges of vaccine distribution. The federal government had learned from past experiences that software problems could potentially damage a president's legacy, leading to a reluctance to invest in software solutions. This abdication of software responsibility resulted in a missed opportunity to effectively address the vaccine distribution crisis and potentially save lives. The lack of accountability and seriousness across trusted institutions during the pandemic was a missed opportunity for the United States to win a crucial battle against the virus.

    • Vaccine inventory tracking failureThe tech industry prioritized self-preservation over helping manage the COVID-19 vaccine distribution process due to fear of political backlash and potential consequences of appearing more competent than the government.

      During the COVID-19 vaccine distribution process, the lack of a unified platform for tracking vaccine inventory across various pharmacies and supply chains was due to a combination of government procurement project failures and tech industry self-preservation. The tech industry, including powerful tech companies, chose to prioritize avoiding political backlash over building a solution to help manage the vaccine distribution. This decision was driven by the fear of being held responsible for the entire vaccination effort and the potential consequences of appearing more competent than the government. Despite the incompetence of the native infrastructure for dealing with public health emergencies, there is a need to recognize and address the underperformance of institutions, particularly health departments, to improve our response to future crises.

    • COVID-19 vaccine distribution politicsPolitical prioritization in vaccine distribution led to misallocation, frustration, and bureaucratic complications, rather than focusing on medical necessity and operational efficiency.

      The COVID-19 vaccine distribution in the United States was complicated by a political tiering system that prioritized certain groups based on political power rather than medical necessity. This system, which varied from place to place and changed frequently, led to misprioritization of the vaccine and a bureaucratic process that was difficult for some, particularly older adults, to navigate. The consequences of this were not only the misallocation of vaccines but also the frustration and enragement of those who felt that human lives were being optimized for political gain. Despite the incredible achievement of producing and distributing the vaccine in record time, the process could have gone more smoothly if the focus had been on medical necessity and operational efficiency rather than societal equity and political considerations.

    • COVID-19 vaccine distribution prioritizationPolitical bickering overshadowed a sensible strategy for vaccine distribution, leading to inefficiencies and missed opportunities. In future emergencies, a more agile and collaborative approach is necessary to leverage expertise and overcome challenges effectively.

      During the COVID-19 vaccine distribution, the prioritization system was overshadowed by political bickering, leading to inefficiencies and a lack of clear solutions. The scarcity of files required a sensible strategy for getting vaccines into people's arms, but instead, there was a focus on who should receive the files first. This issue was not unique to the United States, as many countries faced similar challenges. A functioning system would have identified experts in private industry or civil society and given them the resources and authority to address the problem. The absence of this approach led to missed opportunities and a slower response. The lesson learned is that in future emergencies, it's crucial to adopt a more agile and collaborative approach, leveraging the expertise of various sectors to overcome challenges effectively.

    • Vaccine distribution policiesGovernors' conflicting statements and bureaucratic underperformance led to confusion and logistical challenges during the early stages of COVID-19 vaccine distribution, affecting societal trust and access for underserved communities

      During the early stages of the COVID-19 vaccine rollout in California, there was a significant discrepancy between the allocated vaccines and those actually administered. Governors made confident statements about vaccine distribution policies, only to change course later, leading to confusion for regulatory departments and pharmacies. Pharmacies, which deliver a majority of medications in the US, had to balance societal trust and business results, resulting in some individual acts of defiance against the official policy. Despite the trillions of dollars at stake and the urgency of the situation, the fundraising process for addressing vaccine distribution issues was hindered by bureaucracy and the underperformance of certain industries. The logistical problems persisted beyond vaccine hesitancy, affecting underserved communities and those with limited access to transportation.

    • Decisive action during a crisisBold and decisive action, including investing personal resources, can make a significant difference during a crisis and help move projects forward, even when faced with funding challenges.

      During a crisis, bold and decisive action can make a significant difference. The speaker shares an experience of funding a team to address a societal problem, but faced challenges in securing additional funds. He reflects on the situation and acknowledges that he could have approached fundraising differently, but ultimately, his ability to make quick decisions and invest his own resources played a crucial role in moving the project forward. The speaker also emphasizes the importance of choosing winning and prioritizing human lives over reducing waste and corruption in the context of charitable giving. Despite the challenges, the speaker remains optimistic and encourages a mindset of taking calculated risks and acting decisively when it matters most.

    • Government-Private Sector CollaborationDuring crises, effective collaboration between the government and private sector is crucial for addressing complex issues. Tech industry leaders should engage with the government, understand its complexities, and work together to find solutions.

      The private sector and government must work together more effectively to address complex issues, particularly during crises. The government's competence issues should not be ignored, but criticizing them unproductively can hinder progress. Instead, tech industry leaders should engage with the government, understand its complexities, and collaborate to find solutions. The COVID-19 vaccine distribution effort serves as a prime example of the importance of this collaboration. It's crucial for society to rely on competent individuals and organizations to take action when needed, rather than relying on the government alone. The tech industry has a significant role to play in this, and it's essential to recognize the potential for positive impact when opportunities arise.

    • Crypto Justification DebateThe crypto market's value justification is debated, with critics arguing resources could be better spent and advocates pointing to financial inclusion and new applications, drawing comparisons to the dot-com bubble.

      While the crypto market has seen massive investments totaling trillions of dollars, the question remains whether the value produced justifies the resources staked. Critics argue that the resources could have been better spent on infrastructure or other productive enterprises. However, crypto advocates point to its potential for enabling financial inclusion and the development of new applications, drawing comparisons to the dot-com bubble and its eventual impact. Ultimately, the true value of crypto will be determined by its ability to deliver meaningful applications and use cases that justify the resources invested. Until then, the debate continues.

    • Crypto privacy vs government surveillanceWhile crypto offers potential privacy benefits, extensive financial surveillance practices in traditional banking systems raise concerns about potential privacy invasions on a massive scale. Crypto advocates prioritize financial gains over privacy, leading to a complex debate about technology, privacy, and government regulation.

      While some argue that cryptocurrencies serve as a valuable hedge against potential government overreach and intrusive financial surveillance, others caution that the privacy promises of crypto may not hold up under closer scrutiny. The speaker acknowledges the existence of extensive financial surveillance practices in traditional banking systems, which have grown more sophisticated over the years, raising concerns about potential privacy invasions on a massive scale. Despite these concerns, crypto advocates often prioritize the potential for financial gains over privacy, leading to a complex and nuanced debate about the role of technology, privacy, and government regulation in our financial systems. Ultimately, the speaker calls for a more direct and open conversation about the implications of technology and its relationship to government control, urging us to consider the potential consequences of allowing powerful technological tools to be wielded by the state without proper oversight or accountability.

    • Tech industry ethicsThe tech industry must balance power and ethics, avoiding manipulation for power-seeking purposes, engaging with government for policy changes, and advocating for consensus on privacy and free speech concerns.

      The use of technology as a tool for bypassing legal and ethical concerns, even with the involvement of non-governmental organizations, can lead to unconstitutional, immoral, and illegal actions. The tech industry, despite having the power to achieve results, must be cautious about being manipulated for power-seeking purposes. The current situation illustrates the lack of accountability in institutions and the need for a consensus on addressing privacy and free speech concerns. A possible solution is not just changing laws, but also engaging more with government and advocating for policy changes. It's crucial for the tech industry to have a balanced risk tolerance and not shy away from speaking out when necessary. Ultimately, it's a political decision about how we construct our democracy, and we should strive for optimism and finding solutions that work for everyone.

    • Complexity of financeCulture should not be oversimplified as the sole explanation for differences in finance, instead, focus on underlying incentives and investigate physical facts for accurate understanding.

      While culture may play a role in explaining certain differences between countries or organizations, it's important not to oversimplify complex issues by attributing them solely to culture. Instead, we should look for underlying incentives and investigate the physical facts to get a more accurate understanding of situations. For instance, in finance, the minutiae of banking procedures can reveal crucial information that may be overlooked by the average reporter or investigator. By understanding how institutions and society interact with the physical world, we can uncover hidden truths and expose fraud. It's essential to be thorough and persistent in our investigations, as even the most knowledgeable individuals can sometimes miss critical details. Short sellers, for example, may fail to detect certain frauds due to the difficulty of finding an instrument and another side of the trade. Ultimately, a combination of diligent research, critical thinking, and a willingness to delve into the details can help us uncover the truth and make informed decisions.

    • Money Laundering OperationsSam Bankman-Fried and Alameda Research utilized sophisticated methods to launder tens of billions of dollars, exploiting regulatory loopholes and political connections. However, money laundering can take various forms, and the key difference lies in the origin of the money being laundered.

      Sam Bankman-Fried (SPF) and his organization, Alameda Research, were extremely sophisticated in their money laundering operations, which involved moving tens of billions of dollars around the financial system on behalf of Tether. They were able to do this largely under the full color of law, exploiting regulatory loopholes and political connections. SPF's ability to subordinate Japanese banks and manipulate regulatory licenses was a testament to their sophistication. However, it's important to note that money laundering can take many forms, and what SPF did is not so different from how some real estate transactions work at large scales. The difference lies in the origin of the money being laundered. Another interesting point discussed was the role of culture and personality in negotiations and earnings. Some cultures and personality types are more inclined to negotiate aggressively, leading to disparate outcomes in the business world.

    • Tech industry barriersThe tech industry's evolution has made it harder for young, less resourced individuals to rise to prominence, but the speaker remains optimistic about their potential and the importance of finding a balance between productivity and personal fulfillment.

      The tech industry has evolved, and the barriers to entry have increased, making it more challenging for young, less resourced individuals to rise to prominence. The speaker believes that this is partially due to the fact that established players in the industry do not immediately seize all the value in new frontiers. He also mentions concerns about the impact of certain tech products on people's productivity and happiness. Despite these concerns, the speaker remains optimistic about the potential of young people and continues to be a prolific writer and thinker. He acknowledges his own past time spent on video games and recognizes the potential for video games to be a distraction from true goals, but also sees value in intellectual pursuits and the importance of making progress towards one's values. The speaker's reflections highlight the complexities of navigating the tech industry and the importance of finding a balance between productivity and personal fulfillment.

    • Finance industry's incentive systemThe finance industry's incentive system attracts top minds due to its ability to turn intelligence into financial success, making good writing a valuable skill for good thinkers.

      The concentration of writing talent in finance may be due to the industry's ability to turn intelligence into financial success, creating an incentive system that attracts top minds. Uranium hexafluoride may be a difficult substance to work with, but good writing is a valuable skill that good thinkers possess. In the past, some individuals may have mistakenly believed that certain industries, like software or journalism, would experience a decrease in demand for labor and therefore, fewer opportunities for employment. However, it's essential to have rigor and not blindly follow assumptions, especially when making life-altering decisions. The importance of trusting institutions less and relying more on systemically sound reasoning patterns cannot be overstated. Founders should focus on making their businesses successful instead of getting distracted by peripheral activities. As the president of the United States, ensuring accountability and preparedness for future crises would involve a combination of personnel changes and implementing cultural practices from industries like tech, such as a relentless focus on innovation and problem-solving.

    • Blameless post-mortemsBlameless post-mortems encourage improvement by focusing on facts, learning, and accountability without blame, leading to a culture of continuous improvement and growth.

      The concept of blameless post-mortems can be beneficial for society. Instead of focusing on blame, we should strive for a dispassionate record of what happened, how it led to the outcome, and what could be done better next time. This approach can lead to improvement without causing unnecessary fear of accountability. Individuals should be encouraged to document their experiences truthfully and comprehensively. This is the first step towards learning from past mistakes and avoiding repeating them. Additionally, it's important to remember that everyone goes through various stages in life, and it's essential to find a balance between personal and professional commitments.

    Recent Episodes from Dwarkesh Podcast

    David Reich - How One Small Tribe Conquered the World 70,000 Years Ago

    David Reich - How One Small Tribe Conquered the World 70,000 Years Ago

    I had no idea how wild human history was before chatting with the geneticist of ancient DNA David Reich.

    Human history has been again and again a story of one group figuring ‘something’ out, and then basically wiping everyone else out.

    From the tribe of 1k-10k modern humans who killed off all the other human species 70,000 years ago; to the Yamnaya horse nomads 5,000 years ago who killed off 90+% of (then) Europeans and also destroyed the Indus Valley.

    So much of what we thought we knew about human history is turning out to be wrong, from the ‘Out of Africa’ theory to the evolution of language, and this is all thanks to the research from David Reich’s lab.

    Buy David Reich’s fascinating book, Who We Are How We Got Here.

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Follow me on Twitter for updates on future episodes.

    Sponsor

    This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue.

    If you’re interested in advertising on the podcast, check out this page.

    Timestamps

    (00:00:00) – Archaic and modern humans gene flow

    (00:20:24) – How early modern humans dominated the world

    (00:39:59) – How bubonic plague rewrote history

    (00:50:03) – Was agriculture terrible for humans?

    (00:59:28) – Yamnaya expansion and how populations collide

    (01:15:39) – “Lost civilizations” and our Neanderthal ancestry

    (01:31:32) – The DNA Challenge

    (01:41:38) – David’s career: the genetic vocation



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enAugust 29, 2024

    Joe Carlsmith - Otherness and control in the age of AGI

    Joe Carlsmith - Otherness and control in the age of AGI

    Chatted with Joe Carlsmith about whether we can trust power/techno-capital, how to not end up like Stalin in our urge to control the future, gentleness towards the artificial Other, and much more.

    Check out Joe's sequence on Otherness and Control in the Age of AGI here.

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

    Sponsors:

    - Bland.ai is an AI agent that automates phone calls in any language, 24/7. Their technology uses "conversational pathways" for accurate, versatile communication across sales, operations, and customer support. You can try Bland yourself by calling 415-549-9654. Enterprises can get exclusive access to their advanced model at bland.ai/dwarkesh.

    - Stripe is financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue.

    If you’re interested in advertising on the podcast, check out this page.

    Timestamps:

    (00:00:00) - Understanding the Basic Alignment Story

    (00:44:04) - Monkeys Inventing Humans

    (00:46:43) - Nietzsche, C.S. Lewis, and AI

    (1:22:51) - How should we treat AIs

    (1:52:33) - Balancing Being a Humanist and a Scholar

    (2:05:02) - Explore exploit tradeoffs and AI



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enAugust 22, 2024

    Patrick McKenzie - How a Discord Server Saved Thousands of Lives

    Patrick McKenzie - How a Discord Server Saved Thousands of Lives

    I talked with Patrick McKenzie (known online as patio11) about how a small team he ran over a Discord server got vaccines into Americans' arms: A story of broken incentives, outrageous incompetence, and how a few individuals with high agency saved 1000s of lives.

    Enjoy!

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Follow me on Twitter for updates on future episodes.

    Sponsor

    This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue.

    Timestamps

    (00:00:00) – Why hackers on Discord had to save thousands of lives

    (00:17:26) – How politics crippled vaccine distribution

    (00:38:19) – Fundraising for VaccinateCA

    (00:51:09) – Why tech needs to understand how government works

    (00:58:58) – What is crypto good for?

    (01:13:07) – How the US government leverages big tech to violate rights

    (01:24:36) – Can the US have nice things like Japan?

    (01:26:41) – Financial plumbing & money laundering: a how-not-to guide

    (01:37:42) – Maximizing your value: why some people negotiate better

    (01:42:14) – Are young people too busy playing Factorio to found startups?

    (01:57:30) – The need for a post-mortem



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enJuly 24, 2024

    Tony Blair - Life of a PM, The Deep State, Lee Kuan Yew, & AI's 1914 Moment

    Tony Blair - Life of a PM, The Deep State, Lee Kuan Yew, & AI's 1914 Moment

    I chatted with Tony Blair about:

    - What he learned from Lee Kuan Yew

    - Intelligence agencies track record on Iraq & Ukraine

    - What he tells the dozens of world leaders who come seek advice from him

    - How much of a PM’s time is actually spent governing

    - What will AI’s July 1914 moment look like from inside the Cabinet?

    Enjoy!

    Watch the video on YouTube. Read the full transcript here.

    Follow me on Twitter for updates on future episodes.

    Sponsors

    - Prelude Security is the world’s leading cyber threat management automation platform. Prelude Detect quickly transforms threat intelligence into validated protections so organizations can know with certainty that their defenses will protect them against the latest threats. Prelude is backed by Sequoia Capital, Insight Partners, The MITRE Corporation, CrowdStrike, and other leading investors. Learn more here.

    - This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue.

    If you’re interested in advertising on the podcast, check out this page.

    Timestamps

    (00:00:00) – A prime minister’s constraints

    (00:04:12) – CEOs vs. politicians

    (00:10:31) – COVID, AI, & how government deals with crisis

    (00:21:24) – Learning from Lee Kuan Yew

    (00:27:37) – Foreign policy & intelligence

    (00:31:12) – How much leadership actually matters

    (00:35:34) – Private vs. public tech

    (00:39:14) – Advising global leaders

    (00:46:45) – The unipolar moment in the 90s



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enJune 26, 2024

    Francois Chollet, Mike Knoop - LLMs won’t lead to AGI - $1,000,000 Prize to find true solution

    Francois Chollet, Mike Knoop - LLMs won’t lead to AGI - $1,000,000 Prize to find true solution

    Here is my conversation with Francois Chollet and Mike Knoop on the $1 million ARC-AGI Prize they're launching today.

    I did a bunch of socratic grilling throughout, but Francois’s arguments about why LLMs won’t lead to AGI are very interesting and worth thinking through.

    It was really fun discussing/debating the cruxes. Enjoy!

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Timestamps

    (00:00:00) – The ARC benchmark

    (00:11:10) – Why LLMs struggle with ARC

    (00:19:00) – Skill vs intelligence

    (00:27:55) - Do we need “AGI” to automate most jobs?

    (00:48:28) – Future of AI progress: deep learning + program synthesis

    (01:00:40) – How Mike Knoop got nerd-sniped by ARC

    (01:08:37) – Million $ ARC Prize

    (01:10:33) – Resisting benchmark saturation

    (01:18:08) – ARC scores on frontier vs open source models

    (01:26:19) – Possible solutions to ARC Prize



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enJune 11, 2024

    Leopold Aschenbrenner - China/US Super Intelligence Race, 2027 AGI, & The Return of History

    Leopold Aschenbrenner - China/US Super Intelligence Race, 2027 AGI, & The Return of History

    Chatted with my friend Leopold Aschenbrenner on the trillion dollar nationalized cluster, CCP espionage at AI labs, how unhobblings and scaling can lead to 2027 AGI, dangers of outsourcing clusters to Middle East, leaving OpenAI, and situational awareness.

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Follow me on Twitter for updates on future episodes. Follow Leopold on Twitter.

    Timestamps

    (00:00:00) – The trillion-dollar cluster and unhobbling

    (00:20:31) – AI 2028: The return of history

    (00:40:26) – Espionage & American AI superiority

    (01:08:20) – Geopolitical implications of AI

    (01:31:23) – State-led vs. private-led AI

    (02:12:23) – Becoming Valedictorian of Columbia at 19

    (02:30:35) – What happened at OpenAI

    (02:45:11) – Accelerating AI research progress

    (03:25:58) – Alignment

    (03:41:26) – On Germany, and understanding foreign perspectives

    (03:57:04) – Dwarkesh’s immigration story and path to the podcast

    (04:07:58) – Launching an AGI hedge fund

    (04:19:14) – Lessons from WWII

    (04:29:08) – Coda: Frederick the Great



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enJune 04, 2024

    John Schulman (OpenAI Cofounder) - Reasoning, RLHF, & Plan for 2027 AGI

    John Schulman (OpenAI Cofounder) - Reasoning, RLHF, & Plan for 2027 AGI

    Chatted with John Schulman (cofounded OpenAI and led ChatGPT creation) on how posttraining tames the shoggoth, and the nature of the progress to come...

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

    Timestamps

    (00:00:00) - Pre-training, post-training, and future capabilities

    (00:16:57) - Plan for AGI 2025

    (00:29:19) - Teaching models to reason

    (00:40:50) - The Road to ChatGPT

    (00:52:13) - What makes for a good RL researcher?

    (01:00:58) - Keeping humans in the loop

    (01:15:15) - State of research, plateaus, and moats

    Sponsors

    If you’re interested in advertising on the podcast, fill out this form.

    * Your DNA shapes everything about you. Want to know how? Take 10% off our Premium DNA kit with code DWARKESH at mynucleus.com.

    * CommandBar is an AI user assistant that any software product can embed to non-annoyingly assist, support, and unleash their users. Used by forward-thinking CX, product, growth, and marketing teams. Learn more at commandbar.com.



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enMay 15, 2024

    Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

    Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

    Mark Zuckerberg on:

    - Llama 3

    - open sourcing towards AGI

    - custom silicon, synthetic data, & energy constraints on scaling

    - Caesar Augustus, intelligence explosion, bioweapons, $10b models, & much more

    Enjoy!

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Human edited transcript with helpful links here.

    Timestamps

    (00:00:00) - Llama 3

    (00:08:32) - Coding on path to AGI

    (00:25:24) - Energy bottlenecks

    (00:33:20) - Is AI the most important technology ever?

    (00:37:21) - Dangers of open source

    (00:53:57) - Caesar Augustus and metaverse

    (01:04:53) - Open sourcing the $10b model & custom silicon

    (01:15:19) - Zuck as CEO of Google+

    Sponsors

    If you’re interested in advertising on the podcast, fill out this form.

    * This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue. Learn more at stripe.com.

    * V7 Go is a tool to automate multimodal tasks using GenAI, reliably and at scale. Use code DWARKESH20 for 20% off on the pro plan. Learn more here.

    * CommandBar is an AI user assistant that any software product can embed to non-annoyingly assist, support, and unleash their users. Used by forward-thinking CX, product, growth, and marketing teams. Learn more at commandbar.com.



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe

    Sholto Douglas & Trenton Bricken - How to Build & Understand GPT-7's Mind

    Sholto Douglas & Trenton Bricken - How to Build & Understand GPT-7's Mind

    Had so much fun chatting with my good friends Trenton Bricken and Sholto Douglas on the podcast.

    No way to summarize it, except: 

    This is the best context dump out there on how LLMs are trained, what capabilities they're likely to soon have, and what exactly is going on inside them.

    You would be shocked how much of what I know about this field, I've learned just from talking with them.

    To the extent that you've enjoyed my other AI interviews, now you know why.

    So excited to put this out. Enjoy! I certainly did :)

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. 

    There's a transcript with links to all the papers the boys were throwing down - may help you follow along.

    Follow Trenton and Sholto on Twitter.

    Timestamps

    (00:00:00) - Long contexts

    (00:16:12) - Intelligence is just associations

    (00:32:35) - Intelligence explosion & great researchers

    (01:06:52) - Superposition & secret communication

    (01:22:34) - Agents & true reasoning

    (01:34:40) - How Sholto & Trenton got into AI research

    (02:07:16) - Are feature spaces the wrong way to think about intelligence?

    (02:21:12) - Will interp actually work on superhuman models

    (02:45:05) - Sholto’s technical challenge for the audience

    (03:03:57) - Rapid fire



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe

    Demis Hassabis - Scaling, Superhuman AIs, AlphaZero atop LLMs, Rogue Nations Threat

    Demis Hassabis - Scaling, Superhuman AIs, AlphaZero atop LLMs, Rogue Nations Threat

    Here is my episode with Demis Hassabis, CEO of Google DeepMind

    We discuss:

    * Why scaling is an artform

    * Adding search, planning, & AlphaZero type training atop LLMs

    * Making sure rogue nations can't steal weights

    * The right way to align superhuman AIs and do an intelligence explosion

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Timestamps

    (0:00:00) - Nature of intelligence

    (0:05:56) - RL atop LLMs

    (0:16:31) - Scaling and alignment

    (0:24:13) - Timelines and intelligence explosion

    (0:28:42) - Gemini training

    (0:35:30) - Governance of superhuman AIs

    (0:40:42) - Safety, open source, and security of weights

    (0:47:00) - Multimodal and further progress

    (0:54:18) - Inside Google DeepMind



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe