Logo

    David Reich - How One Small Tribe Conquered the World 70,000 Years Ago

    enAugust 29, 2024
    What was the main topic of the podcast episode?
    Summarise the key points discussed in the episode?
    Were there any notable quotes or insights from the speakers?
    Which popular books were mentioned in this episode?
    Were there any points particularly controversial or thought-provoking discussed in the episode?
    Were any current events or trending topics addressed in the episode?

    Podcast Summary

    • Human EvolutionNew discoveries and technology challenge previous beliefs about human evolution, particularly the relationship between modern humans and extinct groups like Neanderthals and Denisovans, and researchers are exploring alternative explanations such as higher levels of gene flow.

      Our understanding of human evolution, particularly the relationship between modern humans and extinct groups like Neanderthals and Denisovans, is constantly evolving. New discoveries and advancements in technology have led to revisions of previous theories. For instance, it was once believed that modern humans and Neanderthals separated around 500-750,000 years ago, but recent findings from mitochondrial DNA and the Y chromosome suggest a more recent shared ancestry. These discrepancies challenge the current model, and researchers are exploring alternative explanations, such as higher levels of gene flow between modern humans and Neanderthals. Furthermore, modern humans have also been found to have significant substructure, adding complexity to our understanding of human history. The field of ancient DNA research is continuously reshaping our perception of human evolution, and it's essential to remain open to new discoveries and reevaluate existing theories.

    • Neanderthal-human gene flow locationThe exact location and timing of Neanderthal-human gene flow is still unclear, but the Near East is a likely candidate due to its ecological continuity with Africa and Eurasia.

      The exact location and timing of gene flow between Neanderthals and early humans, specifically the modern human lineage, is still unclear. While it's widely accepted that Africa was the cradle of humanity for the majority of human history, the period around 2 million to 500,000 years ago is less clear. During this time, humans were present in both Africa and Eurasia, with parallel increases in brain size. It's possible that the main human lineages were in both places and mixed to form the lineages that led to modern humans. The Near East could be a likely candidate for this gene flow event due to its ecological continuity with Africa and Eurasia, making it a natural place for interactions between different human populations. However, the exact number and size of human groups during this time is still a topic of ongoing research. Despite being in different parts of the world, there was selective pressure for larger brains in both populations, possibly due to environmental pressures or the development of complex social structures.

    • Human Evolution and Cultural InnovationsHuman distinctiveness, including large brains, may not be solely due to genetics but also cultural innovations and gene flow between populations. Frequent migrations and exchanges throughout history have challenged the notion of completely separate populations in Africa and Eurasia.

      The evolution of human distinctiveness, including our large brains, may not be solely due to genetic factors. Instead, it may be the result of cultural innovations and interconnected populations with significant gene flow. The large brain size was likely present before the separation of modern humans and Neanderthals. The idea of distinct human populations in Africa and Eurasia being completely separate for long periods is a strange one, as there have been frequent migrations and exchanges throughout history. The shift in human expansion and dominance around 60,000 years ago might have been driven by cultural innovations like language and information storage. Epigenetic modifications, which are not directly encoded by DNA but can affect gene expression, provide evidence for this theory. Researchers have identified differentially methylated regions in Neanderthal and modern human genomes, which suggest that the vocal tract was a significant area of difference. This discovery, among others, highlights the unexpected findings that continue to emerge from genetics research.

    • Human vocal tract evolutionRecent epigenetic findings suggest significant changes to the human vocal tract occurred within the last few hundred thousand years, allowing for wider sound articulation and contributing to human language development

      Recent discoveries in epigenetics suggest that significant changes to the human vocal tract occurred within the last few hundred thousand years, making it distinct from that of our extinct relatives, the Neanderthals and Denisovans. These changes, which have allowed for the articulation of a wider range of sounds, may have played a crucial role in the development of human language. The timing and causes of these changes are still unclear, but they likely occurred before modern humans expanded out of Africa and dominated other human populations. This discovery challenges the notion that AI can simply be made larger to achieve human-level intelligence, as human distinctiveness lies not only in brain size but also in our unique cultural and social learning abilities.

    • Human expansion out of AfricaHuman expansion out of Africa was a complex process involving interbreeding with other ancient human groups and multiple waves of expansion and replacement, with agriculture and population density potentially contributing to the spread of diseases and the decline of some groups.

      The expansion of modern humans out of Africa and into different parts of Eurasia was a complex process involving interbreeding with other ancient human groups, such as Neanderthals and Denisovans. This process was not a linear one, but rather involved multiple waves of expansion and replacement of different human groups. Additionally, the spread of agriculture and the resulting increase in population density may have led to the spread of diseases, potentially contributing to the decline of some human groups. Overall, human history has involved recurring interactions and conflicts between different groups, with various factors contributing to the success or decline of different populations.

    • Black Death bacterium impact on human historyThe Black Death bacterium, responsible for numerous deaths in western Eurasia over the past 5,000 years, likely originated from step rodents and contributed to cultural shifts and societal transformations, including the Industrial Revolution and the fall of the Roman Empire, and may provide insights into human adaptation to agriculture and early states.

      The Yersinia pestis bacterium, responsible for the Black Death, has been killing significant portions of the population in western Eurasia for the past 5,000 years. Studies on ancient DNA from various time periods and locations reveal that a large fraction of deaths were caused by this agent. The implication is that it may have originated from step rodents and contributed to the disruption and replacement of existing populations, potentially paving the way for cultural shifts and societal transformations. The impact of this disease on human history is significant, with possible consequences such as the Industrial Revolution and the fall of the Roman Empire. Genetic data may also provide insights into how humans adapted to the innovation of agriculture and the emergence of early states, although more research is needed to confirm this.

    • Genetic adaptation in human populationsDespite limited genetic adaptation over 50,000 years, recent research indicates significant changes in the last 10,000 years related to metabolism and immune traits, possibly due to the shift to agriculture

      Over the last 50,000 years, there has been relatively little genetic adaptation across different populations, as evidenced by the lack of significant frequency differences in genetic variants between Europeans, East Asians, and West Africans. This suggests that natural selection may not have been a major driving force in shaping human populations during this time period. However, recent research suggests that there have been significant genetic changes in the last 10,000 years, particularly in relation to metabolism and immune traits. It is unclear why this is the case, but it may be related to the shift from a hunter-gatherer lifestyle to agriculture. The exact nature of population interactions during this period is also still being studied. Overall, the relationship between genetics and human history is complex and continues to be an active area of research.

    • Archaeology and GeneticsGenetic data has challenged traditional archaeological theories, leading to a more nuanced understanding of human history through the reconciliation of archaeological findings and genetic evidence.

      The relationship between archaeological findings and genetic data in understanding human history can be complex and challenging. For instance, the arrival of new material cultures, such as the cordedware complex or the bellbeaker complex, was once believed to be the result of large-scale invasions or migrations. However, this theory was disputed after World War II due to the implications it held for racial ideologies. With the advent of genetic data, it was discovered that there were significant population disruptions in Europe during certain periods, which contradicted the theory of cultural diffusion rather than migration. This finding was met with resistance in the archaeological community, but it ultimately led to a reconciliation and a more nuanced understanding of the subtlety of interaction events between different populations. The ongoing debate revolves around the nature of these interaction events, such as peaceful or violent, and the degree of organization behind them. Overall, the discovery of the genetic data has shown that our understanding of prehistory is incomplete without hard data to support our theories.

    • Human MigrationsHuman migrations were not simple, one-directional processes, but involved complex interactions between groups, including the absorption of women, resulting in the spread of ancestry and cultural practices in unexpected ways

      The expansion of certain human groups, such as the Yamnaya and the Lapita Cultural Complex, was not a simple, one-directional process. Instead, it involved complex interactions between different groups, including the absorption of women from other communities. These interactions resulted in the spread of ancestry and cultural practices, sometimes in unexpected ways. For example, the Yamnaya expansion was initially male-dominated, but later, Yamnaya ancestry was carried further west through the absorption of Yamnaya women into the cordedware community. Similarly, in the Southwest Pacific, the initial expansion of East Asian ancestry was followed by a later, male-driven expansion of Papuan ancestry. These findings challenge the common assumption that human migrations were primarily male-driven and that one group would dominate or replace another. Instead, the evidence suggests that interactions between groups were more complex and dynamic.

    • Neanderthal ancestry and admixtureNeanderthal ancestry in non-African populations was rapidly replaced through a process called admixture, leaving only small traces today, likely driven by the arrival of large numbers of modern humans into new regions.

      The history of human evolution involves a complex interplay of genetic admixture, population dynamics, and environmental factors. For instance, after the Neanderthals and modern humans came into contact, there were significant amounts of Neanderthal ancestry in non-African populations. However, this ancestry was rapidly replaced through a process called admixture, leaving only small traces today. This process was likely driven by the arrival of large numbers of modern humans into new regions, leading to the eventual displacement of local Neanderthal populations. This phenomenon is not limited to humans, as it can be observed in other species as well. The question of how random or deterministic the path to civilization was is still a subject of debate, with some arguing that certain cultural or biological characteristics were in place during the last Ice Age that enabled the rapid development of agriculture and complex societies. Overall, the study of ancient DNA and genetics has shown that everyone in the world is the result of recurrent mixture events throughout history.

    • Human Ancestry ComplexityThe complexity of human ancestry includes significant mixture and branching both in and outside Africa, with potential time spent in Eurasia, diverse ancestries for different populations, and a need to understand recent adaptations.

      The history of human ancestry is far more complex than previously assumed, with significant mixture and branching occurring both in and outside of Africa. The evidence suggests that our lineage may have spent considerable time in Eurasia, and that different populations have diverse ancestries that extend beyond their current geographic locations. The discovery of ancient DNA from Africa would greatly enhance our understanding of human evolution, as it would provide crucial insights into the origins of modern human lineages and their relationships with archaic forms. Additionally, there is a need to better understand how biological adaptations occurred in the last hundreds of thousands of years, as the genetic underpinnings of modern human cognitive and other propensities remain a mystery.

    • Brain processing of visual informationResearchers decode neuronal responses to understand brain's processing of visual information, opening possibilities for genetic data mining and insights into human history

      Researchers have made significant strides in understanding how the brain processes and represents visual information, such as photographs, by decoding the neuronal responses. This discovery, which began with analyzing a monkey's brain, raises intriguing questions about the complexity of the brain and the genome. While we can't yet read the genome to determine how a person looks or develops, researchers are investigating how to mine genetic data to learn about biological adaptation. The human brain and genome are complex systems, and understanding their information content could provide valuable insights into their similarities and differences. Additionally, the persistence of cultural traditions despite the absence of writing for extended periods highlights the importance of studying both genetics and culture in understanding human history.

    • Ancient Indian PopulationsAround 3,800 years ago, three distinct ancestral populations came together in India and began to mix, leading to a complex genetic history with a stable genetic gradient, except for distinct Patel communities with Central Asian ancestry

      The genetic history of populations in India involves a complex interplay of ancient migrations and cultural changes. Around 3,800 years ago, three distinct ancestral populations - farmers, local hunter-gatherers, and steppe pastoralists - came together in the Indian subcontinent and began to mix. However, this process of mixing eventually came to a halt due to cultural changes that solidified the caste system, leading to a stable genetic gradient instead of a collapsing population. This system is unique because it offers a snapshot of ancient genetic diversity that has remained relatively unchanged for thousands of years. One interesting exception to this pattern are the Patel communities, who have distinct genetic ancestry from Central Asia, pushing them off the main genetic gradient. The field of research in this area is constantly evolving, with new techniques and discoveries leading to exciting collaborations between researchers in genetics, archaeology, and history. While some traditional archaeologists may initially be skeptical of new findings, the majority are eager to learn and embrace interdisciplinary approaches to understanding the past.

    • Ancient DNA discoveriesAncient DNA discoveries have revolutionized our understanding of human history, providing new insights into previously limited data areas and revealing unexpected information from the past.

      Our understanding of human history has undergone significant transformations through various scientific revolutions, from carbon dating to ancient DNA and genomic sequencing. One of these revelations was the discovery of extractable DNA from ancient human remains, which was once thought impossible. This unexpected gift from the past has led to a wealth of new insights, particularly in areas with limited data like Africa, South Asia, Australia, and New Guinea. However, there is still much to discover, such as decoupling ancient lineages and sampling from deeper into the past. These advancements will provide valuable insights into individual places and deep time, ultimately enriching our understanding of human history. It's an exciting time for historical research, and the future holds even more promising discoveries.

    Recent Episodes from Dwarkesh Podcast

    David Reich - How One Small Tribe Conquered the World 70,000 Years Ago

    David Reich - How One Small Tribe Conquered the World 70,000 Years Ago

    I had no idea how wild human history was before chatting with the geneticist of ancient DNA David Reich.

    Human history has been again and again a story of one group figuring ‘something’ out, and then basically wiping everyone else out.

    From the tribe of 1k-10k modern humans who killed off all the other human species 70,000 years ago; to the Yamnaya horse nomads 5,000 years ago who killed off 90+% of (then) Europeans and also destroyed the Indus Valley.

    So much of what we thought we knew about human history is turning out to be wrong, from the ‘Out of Africa’ theory to the evolution of language, and this is all thanks to the research from David Reich’s lab.

    Buy David Reich’s fascinating book, Who We Are How We Got Here.

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Follow me on Twitter for updates on future episodes.

    Sponsor

    This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue.

    If you’re interested in advertising on the podcast, check out this page.

    Timestamps

    (00:00:00) – Archaic and modern humans gene flow

    (00:20:24) – How early modern humans dominated the world

    (00:39:59) – How bubonic plague rewrote history

    (00:50:03) – Was agriculture terrible for humans?

    (00:59:28) – Yamnaya expansion and how populations collide

    (01:15:39) – “Lost civilizations” and our Neanderthal ancestry

    (01:31:32) – The DNA Challenge

    (01:41:38) – David’s career: the genetic vocation



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enAugust 29, 2024

    Joe Carlsmith - Otherness and control in the age of AGI

    Joe Carlsmith - Otherness and control in the age of AGI

    Chatted with Joe Carlsmith about whether we can trust power/techno-capital, how to not end up like Stalin in our urge to control the future, gentleness towards the artificial Other, and much more.

    Check out Joe's sequence on Otherness and Control in the Age of AGI here.

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

    Sponsors:

    - Bland.ai is an AI agent that automates phone calls in any language, 24/7. Their technology uses "conversational pathways" for accurate, versatile communication across sales, operations, and customer support. You can try Bland yourself by calling 415-549-9654. Enterprises can get exclusive access to their advanced model at bland.ai/dwarkesh.

    - Stripe is financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue.

    If you’re interested in advertising on the podcast, check out this page.

    Timestamps:

    (00:00:00) - Understanding the Basic Alignment Story

    (00:44:04) - Monkeys Inventing Humans

    (00:46:43) - Nietzsche, C.S. Lewis, and AI

    (1:22:51) - How should we treat AIs

    (1:52:33) - Balancing Being a Humanist and a Scholar

    (2:05:02) - Explore exploit tradeoffs and AI



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enAugust 22, 2024

    Patrick McKenzie - How a Discord Server Saved Thousands of Lives

    Patrick McKenzie - How a Discord Server Saved Thousands of Lives

    I talked with Patrick McKenzie (known online as patio11) about how a small team he ran over a Discord server got vaccines into Americans' arms: A story of broken incentives, outrageous incompetence, and how a few individuals with high agency saved 1000s of lives.

    Enjoy!

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Follow me on Twitter for updates on future episodes.

    Sponsor

    This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue.

    Timestamps

    (00:00:00) – Why hackers on Discord had to save thousands of lives

    (00:17:26) – How politics crippled vaccine distribution

    (00:38:19) – Fundraising for VaccinateCA

    (00:51:09) – Why tech needs to understand how government works

    (00:58:58) – What is crypto good for?

    (01:13:07) – How the US government leverages big tech to violate rights

    (01:24:36) – Can the US have nice things like Japan?

    (01:26:41) – Financial plumbing & money laundering: a how-not-to guide

    (01:37:42) – Maximizing your value: why some people negotiate better

    (01:42:14) – Are young people too busy playing Factorio to found startups?

    (01:57:30) – The need for a post-mortem



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enJuly 24, 2024

    Tony Blair - Life of a PM, The Deep State, Lee Kuan Yew, & AI's 1914 Moment

    Tony Blair - Life of a PM, The Deep State, Lee Kuan Yew, & AI's 1914 Moment

    I chatted with Tony Blair about:

    - What he learned from Lee Kuan Yew

    - Intelligence agencies track record on Iraq & Ukraine

    - What he tells the dozens of world leaders who come seek advice from him

    - How much of a PM’s time is actually spent governing

    - What will AI’s July 1914 moment look like from inside the Cabinet?

    Enjoy!

    Watch the video on YouTube. Read the full transcript here.

    Follow me on Twitter for updates on future episodes.

    Sponsors

    - Prelude Security is the world’s leading cyber threat management automation platform. Prelude Detect quickly transforms threat intelligence into validated protections so organizations can know with certainty that their defenses will protect them against the latest threats. Prelude is backed by Sequoia Capital, Insight Partners, The MITRE Corporation, CrowdStrike, and other leading investors. Learn more here.

    - This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue.

    If you’re interested in advertising on the podcast, check out this page.

    Timestamps

    (00:00:00) – A prime minister’s constraints

    (00:04:12) – CEOs vs. politicians

    (00:10:31) – COVID, AI, & how government deals with crisis

    (00:21:24) – Learning from Lee Kuan Yew

    (00:27:37) – Foreign policy & intelligence

    (00:31:12) – How much leadership actually matters

    (00:35:34) – Private vs. public tech

    (00:39:14) – Advising global leaders

    (00:46:45) – The unipolar moment in the 90s



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enJune 26, 2024

    Francois Chollet, Mike Knoop - LLMs won’t lead to AGI - $1,000,000 Prize to find true solution

    Francois Chollet, Mike Knoop - LLMs won’t lead to AGI - $1,000,000 Prize to find true solution

    Here is my conversation with Francois Chollet and Mike Knoop on the $1 million ARC-AGI Prize they're launching today.

    I did a bunch of socratic grilling throughout, but Francois’s arguments about why LLMs won’t lead to AGI are very interesting and worth thinking through.

    It was really fun discussing/debating the cruxes. Enjoy!

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Timestamps

    (00:00:00) – The ARC benchmark

    (00:11:10) – Why LLMs struggle with ARC

    (00:19:00) – Skill vs intelligence

    (00:27:55) - Do we need “AGI” to automate most jobs?

    (00:48:28) – Future of AI progress: deep learning + program synthesis

    (01:00:40) – How Mike Knoop got nerd-sniped by ARC

    (01:08:37) – Million $ ARC Prize

    (01:10:33) – Resisting benchmark saturation

    (01:18:08) – ARC scores on frontier vs open source models

    (01:26:19) – Possible solutions to ARC Prize



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enJune 11, 2024

    Leopold Aschenbrenner - China/US Super Intelligence Race, 2027 AGI, & The Return of History

    Leopold Aschenbrenner - China/US Super Intelligence Race, 2027 AGI, & The Return of History

    Chatted with my friend Leopold Aschenbrenner on the trillion dollar nationalized cluster, CCP espionage at AI labs, how unhobblings and scaling can lead to 2027 AGI, dangers of outsourcing clusters to Middle East, leaving OpenAI, and situational awareness.

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Follow me on Twitter for updates on future episodes. Follow Leopold on Twitter.

    Timestamps

    (00:00:00) – The trillion-dollar cluster and unhobbling

    (00:20:31) – AI 2028: The return of history

    (00:40:26) – Espionage & American AI superiority

    (01:08:20) – Geopolitical implications of AI

    (01:31:23) – State-led vs. private-led AI

    (02:12:23) – Becoming Valedictorian of Columbia at 19

    (02:30:35) – What happened at OpenAI

    (02:45:11) – Accelerating AI research progress

    (03:25:58) – Alignment

    (03:41:26) – On Germany, and understanding foreign perspectives

    (03:57:04) – Dwarkesh’s immigration story and path to the podcast

    (04:07:58) – Launching an AGI hedge fund

    (04:19:14) – Lessons from WWII

    (04:29:08) – Coda: Frederick the Great



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enJune 04, 2024

    John Schulman (OpenAI Cofounder) - Reasoning, RLHF, & Plan for 2027 AGI

    John Schulman (OpenAI Cofounder) - Reasoning, RLHF, & Plan for 2027 AGI

    Chatted with John Schulman (cofounded OpenAI and led ChatGPT creation) on how posttraining tames the shoggoth, and the nature of the progress to come...

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

    Timestamps

    (00:00:00) - Pre-training, post-training, and future capabilities

    (00:16:57) - Plan for AGI 2025

    (00:29:19) - Teaching models to reason

    (00:40:50) - The Road to ChatGPT

    (00:52:13) - What makes for a good RL researcher?

    (01:00:58) - Keeping humans in the loop

    (01:15:15) - State of research, plateaus, and moats

    Sponsors

    If you’re interested in advertising on the podcast, fill out this form.

    * Your DNA shapes everything about you. Want to know how? Take 10% off our Premium DNA kit with code DWARKESH at mynucleus.com.

    * CommandBar is an AI user assistant that any software product can embed to non-annoyingly assist, support, and unleash their users. Used by forward-thinking CX, product, growth, and marketing teams. Learn more at commandbar.com.



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enMay 15, 2024

    Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

    Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

    Mark Zuckerberg on:

    - Llama 3

    - open sourcing towards AGI

    - custom silicon, synthetic data, & energy constraints on scaling

    - Caesar Augustus, intelligence explosion, bioweapons, $10b models, & much more

    Enjoy!

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Human edited transcript with helpful links here.

    Timestamps

    (00:00:00) - Llama 3

    (00:08:32) - Coding on path to AGI

    (00:25:24) - Energy bottlenecks

    (00:33:20) - Is AI the most important technology ever?

    (00:37:21) - Dangers of open source

    (00:53:57) - Caesar Augustus and metaverse

    (01:04:53) - Open sourcing the $10b model & custom silicon

    (01:15:19) - Zuck as CEO of Google+

    Sponsors

    If you’re interested in advertising on the podcast, fill out this form.

    * This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue. Learn more at stripe.com.

    * V7 Go is a tool to automate multimodal tasks using GenAI, reliably and at scale. Use code DWARKESH20 for 20% off on the pro plan. Learn more here.

    * CommandBar is an AI user assistant that any software product can embed to non-annoyingly assist, support, and unleash their users. Used by forward-thinking CX, product, growth, and marketing teams. Learn more at commandbar.com.



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe

    Sholto Douglas & Trenton Bricken - How to Build & Understand GPT-7's Mind

    Sholto Douglas & Trenton Bricken - How to Build & Understand GPT-7's Mind

    Had so much fun chatting with my good friends Trenton Bricken and Sholto Douglas on the podcast.

    No way to summarize it, except: 

    This is the best context dump out there on how LLMs are trained, what capabilities they're likely to soon have, and what exactly is going on inside them.

    You would be shocked how much of what I know about this field, I've learned just from talking with them.

    To the extent that you've enjoyed my other AI interviews, now you know why.

    So excited to put this out. Enjoy! I certainly did :)

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. 

    There's a transcript with links to all the papers the boys were throwing down - may help you follow along.

    Follow Trenton and Sholto on Twitter.

    Timestamps

    (00:00:00) - Long contexts

    (00:16:12) - Intelligence is just associations

    (00:32:35) - Intelligence explosion & great researchers

    (01:06:52) - Superposition & secret communication

    (01:22:34) - Agents & true reasoning

    (01:34:40) - How Sholto & Trenton got into AI research

    (02:07:16) - Are feature spaces the wrong way to think about intelligence?

    (02:21:12) - Will interp actually work on superhuman models

    (02:45:05) - Sholto’s technical challenge for the audience

    (03:03:57) - Rapid fire



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe

    Demis Hassabis - Scaling, Superhuman AIs, AlphaZero atop LLMs, Rogue Nations Threat

    Demis Hassabis - Scaling, Superhuman AIs, AlphaZero atop LLMs, Rogue Nations Threat

    Here is my episode with Demis Hassabis, CEO of Google DeepMind

    We discuss:

    * Why scaling is an artform

    * Adding search, planning, & AlphaZero type training atop LLMs

    * Making sure rogue nations can't steal weights

    * The right way to align superhuman AIs and do an intelligence explosion

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Timestamps

    (0:00:00) - Nature of intelligence

    (0:05:56) - RL atop LLMs

    (0:16:31) - Scaling and alignment

    (0:24:13) - Timelines and intelligence explosion

    (0:28:42) - Gemini training

    (0:35:30) - Governance of superhuman AIs

    (0:40:42) - Safety, open source, and security of weights

    (0:47:00) - Multimodal and further progress

    (0:54:18) - Inside Google DeepMind



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe