Logo
    Search

    Andy Matuschak - Self-Teaching, Spaced Repetition, & Why Books Don’t Work

    enJuly 12, 2023

    Podcast Summary

    • The importance of engaging with new material for true comprehensionDeliberate practice and interest are crucial for understanding complex concepts and reading difficult texts.

      Memory plays a crucial role in understanding complex concepts, and the deliberate practice of engaging with new material is essential for true comprehension. However, many students are not motivated to learn material that isn't meaningful or relevant to their goals. The education system often focuses on rote memorization and doesn't always help students make connections between new concepts and their interests. The ability to read and understand complex texts requires practice and skill, and many people struggle with this due to a lack of experience or understanding of effective reading strategies. The authors of "How to Read a Book" argue that becoming a more skillful reader can lead to deeper understanding and appreciation of difficult texts when the material aligns with one's interests. Ultimately, the key is to approach learning with curiosity, deliberation, and a willingness to engage deeply with new ideas.

    • Asking questions and self-reflection for effective readingImplement strategies like asking questions, using study guides, and slowing down to enhance learning and deepen understanding of complex subjects.

      Effective reading involves both understanding the material and remembering it. Two reasons why people might struggle with retaining information are forgetting what was learned or never understanding it in the first place. Adler and Van Doren suggest that asking questions and trying to answer them is the first rule of skillful reading. However, when focusing intensely on the material, metacognition, or self-awareness and reflection, can be challenging. To help with this, some strategies include using syllabi or study guides, embedding review questions into textbooks, or slowing down and reading more attentively. These practices can provide feedback and help readers notice when they haven't absorbed the material, ultimately leading to improved understanding and retention. In today's digital age, with an abundance of online resources, developing conscientiousness and self-motivation is crucial for learning. By implementing these strategies, readers can enhance their learning experience and deepen their understanding of complex subjects.

    • Syllabi as valuable learning scaffoldsSyllabi help learners navigate new subjects, but as they gain skills, they should critically evaluate and revise their learning plan based on their progress

      Syllabi can serve as valuable scaffolds for learners, particularly those who are new to a subject or domain. They help learners navigate the material by providing a clear path and structure. However, as learners become more skilled and familiar with the subject matter, they can lean less on the syllabus and rely more on their own judgment and planning. Adjunct questions, a type of scaffolding, can also aid learners by encouraging reflection and deepening understanding of the material. It's important for learners to critically evaluate the necessity of each piece of content and not feel obligated to follow a syllabus blindly. Instead, they should use it as a starting point and iteratively revise their learning plan as they gain more knowledge and skills.

    • The value of reading and experiences in shaping our understandingReading and experiences shape our worldview, but effective knowledge compilation requires repeated exposure and engagement for deeper understanding

      While unschooling allows children to pursue their interests freely, it may limit their exposure to diverse knowledge and opportunities. On the other hand, reading and gaining experiences shape our understanding of the world, even if we forget the specific details. These experiences help us compile knowledge and apply it in various contexts. However, effective knowledge compilation requires repeated exposure and engagement with the material. While forgetting some details may not hinder our growth as interviewers or learners, it is a shame not to have a more rigorous practice for retaining information. Paul Graham's idea of reading and experiences shaping our worldview is true, but the depth of understanding needed depends on the situation.

    • Engaging with information through note-taking and writing improves understanding and retentionActively engaging with complex information through note-taking, writing, and memorization can lead to a deeper understanding and improved recall of difficult concepts.

      Constant engagement with information, whether through note-taking or writing, can help improve understanding and retention of complex material. Tyler Cowen, who consumes large amounts of content without extensive note-taking, demonstrates this through his prolific writing output. Memorization plays a crucial role in this process, as it enables us to recall and connect new information with existing knowledge. Memorization can take various forms, from committing trivia to memory to understanding complex arguments or noticing surprising connections. The ability to recall information quickly and accurately is essential for comprehending difficult concepts and generating insights. So, actively engaging with information and building a strong foundation of knowledge through memorization can lead to a deeper understanding of complex subjects.

    • The Power of Memory in Learning and LifeMemory plays a crucial role in learning and life, helping us understand complex concepts, notice connections, and participate in high-level strategic conversations. It's not limited by the amount of information we can remember.

      Memory plays a crucial role in learning and life in general. The speaker shares his experience of using thorough memorization to gain a deep understanding of the education market and participate in high-level strategic conversations. He also emphasizes the importance of memory in noticing connections and ideas, even if they are not easily described in text. The speaker also discusses the potential value of forgetting and how it may help us generalize insights. He also touches on the idea that memories aren't necessarily lost, but rather, they can be accessed with the right cues. The speaker's reflections challenge the common belief that memorization limits our ability to learn new things, and he shares examples of individuals who have memorized vast amounts of information without negative consequences. In essence, memory is a powerful tool that guides our attention and helps us understand complex concepts, and it is not limited by the amount of information we can remember.

    • The role of past experiences and context in learning new informationHaving a strong foundation of knowledge and understanding the context and relevance of new information are crucial for effective learning and memory retention.

      Our past experiences and familiarity with certain concepts play a significant role in how effectively we learn and remember new information. The concept of spreading activation explains how our brain processes and retrieves information, with the weights on connections determining what is most relevant and useful. Our understanding of a subject and the context in which we learn it can greatly impact our ability to comprehend and remember the material. At Apple, the speaker's engineering background and extensive experience with the company's platform allowed them to learn new things more easily due to the existing knowledge they could connect it to. However, in their current role as a researcher, they are faced with discovering new things and making things that didn't exist before, which requires synthesizing unfamiliar literatures and answering complex questions about what should be done. The speaker also mentioned predictive utility theory, which suggests that the brain predicts the usefulness of an item in a given situation, and the probability of retrieval corresponds to this prediction. This theory highlights the importance of understanding the relevance and utility of the information we are learning, and how it can be applied to real-world problems. Overall, the discussion emphasizes the importance of having a solid foundation of knowledge and understanding the context and relevance of new information to effectively learn and remember it.

    • Effective learning from connections and practiceConnections and practice are essential for effective learning. Immersive experiences help recognize new insights, while explicit practice builds a foundation. Stamina and motivation also play a role.

      Effective learning can come from both explicit practice and immersive experiences, depending on the specific field or subject matter. The Swanson linking example illustrates how recognizing connections between seemingly unrelated things can lead to new insights and discoveries. However, not all knowledge can be gained naturally through immersion alone, especially in fields with rare diagnoses or long tail events. Explicit practice, such as spaced repetition or setting up flashcards, can serve as a valuable tool to build a foundation and bootstrap oneself into a position where naturalistic reinforcement becomes more effective. Stamina, another important factor in learning, can be developed through practice and discipline, but can also be influenced by external factors like social interaction and motivation. Ultimately, the most effective learning strategy often involves a combination of both explicit practice and immersive experiences tailored to the specific learning goals and context.

    • The demands of learning complex topics can be more exhausting than expectedLearning complex topics requires constant focus and attention, but engaging and interactive methods can make the process less draining.

      While learning quantum mechanics or similar complex topics may seem less demanding due to their cognitive indirectness compared to daily work, the constant attention and focus required can be more exhausting than one might expect. This is particularly true when reading textbooks or other material without emotional engagement or curiosity. However, when the demandingness of the material matches one's ability, the learning process can be more engaging and less draining. The use of new mediums like video games or interactive environments could potentially enhance the learning experience by creating a more immersive and conversational environment, but successful implementation remains a challenge. For instance, games like The Witness by Jonathan Blow demonstrate the potential of a medium of action, where learning is not explicit but rather comes from interacting with the environment. However, more research and development are needed to effectively apply this approach to explicit subjects.

    • Exploring different ways to learn tacit knowledgeStreaming videos offer a new way to learn tacit knowledge, but they lack the feedback and context of traditional apprenticeships. Games and interactive resources can provide engaging experiences, but they may not offer the same depth of learning. The education landscape needs more resources for hands-on domains like design and architecture.

      While games like Kerbal Space Program and Zachtronics' titles can offer some educational value, their primary purpose is to provide an engaging experience. On the other hand, resources like Grant's videos and textbooks serve different functions, catering to those who want to learn specific skills like robotics or programming. The active learning experience of NAND to Tetris comes closest to a game, but it's not yet scalable for mass audiences. Apprenticeship remains an effective learning method for acquiring tacit knowledge, but it's not easily replicable. Streaming videos, such as those featuring engineers like George Hotz, can transmit valuable tacit knowledge, but they lack the feedback and context provided in traditional apprenticeships. The education landscape needs more resources in domains like design, architecture, and musical composition, which heavily rely on hands-on feedback and context. Despite the limitations, streaming videos represent an exciting step forward in making tacit knowledge more accessible.

    • The instant feedback loop in programming keeps developers engaged and motivatedProgramming's rapid feedback cycle turns coding challenges into solvable tasks, fostering a sense of progress and satisfaction

      The sense of instantaneous feedback and direct contact with the environment in programming, which makes it feel like a video game, is a key factor in keeping programmers engaged and motivated. This rapid feedback cycle is what makes programming problems feel like solvable challenges, leading to a sense of progress and satisfaction. However, in other fields like design, the search for solutions can feel more like a combinatorial search space with no clear right answer, leading to a lack of the same sense of forward progress. The exploration and experimentation in design can feel aimless and unfulfilling, with no clear sense of getting closer to a solution. Yet, there are designers who remain obsessed with the process, likely due to their skill level and the nature of the problems they're solving, which offer a sense of tractability and a finite, polishable outcome.

    • Exploring Creativity and Problem-Solving Skills through Design ProcessesDesign processes like creating posters, code golf, and endless refactoring help median students engage in inquiry learning and explore creativity and problem-solving skills outside of regular work.

      The design process for creating posters or engaging in activities like code golf and endless refactoring is a way for designers and programmers to explore their creativity and problem-solving skills outside of their regular work. This approach is different from the traditional educational system, which often focuses on making students do things they may not be interested in. Instead, educational tools and methods for median students should be geared towards inquiry learning and transforming the learning experience into something that relates to their goals. This can be achieved through the use of dynamic media representations, such as Cuisenaire rods, and social interaction, which can make learning more engaging and require less willpower. However, the majority of the educational space is focused on addressing the bottom quartile of students, and there is a need for more attention and resources dedicated to creating effective learning tools for the median student.

    • Innovative approaches for underperforming studentsUnderstanding emotional and practical issues can make learning enjoyable and effective for underperforming students, utilizing modern memory systems and problem-solving techniques.

      While there is value in focusing on improving education for high-performing students, there is also a significant group of underperforming students who could benefit greatly from innovative approaches. These students, often disengaged and uninterested, may hold the key to creating groundbreaking inventions if given the right tools and support. However, the challenge lies in addressing the emotional and practical issues that make learning difficult for some students. By understanding and addressing the reasons behind their misery, such as lack of interest or fear of failure, and utilizing modern memory systems and problem-solving techniques, it is possible to make learning a more enjoyable and effective experience for all students. Ultimately, the goal should be to create a learning process that is engaging, fun, and accessible to everyone, regardless of their current performance level.

    • Finding Balance in Structured and Unstructured LearningRecognize the importance of foundational knowledge while embracing flexibility and enjoyment in the learning process. Consider various methods to ensure discipline and balance between structured and unstructured learning.

      While there is value in both disciplined, structured learning and free-flowing, unstructured learning, the speaker finds herself drawn to the tension between the two. She acknowledges the importance of memorizing foundational knowledge, like taxonomies, as a means to build a solid base for further learning. However, she also recognizes the need for flexibility and enjoyment in the learning process. The speaker uses various tools and methods to ensure discipline in her own learning, despite her preference for a more unstructured approach. Ultimately, she aspires to find a balance between the two extremes, recognizing that both structured and unstructured learning have their merits. Additionally, when it comes to education for children, the speaker believes that the various purposes of schooling can be unbundled and considers homeschooling as a potential option, but acknowledges the complexity of the decision and the need for careful consideration.

    • The Complexity of Educating Older ChildrenDewey believed 12-year-olds lack self-direction, education reform has made progress but challenges remain, finding balance between freedom and guidance is complex.

      While providing a quality education for a child can be affordable through alternative methods, such as hiring a group of teachers on an hourly basis, the question of what and when to teach specific subjects becomes more complex as the child grows older. John Dewey, a philosopher known for his education reform efforts, argued that a 12-year-old child doesn't have a fully developed sense of self and can't be entirely free to follow their whims without guidance. Education reform has been a challenging issue historically, but significant progress has been made. For instance, the percentage of high school graduates in the US has increased dramatically, and the bar for a high school education has been raised. However, there are still challenges, and the role of technology in education is an ongoing debate. While there have been many new tools introduced through IT, the broader issue of educational stagnation remains. Ultimately, finding the right balance between freedom and guidance for a child's education is a complex task that requires careful consideration and ongoing adaptation.

    • Challenges in raising intellectual greatness through mass educationMass education has improved access to education for the bottom quartile, but producing intellectual greats remains a challenge, as traditional methods like aristocratic tutoring may be more effective.

      While mass education has significantly improved the educational attainment of the bottom quartile of the population over the last 20 to 30 years, raising the ceiling of intellectual greatness through mass education remains a challenge. The production function for intellectual greatness, such as that of John von Neumann, may not be well-suited to mass education systems. Instead, aristocratic tutoring or exclusive educational institutions have historically produced intellectual giants. However, the opportunity cost for becoming a tutor or teacher is higher now, and the most expensive tutors may not necessarily be the most effective. It's unclear whether we've gotten worse at producing intellectual greats or if the market for tutors and educators has simply changed over time.

    • The correlation between a teacher's intelligence and effectiveness is not definitiveEffective teaching relies on empathy, communication, and care, not just deep subject matter knowledge. Adaptability and improvisation require a certain level of familiarity with the subject.

      While having an inspiring and motivated teacher can greatly impact learning, the importance of a teacher's intelligence or expertise is not definitively correlated with their teaching effectiveness. The pool of available teachers has expanded, but the top talent may not be getting more opportunities. Effective teaching often relies on empathy, communication, and care, rather than deep subject matter knowledge. However, the ability to improvise and adapt to students' questions and answers, which is required for inquiry-oriented classes, does necessitate a certain level of familiarity with the subject matter. The widespread availability of resources like Wikipedia and hypertext has not significantly changed how people write or teach, as these resources often provide comprehensive explanations that obviate the need for extensive exposition on every topic. Instead, writers and teachers can rely on linking to relevant information to enhance their content.

    • Exploring Complex Information with HypertextHypertext is a valuable tool for navigating complex information, particularly in reference materials and personal notes. It helps users quickly locate relevant information and build upon their existing knowledge.

      Hypertext, or the ability to navigate and connect information non-linearly, has been a valuable tool for accessing and understanding complex information since before the digital age. Encyclopedias and other reference materials, such as the Syntopicon, served as early examples of hypertext through their cross-referencing and linking of related information. However, not all types of text are equally well-suited to hypertext. Encyclopedias and dictionaries, which contain large amounts of raw, atomically informational data, benefit greatly from hypertext's navigational capabilities. On the other hand, more narrative or holistic concepts may be more difficult to represent effectively using hypertext. Despite the limitations, hypertext remains a powerful tool for navigating and exploring complex information, particularly in the context of reference materials and personal notes. It can help users quickly locate relevant information and build upon their existing knowledge, making it an essential tool for researchers and writers.

    • The benefits and drawbacks of ephemeral and durable note-takingEphemeral note-taking, like Matt Levin's newsletter approach, allows for context and humor but can be challenging to understand complex topics. Durable note-taking aims for evergreen content but may lack context. A combination of both can be useful.

      The way we document and process information can greatly impact its utility and accessibility. Matt Levin's newsletter approach, which involves repeatedly explaining concepts and linking to past explanations, is an argument for ephemeral and inflected note-taking. This contrasts with more durable note-taking, which aims for evergreen content. However, the benefits and drawbacks of each approach are not yet clear. The use of hypertext and reliance on old blogosphere norms can make understanding complex topics challenging, but it also allows for the addition of context and humor. The combination of both types of note-taking, with a journal for current thoughts and more durable notes for reference, can be a useful approach. The amount of prepping or preparation required for effective note-taking is a balancing act, with an ideal amount providing enough context for future reference without becoming burdensome. As a researcher, it's essential to understand the unique challenges and opportunities of creating products in different contexts, such as startups and academic research.

    • Balance contemplation and action for impactful solutionsAllow time for idea development before rushing to build MVPs. Deep thinking leads to powerful designs and interfaces.

      While it's important to test and build minimum viable products (MVPs) to validate ideas, it's equally crucial to allow time for contemplation and idea development before jumping into action. The speaker shares that historically, following advice to build MVPs quickly has led them to create straightforward solutions, but the most powerful designs and interfaces often arise from deep thinking and the expression of new ideas or primitives. The speaker admits that they often need reminding to take action and carve off a piece of an idea to build, rather than getting lost in theory land. The consequence of shipping a product before it felt ready to scale was learning valuable lessons, but it may not have fully captured the potential of the original idea. Ultimately, the balance between contemplation and action is essential for creating impactful and innovative solutions.

    • Design focus and audience applicability are crucial for effective tools for thoughtEffective tools for thought require a strong design focus and understanding of specific audience applicability for rapid development and wide appeal. Crowdfunded research projects, with their large and diverse audiences, can benefit from this approach.

      Effective tools for thought require a strong focus on design and understanding of audience applicability. The speaker learned this through his experience with a general tool for thought, which while functional, could have been more rapidly developed with a clearer focus on specific use cases and audiences. The allure of tools for thought in Silicon Valley comes from their potential for personal empowerment and expansion of capacity, as well as their meta nature. However, the speaker notes that building such tools is primarily a design problem, and the design ideas of engineers, who make up the majority of those interested, are often not optimal. Crowdfunded research projects, on the other hand, benefit from being understandable and interesting to a large audience, as well as having already made some progress. The speaker discovered this through his own experience with a project that, while general, had shown promising results and applicability to many people.

    • Effective funding strategy for researchersCrowdfunding can be an effective way for researchers to fund their projects, but it may not be sustainable for large teams or institutions. Success depends on building a subscriber base early and considering other funding sources like grants, philanthropic organizations, and high net worth individuals.

      Crowdfunding can be an effective way for researchers to fund their projects, especially during certain stages, but it may not be sustainable for large teams or institutions. This approach has worked for some researchers, like the speaker, who started early and built up a subscriber base over time. However, it may not be suitable for everyone, as it doesn't provide the same level of funding as grants or philanthropic organizations. The speaker also noted that they have reached out to high net worth individuals for smaller projects, but the funding from crowdfunding is essential for their research. Additionally, the speaker acknowledged that they may not be the most successful researcher in their field, but they have found success with crowdfunding. Overall, while crowdfunding can be a valuable tool for researchers, it may not be the only solution, and it's essential to consider the specific circumstances and limitations of each project.

    • Researcher values honest inquiry over marketing successResearcher chooses not to promote work extensively or monetize effectively to prioritize honest inquiry

      The researcher in question values honest inquiry over marketing and financial success. He deliberately chooses not to publish "minimum viable papers" or promote his work extensively, as he believes that marketing can distort the research process and lead to exaggerated claims. He admires other researchers who have large audiences but low patronage, attributing this to their approach to marketing and presentation of their membership offerings. The researcher himself has a Substack where supporters can contribute, but he earns a small fraction of what he could make through ads. He is aware that there are more effective ways to monetize his work, but prioritizes honest inquiry over marketing success.

    • Monetizing Creator Work through Subscription ModelsFraming membership as an offering instead of a tip jar can lead to better results for creators using subscription models. Clear presentation and making subscriber-only content visible to a larger audience are also important considerations.

      The way creators monetize their work through subscription models can significantly impact audience engagement. The speaker shared his experience with Patreon and Substack, noting that framing membership as an offering, rather than a tip jar, can lead to better results. He also discussed the challenge of making subscriber-only content visible to a larger audience and the importance of clear presentation. The speaker expressed conflicted feelings about the exclusivity of subscription models and the potential impact on the reach and virality of his work. He suggested that a more integrated presentation of public and private content could help address this issue. Overall, the conversation highlighted the importance of thoughtfully considering the monetization strategies used in creative work and their potential impact on audience engagement and reach.

    • Exploring the benefits of spaced repetition in knowledge work practicesSpaced repetition practices like generating and sharing working notes help retain and recall non-regular information. Understand the content's shape and express it effectively.

      There are various forms of beneficial knowledge work practices that operate through the principle of spaced repetition, which helps in retaining and recalling information. These practices are often embedded in our daily work routines, such as writing research papers, mentoring students, and participating in medical grand rounds. However, the formal practice of generating and sharing working notes, as explored in the discussion, is unique in that it focuses on material that isn't regularly repeated or firmly tethered to our lives. When it comes to creating a personal website, the optimal amount of effort depends on its intended purpose and desired form. Engineers, in particular, may unnecessarily invest in complex technical infrastructure when a simpler solution would suffice. Ultimately, the key is to understand the shape of the content you wish to share and find a way to express it effectively. My personal website, for instance, was not an enormous project for me, despite its impact on commercial projects. The working notes, while not perfect, served their purpose. The discussion also touched upon the speaker's experience at Apple, but no specific question was asked about it.

    • Interconnected Constraints in Apple's Design and DevelopmentEffective communication and collaboration between teams are essential to create a cohesive product at Apple, where each team's constraints impact one another, and individual leaders have significant decision-making power within their domains.

      At Apple, design and development are highly compartmentalized, with each team focusing on specific areas and constraints. However, these constraints are not isolated, but rather interconnected and influenced by each other. For instance, the thermals team determines power budgets for components, which can impact the design decisions of other teams. Communication and collaboration between teams are crucial to ensure a cohesive product. Despite the complexity and size of the company, Apple's hierarchical structure allows for individual leaders to have significant decision-making power within their domains. This delegation of responsibility enables the company to handle the numerous trade-offs and product developments effectively.

    • Apple's leadership structure and effective delegationEffective leaders focus on critical areas, delegate others, and maintain oversight. Some areas, like quantum physics, may not benefit significantly from spaced repetition due to cultural knowledge requirements and lack of early adopters.

      Effective leadership involves focusing on a select few critical areas while delegating the rest. This was exemplified in the discussion about Apple's management structure, where leaders were responsible for a small percentage of decisions they directly controlled, a larger percentage they kept an eye on, and the rest they delegated. This structure allows leaders to maintain focus on what truly matters while ensuring that other tasks are still addressed. Another key point from the conversation was the efficient lack of adoption of spaced repetition in certain areas, such as learning quantum physics. The reasons for this include the need for substantial, difficult-to-transmit cultural knowledge and the lack of early adopters to explore and refine its use in these fields. Additionally, the benefits of spaced repetition may not be as significant in these areas as in others, such as language learning or medical studies, where its use is more widespread. Overall, the discussion provided valuable insights into effective leadership and the adoption of new technologies in various fields.

    Recent Episodes from Dwarkesh Podcast

    Tony Blair - Life of a PM, The Deep State, Lee Kuan Yew, & AI's 1914 Moment

    Tony Blair - Life of a PM, The Deep State, Lee Kuan Yew, & AI's 1914 Moment

    I chatted with Tony Blair about:

    - What he learned from Lee Kuan Yew

    - Intelligence agencies track record on Iraq & Ukraine

    - What he tells the dozens of world leaders who come seek advice from him

    - How much of a PM’s time is actually spent governing

    - What will AI’s July 1914 moment look like from inside the Cabinet?

    Enjoy!

    Watch the video on YouTube. Read the full transcript here.

    Follow me on Twitter for updates on future episodes.

    Sponsors

    - Prelude Security is the world’s leading cyber threat management automation platform. Prelude Detect quickly transforms threat intelligence into validated protections so organizations can know with certainty that their defenses will protect them against the latest threats. Prelude is backed by Sequoia Capital, Insight Partners, The MITRE Corporation, CrowdStrike, and other leading investors. Learn more here.

    - This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue.

    If you’re interested in advertising on the podcast, check out this page.

    Timestamps

    (00:00:00) – A prime minister’s constraints

    (00:04:12) – CEOs vs. politicians

    (00:10:31) – COVID, AI, & how government deals with crisis

    (00:21:24) – Learning from Lee Kuan Yew

    (00:27:37) – Foreign policy & intelligence

    (00:31:12) – How much leadership actually matters

    (00:35:34) – Private vs. public tech

    (00:39:14) – Advising global leaders

    (00:46:45) – The unipolar moment in the 90s



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enJune 26, 2024

    Francois Chollet, Mike Knoop - LLMs won’t lead to AGI - $1,000,000 Prize to find true solution

    Francois Chollet, Mike Knoop - LLMs won’t lead to AGI - $1,000,000 Prize to find true solution

    Here is my conversation with Francois Chollet and Mike Knoop on the $1 million ARC-AGI Prize they're launching today.

    I did a bunch of socratic grilling throughout, but Francois’s arguments about why LLMs won’t lead to AGI are very interesting and worth thinking through.

    It was really fun discussing/debating the cruxes. Enjoy!

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Timestamps

    (00:00:00) – The ARC benchmark

    (00:11:10) – Why LLMs struggle with ARC

    (00:19:00) – Skill vs intelligence

    (00:27:55) - Do we need “AGI” to automate most jobs?

    (00:48:28) – Future of AI progress: deep learning + program synthesis

    (01:00:40) – How Mike Knoop got nerd-sniped by ARC

    (01:08:37) – Million $ ARC Prize

    (01:10:33) – Resisting benchmark saturation

    (01:18:08) – ARC scores on frontier vs open source models

    (01:26:19) – Possible solutions to ARC Prize



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enJune 11, 2024

    Leopold Aschenbrenner - China/US Super Intelligence Race, 2027 AGI, & The Return of History

    Leopold Aschenbrenner - China/US Super Intelligence Race, 2027 AGI, & The Return of History

    Chatted with my friend Leopold Aschenbrenner on the trillion dollar nationalized cluster, CCP espionage at AI labs, how unhobblings and scaling can lead to 2027 AGI, dangers of outsourcing clusters to Middle East, leaving OpenAI, and situational awareness.

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Follow me on Twitter for updates on future episodes. Follow Leopold on Twitter.

    Timestamps

    (00:00:00) – The trillion-dollar cluster and unhobbling

    (00:20:31) – AI 2028: The return of history

    (00:40:26) – Espionage & American AI superiority

    (01:08:20) – Geopolitical implications of AI

    (01:31:23) – State-led vs. private-led AI

    (02:12:23) – Becoming Valedictorian of Columbia at 19

    (02:30:35) – What happened at OpenAI

    (02:45:11) – Accelerating AI research progress

    (03:25:58) – Alignment

    (03:41:26) – On Germany, and understanding foreign perspectives

    (03:57:04) – Dwarkesh’s immigration story and path to the podcast

    (04:07:58) – Launching an AGI hedge fund

    (04:19:14) – Lessons from WWII

    (04:29:08) – Coda: Frederick the Great



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enJune 04, 2024

    John Schulman (OpenAI Cofounder) - Reasoning, RLHF, & Plan for 2027 AGI

    John Schulman (OpenAI Cofounder) - Reasoning, RLHF, & Plan for 2027 AGI

    Chatted with John Schulman (cofounded OpenAI and led ChatGPT creation) on how posttraining tames the shoggoth, and the nature of the progress to come...

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

    Timestamps

    (00:00:00) - Pre-training, post-training, and future capabilities

    (00:16:57) - Plan for AGI 2025

    (00:29:19) - Teaching models to reason

    (00:40:50) - The Road to ChatGPT

    (00:52:13) - What makes for a good RL researcher?

    (01:00:58) - Keeping humans in the loop

    (01:15:15) - State of research, plateaus, and moats

    Sponsors

    If you’re interested in advertising on the podcast, fill out this form.

    * Your DNA shapes everything about you. Want to know how? Take 10% off our Premium DNA kit with code DWARKESH at mynucleus.com.

    * CommandBar is an AI user assistant that any software product can embed to non-annoyingly assist, support, and unleash their users. Used by forward-thinking CX, product, growth, and marketing teams. Learn more at commandbar.com.



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe
    Dwarkesh Podcast
    enMay 15, 2024

    Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

    Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

    Mark Zuckerberg on:

    - Llama 3

    - open sourcing towards AGI

    - custom silicon, synthetic data, & energy constraints on scaling

    - Caesar Augustus, intelligence explosion, bioweapons, $10b models, & much more

    Enjoy!

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Human edited transcript with helpful links here.

    Timestamps

    (00:00:00) - Llama 3

    (00:08:32) - Coding on path to AGI

    (00:25:24) - Energy bottlenecks

    (00:33:20) - Is AI the most important technology ever?

    (00:37:21) - Dangers of open source

    (00:53:57) - Caesar Augustus and metaverse

    (01:04:53) - Open sourcing the $10b model & custom silicon

    (01:15:19) - Zuck as CEO of Google+

    Sponsors

    If you’re interested in advertising on the podcast, fill out this form.

    * This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue. Learn more at stripe.com.

    * V7 Go is a tool to automate multimodal tasks using GenAI, reliably and at scale. Use code DWARKESH20 for 20% off on the pro plan. Learn more here.

    * CommandBar is an AI user assistant that any software product can embed to non-annoyingly assist, support, and unleash their users. Used by forward-thinking CX, product, growth, and marketing teams. Learn more at commandbar.com.



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe

    Sholto Douglas & Trenton Bricken - How to Build & Understand GPT-7's Mind

    Sholto Douglas & Trenton Bricken - How to Build & Understand GPT-7's Mind

    Had so much fun chatting with my good friends Trenton Bricken and Sholto Douglas on the podcast.

    No way to summarize it, except: 

    This is the best context dump out there on how LLMs are trained, what capabilities they're likely to soon have, and what exactly is going on inside them.

    You would be shocked how much of what I know about this field, I've learned just from talking with them.

    To the extent that you've enjoyed my other AI interviews, now you know why.

    So excited to put this out. Enjoy! I certainly did :)

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. 

    There's a transcript with links to all the papers the boys were throwing down - may help you follow along.

    Follow Trenton and Sholto on Twitter.

    Timestamps

    (00:00:00) - Long contexts

    (00:16:12) - Intelligence is just associations

    (00:32:35) - Intelligence explosion & great researchers

    (01:06:52) - Superposition & secret communication

    (01:22:34) - Agents & true reasoning

    (01:34:40) - How Sholto & Trenton got into AI research

    (02:07:16) - Are feature spaces the wrong way to think about intelligence?

    (02:21:12) - Will interp actually work on superhuman models

    (02:45:05) - Sholto’s technical challenge for the audience

    (03:03:57) - Rapid fire



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe

    Demis Hassabis - Scaling, Superhuman AIs, AlphaZero atop LLMs, Rogue Nations Threat

    Demis Hassabis - Scaling, Superhuman AIs, AlphaZero atop LLMs, Rogue Nations Threat

    Here is my episode with Demis Hassabis, CEO of Google DeepMind

    We discuss:

    * Why scaling is an artform

    * Adding search, planning, & AlphaZero type training atop LLMs

    * Making sure rogue nations can't steal weights

    * The right way to align superhuman AIs and do an intelligence explosion

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here.

    Timestamps

    (0:00:00) - Nature of intelligence

    (0:05:56) - RL atop LLMs

    (0:16:31) - Scaling and alignment

    (0:24:13) - Timelines and intelligence explosion

    (0:28:42) - Gemini training

    (0:35:30) - Governance of superhuman AIs

    (0:40:42) - Safety, open source, and security of weights

    (0:47:00) - Multimodal and further progress

    (0:54:18) - Inside Google DeepMind



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe

    Patrick Collison (Stripe CEO) - Craft, Beauty, & The Future of Payments

    Patrick Collison (Stripe CEO) - Craft, Beauty, & The Future of Payments

    We discuss:

    * what it takes to process $1 trillion/year

    * how to build multi-decade APIs, companies, and relationships

    * what's next for Stripe (increasing the GDP of the internet is quite an open ended prompt, and the Collison brothers are just getting started).

    Plus the amazing stuff they're doing at Arc Institute, the financial infrastructure for AI agents, playing devil's advocate against progress studies, and much more.

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

    Timestamps

    (00:00:00) - Advice for 20-30 year olds

    (00:12:12) - Progress studies

    (00:22:21) - Arc Institute

    (00:34:27) - AI & Fast Grants

    (00:43:46) - Stripe history

    (00:55:44) - Stripe Climate

    (01:01:39) - Beauty & APIs

    (01:11:51) - Financial innards

    (01:28:16) - Stripe culture & future

    (01:41:56) - Virtues of big businesses

    (01:51:41) - John



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe

    Tyler Cowen - Hayek, Keynes, & Smith on AI, Animal Spirits, Anarchy, & Growth

    Tyler Cowen - Hayek, Keynes, & Smith on AI, Animal Spirits, Anarchy, & Growth

    It was a great pleasure speaking with Tyler Cowen for the 3rd time.

    We discussed GOAT: Who is the Greatest Economist of all Time and Why Does it Matter?, especially in the context of how the insights of Hayek, Keynes, Smith, and other great economists help us make sense of AI, growth, animal spirits, prediction markets, alignment, central planning, and much more.

    The topics covered in this episode are too many to summarize. Hope you enjoy!

    Watch on YouTube. Listen on Apple PodcastsSpotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

    Timestamps

    (0:00:00) - John Maynard Keynes

    (00:17:16) - Controversy

    (00:25:02) - Fredrick von Hayek

    (00:47:41) - John Stuart Mill

    (00:52:41) - Adam Smith

    (00:58:31) - Coase, Schelling, & George

    (01:08:07) - Anarchy

    (01:13:16) - Cheap WMDs

    (01:23:18) - Technocracy & political philosophy

    (01:34:16) - AI & Scaling



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe

    Lessons from The Years of Lyndon Johnson by Robert Caro [Narration]

    Lessons from The Years of Lyndon Johnson by Robert Caro [Narration]

    This is a narration of my blog post, Lessons from The Years of Lyndon Johnson by Robert Caro.

    You read the full post here: https://www.dwarkeshpatel.com/p/lyndon-johnson

    Listen on Apple Podcasts, Spotify, or any other podcast platform. Follow me on Twitter for updates on future posts and episodes.



    Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe

    Related Episodes

    Cognitive Psychology & Neuroeducation with Prof. Michelle Miller, PhD

    Cognitive Psychology & Neuroeducation with Prof. Michelle Miller, PhD

    Dive into the science of Cognitive Psychology together with Prof. Michelle Miller, PhD! Find out about the newest development in this exciting field! Discover how Cognitive Psychology can help us learn! Get bullet-proof strategies on how to succeed in your exams!

    About the Podcast Guest:

    Michelle Miller, Ph.D is a Professor at the Department of Psychological Sciences & President's Distinguished Teaching Fellow

    Michelle's Books:

    https://www.hup.harvard.edu/catalog.php?isbn=9780674660021

    For the Internet generation, educational technology designed with the brain in mind offers a natural pathway to the pleasures and rewards of deep learning. Drawing on neuroscience and cognitive psychology, Michelle Miller shows how attention, memory, critical thinking, and analytical reasoning can be enhanced through technology-aided approaches.

    https://wvupressonline.com/remembering-and-forgetting-in-the-age-of-technology 

    wvupressonline.com
     
    Remembering and Forgetting in the Age of Technology Michelle D. Miller

    Saundra Maguire's book: https://www.amazon.com/Teach-Students-How-Learn-Metacognition/dp/162036316X 

    Rebecca Campbell: https://www.rebeccacampbellphd.com/about 

    Michelle's website and blog: michellemillerphd.com

    Michelle's free Substack newsletter (coming soon in 2023!)

    https://michellemillerphd.substack.com/p/coming-soon

    michellemillerphd.substack.com
    This is Michelle Miller's R3 Newsletter, a newsletter about Research, Resources, and Reflections on Teaching.

    Michelle's roundup of classic books on psychology: https://www.michellemillerphd.com/fifteen-must-read-classic-books-on-human-psychology-and-what-im-reading-and-writing-next/

    About the podcast host:

    The Neurocareers podcast is brought to you by The Institute of Neuroapproaches ( https://www.neuroapproaches.org/ ) and its founder - Milena Korostenskaja, Ph.D. (Dr. K) - a neuroscience educator, neuroscience research consultant, and career coach for students and recent graduates in neuroscience and neurotechnologies. As a professional coach with a background in the field, Dr. K understands the unique challenges and opportunities facing students in this field. Whether you're looking to apply for jobs, build your professional network, or develop your research skills, Dr. K can provide personalized coaching and support to help you succeed. What you'll get:

    • One-on-one coaching sessions to help you identify and pursue your career goals;
    • Tailored guidance on job search strategies, resume and cover letter development, and interview preparation;
    • Access to a network of professionals in the field of neuroscience and neurotechnologies;
    • Ongoing support and guidance to help you stay on track and achieve your goals.

    You can always schedule a free neurocareer consultation/coaching session with Dr. K at: https://neuroapproaches.as.me/free-neurocareer-consultation

    Also, find your support in the Your Support Station podcast with Dr. K here:

    https://www.neuroapproaches.org/podcast-2

    S1:E29: What dementia ACTUALLY looks like

    S1:E29: What dementia ACTUALLY looks like

    "He described all the symptoms of dementia, but then said his mom didn't have dementia." What does dementia ACTUALLY look like? Why is it so challenging to stage someone? Let's find out in this episode of my podcast—the final episode of 2019! Sponsored by Flying Angels, flyingangels.com

    Are you enjoying my podcast? Please rate & review! Learn more about how I can help your organization here: https://rachaelwonderlin.com/workwithme/