Logo
    Search

    How YC fosters AI Innovation with Garry Tan

    enMay 23, 2024

    Podcast Summary

    • From Founder to Investor: Gary Tan's Journey through Y CombinatorGary Tan's success story showcases the value of Y Combinator, which helped him transform from a founder to an investor during the startup scene's infancy. Early successes like Airbnb and Dropbox proved its worth, and today, Y Combinator funds 15-25 companies per group partner.

      Gary Tan's journey from founding a company to becoming an investor at Y Combinator was influenced by the accessibility of information and resources provided by Paul Graham's essays and the early days of the startup scene. In 2008, when Gary went through Y Combinator, the startup ecosystem was still in its infancy, and there was skepticism from the investor community. However, the success stories of companies like Airbnb and Dropbox proved that Y Combinator was a valuable resource for entrepreneurs. Some notable companies from Gary's batch included Mike Montano's company, which was acquired by Twitter, and Chris Golda's startup in comment analytics. Today, Y Combinator has expanded significantly, with 14 group partners funding between 15-25 companies each. Gary's experience is a testament to how the startup scene has evolved, making it more accessible for aspiring entrepreneurs to learn and succeed.

    • Building relationships and staying persistent in tech can lead to significant impactsStaying the course, supporting technically skilled individuals, and identifying the next generation of tech visionaries are keys to success in the tech industry

      Building relationships and staying persistent in the tech industry can lead to significant impacts and opportunities in the long run. D Scott Phoenix, a veteran in Silicon Valley, shared his experiences of starting a company with friends over pizza and beers, and later selling it to Google. He emphasized the importance of longevity and early non-zero sum actions in shaping one's career and reputation. The social media space in the early 2000s serves as a parallel example, with numerous failed attempts before Facebook's success. The tech industry is filled with similar stories, and the common thread is the importance of staying the course and supporting technically skilled individuals. In the context of Y Combinator's current focus on AI-centric companies, the secret to success might be in identifying and nurturing the next generation of tech visionaries.

    • Combining technical expertise and domain knowledge leads to innovative applicationsFounders with a technical background and deep understanding of a specific domain can create profitable businesses by transforming powerful APIs into innovative applications. Direct customer interaction is crucial for gaining insights into the market.

      The combination of technical expertise and deep understanding of specific domains can lead to innovative applications of large language models. The founders of a company gained access to a powerful API and, with a lawyer's perspective on time and motion work, transformed it into a profitable business. This partnership between a technical expert and a domain expert is a common pattern among successful startups. Moreover, the importance of direct interaction with customers, founders, and even litigators, rather than relying on second or third-hand information, is emphasized for gaining insights into the market. In the context of Y Combinator batches, the increasing number of AI-focused startups indicates the growing interest and potential in this field. By examining these startups, common themes and trends can be identified, providing valuable insights into the current state and future direction of AI innovation.

    • Companies Shift Focus to AI Applications70% of new tech companies focus on AI, 2/3rds of these focus on practical applications, AI's direct problem-solving capabilities drive revenue growth, disruption opens up new possibilities, closed-source and open-source approaches coexist

      We're witnessing a significant shift in technology with the rise of AI and machine learning, which is acting as a lens for founders to build companies. Approximately 70% of these companies are related to AI, with 2/3rds of them focusing on practical applications. The technology itself can be seen as a concentrated form of intelligence, though it's currently limited and requires careful implementation. The growth of these early-stage startups, many of which are still small teams, is remarkable, with revenue growth increasing significantly over short periods of time. This growth can be attributed to the direct problem-solving capabilities of AI, which can often outperform human efforts or traditional outsourcing solutions. The disruption brought about by AI is opening up new possibilities and creating a mandate for companies to explore AI applications, regardless of their understanding or expertise in the field. Additionally, there's a parallel trend of closed-source and open-source approaches, with both coexisting and influencing each other in the ecosystem.

    • YC's culture of scrappiness and shippingYC encourages founders to focus on near-term milestones and tangible progress, fostering an environment that helps big ideas become achievable goals through weekly accountability and social pressure.

      Y Combinator (YC) fosters a culture of scrappiness and shipping, pushing founders to focus on near-term milestones and tangible progress. While some companies may explore building foundational models, YC's strength lies in its ability to help founders see the potential for big ideas and push them towards achievable goals. This culture is rooted in the experiences of successful companies like Cruise, which started with a simple demo and attracted talent and capital based on tangible progress. The weekly accountability of group office hours adds social pressure, encouraging founders to meet their goals and keep moving forward. Ultimately, the ability to run fast and consistently in the startup marathon sets the stage for compounding success.

    • Startups vs. Established Companies: The Unexpected BattleSurprisingly, some startups outperform established companies, but excessive funding can hinder decision-making and flexibility.

      The startup world can be compared to mice running around the feet of an elephant, where the elephant represents the large, established companies. Deciding which startups will succeed and which will be "stomped" is not always clear. Surprisingly, some startups that were assumed to have no chance against incumbents have managed to win or do well. For instance, companies like Scale, which could have been easily replicated by tech giants like Google or Amazon, instead managed to succeed. However, the elephant is not as strong as it used to be due to the immense resources and inability to adapt quickly to new challenges. This creates opportunities for startups, but raising too much money can potentially lead to making the wrong decisions and being unable to cut costs when necessary. The Y Combinator (YC) program has seen shifts over time, with fewer mergers and a trend towards older founders joining the program. The average age of YC founders is now around 27 years old.

    • Impact of market openness and funding landscape on founder demographicsMarket openness influences the age and expertise of successful founders. Younger, innovative teams thrive during market openness, while older, experienced founders may have an advantage in closed markets. Funding landscape also plays a role, with more funding leading to younger, highly technical teams and less funding resulting in less capital raised.

      The age and demographics of tech startup founders are influenced by the openness and dynamics of markets. During periods of market openness, younger founders with innovative ideas can thrive. Conversely, when markets are more closed and require domain expertise, older and more experienced founders may have an advantage. Additionally, the funding landscape can also impact the profile of founders. Historically, increases in funding amounts by Y Combinator have led to waves of innovative, young, and highly technical teams. Conversely, during periods of lower funding, the median YC startup may raise less money. Overall, the tech ecosystem is constantly evolving, and the ability to adapt and navigate these changes is crucial for success. Being in person and immersed in the SF ecosystem can provide valuable resources and opportunities for startups, helping them go from 0 to 1.

    • The unique dynamic in San Francisco fosters collaboration and innovation in the tech industrySurrounding oneself with intelligent and ambitious individuals in SF leads to valuable discussions and advancements, as emphasized by Paul Graham and Gary's leadership.

      Being surrounded by intelligent and ambitious individuals is crucial for driving innovation and progress in the tech industry. The unique dynamic in San Francisco allows for this collaboration between toolmakers, researchers, and industry practitioners, leading to valuable discussions and advancements. Paul Graham, widely credited for his role in revitalizing Y Combinator, emphasizes the importance of YC as a beacon for those who believe in using technology to solve problems. He envisions YC as an institution that fosters a movement of like-minded individuals and provides them with the resources and network to create and manifest their ideas. In addition, Gary's leadership in San Francisco sets an expectation for excellence and encourages development across various communities, further contributing to the city's ambition and growth.

    • Advocating for Quality EducationEffective governments provide essential services, including quality education. Lack of access to math education in public schools can hinder future potential. Speak out, vote for initiatives, and use resources to extend educational opportunities to all.

      Effective government plays a crucial role in providing essential services that private enterprise cannot, such as quality education. The speaker, a Stanford graduate and tech creator, emphasizes the importance of algebra education for children, especially in a diverse city like San Francisco, which attracts ambitious people from around the world. However, they lament that the lack of access to math education in public schools can hinder the next generation's potential to contribute to the vibrant community. The speaker calls for open dialogue and encourages individuals to speak out and vote for initiatives that extend educational opportunities to all. In essence, the speaker advocates for the extension of ladders for upward mobility and access to education for everyone, regardless of their socio-economic background. The speaker also highlights the importance of being informed and using resources like voter guides to make informed decisions.

    • Open conversations about education and access in competition mathNormal humans are engaging in nuanced discussions about education and access in competition math, fostering civic discourse and ensuring equal opportunities for all.

      There's a need for more open conversations about education and access, particularly in the realm of competition math, without framing it as an issue of racism or other complexities. Normal human beings, trying to live their lives, have put blinders on these issues due to their complexity and potential controversy. However, there's a growing trend of people speaking up and engaging in civic discourse with nuance, making it possible to discuss these topics effectively. Gary's YouTube channel is an example of this, focusing on AI and other related topics while fostering a grassroots level of civic engagement. Overall, it's essential to have open and nuanced discussions about education and access to ensure that everyone has an equal opportunity to succeed. To keep up with these conversations, follow the No Priors Pryors podcast on Twitter, subscribe to their YouTube channel, and sign up for emails or find transcripts at nodashpriors.com.

    Recent Episodes from No Priors: Artificial Intelligence | Machine Learning | Technology | Startups

    State Space Models and Real-time Intelligence with Karan Goel and Albert Gu from Cartesia

    State Space Models and Real-time Intelligence with Karan Goel and Albert Gu from Cartesia
    This week on No Priors, Sarah Guo and Elad Gil sit down with Karan Goel and Albert Gu from Cartesia. Karan and Albert first met as Stanford AI Lab PhDs, where their lab invented Space Models or SSMs, a fundamental new primitive for training large-scale foundation models. In 2023, they Founded Cartesia to build real-time intelligence for every device. One year later, Cartesia released Sonic which generates high quality and lifelike speech with a model latency of 135ms—the fastest for a model of this class. Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @krandiash | @_albertgu Show Notes:  (0:00) Introduction (0:28) Use Cases for Cartesia and Sonic  (1:32) Karan Goel & Albert Gu’s professional backgrounds (5:06) Steady State Models (SSMs) versus Transformer Based Architectures  (11:51) Domain Applications for Hybrid Approaches  (13:10) Text to Speech and Voice (17:29) Data, Size of Models and Efficiency  (20:34) Recent Launch of Text to Speech Product (25:01) Multimodality & Building Blocks (25:54) What’s Next at Cartesia?  (28:28) Latency in Text to Speech (29:30) Choosing Research Problems Based on Aesthetic  (31:23) Product Demo (32:48) Cartesia Team & Hiring

    Can AI replace the camera? with Joshua Xu from HeyGen

    Can AI replace the camera? with Joshua Xu from HeyGen
    AI video generation models still have a long way to go when it comes to making compelling and complex videos but the HeyGen team are well on their way to streamlining the video creation process by using a combination of language, video, and voice models to create videos featuring personalized avatars, b-roll, and dialogue. This week on No Priors, Joshua Xu the co-founder and CEO of HeyGen,  joins Sarah and Elad to discuss how the HeyGen team broke down the elements of a video and built or found models to use for each one, the commercial applications for these AI videos, and how they’re safeguarding against deep fakes.  Links from episode: HeyGen McDonald’s commercial Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil |  @joshua_xu_ Show Notes:  (0:00) Introduction (3:08) Applications of AI content creation (5:49) Best use cases for Hey Gen (7:34) Building for quality in AI video generation (11:17) The models powering HeyGen (14:49) Research approach (16:39) Safeguarding against deep fakes (18:31) How AI video generation will change video creation (24:02) Challenges in building the model (26:29) HeyGen team and company

    How the ARC Prize is democratizing the race to AGI with Mike Knoop from Zapier

    How the ARC Prize is democratizing  the race to AGI with Mike Knoop from Zapier
    The first step in achieving AGI is nailing down a concise definition and  Mike Knoop, the co-founder and Head of AI at Zapier, believes François Chollet got it right when he defined general intelligence as a system that can efficiently acquire new skills. This week on No Priors, Miked joins Elad to discuss ARC Prize which is a multi-million dollar non-profit public challenge that is looking for someone to beat the Abstraction and Reasoning Corpus (ARC) evaluation. In this episode, they also get into why Mike thinks LLMs will not get us to AGI, how Zapier is incorporating AI into their products and the power of agents, and why it’s dangerous to regulate AGI before discovering its full potential.  Show Links: About the Abstraction and Reasoning Corpus Zapier Central Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @mikeknoop Show Notes:  (0:00) Introduction (1:10) Redefining AGI (2:16) Introducing ARC Prize (3:08) Definition of AGI (5:14) LLMs and AGI (8:20) Promising techniques to developing AGI (11:0) Sentience and intelligence (13:51) Prize model vs investing (16:28) Zapier AI innovations (19:08) Economic value of agents (21:48) Open source to achieve AGI (24:20) Regulating AI and AGI

    The evolution and promise of RAG architecture with Tengyu Ma from Voyage AI

    The evolution and promise of RAG architecture with Tengyu Ma from Voyage AI
    After Tengyu Ma spent years at Stanford researching AI optimization, embedding models, and transformers, he took a break from academia to start Voyage AI which allows enterprise customers to have the most accurate retrieval possible through the most useful foundational data. Tengyu joins Sarah on this week’s episode of No priors to discuss why RAG systems are winning as the dominant architecture in enterprise and the evolution of foundational data that has allowed RAG to flourish. And while fine-tuning is still in the conversation, Tengyu argues that RAG will continue to evolve as the cheapest, quickest, and most accurate system for data retrieval.  They also discuss methods for growing context windows and managing latency budgets, how Tengyu’s research has informed his work at Voyage, and the role academia should play as AI grows as an industry.  Show Links: Tengyu Ma Key Research Papers: Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training Non-convex optimization for machine learning: design, analysis, and understanding Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss Larger language models do in-context learning differently, 2023 Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning On the Optimization Landscape of Tensor Decompositions Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @tengyuma Show Notes:  (0:00) Introduction (1:59) Key points of Tengyu’s research (4:28) Academia compared to industry (6:46) Voyage AI overview (9:44) Enterprise RAG use cases (15:23) LLM long-term memory and token limitations (18:03) Agent chaining and data management (22:01) Improving enterprise RAG  (25:44) Latency budgets (27:48) Advice for building RAG systems (31:06) Learnings as an AI founder (32:55) The role of academia in AI

    How YC fosters AI Innovation with Garry Tan

    How YC fosters AI Innovation with Garry Tan
    Garry Tan is a notorious founder-turned-investor who is now running one of the most prestigious accelerators in the world, Y Combinator. As the president and CEO of YC, Garry has been credited with reinvigorating the program. On this week’s episode of No Priors, Sarah, Elad, and Garry discuss the shifting demographics of YC founders and how AI is encouraging younger founders to launch companies, predicting which early stage startups will have longevity, and making YC a beacon for innovation in AI companies. They also discussed the importance of building companies in person and if San Francisco is, in fact, back.  Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @garrytan Show Notes:  (0:00) Introduction (0:53) Transitioning from founder to investing (5:10) Early social media startups (7:50) Trend predicting at YC (10:03) Selecting YC founders (12:06) AI trends emerging in YC batch (18:34) Motivating culture at YC (20:39) Choosing the startups with longevity (24:01) Shifting YC found demographics (29:24) Building in San Francisco  (31:01) Making YC a beacon for creators (33:17) Garry Tan is bringing San Francisco back

    The Data Foundry for AI with Alexandr Wang from Scale

    The Data Foundry for AI with Alexandr Wang from Scale
    Alexandr Wang was 19 when he realized that gathering data will be crucial as AI becomes more prevalent, so he dropped out of MIT and started Scale AI. This week on No Priors, Alexandr joins Sarah and Elad to discuss how Scale is providing infrastructure and building a robust data foundry that is crucial to the future of AI. While the company started working with autonomous vehicles, they’ve expanded by partnering with research labs and even the U.S. government.   In this episode, they get into the importance of data quality in building trust in AI systems and a possible future where we can build better self-improvement loops, AI in the enterprise, and where human and AI intelligence will work together to produce better outcomes.  Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @alexandr_wang (0:00) Introduction (3:01) Data infrastructure for autonomous vehicles (5:51) Data abundance and organization (12:06)  Data quality and collection (15:34) The role of human expertise (20:18) Building trust in AI systems (23:28) Evaluating AI models (29:59) AI and government contracts (32:21) Multi-modality and scaling challenges

    Music consumers are becoming the creators with Suno CEO Mikey Shulman

    Music consumers are becoming the creators with Suno CEO Mikey Shulman
    Mikey Shulman, the CEO and co-founder of Suno, can see a future where the Venn diagram of music creators and consumers becomes one big circle. The AI music generation tool trying to democratize music has been making waves in the AI community ever since they came out of stealth mode last year. Suno users can make a song complete with lyrics, just by entering a text prompt, for example, “koto boom bap lofi intricate beats.” You can hear it in action as Mikey, Sarah, and Elad create a song live in this episode.  In this episode, Elad, Sarah, And Mikey talk about how the Suno team took their experience making at transcription tool and applied it to music generation, how the Suno team evaluates aesthetics and taste because there is no standardized test you can give an AI model for music, and why Mikey doesn’t think AI-generated music will affect people’s consumption of human made music.  Listen to the full songs played and created in this episode: Whispers of Sakura Stone  Statistical Paradise Statistical Paradise 2 Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @MikeyShulman Show Notes:  (0:00) Mikey’s background (3:48) Bark and music generation (5:33) Architecture for music generation AI (6:57) Assessing music quality (8:20) Mikey’s music background as an asset (10:02) Challenges in generative music AI (11:30) Business model (14:38) Surprising use cases of Suno (18:43) Creating a song on Suno live (21:44) Ratio of creators to consumers (25:00) The digitization of music (27:20) Mikey’s favorite song on Suno (29:35) Suno is hiring

    Context windows, computer constraints, and energy consumption with Sarah and Elad

    Context windows, computer constraints, and energy consumption with Sarah and Elad
    This week on No Priors hosts, Sarah and Elad are catching up on the latest AI news. They discuss the recent developments in AI music generation, and if you’re interested in generative AI music, stay tuned for next week’s interview! Sarah and Elad also get into device-resident models, AI hardware, and ask just how smart smaller models can really get. These hardware constraints were compared to the hurdles AI platforms are continuing to face including computing constraints, energy consumption, context windows, and how to best integrate these products in apps that users are familiar with.  Have a question for our next host-only episode or feedback for our team? Reach out to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil  Show Notes:  (0:00) Intro (1:25) Music AI generation (4:02) Apple’s LLM (11:39) The role of AI-specific hardware (15:25) AI platform updates (18:01) Forward thinking in investing in AI (20:33) Unlimited context (23:03) Energy constraints

    Cognition’s Scott Wu on how Devin, the AI software engineer, will work for you

    Cognition’s Scott Wu on how Devin, the AI software engineer, will work for you
    Scott Wu loves code. He grew up competing in the International Olympiad in Informatics (IOI) and is a world class coder, and now he's building an AI agent designed to create more, not fewer, human engineers. This week on No Priors, Sarah and Elad talk to Scott, the co-founder and CEO of Cognition, an AI lab focusing on reasoning. Recently, the Cognition team released a demo of Devin, an AI software engineer that can increasingly handle entire tasks end to end. In this episode, they talk about why the team built Devin with a UI that mimics looking over another engineer’s shoulder as they work and how this transparency makes for a better result. Scott discusses why he thinks Devin will make it possible for there to be more human engineers in the world, and what will be important for software engineers to focus on as these roles evolve. They also get into how Scott thinks about building the Cognition team and that they’re just getting started.  Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @ScottWu46 Show Notes:  (0:00) Introduction (1:12) IOI training and community (6:39) Cognition’s founding team (8:20) Meet Devin (9:17) The discourse around Devin (12:14) Building Devin’s UI (14:28) Devin’s strengths and weakness  (18:44) The evolution of coding agents (22:43) Tips for human engineers (26:48) Hiring at Cognition

    OpenAI’s Sora team thinks we’ve only seen the "GPT-1 of video models"

    OpenAI’s Sora team thinks we’ve only seen the "GPT-1 of video models"
    AI-generated videos are not just leveled-up image generators. But rather, they could be a big step forward on the path to AGI. This week on No Priors, the team from Sora is here to discuss OpenAI’s recently announced generative video model, which can take a text prompt and create realistic, visually coherent, high-definition clips that are up to a minute long. Sora team leads, Aditya Ramesh, Tim Brooks, and Bill Peebles join Elad and Sarah to talk about developing Sora. The generative video model isn’t yet available for public use but the examples of its work are very impressive. However, they believe we’re still in the GPT-1 era of AI video models and are focused on a slow rollout to ensure the model is in the best place possible to offer value to the user and more importantly they’ve applied all the safety measures possible to avoid deep fakes and misinformation. They also discuss what they’re learning from implementing diffusion transformers, why they believe video generation is taking us one step closer to AGI, and why entertainment may not be the main use case for this tool in the future.  Show Links: Bling Zoo video Man eating a burger video Tokyo Walk video Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @_tim_brooks l @billpeeb l @model_mechanic Show Notes:  (0:00) Sora team Introduction (1:05) Simulating the world with Sora (2:25) Building the most valuable consumer product (5:50) Alternative use cases and simulation capabilities (8:41) Diffusion transformers explanation (10:15) Scaling laws for video (13:08) Applying end-to-end deep learning to video (15:30) Tuning the visual aesthetic of Sora (17:08) The road to “desktop Pixar” for everyone (20:12) Safety for visual models (22:34) Limitations of Sora (25:04) Learning from how Sora is learning (29:32) The biggest misconceptions about video models