323. Here’s Why All Your Projects Are Always Late — and What to Do About It

Podcast Summary

• The Challenges of Finishing Projects on Time and Within Budget: A Case Study of the Second Avenue Subway in NYC: People often underestimate project complexities and their own abilities, leading to unrealistic expectations, missed deadlines, and frustration. To improve project management, learn from past experiences, adjust timelines and budgets as needed, and break down large projects into smaller tasks with clear goals.

  The Second Avenue Subway in New York City, first proposed in 1968, has become a notorious example of a project that has never been completed on time. Despite numerous attempts, fiscal crises and poor rock quality caused delays and massive cost overruns. The story is a reminder of how challenging it can be to finish projects on time and within budget. Psychologist Roger Bueller, who studies social cognition and judgment, shares his personal experience with this phenomenon and how he's observed it in others. He's found that people often underestimate the time and resources required to complete projects, leading to missed deadlines and frustration. Bueller's research suggests that people tend to overestimate their abilities and underestimate the complexity of tasks, leading to unrealistic expectations and poor planning. To improve project management, it's essential to learn from past experiences, especially failures, and be open to adjusting timelines and budgets as needed. Additionally, breaking down large projects into smaller tasks and setting clear goals can help keep projects on track.

• The Planning Fallacy: Underestimating Project Completion Time: People tend to underestimate project completion time due to idealized mental simulations, leading to a gap between intentions and behaviors (Planning Fallacy). Awareness of this bias can help improve planning and increase productivity.

  The planning fallacy is a common tendency for individuals to underestimate the time it will take to complete a project, despite knowing that similar projects have typically taken longer in the past. This optimistic prediction is influenced by mental simulations that can be idealized and oversimplified, leading to a gap between intentions and behaviors. The planning fallacy, as discovered by researchers like Roger Bueller and named by psychologists Daniel Kahneman and Amos Tversky, affects various domains, from academic projects to everyday activities. It's essential to understand this bias to improve planning and increase productivity. Additionally, the optimism bias, which is related to the planning fallacy, has positive aspects, such as driving motivation and better health. However, it can also lead to unrealistic expectations and disappointment. Being aware of these biases can help us plan more effectively and realistically.

• Impact of Optimism Bias on Estimates and Preparedness: Optimism bias, while beneficial, can lead to inaccurate estimates and lack of preparedness for negative outcomes. It's a flexible adaptive trait, but can hinder project completion on time and within budget. Researchers are exploring ways to harness it for productivity.

  Our natural optimism bias, while beneficial for survival and progress, can also lead to inaccurate estimates and a lack of preparedness for potential negative outcomes. This bias, according to research, is not a design flaw but rather an adaptive trait with flexible responses to the environment. For instance, firefighters under stress show increased sensitivity to negative information. The optimism bias may have played a role in human achievements, but it can also hinder us from completing projects on time and within budget. Researchers like Katherine Milkman are exploring ways to harness this bias and use it to improve productivity.

• The Planning Fallacy and Its Influencing Factors: The Planning Fallacy, caused by overconfidence and coordination neglect, leads to underestimation of project time and resources. Procrastination and distractions in the digital age worsen this effect. Using project management tools like Asana can help mitigate these challenges.

  The planning fallacy, a common phenomenon where people underestimate the time and resources needed to complete a project, is influenced by individual biases like overconfidence and coordination neglect, especially in group projects. Overconfidence arises from societal reinforcement, making us feel better about ourselves when we express confidence. Coordination neglect is the failure to consider the challenges of coordinating work with others, making the planning fallacy more pronounced in larger teams. Procrastination, caused by impulse control failures, can worsen the planning fallacy by delaying project start times. The digital age, with its constant distractions and information overload, further tests our impulse control and contributes to decreased productivity. To combat these challenges, tools like Asana can help teams work together more effectively and efficiently.

• Google and Facebook's Early Product Managers Struggled with Inefficient Project Management Tools: Despite having resources and talent, tech giants like Google and Facebook faced inefficiencies in project management. Frustrated product managers built their own tools to streamline workflow, leading to the creation of Asana.

  Even at tech giants like Google and Facebook, where resources and talent are abundant, there was a need for effective project management tools. Rosenstein, an early product manager at Google, envisioned spending his time collaborating with engineers and designers, but instead found himself bogged down in administrative tasks. Despite discovering that other companies faced similar issues, he couldn't find a satisfactory solution. Frustrated, he started building his own software on nights and weekends, which grew virally within Google. When he joined Facebook, he found similar challenges and, with co-founder Dustin Moskowitz, built an internal tool to manage their workflow. Realizing the potential for a more general solution, they left Facebook to start Asana and develop it further. This experience underscores the importance of efficient project management tools, even in organizations with significant resources.

• Acknowledging unintended consequences in software design: Underestimating project time and cost is common, but accepting and addressing unintended consequences can lead to successful software design

  Creating software to solve the problem of distraction caused by software comes with unintended consequences. The planning fallacy, identified by psychologists Danny Kahneman and Amos Tversky, occurs when people underestimate the time and cost of projects, focusing too much on the unique aspects of the project and not enough on similar projects' durations. This can lead to significant delays and unexpected expenses. However, instead of ignoring the unintended consequences or becoming overly cautious, the middle ground is to acknowledge them and design through them. As the creator of productivity software, Dustin Moskovitz, learned the hard way when it took three years to launch the first version of Asana, despite initial optimism. He believes that mastering impulse control and fighting distraction are key to success, but acknowledges that software, including his own, can contribute to these issues. Therefore, it's essential to accept that there will always be unintended consequences and proactively address them in the design process.

• Analyzing Past Projects for Improvement: Effectively managing projects and making informed decisions involves tracking historical performance and adjusting plans accordingly to improve overall success, reduce cost and time overruns.

  Effective project management and decision-making involve tracking historical performance and adjusting plans accordingly. Yael Grushka-Kakain, a professor at the University of Virginia's Darden School of Business, emphasizes the importance of analyzing past projects and their actual performance compared to planned outcomes. This approach not only raises awareness of the issue within the organization but also leads to improvement. Meteorologists serve as an excellent example of this principle. By tracking and scoring their forecasts and outcomes, they have significantly increased their accuracy and gained public trust. Infrastructure projects, with their massive budgets and schedules, often suffer from cost and time overruns. Economic geographer Bent Flyvbjerg has estimated that infrastructure projects with a budget above $1 billion add up to between $6 and $9 trillion a year globally. He found that 80 to 90% of these projects experience cost and schedule overruns. To combat the planning fallacy and improve overall performance, Grushka-Kakain advocates for the use of data and historical analysis. More governments and private firms are publishing planned and actual deadlines and budgets, which allows for better tracking and evaluation of project performance. By embracing data and focusing on historical performance, organizations can work towards reducing cost and time overruns and improving overall project success.

• Mega projects face consistent cost overruns, schedule delays, and benefits shortfalls: Strategic misrepresentation and optimism bias contribute to mega projects' cost overruns and schedule delays. Countries like the UK and Denmark require accounting for historical underestimations, but proper incentives for contractors are necessary to avoid perverse outcomes.

  Mega projects consistently experience cost overruns, schedule delays, and benefits shortfalls, a trend that has persisted for over a century. The causes include strategic misrepresentation, where planners deliberately underestimate costs and overestimate benefits to increase their chances of getting approved, and optimism bias, where people tend to overestimate what can be achieved realistically. To address this issue, some countries like the UK and Denmark have made it mandatory to use a methodology that accounts for historical underestimations in budgets and schedules. However, this approach could create perverse incentives for contractors if not properly incentivized. The key for commissioning entities is to be aware of these trends and factors, and to take proactive steps to mitigate them.

• Combating the planning fallacy in large projects: Incentivize targets and use data-driven algorithms for accurate forecasting in large projects to overcome the planning fallacy, but challenges include resistance to algorithms and difficulty in finding accurate references.

  Combating the planning fallacy in large projects requires a two-pronged approach: incentivizing those involved to meet targets and using data-driven algorithms for forecasting. The mandatory use of this methodology since 2004 in certain infrastructure projects has shown promising results, with reasonably accurate projections and small cost overruns. However, the implementation of these solutions is not without challenges. People are averse to using algorithms for forecasting due to various reasons, and finding accurate references for data-driven forecasting can be difficult. Amazon's success in using algorithms to make accurate predictions highlights the potential solution, but the application of data and algorithms requires careful identification of similar projects and the use of mathematical and statistical models, which may not come naturally to everyone. Despite these challenges, continuous learning and the adoption of data-driven approaches are crucial steps towards overcoming the planning fallacy.

• Large infrastructure projects with overly optimistic plans can lead to significant cost overruns and delays: Realistic planning and budgeting are crucial for large infrastructure projects to avoid unexpected costs and challenges

  Overly optimistic plans and deadlines for large infrastructure projects, such as the Second Avenue Subway in New York City, can lead to significant cost overruns and delays. This is often due to a combination of unexpected developments and deliberate deception by elected officials who want to show progress to their constituents. As a result, the next person in charge may face unexpected costs and challenges in completing the project. The Second Avenue Subway, which has cost $4.5 billion and is still not complete, serves as a cautionary tale about the importance of realistic planning and budgeting in large infrastructure projects.

• Exploring the hidden side of life with Freakonomics Radio: Freakonomics Radio delves into the unconventional, revealing the hidden side of everyday life through engaging storytelling and innovative research.

  You can access this informative and entertaining content through various platforms, including Apple Podcasts, podcast portals, NPR stations, SiriusXM, and Spotify. The team behind the show includes Alvin Meloth, Allison Hockenberry, Greg Rizolsky, Stephanie Tam, Max Miller, Merritt Jacob, Vera Carruthers, Harry Huggins, and Brian Gutierrez. The music throughout the episodes is composed by Luis Guerra. The archive of every episode can be found on Freakonomics.com, where you can also read transcripts and find links to underlying research. Don't miss out on the latest episodes and insights – follow Freakonomics Radio on Twitter, Facebook, LinkedIn, or via email.