Mastering Specific Use Cases with Generative AI

This episode features an interview with Mikey Chang, Group Product Manager at Niantic. 

In this episode, Mikey reflects with us on the vast user base and feedback he deals with at Niantic, emphasizing a data-driven approach, A/B testing, and the delicate balance of making incremental changes. Mikey provides insights into the unique dynamics of the gaming industry's business-to-consumer model, where users are both entertained and motivated to exercise, highlighting the challenges he faces in achieving this balance.

Host takeaways:

Kyle: 

• Ready Up Feature Success Metrics: Mikey emphasized the importance of focusing on measurable outcomes such as retention, revenue, and real-world social impact when evaluating the success of the Ready Up feature. Clear metrics helped mitigate concerns about potential misuse and provided a basis to demonstrate the feature's effectiveness.
• Managing User Input Noise: Mikey highlighted the challenge of filtering out noise from extensive user input in product development. Utilizing measurable outputs and data-driven approaches helped Mikey stay focused on important aspects without compromising core values or altering the product's vision.
• Incremental Product Adjustments: Mikey's approach involved making incremental tweaks to the game based on data-driven insights rather than implementing significant changes. Balancing the need for innovation with user satisfaction was crucial to avoid upsetting the user base.

Lukasz:

• Differences in User Scale: It’s hard to not highlight the vast difference in the number of users and feedback between Mikey's gaming world and Zuora's business model. Despite fewer users, Zuora remains data-driven, but the dynamics of user feedback and testing strategies differ significantly from Mikey's context.
• Context-Aware A/B Testing: I noted the necessity for context-aware A/B testing in Mikey's environment, where understanding the specific context is crucial for effective testing. Achieving a delicate balance in tweaking elements and minimizing negative consequences was crucial for long-standing success.
• Monitoring User Feedback: I’m curious about Mikey's validation process for hypotheses and speculate that monitoring user feedback on message boards might be an organic validation method. Our conversation highlighted the challenge of ensuring that data-backed hypotheses align with the actual user experience and how user feedback plays a role in this validation process.

Segment Timestamps:

*(00:26) - Ship it or Skip it

*(06:38) - PM Power Moves

*(37:57) - Host Takeaways

Links:

• Connect with Lukasz Weber & Kyle Kolich on LinkedIn
• Learn more about Zuora

>> Connect with Lukasz Weber & Kyle Kolich 

>> Learn more about Zuora

This episode features an interview with Dave Bottoms, General Manager & VP of Product at Upwork.

In this episode, Dave shares with us:

• His VC approach (40/30/30) to product frameworks
• The importance of a growth team focused on achieving a specific ROI
• How to balance customer feedback and innovation
• How to establish clear principles to guide decision-making

Host takeaways:

Kyle: 

• Dave really stresses the importance of adopting a framework to your product leadership, and I appreciate his venture capitalist mindset (40/30/30) when it comes to allocating resources strategically across the product portfolio.
• Our conversation also highlights the importance of a growth team focused on moving metrics to achieve a specific ROI. Dave helped shed light on the role of this team in driving measurable results and contributing to the overall success of the product.

Lukasz:

• I like the top-down perspective in managing larger products and how Dave acknowledges the value of working with experienced senior individuals. He emphasizes that product management guidelines are not rigid rules but adaptable principles that should be adjusted based on the specific situation.
• Clear principles, though sounding lofty, simplify decision-making by providing a clear direction. Dave drove home that when principles are well-defined and communicated, they serve as a framework that not only guides the PM but also aids other teams. 

Segment Timestamps: 

*(00:25) - Ship it or Skip it

*(04:05) - PM Power Moves

*(28:01) - Host Takeaways

Links:

• Connect with Lukasz Weber & Kyle Kolich on LinkedIn
• Learn more about Zuora

>> Connect with Lukasz Weber & Kyle Kolich 

>> Learn more about Zuora

This episode features an interview with Vrushali Paunikar, Chief Product Officer at Carta.

In this episode, Vrushali shares with us:

• Why she’s a disciple of Luca Pacioli
• How to get buy-in from your CEO
• How she secured the green light for what is now a $100 million fund
• And so much more!

Host takeaways:

Kyle: 

• It’s vital to know where your products come from. When Vrushali mentioned Luca Pacioli, my ears perked up! It’s so important to know the material and get into the history of where your products come from and why they matter. This allows you much deeper insight into how users might utilize your products.
• Purpose-built products. Find your product niche and cultivate and validate it through customer feedback. You can create new value from “old” products or ideas through a new verticalization approach.

Lukasz:

• “Nothing beats a small, laser-focused team aligned on a singular vision.” It all comes down to focus and fundamentals, and Vrushali’s team was able to utilize fund accountants within Carta to be laser-focused with her team on what’s working for the product outcome. This allowed Vrushali and her team to iterate and prove out their product very quickly.
• The foundation is key. You don’t want to rebuild products that already exist. Instead, apply it to a different vertical and go from there.
• Product marketing vs product management. Should product managers and product marketers? We’ve been wrestling with that idea for a while, but it’s an interesting topic to discuss across different companies.

Segment Timestamps:

*(00:32) - Ship it or Skip it

*(05:07) - PM Power Moves

*(38:17) - Host Takeaways

>> Connect with Lukasz Weber & Kyle Kolich 

>> Learn more about Zuora

This episode features an interview with Avantika Gomes, Group Product Manager at Figma. 

In this episode, Avantika shares with us:

• Assigning criteria for running beta successfully
• Building products for an expanding audience
• The importance of listening to your user base
• How collaboration can help reduce inefficiencies
• And so much more!

Host takeaways:

Kyle: 

• Being crisp on the problem. Collaboration is key, so when we have a good knowledge base and understand our personas we better understand their pain points and know where to start with mending them.
• Shipping fast and breaking stuff isn’t always the answer. Those of us who have been in the product space for a while have heard “ship it fast, break it fast” our whole careers, but Avantika invites us into a new way of thinking: thoughtful product development and the benefits it brings.

Lukasz:

• Recognizing there are other personas engaging with your products. Our products are built for certain people, so when that pool of people starts to grow, we have to understand who those people are, what they want, and how our products can meet their needs.
• Shipping fast and failing past depends on your product. If we think about an issue that might arise on Facebook, if something breaks, you may not even notice it as a user. But if you experience an issue with an MRI machine while you’re getting a scan, that can affect your life immediately. So, it’s important to be aware of the stakes at hand in terms of shipping fast or not.
• Focus on the right things when hiring. We can say we want to be inclusive and lead a more diverse team all day long, but we need to be aware of the exact parameters we’re using and ask ourselves: how can we hire and retain the best people for the job? How can we help them learn, grow, and flourish?

Segment Timestamps: 

*(00:29) - Ship it or Skip it

*(03:44) - PM Power Moves

*(29:07) - Host Takeaways

>> Connect with Lukasz Weber & Kyle Kolich 

>> Learn more about Zuora

This episode features an interview with Saumil Shrivastava, Director of Product Management on the AI Platform team at Microsoft. 

In this episode, Saumil shares with us:

• The importance of community feedback
• How best to enable customers to build LLM apps quickly
• When products need to be scenario-based and domain-specific
• And much more!

Host takeaways:

Kyle: 

• Security and privacy are gravitating toward LLMs. When we look at LLMs for industries such as healthcare, we see the models become more vertical and specialized so we need to account for that in how we build.
• Strong PMs are good problem solvers. As PMs, we have to be able to solve complex problems at scale with technology. And since we love to do this so much anyway, it shouldn’t be a surprise. However, when the needs of the products we build change, it’s up to us to provide ideas for solutions.

Lukasz:

• We’re in an exploration phase. This also means that there are a lot of unknowns, so we’re often working off an instance-by-instance basis and not a generalized set of rules. The testing starts now and it might take longer than we thought historically.
• The obstacle is the way…sometimes. While I don’t believe every obstacle is the way, I do concur with Saumil that organizationally if there is pushback, it’s wise to push back further and ask questions as to why there are obstacles and then work to create effective solutions. Sometimes it’s time to explore different options and ways of doing things.

Segment Timestamps: 

*(00:31) - Ship it or Skip it

*(04:15) - PM Power Moves

*(25:52) - Host Takeaways

>> Connect with Lukasz Weber & Kyle Kolich 

>> Learn more about Zuora

This episode features an interview with Thomas Fenrich, VP of Product Management at IKEA.

In this episode, Thomas shares with us:

• Why AI needs to be grounded and purposeful
• The importance of getting back to the fundamentals
• Communicating with impact
• The endless benefits of testing internally FIRST
• The power of small, fast releases
• And so much more!

Host takeaways:

Kyle: 

• AI needs to be grounded: AI isn’t the end all be all. We need to be asking ourselves, How does AI actually help us in our businesses? How is it helping the customer? The answers will provide us with focus and allow us to approach our products and customers more thoughtfully.
• Go back to the fundamentals: When launching a new product, it’s important to go back to basics - who are we building it for? Why are we building it? Again, taking a step back to be intentional about what we’re doing helps us, our teams, AND our customers.
• PMs should communicate with impact: Communication, and over-communication at that, is critical for how we talk about our products and making sure we’re delivering the right words and the right message to the right people.

Lukasz:

• The latest and greatest is exciting: At this point, I’m pretty sure all engineers want to be working with AI. And why not, right? It’s the new, shiny toy providing much to be discovered. What we can do as leaders, is help channel the excitement of the new and  useful by continuously learning about it, making sure we have proper bats so we can drive proper business.
• Test internally first: When we’re testing new products, it’s wise to be sure we’re testing internally first. We don’t want to test on the consumer and have to back pedal. Let’s make sure our stuff works first.
• Incremental releases that are small and fast: When we focus on releasing incremental changes fast, we have the opportunity to experiment easily and test things out. This requires minimal effort but can have a large impact. It allows our teams to move really quickly and respond to customers in better ways. This also builds a tremendous uptick in excitement about the product.

Segment Timestamps: 

*(01:32) - Ship it or Skip it

*(03:47) - PM Power Moves

*(33:37) - Host Takeaways

Links:

• Connect with Lukasz Weber & Kyle Kolich on LinkedIn
• Learn more about Zuora

>> Connect with Lukasz Weber & Kyle Kolich 

>> Learn more about Zuora

This episode combines three of our favorite episodes to date, featuring interviews with:

• Krishan Gupta, Product Manager at Google (Google Nest)
• Eugene Song, VP of Product Management at BrainPOP
• And Bjorn Morton, Sr. Director of Product Development at Gannett

Episode Timestamps: 

*(00:15) - Krishan Gupta, Google

*(08:19) - Eugene Song, BrainPop

*(20:48) - Bjorn Morton, Gannett

Check out the resources for this episode:

Listen to the full episodes with each of our awesome guests:

• Krishan Gupta, AI Empowered PM: How LLMs Reshape Product Management
• Eugene Song, Exploring the Impact of Intangible Value that Makes Products Meaningful
• Bjorn Morton, Optimizing the Consumption Experience for Subscribers

Learn More:

• Connect with Lukasz Weber & Kyle Kolich on LinkedIn
• Learn more about Zuora

This episode features music written, performed, and produced by Eugene Song, VP of Product Management at BrainPOP.

>> Connect with Lukasz Weber & Kyle Kolich 

>> Learn more about Zuora

This episode features an interview with Micah Peterson, Chief Product Evangelist at ProcedureFlow, the next-generation Knowledge Management software designed to help companies visualize and navigate processes.

In this episode, Micah shares with us:

• His simple but actionable tips for reducing ambiguity for customers
• What it takes to make consumable and democratized knowledge repositories
• Looking at customer obsession in a whole new way
• And so much more

Host takeaways:

Kyle: 

• Be aware of where your processes break down: The rubber really hits the road where the links in our processes are weakest. Learning where those weak links are and educating our employees on how to navigate those sore spots through fostering a relationship with our customers during those moments is key to better customer success.
• Leverage your SMEs: Your community of users can help improve your products. Get their feedback and better understand what you can do better through their direct feedback.
• Don’t be afraid to say “no”: When we think of customer obsession, the words “yes” “of course” and “always” tend to come to mind first, but Micah encourages us to think of the 99% and to say no to updates that don’t serve the bigger body of customers or the future of the product.

Lukasz:

• People will focus on confusion, so don’t be confusing: Standardizing and removing ambiguity out of the conversation forces customers to focus on the flow, on procedure, and on the business at large.
• Always ask, is this the right thing?: Prioritizing what we really want to achieve when all we want to do is what’s easiest, because oftentimes what’s easy is a distraction and not adding anything to the product.
• Prioritize requests: When we can provide various perspectives on dimensions by looking at different markets, we can then put on the weights in a more strategic way. Because when we can rationalize scoring, it allows us to cut across dimensions and ease any tension between the managers in those markets.

Segment Timestamps: 

*(01:00) - Never Have I Ever

*(09:16) - Innovation, Therapy & Other Wild Tales

*(26:28) - PM Power Moves

*(31:27) - Ship it or Skip it

*(35:58) - Host Takeaways

Links:

• Connect with Lukasz Weber & Kyle Kolich on LinkedIn
• Learn more about Zuora

>> Connect with Lukasz Weber & Kyle Kolich 

>> Learn more about Zuora

This episode features an interview with Randy Mercer, Chief Product Officer at 1WorldSync.

In this episode, Randy tells us about 1WorldSync’s secret sauce for AI use cases as well as how to effectively generate compelling content with the help of AI. Randy also shares his best practices for mergers and acquisitions.

Key Takeaways:

Kyle: 

• Content management: the benefit value prop: Tying your technology to a tangible ROI you can showcase, makes your products pop.
• How retailers fit into the supply chain: Having influence over SEO and keywords to maximize a product and its value in the online marketplace.
• Determining culture fit for acquisition success: Even though this is overlooked in the acquisition process sometimes, it is most definitely a valuable part of the acquisition equation. Part of the deal is getting people, so understanding how those individuals deal with things, what their processes are, and how they interact with one another is vital.

Lukasz: 

• Injecting AI into various parts of the business: AI can’t solve all of our problems or help us with everything, but figuring out which aspects of the business AI can help with can prove valuable over and over when used for appropriate use cases.
• Build vs buy: When acquiring companies is for the sake of filling functional capabilities, it allows more education and crossfeed for the entire product. It helps to drive the relevance of the bigger picture of the product.

Segment Timestamps: 

*(00:36) - Never Have I Ever

*(06:48) - Innovation, Therapy & Other Wild Tales

*(20:34) - PM Power Moves

*(32:18) - Ship it or Skip it

*(35:09) - Host Takeaways

Links:

• Connect with Lukasz Weber & Kyle Kolich on LinkedIn
• Connect with Randy Mercer on LinkedIn
• Learn more about 1WorldSync
• Learn more about Zuora

>> Connect with Lukasz Weber & Kyle Kolich 

>> Learn more about Zuora

This episode features an interview with Bjorn Morton, Sr. Director of Product Development at Gannett.

In this episode, Bjorn gives us an inside look at basic vs smarter models of delivering products to subscribers, helps us understand why print isn’t dead, and the importance of continuing to understand the customer experience. Bjorn also shares his perspective on the evolution of physical media into digital media, and how to continue optimizing the consumption experience for loyal subscribers.

Key Takeaways:

Kyle: 

• Basic vs smarter models: how it used to be vs how it is: Models should be subscriber specific, it’s not one size fits all.
• Print isn’t dead: The journey of converting physical products to digital products doesn’t have to be mutually exclusive - let’s make a better experience for the subscriber overall, whether they use a physical or digital product.

Lukasz: 

• Continue to figure out what the customer experience is: How do you actually deliver the service? Especially if it's a physical product in this digital world. How do you bring in the right readers and subscribers with the right packaging, pricing, etc? Look at innovating current products instead of making something totally new. Making the customer experience better is a process.
• Crypto can be a valuable perk to subscribers: There are many reasons as to why, but crypto is mostly valuable because it enhances the customer experience and makes the process of purchasing products smooth by working as a cash equivalent, easing the customer journey.

Segment Timestamps: 

*(00:41) - Never Have I Ever

*(09:02) - Innovation, Therapy & Other Wild Tales

*(22:49) - PM Power Moves

*(31:23) - Ship it or Skip it

*(36:41) - Host Takeaways

Links:

• Connect with Lukasz Weber & Kyle Kolich on LinkedIn
• Connect with Bjorn Morton
• Learn more about Gannett
• Learn more about Zuora

>> Connect with Lukasz Weber & Kyle Kolich 

>> Learn more about Zuora