Podcast Summary
Large Enterprise Adoption of LLMs and GNAI: Early enterprise adoption of LLMs and GNAI focuses on productivity gains and cost optimization. Future goals include improving customer interactions and creating new product offerings. Talent with MLOps or LLM instrumentation expertise is in high demand.
While the adoption of large language models (LLMs) and generative AI (GNAI) is growing, the business case for investing in operationalizing these technologies within large enterprise organizations is still being explored. The early stages of adoption are focused on productivity gains through simple solutions and optimizing cost bases. However, as we move further, the goal is to improve customer interactions through applications like customer support, HR automation, and legal automation. The aspirational goal is to create new product offerings and monetization opportunities, but we are still at least 18 to 24 months away from seeing these trends. From a talent perspective, there is a clear gap in the market for developers with expertise in MLOps or LLM instrumentation, making it an attractive area for companies looking to acquire and retain top talent. Additionally, developers are drawn to the fresh and challenging nature of this field.
AI tool implementation challenges: Despite advancements in AI tools, large-scale implementation remains a challenge due to issues like talent gap, data quality, availability, and extensive prompt engineering and integration. Approx. 25-30% of applications don't make it to production due to non-technical reasons.
While AI tools are making significant strides in areas like prompt engineering, image and text generation, and code generation, reaching production at scale remains a significant challenge. Talent gap is a top issue identified by experts, and the developer audience is particularly ripe for AI tools targeted at developer tooling. However, achieving large-scale implementation requires addressing challenges such as data quality and availability, as well as the need for extensive prompt engineering and integration. Companies like MongoDB are working to help AI tools better understand their products by providing large amounts of data and natural language prompts. But this requires a shift in mindset and brings its own challenges of integration, scale, and maintenance. Data foundation and data maturity are crucial for taking even simple use cases into production, and the challenges become significantly different when considering enterprise-focused solutions. Approximately 25-30% of applications do not make it from ideation to deployment at scale due to non-technical reasons.
Model implementation challenges in enterprises: Enterprises face challenges in implementing generative AI at scale due to data governance, model life cycle management, and model selection. Standardization and experimentation in a safe environment are necessary for effective model selection.
While the use of generative AI via cloud providers offers many benefits, such as easy access to advanced models and standardization, there are still significant challenges for enterprises in implementing it at scale. These challenges include ensuring data governance, managing the life cycle of models, and making the right model selection for specific use cases. The need for standardization arises from the fact that enterprises typically invest in a large number of applications across various functions, requiring a platform that offers choice and optionality in terms of models. The process of model design and selection is currently more art than science, with a growing number of models available and varying in types such as text, image, audio, and video. The ability to experiment with models in a safe space, like a model garden, is crucial for making the best choice for a particular use case. However, the lack of determinism in these models and the need for trial and error add to the complexity of implementation.
Machine learning model production: Transitioning a machine learning model to production involves estimating workload, deciding on the most cost-effective and suitable model, dealing with obsolescence risk, ongoing maintenance, and benchmarking different models.
Implementing and managing machine learning models involves a significant amount of experimentation, iteration, and ongoing effort. While choosing a model for experimentation requires considering factors like fit and business value, transitioning that model to production brings new challenges such as estimating workload, deciding on the most cost-effective and suitable model for the long term, and dealing with the obsolescence risk of new models. The training and fine-tuning process can be time-consuming and require manual intervention, with windows for tuning provided by companies offering these models. After deployment, ongoing maintenance and upkeep are necessary considerations, including monitoring performance and addressing any potential issues. Additionally, the output from machine learning models can vary greatly, making it essential to benchmark and compare different models for specific use cases. The process of implementing machine learning models is an ongoing journey, requiring continuous evaluation and adaptation to new technologies and improvements.
AI model selection automation: AI is now used to benchmark and train AI models for automating the selection process. Companies increasingly rely on AI Managed Service Providers for help in model evaluation, data foundation, and fine-tuning.
The landscape of AI model selection, customization, and maintenance has evolved significantly. Early methods involved checking the similarity of models and making small tweaks for best practices. However, with the vast number of models available, automation is necessary. Now, AI is used to benchmark and train AI models. This process is complex and requires ongoing effort, but models today are more powerful, reducing the need for extensive optimization. Fine-tuning and customization are still required, but the cycle is faster than before. Companies are also turning to AI Managed Service Providers (MSPs) for help in model evaluation, data foundation, and fine-tuning. This trend is expected to continue as businesses aim to accelerate the adoption of AI solutions.
AI advancements: Expect enhancements in customization, control, cost optimization, and database integration leading to faster testing-production gap and new use cases, but regulatory and safety considerations and costs are important factors to consider.
The Gen AI space is expected to see significant advancements in the next year, particularly in areas of enhanced customization and control, cost optimization, and database integration. These developments could lead to a decrease in the gap between testing and production, and the emergence of new use cases. However, the regulatory and safety aspects of AI are not to be ignored, and the costs involved will become increasingly important. The adoption of AI is inevitable for most companies, and the challenge lies in balancing the potential benefits against the risks and costs. The managed service providers are expected to play a crucial role in this process by offering end-to-end solutions and orchestration capabilities. Overall, the Gen AI space is in a period of rapid growth, and the next year is expected to bring significant progress and innovation.
MongoDB, Google Cloud resources for Gen AI: Explore MongoDB's developer center for articles, how-tos, and podcasts, or experiment with Google Cloud's Gen AI models and connect with their team for discussions.
Both MongoDB and Google Cloud offer valuable resources for those interested in Gen AI. MongoDB, represented by Toni Mirovaeva, invites you to explore their developer center filled with articles, how-tos, and even a podcast, the MortgageB podcast, where you can learn from their team, including Shane. Google Cloud, represented by Miku Ja, encourages you to experiment with their Gen AI models and connect with her on LinkedIn for further discussions. Both teams are dedicated to fostering a community of learning and exploration in the field of Gen AI. So, whether you're just starting out or looking to deepen your understanding, be sure to check out the resources from MongoDB and Google Cloud.