Logo
    Search

    Making ETL pipelines a thing of the past

    enJune 18, 2024

    Podcast Summary

    • Technology RolesIndividuals with diverse backgrounds can succeed in technology, adaptability and continuous learning are crucial for success in technology roles.

      Cassie Shum's journey from a biology background to becoming the VP of Field Engineering at Relational AI highlights the evolving nature of technology roles and the importance of adaptability and continuous learning. Cassie shared how her initial fascination with the software behind research programs led her to pursue a master's degree in computer science and eventually a career in software engineering. Her diverse experience at Thoughtworks, where she learned various aspects of technology and leadership, prepared her for her current role at Relational AI, where she applies her skills to implement their AI co-processor for data clouds and language models in customer environments. This conversation underscores the potential for individuals with diverse backgrounds to succeed in technology and the importance of staying curious and open to new opportunities.

    • Developer productivity with AIAI tools like Copilot can assist developers but the focus should be on outcomes, not just productivity. Relational AI's AI co-processor aims to help organizations extract insights from large data and accomplish various use cases.

      While Generative AI tools like Copilot can assist developers in their work by providing suggestions and quicker understanding of large amounts of data, they are not the end-all solution for boosting developer productivity. Instead, the focus should be on the actual outcomes that engineering organizations aim to accomplish, such as delivering valuable features to customers. The value of developer productivity lies in the ability to produce outcomes that benefit the organization. When it comes to Relational AI's offering of an AI co-processor for data clouds and language models, it aims to address the pain point of extracting insights from large and siloed data by providing an additional tool to help organizations make sense of their data and accomplish various use cases like entity resolution, customer 360, and graph analytics. The AI co-processor is designed to work in conjunction with existing data warehouses, allowing organizations to make the most of their data and gain valuable insights to drive better outcomes.

    • Relational AIBringing AI tools directly to the data instead of moving it allows for data security, governance, and cost savings while enhancing data value and utility

      A new industry trend is emerging where instead of moving data out of an ecosystem to apply AI and analytics, the focus is on bringing AI tools directly to the data. This approach, known as relational AI, allows data to remain in its original location while still enabling the use of various AI tools such as business rules, predictive analytics, and prescriptive analytics. This not only saves time, energy, and resources but also removes constraints related to data security and governance. The use of a knowledge graph adds metadata, semantics, and rules to the data, enhancing its value and utility in the modern data stack. This shift towards bringing AI to the data is an exciting development for businesses that have previously felt unable to use AI due to data privacy concerns or the complexity of moving data. While there is still a need for traditional ETL pipelines to handle legacy data, the focus should be on modernizing the data stack and keeping all data in one place with a clear understanding of its nature.

    • Knowledge graphs for business logicUsing knowledge graphs to bring business logic closer to data increases accuracy, precision, and efficiency, while preventing errors and duplication. Domain-specific modeling and data cleanliness are crucial for effective Relational AI systems.

      Bringing essential business logic closer to your data through the use of knowledge graphs can lead to increased accuracy, precision, and efficiency. This approach helps prevent errors and duplication, ensuring that the data AI draws on is specific to your business and domain. Additionally, the importance of domain-specific modeling and data cleanliness cannot be overstated when working with Relational AI systems. Companies like Relational AI provide tools and knowledge engineers to help model concepts within organizations and weave in existing data, enhancing the overall fidelity and effectiveness of the system. Ultimately, the success of any AI solution relies on a strong foundation of clean, domain-specific data.

    • Field Engineering Role in Product DevelopmentField engineers bridge the gap between customers and product teams, ensuring customer satisfaction and driving product evolution towards self-service solutions through continuous feedback and collaboration with specialists.

      Field engineering plays a crucial role in the continuous development and improvement of a new and innovative product. Field engineers act as the bridge between customers and product teams, understanding the unique needs and ecosystems of each customer while also being experts in the product itself. Their primary goal is to ensure customer satisfaction, making the product journey as easy as possible. However, the ultimate goal is to evolve the product into a self-service, repeatable solution, eventually phasing out the need for close handholding. This feedback loop between customers and product teams is essential for the product's growth and maturity, making field engineering a vital part of the process. Additionally, field engineers work with a team of specialists to stay updated on new technologies and adapt to the ever-changing landscape of the industry.

    • Advanced analytics implementationA successful analytics strategy involves a clear focus on outcomes, collaboration between teams and partners, and a commitment to continuous learning and engagement with the community.

      Successful implementation of advanced analytics, such as graph analytics, prescriptive analytics, and predictive analytics, requires a collaborative effort between specialized teams and partners. Cassie, an expert in the field, emphasizes the importance of maintaining focus on the desired outcome when applying these technologies. She shares an example of using AI for generating recommendation letters as a fun and exciting application, highlighting the time and effort saved. Furthermore, Cassie underscores the importance of partnerships, with companies like Thoughtworks, to ensure a comprehensive understanding of various customer ecosystems and technologies. Lastly, she encourages engagement from the community, as seen in the recognition of Antoni Muroff for his excellent answer on Stack Overflow, which helped 63,000 people with a similar question. In essence, the key takeaway is that a successful analytics strategy involves a clear focus on outcomes, collaboration between teams and partners, and a commitment to continuous learning and engagement with the community.

    • Community engagementEncourage community participation and feedback, listeners' suggestions can lead to valuable discussions, and ratings and reviews help spread the word about the podcast

      Learning from this podcast episode is the importance of community engagement and feedback. The hosts acknowledged that they may have overlooked suggestions from listeners to discuss certain topics less, but they encouraged listeners to continue sharing their thoughts and ideas. In fact, some listeners have even contributed to the blog. The hosts also reminded listeners to leave ratings and reviews to help spread the word about the podcast. Cassidy Williams, Ryan Donovan, and Cassie Shum shared their contact information for those who want to reach out directly. Lastly, they mentioned that relational AI is undergoing a website overhaul and the new content will be available soon at relational.ai. Overall, the hosts emphasized the value of active participation and communication within the community.

    Recent Episodes from The Stack Overflow Podcast

    How to build open source apps in a highly regulated industry

    How to build open source apps in a highly regulated industry

    Before Medplum, Reshma founded and exited two startups in the healthcare space – MedXT (managing medical images online acquired by Box) and Droplet (at-home diagnostics company acquired by Ro). Reshma has a B.S. in computer science and a Masters of Engineering from MIT.

    You can learn more about Medplum here and check out their Github, which has over 1,200 stars, here.

    You can learn more about Khilnani on her website, GitHub, and on LinkedIn.

    Congrats to Stack Overflow user Kvam for earning a Lifeboat Badge with an answer to the question: 

    What is the advantage of using a Bitarray when you can store your bool values in a bool[]?

    A very special 5-year-anniversary edition of the Stack Overflow podcast!

    A very special 5-year-anniversary edition of the Stack Overflow podcast!

    Cassidy reflect on her time as a CTO of a startup and how the shifting environment for funding has created new pressures and incentives for founders, developers, and venture capitalists.

    Ben tries to get a bead on a new Moore’s law for the GenAI era: when will we start to see diminishing returns and fewer step factor jumps? 

    Ben and Cassidy remember the time they made a viral joke of a keyboard!

    Ryan sees how things goes in cycles. A Stack Overflow job board is back! And what do we make of the trend of AI assisted job interviews where cover letters and even technical interviews have a bot in the background helping out.

    Congrats to Erwin Brandstetter for winning a lifeboat badge with an answer to this question:  How do I convert a simple select query like select * from customers into a stored procedure / function in pg?

    Say goodbye to "junior" engineering roles

    Say goodbye to "junior" engineering roles

    How would all this work in practice? Of course, any metric you set out can easily become a target that developers look to game. With Snapshot Reviews, the goal is to get a high level overview of a software team’s total activity and then use AI to measure the complexity of the tasks and output.

    If a pull request attached to a Jira ticket is evaluated as simple by the system, for example, and a programmer takes weeks to finish it, then their productivity would be scored poorly. If a coder pushes code changes only once or twice a week, but the system rates them as complex and useful, then a high score would be awarded. 

    You can learn more about Snapshot Reviews here.

    You can learn more about Flatiron Software here.

    Connect with Kirim on LinkedIn here.

    Congrats to Stack Overflow user Cherry who earned a great question badge for asking: Is it safe to use ALGORITHM=INPLACE for MySQL?

    Making ETL pipelines a thing of the past

    Making ETL pipelines a thing of the past

    RelationalAI’s first big partner is Snowflake, meaning customers can now start using their data with GenAI without worrying about the privacy, security, and governance hassle that would come with porting their data to a new cloud provider. The company promises it can also add metadata and a knowledge graph to existing data without pushing it through an ETL pipeline.

    You can learn more about the company’s services here.

    You can catch up with Cassie on LinkedIn.

    Congrats to Stack Overflow user antimirov for earning a lifeboat badge by providing a great answer to the question: 

    How do you efficiently compare two sets in Python?

    The world’s most popular web framework is going AI native

    The world’s most popular web framework is going AI native

    Palmer says that a huge percentage of today’s top websites, including apps like ChartGPT, Perplexity, and Claude, were built with Vercel’s Next.JS. 

    For the second goal, you can see what Vercel is up to with its v0 project, which lets developers use text prompts and images to generate code. 

    Third, the Vercel AI SDK, which aims to to help developers build conversational, streaming, and chat user interfaces in JavaScript and TypeScript. You can learn more here.

    If you want to catch Jared posting memes, check him out on Twitter. If you want to learn more abiout the AI SDK, check it out 

    here.

    A big thanks to Pierce Darragh for providing a great answer and earning a lifeboat badge by saving a question from the dustinbin of history. Pierce explained: How you can split documents into training set and test set

    Can software startups that need $$$ avoid venture captial?

    Can software startups that need $$$ avoid venture captial?

    You can find Shestakofsky on his website or check him out on X.

    Grab a copy of his new book: Behind the Startup: How Venture Capital Shapes Work, Innovation, and Inequality. 

    As he writes on his website, the book:

    Draws on 19 months of participant-observation research to examine how investors’ demand for rapid growth created organizational problems that managers solved by combining high-tech systems with low-wage human labor. The book shows how the burdens imposed on startups by venture capital—as well as the benefits and costs of “moving fast and breaking things”—are unevenly distributed across a company’s workforce and customers. With its focus on the financialization of innovation, Behind the Startup explains how the gains generated by tech startups are funneled into the pockets of a small cadre of elite investors and entrepreneurs. To promote innovation that benefits the many rather than the few, Shestakofsky argues that we should focus less on fixing the technology and more on changing the financial infrastructure that supports it.

    A big thanks to our user of the week, Parusnik, who was awarded a Great Question badge for asking: How to run a .NET Core console application on Linux?

    An open-source development paradigm

    An open-source development paradigm

    Temporal is an open-source implementation of durable execution, a development paradigm that preserves complete application state so that upon host or software failure it can seamlessly migrate execution to another machine. Learn how it works or dive into the docs. 

    Temporal’s SaaS offering is Temporal Cloud.

    Replay is a three-day conference focused on durable execution. Replay 2024 is September 18-20 in Seattle, Washington, USA. Get your early bird tickets or submit a talk proposal!

    Connect with Maxim on LinkedIn.

    User Honda hoda earned a Famous Question badge for SQLSTATE[01000]: Warning: 1265 Data truncated for column.