Q&A on Generative AI in Insurance With Our AI Engineer

We recently conducted a Q&A with Drew Das, AI Engineer at Sixfold. Our discussion covered various topics, including the nuances of developing generative AI tools for marketing versus the insurance sector, advice for those aspiring to enter the field, and tools he enjoys using.

Let's start with your background and how you got into the world of AI?

I've always been in tech. I started in web development during high school, launching a business creating WordPress sites. I studied at UC Davis, where I also worked in the college IT department. I've spent nearly 10 years working in Silicon Valley, primarily in web development.

My first introduction to conversational interfaces was in 2017 with a company focused on chatbots. This early experience involved developing chatbots for job applications, targeting sectors like truck driving and foreign workers. We aimed to simplify the application process for those uncomfortable with traditional job sites.

I've also worked in cybersecurity, food delivery, and at Jasper, a startup in generative AI. At Jasper, I was the lead of content and involved in launching Jasper Chat, a chatbot product developed in under four days. This product became a primary feature for users. I also worked on an AI-based text editor, a first-of-its-kind product that helped introduce many to generative AI.

‍‍Could you describe in your own words what an AI engineer does? Specifically, what are your daily tasks and responsibilities?

The role of an AI engineer is still evolving, as AI is a relatively new field. Previously, we had machine learning engineers and software engineers with distinct functions. Machine learning engineers focused on training computer systems for making predictions using statistical methods. Software engineers, on the other hand, worked on translating business logic into application code, often using APIs provided by the machine learning team.

An AI engineer's role is broader than that of a machine learning engineer. It involves working with various AI tools, like ML models, vector databases, and advanced techniques. An essential skill for AI engineers is prompt engineering and understanding how these systems integrate. The primary objective is to combine these systems to create software that operates on top of data, rather than just converting business logic into code. This involves building a layer above data designed to emulate human behavior.

For example, in text matching, the goal is to accelerate tasks typically done by humans, such as researching and compiling data to make predictions. AI engineers strive to create systems that can perform these tasks as efficiently as humans.

Could you give some insights into what's on a typical to-do list for you?

Currently, my main focus is on improving text matching accuracy in our system. My task is to implement Retrieval-Augmented Generation (RAG) techniques, which include hybrid search, re-ranking, and new methods of data embedding. These techniques aim to improve our text matching accuracy. This task involves a lot of experimentation, implementing different systems, and optimizing them for better performance.

How can you in a simple way explain the difference between a purpose-built AI tool and a generic AI tool?

General AI tools, like GPT-3, are versatile. They can adapt to various tasks, such as classification or auto-completion. The interface is straightforward: input text, get text out. However, when you want the AI to use a specific language or guide a user in a certain way, prompt engineering becomes essential. People customize these general systems with specific instructions, but this can be cumbersome and has its limitations.

Purpose-built AI like Sixfold comes into play when you start fine-tuning the systems. This involves feeding them examples of data to achieve a desired tone, style, or structure. Additionally, building a knowledge retrieval system on your proprietary data can make your AI unique, providing access to information that other systems don't have. Customizing AI systems is challenging. It's one thing to create a demo or a cool AI video, but running these systems in a production environment, especially in enterprise-grade products, requires them to respond quickly, like within two seconds. So, a lot of work goes into customizing an AI system for practical, real-world applications.

What’s your take on transparency in AI, particularly with large language models? 

I believe transparency is crucial in AI, both within and outside organizations. Particularly, I'm referring to how AI tools are built and used. Although we're not yet at a point where AI systems are making all decisions autonomously, the trend is moving towards AI taking over more complex tasks. Take self-driving cars as an example: in the future, human-driven cars might be considered less trustworthy or even costlier to insure compared to AI-driven ones, potentially limiting human driving to specific scenarios like racetracks.

In such cases, it becomes vital for the AI systems controlling these cars to be transparent. We need to understand their training data, be aware of their limitations, and identify potential risks, especially since these systems significantly impact society. This transparency is essential because AI systems are not deterministic; their output heavily depends on the quality of the input data. Understanding what an AI system has been trained on gives us a clearer picture of its capabilities and limitations, which is essential as these systems become more integral to our daily lives. That's how I view transparency in AI.

What excites you the most about Sixfold?

At my last role, my focus was on solving marketing problems, specifically automating marketing systems and content generation. Marketing is inherently subjective, which makes it challenging to capture the right flavor of content. One dilemma in this field is the risk of producing generic or misleading content, which can be detrimental to society. 

The challenges in my current role are more complex. It's about understanding and utilizing the reasoning capabilities of the model, which involves deducing insights from a given set of data. This differs from just altering the tone of content for marketing purposes. 

For example, in marketing, success is often measured by a feedback loop, like how much generated content is retained in a final document, indicating user preference. However, in my current role, the metrics are different. We have a known 'ground truth' or a specific outcome we aim to achieve, and the goal is to develop a system that consistently aligns with this known outcome. This requires a higher level of accuracy and a different approach compared to marketing, where the outcomes are more subjective and based on individual perception.

‍‍How do you stay updated with the advancements in AI and large language models? Are there any newsletters or blogs you follow?

It's one of the biggest challenges in this field! You might spend months developing something innovative, only to find it's made obsolete by a new development the following week.

It can be frustrating, but it's also exhilarating. You learn to not get too attached to your work and treat it as part of a learning journey. The field is dynamic; each week brings something new that could either render your current work obsolete or introduce an exciting new method.

However, it's crucial for companies to remain disciplined and not get constantly sidetracked by every new innovation. Deciding whether to try out new technologies and determining their integration importance requires careful consideration.

As for staying informed, I don't stick to specific newsletters or blogs. I prefer a more hands-on approach, as I'm not from an academic background. I find YouTube content especially useful for seeing how new things are implemented. It's more about application for me – I need to see something in action to understand it. So, I explore various sources like Hacker News and YouTube, or anything relevant I come across on a particular topic.

Do you share your own work or insights publicly?

Actually, I don't engage much in open source work or sharing my projects publicly. After work, I prefer to disconnect and focus on other interests, like learning guitar. It's about maintaining a healthy balance. I'm fortunate that my work aligns closely with my interests. Over the past three years, I've had the opportunity to explore new techniques and projects that I've been curious about, right in my professional environment. For instance, this week I'm working on an advanced retrieval system, something I've always wanted to try.

Considering young engineers or students interested in entering this field, do you have any advice or recommendations for them?

My main advice is that simply following tutorials and reading books isn't enough to truly learn. The key is to build something. You need to apply what you've learned, either in a professional setting or through a personal project. In technology, and especially in AI, hands-on experience is essential. The possibilities with AI are vast. Building on top of existing AI systems is surprisingly accessible.

For instance, I'm currently working on an AI-based pet project. It's essentially a photo translator using a Raspberry Pi device, which has a computer, display, and camera. The idea is that you take a picture of something, and the system uses GPT for vision to describe what it sees. Then, using that description, it generates a DALLE 3 image related to the object and displays it on the screen. I call it the 'Unreal Camera.' For example, if you take a picture of a dog, it creates an artistic interpretation of that dog. It essentially presents a graphic version of whatever you photograph.

This kind of project would have been impossible to undertake alone a few years ago; it would have required a whole team and several years. Now, thanks to the power of AI, I was able to build it in just two days. So, my recommendation for anyone entering this field is to start building something practical and useful. That's the best way to learn and understand the potential of AI. 

Do you have a favorite AI tool at the moment?

I primarily use ChatGPT and a fascinating app called Perplexity, which is great for research. Perplexity is unique in how it can visit different websites and compile data. Another tool I frequently use is GitHub Copilot for coding. It's incredibly helpful, as it assists in writing code. These tools, especially Copilot, have been instrumental in my work. 

Thanks for your time, Drew! We’ll let you get back to your AI tasks now.

If you’d like an opportunity to work at Sixfold, check out our vacancies.