Sixfold Resources

We recently sat down for a quick Q&A with Stewart Hu, AI scientist at Sixfold. Our conversation ranged from his career journey to how he stays current in the field, as well as the tasks on his daily agenda.

Let’s get this started! In your own words, what does your job as an AI scientist involve?

AI scientists engage in a lot of practical work. Despite our 'scientist' title, our roles often overlap with those of developers or research engineers. In fact, over 50% of our tasks are typical software engineering activities. We develop software grounded in foundational models, employing a range of techniques, not just AI.

Previously, AI encompassed anything linked to machine learning, but now it's more commonly associated with large language models like GPT. Our role includes integrating these models into software applications, utilizing models such as GPT-4, and even fine-tuning our custom models. Additionally, we apply both traditional machine learning and deep learning methods. This involves creating classifiers with techniques predating neural networks, like gradient boosting machines or random forests. At our core, we are software engineers crafting machine learning algorithms to address real-world challenges.

How did you get into the world of Generative AI? 

My fascination with AI really took off with GPT-3's emergence. But it was the debut of the stable diffusion model in August 2022 that truly captivated me. This revelation prompted me to pivot my career towards a tech startup specializing in deep learning and AI.

In the early stages of my career, I worked as a software engineer. This was followed by a ten-year journey in data science, beginning with statistical learning and gradually evolving into machine learning, deep learning, and finally AI. Essentially, I devoted my first decade to hardcore software development, and the next decade explored the realms of data science and machine learning.

Could you give some insights into what's on a typical to-do list for you?

My work is basically divided into three key areas.

Firstly, there's data management: sourcing appropriate data, organizing it properly, and conducting thorough analyses. A major chunk of our time is dedicated to dealing with data - acquiring, scrutinizing, and delving into it.

Secondly, I engage in software development, where my goal is to craft software that's not only reusable but also adaptable to growing complexities. This involves strategic software design to ensure it can be easily scaled up.

The third area is AI, particularly focusing on 'retrieval augmented generation’ . This entails extracting pertinent details from extensive document collections to accurately contextualize models like GPT-4. My day-to-day involves juggling these three components.

How would you distinguish a purpose-built AI tool from a generic one?

AI often gets hyped up with flashy demos requiring little coding. However, Sixfold is a purpose-built gen AI tool, our focus is on crafting solutions that address real-world business problems, not just making eye-catching demos. We use AI to make underwriters work faster, more accurate, and enjoyable. By taking over repetitive tasks, AI allows underwriters to focus on the more engaging aspects of their job.

Our platform is built with a strong emphasis on accountability, not just on interpretability or explainability. This means our solutions cite sources when making recommendations and provide actual source documents for our classifications. It's a practical, business-centric approach that boosts confidence in underwriting decisions.

What excited you the most about joining Sixfold?

Two things particularly drew me to Sixfold. First, the experienced team leading the company. The founders have a proven track record of creating substantial business value, blending tech knowledge with sharp business insight. Second, on a personal level, my wife has been in the insurance industry for over ten years, and I've always found it fascinating. Joining Sixfold presented a chance to dive deeper into this sector. 

It was the combination of the seasoned leadership and the company's expertise in insurance and underwriting that ultimately convinced me to become part of the team.

How do you stay engaged with the AI community? 

My go-to resource is X (formerly known as Twitter), where I've created a list named ‘AI Signals.' This list features over 100 experts deeply engaged in the field, tackling everything from fine-tuning models to enhancing the speed of large language model inference. While some of these individuals may not be widely known, their insights are incredibly valuable. 

Previously, I would follow arXiv for academic papers, GitHub for trending repositories, and Papers with Code to find research papers with their corresponding code. However, X has become my most essential tool. I regularly check updates from my list there to keep up-to-date with the latest developments.

That sounds like a great list! Can we share it with the readers?‍

Of course, happy to share it - here you go! 

How can people best follow your work?

I haven't been active on my blog lately, but I do maintain a GitHub repository named 'LLM Notes.' It serves as a practical guide for data scientists and machine learning practitioners. This repository is a compilation of the knowledge and insights I've gathered throughout my career. A few months back, I uploaded a wealth of information there, including lessons learned, common pitfalls, and personal experiences. It's a good resource for anyone interested in the field. 

Thanks for your time, Stewart! We’ll let you get back to your to-do list now.  

If you’d like an opportunity to work at Sixfold, check out our vacancies.

We recently conducted a Q&A with Drew Das, AI Engineer at Sixfold. Our discussion covered various topics, including the nuances of developing generative AI tools for marketing versus the insurance sector, advice for those aspiring to enter the field, and tools he enjoys using.

Let's start with your background and how you got into the world of AI?

I've always been in tech. I started in web development during high school, launching a business creating WordPress sites. I studied at UC Davis, where I also worked in the college IT department. I've spent nearly 10 years working in Silicon Valley, primarily in web development.

My first introduction to conversational interfaces was in 2017 with a company focused on chatbots. This early experience involved developing chatbots for job applications, targeting sectors like truck driving and foreign workers. We aimed to simplify the application process for those uncomfortable with traditional job sites.

I've also worked in cybersecurity, food delivery, and at Jasper, a startup in generative AI. At Jasper, I was the lead of content and involved in launching Jasper Chat, a chatbot product developed in under four days. This product became a primary feature for users. I also worked on an AI-based text editor, a first-of-its-kind product that helped introduce many to generative AI.

‍‍Could you describe in your own words what an AI engineer does? Specifically, what are your daily tasks and responsibilities?

The role of an AI engineer is still evolving, as AI is a relatively new field. Previously, we had machine learning engineers and software engineers with distinct functions. Machine learning engineers focused on training computer systems for making predictions using statistical methods. Software engineers, on the other hand, worked on translating business logic into application code, often using APIs provided by the machine learning team.

An AI engineer's role is broader than that of a machine learning engineer. It involves working with various AI tools, like ML models, vector databases, and advanced techniques. An essential skill for AI engineers is prompt engineering and understanding how these systems integrate. The primary objective is to combine these systems to create software that operates on top of data, rather than just converting business logic into code. This involves building a layer above data designed to emulate human behavior.

For example, in text matching, the goal is to accelerate tasks typically done by humans, such as researching and compiling data to make predictions. AI engineers strive to create systems that can perform these tasks as efficiently as humans.

Could you give some insights into what's on a typical to-do list for you?

Currently, my main focus is on improving text matching accuracy in our system. My task is to implement Retrieval-Augmented Generation (RAG) techniques, which include hybrid search, re-ranking, and new methods of data embedding. These techniques aim to improve our text matching accuracy. This task involves a lot of experimentation, implementing different systems, and optimizing them for better performance.

How can you in a simple way explain the difference between a purpose-built AI tool and a generic AI tool?

General AI tools, like GPT-3, are versatile. They can adapt to various tasks, such as classification or auto-completion. The interface is straightforward: input text, get text out. However, when you want the AI to use a specific language or guide a user in a certain way, prompt engineering becomes essential. People customize these general systems with specific instructions, but this can be cumbersome and has its limitations.

Purpose-built AI like Sixfold comes into play when you start fine-tuning the systems. This involves feeding them examples of data to achieve a desired tone, style, or structure. Additionally, building a knowledge retrieval system on your proprietary data can make your AI unique, providing access to information that other systems don't have. Customizing AI systems is challenging. It's one thing to create a demo or a cool AI video, but running these systems in a production environment, especially in enterprise-grade products, requires them to respond quickly, like within two seconds. So, a lot of work goes into customizing an AI system for practical, real-world applications.

What’s your take on transparency in AI, particularly with large language models? 

I believe transparency is crucial in AI, both within and outside organizations. Particularly, I'm referring to how AI tools are built and used. Although we're not yet at a point where AI systems are making all decisions autonomously, the trend is moving towards AI taking over more complex tasks. Take self-driving cars as an example: in the future, human-driven cars might be considered less trustworthy or even costlier to insure compared to AI-driven ones, potentially limiting human driving to specific scenarios like racetracks.

In such cases, it becomes vital for the AI systems controlling these cars to be transparent. We need to understand their training data, be aware of their limitations, and identify potential risks, especially since these systems significantly impact society. This transparency is essential because AI systems are not deterministic; their output heavily depends on the quality of the input data. Understanding what an AI system has been trained on gives us a clearer picture of its capabilities and limitations, which is essential as these systems become more integral to our daily lives. That's how I view transparency in AI.

What excites you the most about Sixfold?

At my last role, my focus was on solving marketing problems, specifically automating marketing systems and content generation. Marketing is inherently subjective, which makes it challenging to capture the right flavor of content. One dilemma in this field is the risk of producing generic or misleading content, which can be detrimental to society. 

The challenges in my current role are more complex. It's about understanding and utilizing the reasoning capabilities of the model, which involves deducing insights from a given set of data. This differs from just altering the tone of content for marketing purposes. 

For example, in marketing, success is often measured by a feedback loop, like how much generated content is retained in a final document, indicating user preference. However, in my current role, the metrics are different. We have a known 'ground truth' or a specific outcome we aim to achieve, and the goal is to develop a system that consistently aligns with this known outcome. This requires a higher level of accuracy and a different approach compared to marketing, where the outcomes are more subjective and based on individual perception.

‍‍How do you stay updated with the advancements in AI and large language models? Are there any newsletters or blogs you follow?

It's one of the biggest challenges in this field! You might spend months developing something innovative, only to find it's made obsolete by a new development the following week.

It can be frustrating, but it's also exhilarating. You learn to not get too attached to your work and treat it as part of a learning journey. The field is dynamic; each week brings something new that could either render your current work obsolete or introduce an exciting new method.

However, it's crucial for companies to remain disciplined and not get constantly sidetracked by every new innovation. Deciding whether to try out new technologies and determining their integration importance requires careful consideration.

As for staying informed, I don't stick to specific newsletters or blogs. I prefer a more hands-on approach, as I'm not from an academic background. I find YouTube content especially useful for seeing how new things are implemented. It's more about application for me – I need to see something in action to understand it. So, I explore various sources like Hacker News and YouTube, or anything relevant I come across on a particular topic.

Do you share your own work or insights publicly?

Actually, I don't engage much in open source work or sharing my projects publicly. After work, I prefer to disconnect and focus on other interests, like learning guitar. It's about maintaining a healthy balance. I'm fortunate that my work aligns closely with my interests. Over the past three years, I've had the opportunity to explore new techniques and projects that I've been curious about, right in my professional environment. For instance, this week I'm working on an advanced retrieval system, something I've always wanted to try.

Considering young engineers or students interested in entering this field, do you have any advice or recommendations for them?

My main advice is that simply following tutorials and reading books isn't enough to truly learn. The key is to build something. You need to apply what you've learned, either in a professional setting or through a personal project. In technology, and especially in AI, hands-on experience is essential. The possibilities with AI are vast. Building on top of existing AI systems is surprisingly accessible.

For instance, I'm currently working on an AI-based pet project. It's essentially a photo translator using a Raspberry Pi device, which has a computer, display, and camera. The idea is that you take a picture of something, and the system uses GPT for vision to describe what it sees. Then, using that description, it generates a DALLE 3 image related to the object and displays it on the screen. I call it the 'Unreal Camera.' For example, if you take a picture of a dog, it creates an artistic interpretation of that dog. It essentially presents a graphic version of whatever you photograph.

This kind of project would have been impossible to undertake alone a few years ago; it would have required a whole team and several years. Now, thanks to the power of AI, I was able to build it in just two days. So, my recommendation for anyone entering this field is to start building something practical and useful. That's the best way to learn and understand the potential of AI. 

Do you have a favorite AI tool at the moment?

I primarily use ChatGPT and a fascinating app called Perplexity, which is great for research. Perplexity is unique in how it can visit different websites and compile data. Another tool I frequently use is GitHub Copilot for coding. It's incredibly helpful, as it assists in writing code. These tools, especially Copilot, have been instrumental in my work. 

Thanks for your time, Drew! We’ll let you get back to your AI tasks now.

If you’d like an opportunity to work at Sixfold, check out our vacancies.

Dive into our Q&A with Brian Moseley, Co-Founder and CTO at Sixfold. We're chatting about his tech journey, from the early days at Sega to his recent tenure at American Express, and what led him to Sixfold. Brian also shares insights into the team's 'chill, pragmatic, and adaptable' culture and talks about his passion for Warhorn, his own gaming community project.

What’s your career path and how did it lead you to becoming Sixfold's CTO?

My software career began in college when I joined my friend’s community and media startup; we were among the first to experiment with web-based advertising. Through this venture, I met many tech professionals in San Francisco, who got me excited about the emerging web scene there. Ultimately, I decided to leave school and move across the country to work at Sega of America, focusing on database administration for their website.

From there, I joined several early-stage startups, including Critical Path, where I was the 17th employee. I experienced growth through its IPO, becoming an engineering manager and software architect. Later, I worked at the Open Source Applications Foundation, implementing a number of obscure internet protocols as the lead of an open source calendaring server.

As my career progressed, I found myself being the first engineering hire at a startup and then the first employee of another, so the logical next step was to become a cofounder and the CTO of Hoodline, a hyperlocal journalism company. Unfortunately, we did not succeed in reinventing local news, so I decided to try my hand at a big company and started working for American Express. 

I made the leap to full-time engineering management at Amex and eventually spent five years as the firm’s Head of Developer Experience, serving a global crew of 10,000 software engineers. Working in a big company was an incredibly educational part of my career, but I increasingly had the itch to build something from scratch again. So, Alex’s pitch got my attention and ultimately led to me cofounding Sixfold AI.

What originally attracted you to Sixfold?

It was a combination of three things:

1️⃣ I’d been looking for some time for an opportunity to build a company, team, and product from scratch.

2️⃣ I had a great time working with Alex briefly back in the early 2000s, so the prospect of cofounding with him was super appealing.

3️⃣ It seemed like exactly the right moment to go after the opportunities unlocked by generative AI in vertical business domains. Everything I saw was focused on LLM infrastructure and tooling, but nobody was talking in public about using generative AI to solve problems for business users like insurance underwriters.‍

‍How is Sixfold different from other players in the field?

We aren’t encumbered with legacy technology, organizations, or ways of working. We’re building our team and product from the ground up, which means we get to experiment freely and iterate quickly.

Another aspect that sets us apart is our commitment to ensuring our product is something users look forward to, not something they complain about with each use. The complexity and challenging nature of many enterprise software interfaces can be daunting. That's why we focus on user-friendliness and intuitive design.

We value user delight! Sixfold is not only effective; it's also fun to use.

What aspects of Sixfold and its technology are you most passionate about and find particularly exciting?

I love all of my children equally 🙃
‍
That said, as you might expect, I’m engrossed in the rapidly developing craft of generative AI engineering. What we’re doing goes way beyond basic “chat with your document” scenarios. We have a whole new set of primitives to assemble into complex systems - and they are evolving so fast it’s hard to keep up! It’s particularly interesting to apply these techniques in near real time as part of the live user experience. 

Our AI pipelines are hybrids of batch and streaming, with the work distribution and scheduling needs of batch processing and the resiliency and throughput of a real-time system. “Notebook AI” solutions don’t scale or meet our availability and performance requirements without a lot of additional engineering.‍

How would you characterize the dynamic and strengths of the team?

‍Our founding team has so much experience starting and scaling companies, we’ve all done zero to one many times. That doesn’t mean we don’t make mistakes, but they tend to be new mistakes, not many of the ones we’ve seen before. And my co-founders are teaching me everything there is to know about insurance.

The core of our tech team is a group of veteran startup hands with deep experience in their functional areas who can all flex into team building and leadership as we grow. We built on this core with a group of 'smart and gets things done' engineers who can build fast and iterate quickly, especially the closer they get to the AI layer of our stack.

Ultimately, our tech team is able to build, test, and learn quickly while keeping the quality bar high. And our go-to-market team keeps us oriented, fusing an understanding of what our customers need with a compelling vision for the future of underwriting.

How do you contribute to the team's growth and development?

As a hands-on-keyboard software engineer for nearly 30 years, I lead our product and platform development from the front. I pair with our engineers to help them develop their core software skills, which include architecture, design, coding, troubleshooting, and operating in production. This also provides an opportunity for them to build mentorship skills, as there is much I can learn from them as well.

I strive to create opportunities for our team members to step outside their comfort zones. Most of our engineers arrived with a specialization in some technical area, whether it be front-end development, infrastructure, or AI. However, our team is most effective when any member can work on any feature, regardless of where it falls in our stack. Therefore, I encourage each engineer to become proficient in all our languages and frameworks. For example, an AI engineer might make changes to our core domain model in Ruby, or a front-end engineer could add a new step to our AI pipeline using Python.

What is an achievement you are particularly proud of that the team has accomplished since the launch?

I’m most proud that we’ve been able to make the product work the way we envisioned it. It’s not just about our accuracy, which is great and getting even better. Remember the first time you experienced the magic of ChatGPT? Underwriters have that same feeling when they use Sixfold AI. They don’t have to read a hundred pages of tedious documents anymore just to find the three hidden nuggets that show the risk in an insurance case. They can sit back and let Sixfold find the nuggets for them in a matter of minutes. Magic!

In three words, how would you describe the team's culture?

Chill. I don’t mean we’re not working hard or that we don’t have high intensity about our venture. But you can tell that most of us have been in startups before because people handle the pressure really well. Nobody’s freaking out or creating negative vibes. We all have each others’ backs.
‍
Pragmatic. We aren’t striving for the perfect user experience or the most scalable systems right out of the gate. We’ll get there over time as we prove our right to be in business by delivering things customers want to pay for. We push ourselves to make “two-way door” decisions a little bit faster knowing that we can always iterate or reverse course later.

Adaptable. We develop in quick sprints and frequently stop to check in on what’s most important right now. Some weeks, that’s building features. Others, it’s improving resilience or shoring up security. At this stage of the company, every next customer engagement brings new problem statements and requirements. While we’ve introduced a measure of longer-term planning, we’re still keeping our heads on a swivel (as my high school football coach would say) to make sure our vision, roadmap, and near-term development plans evolve in the face of constant market discoveries.

How do you stay updated with the advancement in AI and LLMs, any particular newsletters or blogs that you like to follow?

I read 'Ben's Bites' daily, a resource recommended by Alex (our CEO) when I first joined. It's even included in our welcome email to all new team members. In addition, I follow 'The Information', particularly their newsletter 'AI Agenda', which I find very insightful and full of links to deeper reading.

Our team is also a significant source of discovery. Team members are constantly tuned into various niches and aspects of our industry, sharing relevant links and resources in Slack. We’re reading each other's stuff all the time.

Could you share some of your favorite tech tools or frameworks that you rely on heavily?

iTerm and Visual StudioCode - This is where I write and run my code. Others love their all-in-one commercial IDEs, but I do just fine with these tools.

Ruby and Rails - I’ve been building Rails apps and writing scripts and command line tools in Ruby since 2007. It just keeps getting better! I spend a lot of time with JavaScript as well, and it’s fine, you can’t really get away from it in the modern world, but I’ll always reach for Ruby when I can.

React - Again, I have been doing it since near the beginning, and it still scratches the itch. Haven’t had any reason to embrace the Hotwire part of the Rails omakase experience or to dabble in any of the post-React web frameworks. Maybe somebody out there will finally make me see the light?

Linear - Modern project management for product development that strikes an almost-perfect balance between the “all batteries in the universe included” and “bring your own batteries” approaches of pretty much everything else out there.

Mermaid - Being able to create a diagram by writing code has 10 x'ed my output of diagrams, which improves the clarity of my engineering dialogues.

Do you have a favorite AI solution that you would like to highlight?

Not yet! The gen AI ecosystem is still immature and changing rapidly. The popular tools and frameworks don’t always prioritize qualities we require for production usage - stable APIs, for example, or the ability to bring your own observability tools or work distribution systems. We’re having a better experience stripping down to the basics and devising our own abstractions over AI primitives (model and vector store APIs, for example) and in-house utilities.
‍
Ask me again in 12 months - I may well have a different answer!

Any fun tech projects that you're working on at the moment (non-Sixfold-related)? 

Yeah, there are two main things I'm focusing on outside of work. Firstly, I've recently picked up World of Warcraft again after about a year's break. It's been a fun way to spend time outside of building our company.

More to the point, I started a website back in 2001 called Warhorn, which I'm still running today, more than 20 years later. It's essentially a marketplace for tabletop role-playing games, board games, collectible card games, miniatures, word games, and more. I originally created it out of my love for tabletop role-playing, particularly tournament-style games like Dungeons and Dragons. It was born from the need to streamline the logistics, scheduling, and signups for large gaming events. 

The site was initially written in PHP in 2001, then I rewrote it in Ruby on Rails in 2011 and added a React front end around 2016 or 2017. Warhorn has been my laboratory for learning new technologies, such as GraphQL. It's a SaaS offering, which has helped me familiarize myself with various cloud providers and vendor systems. This experience has directly influenced my decisions at Sixfold and at previous jobs.

Warhorn is not just my tech lab but also my way of contributing to the gaming community. It's a long-standing project with over 25,000 monthly active users. I've kept it running as a passion project, not for commercial purposes. It's a creative outlet for me, similar to painting on canvas. It's an enjoyable aspect of my life where my profession and hobby intersect, providing a way to relax and be creative.

Warhorn seems absolutely awesome! Thanks, Brian, this was a lot of fun. 

‍

Are you excited about the opportunity to work at Sixfold?

If you’d like an opportunity to work with us, check out our vacancies.