Sixfold Content

Sixfold's AI Governance Program is an ongoing internal process for identifying, assessing, and mitigating the risks of deploying AI. It is organized into five focus areas with named owners, defined deliverables, and a regular cadence of reviews. 

This post explains what it covers and why we built it the way we did.

Why we established this now

Responsible AI is not a new territory for Sixfold. We have been thinking carefully about how we build and deploy AI in insurance underwriting since the day we launched in May 2023. 

The honest answer to "why now?" is this: we wanted to formalize what we were already doing, and the regulatory environment gave us both the mandate and the structure to do it well.

The reality is that AI legislation is different from most compliance frameworks insurers and vendors are familiar with. AI compliance is process-based. It is about building the organizational muscle to understand whether your systems could cause harm, and then evaluating that question consistently over time.

That kind of ongoing work requires real ownership, clear processes, and people who are explicitly accountable for staying on top of it. We also made a deliberate choice not to chase specific legislation line by line. AI laws are still in progress across jurisdictions and contested at every level. Instead, we anchored the program around established industry reference points:

1. The NIST AI Risk Management Framework (AI RMF 1.0), an industry framework that gives durable guidance for risk and oversight.
2. The EU AI Act's high-risk provider requirements (Articles 9-17), which has concrete expectations on documentation, testing, and accountability

That approach is more durable and more honest about where regulation actually is today.

From principles to practice

Sixfold has had a set of responsible AI principles since early on, we formalized and launched our first Responsible AI approach in 2024 and established and launched an updated version in 2025. What the governance program does is translate those principles into organized action.

"Principles are well and good, but without action words they're just hot air."
‍
– Noah Grosshandler, Product Manager and AI Governance Program Lead at Sixfold

The program gives us specific owners, defined deliverables, and a cadence for revisiting each area. It creates the connective tissue between what we believe and what we actually do.  And critically, it builds the structure to change over time, because the landscape will constantly move forward, our product will change, and what it means to apply those principles responsibly will change with them.

We are building toward a complete Annex IV documentation package. That is the concrete output regulators and enterprise customers ask for. The program also addresses requirements to prevent algorithmic discrimination in much of the emerging global AI legislation, especially relevant to our customers operating in the life and health space. Bias and fairness testing is part of the program, with methodology tailored to each line of business. 

How the program is structured

The program is organized around five focus areas, each with its own owners and activities:

General governance covers the program itself: keeping the right people informed, running annual retrospectives, and ensuring continuity when things change.

Risk and compliance is responsible for identifying and tracking AI-specific risks, reporting on them quarterly, and handling any ad hoc mitigation or incident response as needed.

Data and model governance focuses on the data used to develop and train our systems. This means understanding where data comes from, confirming it is used ethically, and being able to clearly explain how system behavior relates to the inputs that shaped it.

Ethics and responsible AI covers fairness, explainability, and human oversight. It is the function that asks where humans should stay in the loop, what our position is on where AI should and should not operate, and whether the way our system works is genuinely ethical, not just technically sound.

Security and privacy handles the technical infrastructure: making sure the underlying systems meet the security and privacy requirements of what we are building.

Each area has its own cadence. Risk and compliance runs quarterly reviews. The ethics function does the same, looking at new functionality and evaluating human oversight design. Between those cycles, the full group of program owners meets bi-weekly to stay coordinated.

"The different areas of focus don't need to change when a new regulation drops or a new product drops, we can slot new requirements into the existing structure."
‍
– Noah Grosshandler, Product Manager and Facilitator of the AI Governance Program

One thing worth being explicit about: having dedicated program owners does not mean this is only their responsibility. At the end of the day, the company is accountable. Every person at Sixfold is responsible for building software that is honest, transparent, and ethical. The program owners exist to maintain the expertise and the process rigor that makes that possible at scale.

What this means for our customers

We have customers in North America, EMEA, South America and Australia today. Across the globe our customers face their own compliance pressures as deployers, and we can support them with that. Many of them are being asked by their own regulators and partners to demonstrate responsible AI practices, and most are still working out what that means for them.

The governance program helps us support our customers directly: 

• When a customer receives a compliance questionnaire about their AI vendor, we will have clear and documented answers. 
• When they are trying to build their own responsible AI stance, we can share what we have learned. We are not just handing over a package of documents; we are trying to be a resource to support them build out their own AI Governance practices. 
• Our customer success team works with each customer individually, because our customers operate across different jurisdictions, different product lines, and different regulatory environments. There is no one-size-fits-all approach here.

"The company is accountable. Just because there is this program and this charter doesn't mean that it is not every single person at this company's job to make sure that what we're building is built ethically, built honestly, built transparently."
‍
– Noah Grosshandler, Product Manager and AI Governance Program Lead 

What comes next

In the near term, we are focused on achieving the right baseline: making sure all documentation is in place, all processes are codified, and we are ready for the enforcement deadlines that matter.

The longer-term goal is bigger. We want to continue to be at the forefront of what responsible AI development actually looks like in insurance, not just compliant on paper but genuinely ahead of the problem. That means working with bodies like the NAIC to share what has worked and what has not. And it means making sure the program itself stays modular and adaptable, so when new regulations drop or our product evolves, we can slot in new requirements without rebuilding from scratch.

We built the governance program because Responsible AI is a core part of Sixfold. The structure the program provides is how we make sure that commitment holds.

Questions about Sixfold's AI governance approach? Reach out to your customer success representative or get in touch with our team. 

Learn more about our commitment to being a Responsible AI organization here. 

━━━

FAQ

What is an AI Governance Program? An AI Governance Program is a structured and ongoing internal process for identifying, assessing and mitigating risks. Sixfold developed the program to mitigate the risks of deploying our AI in insurance underwriting. It is organized into five focus areas, each with named owners, defined deliverables, and a regular review cadence.

Why did Sixfold create a formal AI governance program? To formalize responsible AI practices it had been following since launch, and to meet the requirements of emerging AI legislation. AI compliance is process-based and requires ongoing evaluation.

Is the AI Governance Program required by law? Not by a single law, but most emerging AI regulations require some form of oversight program. Rather than chasing specific legislation, Sixfold anchored the program in established frameworks like NIST AI RMF, which is more durable given how frequently AI laws are still changing.

How often does Sixfold review its AI governance program? Program owners meet bi-weekly. Risk and compliance runs quarterly reviews, as does the ethics and responsible AI function. Incident response is handled ad hoc as needed.

How does the AI Governance Program help insurance customers with their own compliance? Sixfold gives customers documented answers to vendor compliance questionnaires and a framework they can learn from as they build their own responsible AI practices. Sixfold's customer success team works with each customer individually given differences in jurisdiction and product line.

Who is responsible for AI governance at Sixfold? The whole company. Named owners across the five focus areas maintain the expertise and process rigor, but every person at Sixfold is accountable for building software that is honest, transparent, and ethical.

As AI becomes more embedded in the insurance underwriting process, carriers, vendors, and regulators share a growing responsibility to ensure these systems remain fair and unbiased.

At Sixfold, our dedication to building responsible AI means regularly exploring new and thoughtful ways to evaluate fairness.¹

We sat down with Elly Millican, Responsible AI & Regulatory Research Expert, and Noah Grosshandler, Product Lead on Sixfold's Life & Health team, to discuss how Sixfold is approaching fairness testing in a new way.

Fairness As AI Systems Advance

Fairness in insurance underwriting isn’t a new concern, but testing for it in AI systems that don’t make binary decisions is.

At Sixfold, our Underwriting AI for life and health insurers don’t approve or deny applicants. Instead, it analyzes complex medical records and surface relevant information based on each insurer's unique risk appetite. This allows underwriters to work much more efficiently and focus their time on risk assessment, not document review.

“We needed to develop new methodologies for fairness testing that reflect how Sixfold works.”
‍
— Elly Millican, Responsible AI & Regulatory Research Expert

While that’s a win for underwriters, it complicates fairness testing. When your AI produces qualitative outputs such as facts and summaries, rather than scores and decisions, most traditional fairness metrics won’t work. Testing for fairness in this context requires an alternative approach.
‍

“The academic work around fairness testing is very focused on traditional predictive models, however Sixfold is doing document analysis,” explains Millican. “We needed to develop new methodologies for fairness testing that reflect how Sixfold works.”

“The academic work around fairness testing is very focused on traditional predictive models, however Sixfold is doing document analysis,” explains Millican. “We needed to develop new methodologies for fairness testing that reflect how Sixfold works.”

“Even selecting which facts to pull and highlight from medical records in the first place comes with the opportunity to introduce bias. We believe it’s our responsibility to test for and mitigate that,” Grosshandler adds.

While regulations prohibit discrimination in underwriting, they rarely spell out how to measure fairness in systems like Sixfold’s. That ambiguity has opened the door for innovation, and for Sixfold to take initiative on shaping best practices and contributing to the regulatory conversation.

A New Testing Methodology

To address the challenge of fairness testing in a system with no binary outcomes, Sixfold is developing a methodology rooted in counterfactual fairness testing. The idea is simple: hold everything constant except for a single demographic attribute and see if and how the AI’s output changes.²

“Ultimately we want to validate that medically similar cases are treated the same when their demographic attributes differ,”
‍
— Noah Grosshandler, Product Manager @Sixfold

“We start with an ‘anchor’ case and create a ‘counterfactual twin’ who is identical in every way except for one detail, like race or gender. Then we run both through our pipeline to see if the medical information that’s presented in Sixfold varies in a notable or concerning way” Millican explains.

“Ultimately we want to validate that medically similar cases are treated the same when their demographic attributes differ,” Grosshandler states.

Proof-of-Concept

For the initial proof-of-concept, the team is focused on two key dimensions of Sixfold’s Life & Health pipeline.

1. Fact Extraction Consistency
Does Sixfold extract the same facts from medically identical underwriting case records that differ only in one protected attribute?

2. Summary Framing and Content Consistency
Does Sixfold produce diagnosis summaries with equivalent clinical content and emphasis for medically identical underwriting cases?

“It’s not just about missing or added facts, sometimes it’s a shift in tone or emphasis that could change how a case is perceived,” Millican explains. “We want to be sure that if demographic details are influencing outputs, it’s only when clinically appropriate. Otherwise, we risk surfacing irrelevant information that could skew decisions.”

Expanding the Scope

While the team’s current focus is on foundational fairness markers (race and gender), the methodology is designed to evolve. Future testing will likely explore proxy variables such as ZIP codes, names, and socioeconomic indicators, which might implicitly shape model behavior.

“We want to get into cases where the demographic signal isn’t explicit, but the model might still infer something. Names, locations, insurance types, all of these could serve as proxies that unintentionally influence outcomes,” Millican elaborates.

The team is also thinking ahead to version control for prompts and model updates, ensuring fairness testing keeps pace with an evolving AI stack.

“We’re trying to define what fairness means for a new kind of AI system,” explains Millican. “One that doesn’t give a single output, but shapes what people see, read, and decide.”

Sixfold isn’t just testing for fairness in isolation, it’s aiming to contribute to a broader conversation on how LLMs should be evaluated in high-stakes contexts like insurance, healthcare, finance, and more.

That’s why Sixfold is proactively bringing this work to the attention of regulatory bodies. By doing so, we hope to support ongoing standards development in the industry and help others build responsible and transparent AI systems.

“This work isn’t just about evaluating Sixfold, it’s about setting new standards for a new category of AI." Grosshandler concludes.

“This work isn’t just about evaluating Sixfold, it’s about setting new standards for a new category of AI. Regulators are still figuring this out, so we’re taking the opportunity to contribute to the conversation and help shape how fairness is monitored in systems like ours,” Grosshandler concludes.

Positive Regulatory Feedback

When we recently walked through our testing methodology and results with a group of regulators focused on AI and data, the feedback was both thoughtful and encouraging. They didn’t shy away from the complexity, but they clearly saw the value in what we’re doing.

“The fact that it’s hard shouldn’t be a reason not to try. What you’re doing makes sense... You’re scrutinizing something that matters.” said one senior policy advisor.

“The fact that it’s hard shouldn’t be a reason not to try. What you’re doing makes sense... You’re scrutinizing something that matters.”

— Senior Policy Advisor

One of the key themes that came up during the meeting was the unique nature of generative AI, and why it demands a different kind of oversight. As one senior actuary and behavioral data scientist put it: “Large language models are more qualitative than quantitative... A lot of technical folks don’t really get qualitative. They’re used to numbers. The more you can explain how you test the language for accuracy, the more attention it will get.”

That comment really resonated. It reflects the heart of our approach, we’re not just tracking metrics. We’re evaluating how language evolves, how facts can shift, and how risk is framed and communicated depending on the inputs.

The Road Ahead

Fairness in AI isn’t a fixed destination, it’s an ongoing commitment. Sixfold’s work in developing and refining fairness and bias testing methodologies reflects that mindset.

As more organizations turn to LLMs to analyze and interpret sensitive information, the need for thoughtful, domain-specific fairness methods will only grow. At Sixfold, we’re proud to be at the forefront of that work.

Footnotes

¹While internal reviews have not surfaced evidence of systemic bias, Sixfold is committed to continuous testing and transparency to ensure that remains the case as we expand and refine our AI systems.

²To ensure accuracy, cases involving medically relevant demographic traits, like pregnancy in a gender-flipped case, are filtered out. The methodology is designed to isolate unfair influence, not obscure legitimate medical distinctions.

Today, we’re excited to introduce the first-ever AI Accuracy Validator built for insurance underwriting. 

This application provides our commerical insurance customers with a transparent and comprehensive way to evaluate Sixfold’s accuracy—reinforcing our commitment to bring reliable and trustworthy risk assessments to underwriters.

Why did we build this?

For an AI solution to truly add value in underwriting, it needs to be both efficient and accurate. Many claim to be both—but is there proof?

For an AI solution to truly add value in underwriting, it needs to be both efficient and accurate. Many claim to be both—but is there proof?

Measuring efficiency can be fairly straightforward—reducing manual work, processing submissions faster, and automating repetitive tasks all provide clear benchmarks. But accuracy? That’s a completely different challenge.

How does it work?

The Accuracy Validator compares Sixfold’s AI-generated insights to the ideal version—what an experienced underwriter at the carrier would expect. It checks for accuracy, scores the results, and provides feedback to improve alignment with human analysis.

Here is a video overview from Lana, Head of Product at Sixfold, on how the validator works:

AI that speaks Underwriter

‍‍For AI solutions built for underwriters, accuracy isn’t about finding a single “correct” answer—it’s about reasoning like an underwriter. Take a risk summary as an example, an AI-constructed risk summary shouldn’t just condense information; it should highlight the key risk factors that matter to each carrier.

But what happens if an AI summary leaves out a key risk detail? How do you measure how off it is? What do you compare it to? And when a model is updated, how do you know it’s actually improving accuracy—not just changing the output?

So we started searching for an evaluation tool that could help us answer these questions — but nothing existed.

These were the questions we asked ourselves. So we started searching for an evaluation tool that could help us answer these questions — but nothing existed. It wasn’t just that we couldn’t find the right tool—we realized the industry wasn’t even thinking about accuracy in an insurance-underwriting-specific way.

So, we built it. With this capability in place, we can continuously improve Sixfold’s output, ensuring underwriters receive factually correct, reliable, and actionable insights for every risk assessment.

Benefit #1 - Track progress over time

Evaluating AI accuracy isn’t just a one-time task—it’s about ensuring consistency and continuous improvement. With clear benchmark metrics, insurers can easily track progress and see how Sixfold’s AI aligns with their underwriting standards over time.

Accuracy benchmarks help insurers assess Sixfold’s performance during the pilot phase, ensuring it delivers value to the underwriting team before moving to full implementation.

Considering a Sixfold pilot? Accuracy benchmarks help insurers assess Sixfold’s performance during the pilot phase, ensuring it delivers value to the underwriting team before moving to full implementation. Want to keep tabs on accuracy? No problem. We offer on-demand reports to give our customers a real-time look at how well our AI is performing, whenever they need it.

Benefit #2 - Confident AI adoption

From day one, our goal has been to build an underwriting AI solution that users trust. If underwriters can’t trust Sixfold’s insights, why would they rely on them for critical decisions?

Even in low-stakes tasks, AI’s accuracy isn’t always guaranteed. Take general-purpose LLMs—they handle simple research tasks and tasks such as summarizing reports, but even then, you might find yourself second-guessing their output. They’re right sometimes—but how often? And can you tell when they’re not?

The result? More confident decisions, stronger justifications, and a clearer business case for when to quote—and when not to.

That kind of guesswork isn’t good enough for underwriting. The high-stakes decisions underwriters make every day demand high-stakes trust.

With transparent accuracy reporting, underwriters know exactly how reliable Sixfold’s insights are. The result? More confident decisions, stronger justifications, and a clearer business case for when to quote—and when not to.

Benefit #3 - Audit-ready records

To support insurers’ audit and compliance needs, we conduct regular assessments using this application — both after code updates and at scheduled intervals—to prevent model drift and ensure reliability. This process helps identify inconsistencies and flag any deviations from expected results before they impact underwriting decisions.

The Accuracy Validator generates a transparent, audit-ready log for each assessment, allowing insurers to:

✅ Verify the reasoning behind AI-generated insights and decisions.

✅ Monitor model performance over time to proactively address potential drift.

✅ Demonstrate compliance with regulatory requirements by providing clear, documented AI processes

Feedback from customers

As we’ve started to introduce this capability to insurers, the response has been overwhelmingly positive. Some have even asked if they can use it to evaluate some of their other AI applications — a very clear proof of its value from day one. Others have asked to use the Accuracy Validator outside of AI applications to monitor overall underwriting accuracy.

Another key feedback we’ve received is that no other AI solution offers this level of structured performance measurement and tracking.

Another key feedback we’ve received is that no other AI solution offers this level of structured performance measurement and tracking. Sixfold is the first to give insurers a clear way to validate AI impact and track results over time in underwriting.

Curious to learn how you can get started with Sixfold? Check out the FAQ section to learn more about our pilot program, designed to help insurers fully assess the value of Sixfold before scaling up.

Reach out with any additional questions!

In the hands of insurers, AI can drive great efficiency —safely and responsibly. We recently sat down with Matt Kelly, Data Strategy & Security expert and counsel at Debevoise & Plimpton, to explore how insurers can achieve this.

Matt has played a key role in developing Sixfold’s 2024 Responsible AI Framework. With deep expertise in AI governance, he has led a growing number of insurers through AI implementations as adoption accelerates across the insurance industry.

To support insurers in navigating the early stages of compliance evaluation, he outlined four key steps:‍

Step 1: Define the Type of Vendor

Before getting started, it’s important to define what type of AI vendor you’re dealing with. Vendors come in various forms, and each type serves a different purpose. Start by asking these key questions:

• Are they really an AI vendor at all? Practically all vendors use AI (or will do so soon) – even if only in the form of routine office productivity tools and CRM suites. The fact that a vendor uses AI does not mean they use it in a way that merits treating them as an “AI vendor.” If the vendor’s use of AI is not material to either the risk or value proposition of the service or software product being offered (as may be the case, for instance, if a vendor uses it only for internal idea generation, background research, or for logistical purposes), ask yourself whether it makes sense to treat them as an AI vendor at all.  
• Is this vendor delivering AI as a standalone product, or is it part of a broader software solution? You need to distinguish between vendors that are providing an AI system that you will interact with directly, versus those who are providing a software solution that leverages AI in a way that is removed from any end users. 
• What type of AI technology does this vendor offer? Are they providing or using machine learning models, natural language processing tools, or something else entirely? Have they built or fine-tuned any of their AI systems themselves, or are they simply built atop third-party solutions?
• How does this AI support the insurance carrier’s operations? Is it enhancing underwriting processes, improving customer service, or optimizing operational efficiency?

Pro Tip: Knowing what type of AI solution you need and what the vendor provides will set the stage for deeper evaluations. Map out a flowchart of potential vendors and their associated risks. 

Step 2: Identify the Risks Associated with the Vendor

Regulatory and compliance risks are always present when evaluating AI vendors, but it’s important to understand the specific exposures for each type of implementation. Some questions to consider are: 

• Are there specific regulations that apply? Based on your expected use of the vendor, are there likely to be specific regulations that would need to be satisfied in connection with the engagement (as would be the case, for instance, with using AI to assist with underwriting decisions in various jurisdictions)? 
• What are the data privacy risks? Does the vendor require access to sensitive information – particularly personal information or material nonpublic information – and if so, how do they protect it? Can a customer’s information easily be removed from the underlying AI or models?
• How explainable are their AI models? Are the decision-making processes clear, are they well documented, and can the outputs be explained to and understood by third parties if necessary?
• What cybersecurity protocols are in place? How does the vendor ensure that AI systems (and your data) are secure from misuse or unauthorized access?
• How will things change? What has the vendor committed to do in terms of ongoing monitoring and maintenance? How will you monitor compliance and consistency going forward?  

Pro Tip: A good approach is to create a comprehensive checklist of potential risks for evaluation. For each risk that can be addressed through contract terms, build a playbook that includes key diligence questions, preferred contract clauses, and acceptable backup options. This will help ensure all critical areas are covered and allow you to handle each risk with consistency and clarity.

Step 3: Evaluate How Best to Mitigate the Identified Risks 

Your company likely has processes in place to handle third-party risks, especially when it comes to data protection, vendor management, and quality control. However, not all risks may be covered, and they may need new or different mitigations. Start by asking:

• What existing processes already address AI vendor risks? For example, if you already have robust data privacy policies, consider whether those policies cover key AI-related risks, and if so, ensure they are incorporated into the AI vendor review process.
• Which risks remain unresolved? Identify the gaps in your current processes to identify unique residual risks – such as algorithmic biases or the need for external audits on AI models – that will require new and ongoing resource allocations.
• How can we mitigate the residual risks? Rather than relying solely on contractual provisions and commercial remedies, consider alternative methods to mitigate residual risks, including data access controls and other technical limitations. For instance, when it comes to sharing personal or other protected data, consider alternative means (including the use of anonymized, pseudonymized, or otherwise abstracted datasets) to help limit the exposure of sensitive information.

Pro Tip: You don’t always need to reinvent the wheel. Look at existing processes within your organization, such as those for data privacy, and determine if they can be adapted to cover AI-specific risks.

Step 4: Establish a Plan for Accepting and Governing Remaining Risks

Eliminating all AI vendor risks cannot be the goal. The goal must be to identify, measure, and mitigate AI vendor risks to a level that is reasonable and that can be accepted by a responsible, accountable person or committee. Keep these final considerations in mind:

• How centralized is your company’s decision-making process? Some carriers may have a centralized procurement team handling all AI vendor decisions, while others may allow business units more autonomy. Understanding this structure will guide how risks are managed.
• Who is accountable for evaluating and approving these risks? Should this decision be made by a procurement team, the business unit, or a senior executive? Larger engagements with greater risks may require involvement from higher levels of the company.
• Which risks are too significant to be accepted? In any vendor engagement, some risks may simply be unacceptable to the carrier. For example, allowing a vendor to resell policyholder information to third parties would often fall into this category. Those overseeing AI vendor risk management usually identify these types of risks instinctively, but clearly documenting them helps ensure alignment among all stakeholders, including regulators and affected parties.  

One-Process-Fits-All Doesn’t Apply 

As AI adoption grows in insurance, taking a strategic approach can help simplify review processes and prioritize efforts. These four steps provide the foundation for making informed, secure decisions from the start of your AI implementation project.

Evaluating AI vendors is a unique process for each carrier that requires clarity about the type of vendor, understanding the risks, identifying the gaps in your existing processes, and deciding how to mitigate the remaining risks moving forward. Each organization will have a unique approach based on its structure, corporate culture, and risk tolerance.

“Every insurance carrier that I’ve worked with has its own unique set of tools and rules for evaluating AI vendors, what works for one may not be the right fit for another.”

- Matt Kelly, Counsel at Debevoise & Plimpton. 

With the rise of AI solutions in the Insurance market, questions around AI regulations and compliance are increasingly at the forefront. Key questions such as “What happens when we use data in the context of AI?” and “What are the key focus areas in the new regulations?” are top of mind for both consumers and industry leaders.

To address these topics, Sixfold’s founder and CEO, Alex Schmelkin, hosted the webinar “How to Secure Your AI Compliance Team’s Approval”. Joined by industry experts Jason D. Lapham, Deputy Commissioner for P&C Insurance for the State of Colorado, and Matt Kelly, Data Strategy & Security Counsel at Debevoise & Plimpton, the discussion provided essential insights into navigating AI regulations and compliance.

Here are the key insights from the session:

AI Regulation Developments: Colorado Leads the Way in the U.S

“There’s a requirement in almost any regulatory regime to protect consumer data. But now, what happens when we start using that data in AI? Are things different?” — Alex Schmelkin

Both nationally and globally, AI regulations are being implemented. In the U.S., Colorado became the first state to pass a law and implement regulations related to AI in the insurance sector. Jason Lapham explained that the key components of this legislation revolve around two major requirements:

1. Governance and Risk Management Frameworks: Companies must establish robust frameworks to manage the risks associated with AI and predictive models.
2. Quantitative Testing: Businesses must test their AI models to ensure that outcomes generated from non-traditional data sources (e.g., external consumer data) do not lead to unfairly discriminatory results. The legislation also mandates a stakeholder process prior to adopting rules.

Initially, the focus was on life insurance, as it played a critical role in shaping the legislative process. The first regulation, implementing Colorado’s Bill 169, adopted in late 2023, addressed governance and risk management. This regulation applies to life insurers across all practices, and the Regulatory Agency received the first reports this year from companies using predictive models and external consumer data sources.

So, what’s the next move for the first-moving state in terms of AI regulations? Colorado Division of Insurance is developing a framework for quantitative testing to help insurers assess whether their models produce unfairly discriminatory outcomes. Insurers are expected to take action if their models do lead to such outcomes.

Compliance Approach: Develop Governance Programs

“When we’re discussing with clients, we say focus on the operational risk side, and it will get you largely where you need to be for most regulations out there.” — Matt Kelly

With AI regulations differing across U.S. states and globally, companies face challenges. Matt Kelly described how his team at Debevoise & Plimpton navigate these challenges by building a framework that prioritizes managing operational risk related to technology. Their approach involves asking questions such as :

• What AI is being used?
• What risks are associated with its use?
• How is the company governing or mitigating those risks?

By focusing on these questions, companies can develop strong governance programs that align with most regulatory frameworks. Kelly advises clients to center their efforts on addressing operational risks, which takes them a long way toward compliance.

The Four Pillars of AI Compliance 

Across different AI regulatory regimes, four common themes emerge:

1. Transparency and Accountability: Companies must understand and clearly explain their AI processes. Transparency is a universal requirement.
2. Ethical and Fair Usage: Organizations must ensure their AI models do not introduce bias and must be able to demonstrate fairness.
3. Consumer Protection: In all regulatory contexts, protecting consumer data is essential. With AI, this extends to ensuring models do not misuse consumer information.
4. Governance Structure: Insurance companies are responsible for ensuring that they—and any third-party model providers—comply with AI regulations. While third-party providers play a role, carriers are ultimately accountable.

Matt Kelly emphasizes that insurers can navigate these four themes successfully by establishing the right frameworks and governance structures. 

Protection vs. Innovation: Striking the Right Balance 

“We tend not to look at innovation as a risk. We see it as aligned with protecting consumers when managed correctly.” — Matt Kelly

Balancing consumer protection with innovation is crucial for insurers. When done correctly, these goals align. Matt noted that the focus should be on leveraging technology to improve services without compromising consumer rights.

One major concern in insurance is unfair discrimination, particularly in how companies categorize risks using AI and consumer data. Regulators ask whether these categorizations are justified based on coverage or risk pool considerations, or whether they are unfairly based on unrelated characteristics. Aligning these concerns with technological innovation can lead to more accurate and fair coverage decisions while ensuring compliance with regulatory standards.

Want to learn more? 

Watch the full webinar recording and download Sixfold’s Responsible AI framework for Sixfold’s approach to safe AI usage. 

‍

These days, my professional life is dedicated to one focused part of the global business landscape: the untamed frontier where cutting-edge AI meets insurance. 

I have conversations with insurers around the world about where it’s all going and how AI will work under new global regulations. And one thing never ceases to amaze me: how often I end up addressing the same misconceptions. 

Some confusion is understandable (if not inevitable) considering the speed with which these technologies are evolving, the hype from those suddenly wanting a piece of the action, and some fear-mongering from an old guard seeking to maintain the status quo. So, I thought I’d take a moment to clear the air and address six all-too-common myths about AI in insurance.

Myth 1: You’re not allowed to use AI in insurance

Yes, there’s a patchwork of emerging AI regulations—and, yes, in many cases they do zero-in specifically on insurance—but they do not ban its use. From my perspective, they do just the opposite: They set ground rules, which frees carriers to invest in innovation without fear they are developing in the wrong direction and will be forced into a hard pivot down the line. 

Sixfold has actually increased customers (by a lot) since the major AI regulations in Europe and elsewhere were announced. So, let’s put this all-too-prevalent misconception to bed once and for all. There are no rules prohibiting you from implementing AI into your insurance processes.

Myth 2: AI solutions can’t secure customer data

As stated above, there are no blanket prohibitions on using customer data in AI systems. There  are, however, strict rules dictating how data—particularly PII and PHI—must be managed and secured. These guidelines aren’t anything radically new to developers with experience in highly regulated industries. 

Security-first data processes have been the norm since long before LLMs went mainstream. These protocols protect crucial personal data in applications that individuals and businesses use every day without issue (digital patient portals, browser-based personal banking, and market trading apps, just to name a few). These same measures can be seamlessly extended into AI-based solutions.

Myth 3: “My proprietary data will train other companies’ models”

No carrier would ever allow its proprietary data to train models used by competitors. Fortunately, implementing an LLM-powered solution does not mean giving up control of your data—at least with the right approach. 

A responsible AI vendor helps their clients build AI solutions trained on their unique data for their exclusive use, as opposed to a generic insurance-focused LLM to be used by all comers. This also means allowing companies to maintain full control over their submissions within their environment so that when, for example, a case is deleted, all associated artifacts and data are removed across all databases.

At Sixfold, we train our base models on public and synthetic (AKA, “not customer”) data. We then copy these base models into dedicated environments for our customers and all subsequent training and tuning happens in the dedicated environments. Customer guidelines and data never leave the dedicated environment and never make it back to the base models.

Let’s kill this one: Yes, you can use AI and still maintain control of your data.

Myth 4: There’s no way to prevent LLM hallucinations

We’ve all seen the surreal AI-generated images lurching up from the depths of the uncanny valley—hands with too many fingers, physiology-defying facial expressions, body parts & objects melded together seemingly at random. Surely, we can’t use that technology for consequential areas like insurance. But I’m here to tell you that with the proper precautions and infrastructure, the impact of hallucinations can be greatly minimized, if not eliminated.

Mitigation is achieved using a myriad of tactics such as using models to auto-review generated content, incorporating user feedback to identify and correct hallucinations, and conducting manual reviews to ensure quality by comparing sample outputs against ground truth sets.

Myth 5: AIs run autonomously without human oversight

Even if you never watched The Terminator, The Matrix, 2001: A Space Odyssey, or any other movie about human-usurping tech, it’d be reasonable to have some reservations about scaled automation. There’s a lot of fearful talk out there about humans ceding control in important areas to un-feeling machines. However, that’s not where we’re at, nor is it how I see these technologies developing. 

Let’s break this one down.

AI is a fantastic and transformative technology, but even I—the number one cheerleader for AI-powered insurance—agree we shouldn’t leave technology alone to make consequential decisions like who gets approved for insurance and at what price. But even if I didn’t feel this way, insurtechs are obliged to comply with new regulations (e.g., the EU AI Act and the California Department of Insurance), that tilt towards avoiding fully automated underwriting and require, at the very least, that humans overseers can audit and review decisions. 

When it comes to your customers’ experience, AI opens the door to more human engagement, not less. In my view, AI will free underwriters from banal, repetitive data work (which machines handle better anyway) so that they can apply uniquely human skills in specialized or complex use cases they previously wouldn’t have had the bandwidth to address.

Myth 6: Regulations are still being written, it’s better to wait for them to settle

I hear this one a lot. I understand why people arrive at this view. My take? You can’t afford to sit on the sidelines!

To be sure, multiple sets of AI regulations are taking root at different governmental levels, which adds complexity. But here’s a little secret from someone paying very close attention to emerging AI rulesets: there’s very little daylight between them. 

Here’s the thing: regulators worldwide attend the same conferences, engage with the same stakeholders, and read the same studies & whitepapers. And they all watching what each other is doing. As a result, we’re arriving at a global consensus focused on three main areas: data security, transparency, and auditability. 

The global AI regulatory landscape is, like any global regulatory landscape, complex; but I’m here to tell you it’s not nearly as uneven or even close to unmanageable as you may fear. 

Furthermore, if an additional major change were to be introduced, it wouldn't suddenly take effect. That’s by design. Think of all the websites and digital applications that launched—and indeed, thrived—in the six-year window between when GDPR was introduced in 2012 to when it became enforceable in 2018. Think of everything that would have been lost if they had waited until GDPR was firmly established before moving forward.

My entire career has been spent in fast-moving cutting-edge technologies. And I can tell you from experience that it’s far better to deploy & iterate than to wait for regulatory Godot to arrive. Jump in and get started!

There are more myths to bust! Watch our compliance webinar

The regulations coming are not as odious or as unmanageable as you might fear—particularly when you work with the right partners. I hope I’ve helped overcome some misconceptions as you move forward on your AI journey.

Want to learn more about AI insurance and compliance? Watch the replay of our compliance webinar featuring a discussion between myself; Jason D. Lapham, the Deputy Commissioner for P&C Insurance the Colorado Division of Insurance; and Matt Kelly a key member of Debevoise & Plimpton’s Artificial Intelligence Group. We're discussing the global regulatory landscape and how AI models should be evaluated regarding compliance, data usage, and privacy.

‍

This article was originally posted on LinkedIn