Sixfold Content

The EU AI Act sets rules for companies that develop and use AI in the European Union. It entered into force in 2024, with different requirements scheduled to apply over time.

In June 2026, the European Parliament and Council adopted the Digital Omnibus on AI, a package of targeted amendments to the EU AI Act. It moves the compliance date for stand-alone high-risk AI systems from August 2, 2026, to December 2, 2027. The update is relevant to Sixfold because the Act identifies certain uses of AI in life and health insurance as high-risk.

The updated timeline gives Sixfold more time to incorporate the standards and guidance that the European standards bodies and the Commission are still developing. Sixfold’s AI Governance Program and the work already underway continue. Give our customers concrete answers on AI governance.

What Changed

The EU AI Act uses a risk-based approach. Systems that could have a greater effect on people’s safety or fundamental rights are subject to stricter requirements.

Annex III specifically lists AI systems intended for risk assessment and pricing in relation to natural persons in life and health insurance. It does not classify every use of AI in insurance or underwriting as high-risk.

The Act also distinguishes between providers, which develop AI systems, and deployers, which use them. Sixfold is a provider under the Act. Its focus includes documentation, testing, governance, and giving customers clear information about how its solution is developed and managed.

How Sixfold Stays Consistent

Sixfold designed its AI Governance Program to remain useful as requirements develop. The program uses several reference points:

1. The NIST AI Risk Management Framework, which provides guidance for identifying and managing AI risk.
2. The EU AI Act’s high-risk provider requirements, which set detailed expectations for documentation, testing, and accountability.
3. International AI management frameworks that provide additional guidance for governing AI responsibly.

Together, these reference points give Sixfold a consistent structure for managing AI risk. Sixfold treats the EU AI Act’s high-risk provider requirements as a demanding benchmark.

What Sixfold is Doing Today

Sixfold’s AI Governance Program is an ongoing internal process for identifying, assessing, and addressing the risks involved in developing and deploying AI.

The program covers five areas: general governance, risk and compliance, data and model governance, ethics and Responsible AI, and security and privacy. Each area has named owners, defined deliverables, and a regular review schedule.

Risk and compliance reviews take place quarterly. The ethics function reviews new functionality and human oversight design each quarter. The full group of program owners meets every two weeks.

Sixfold is also preparing technical documentation aligned with Annex IV of the EU AI Act. This work brings together information about the solution, its intended use, governance, testing, validation, and deployment procedures. It gives customers and regulators a clearer view of how the solution is developed and managed.

Bias and fairness testing is another part of the program, with the methodology adapted for each line of business. The approved amendments permit special category personal data to be processed when strictly necessary to detect and correct bias, with appropriate safeguards, in both high-risk and non-high-risk AI systems. This provides greater clarity for providers conducting bias testing, and Sixfold will continue incorporating additional guidance as it is published.

Structure for New Requirements

Europe is not the only jurisdiction updating its AI rules. Colorado repealed and replaced its original 2024 AI Act with SB26-189, which was signed in May 2026 and takes effect January 1, 2027. The new law drops the algorithmic discrimination duty of care in favor of a transparency and disclosure based framework. It encompasses Automated Decision-Making Technology, or ADMT, rather than AI only, and focuses on areas including developer documentation, consumer notices, recordkeeping, and consumer rights.

The details differ across jurisdictions, but Sixfold can assess new and changing requirements through the structure it already has in place: defined roles, documentation, testing, and regular reviews.That is the practical benefit of a program grounded in established, stable frameworks. Sixfold can incorporate new requirements without rebuilding its governance approach every time a law changes.

Customers & Responsible AI

Sixfold works with insurance customers to give them a clear understanding of how its solution is developed, tested, and governed. This includes documentation on intended use, testing methods, safeguards, human oversight, and the processes Sixfold uses to manage AI risk.

Customers can bring questions about Sixfold’s governance practices directly to the team. Sixfold can provide the supporting information needed for conversations with procurement, risk, compliance, and other internal stakeholders. That gives customers concrete answers about their AI provider instead of broad statements about Responsible AI.

Questions about Sixfold’s AI governance approach? Reach out to your customer success representative or get in touch with the Sixfold team.

Learn more about Sixfold’s AI Governance Program and its broader Responsible AI approach.

━━━

Frequently Asked Questions

When do the EU AI Act’s high-risk requirements apply?

Under the amendments approved in June 2026, requirements for stand-alone high-risk AI systems will apply from December 2, 2027. Requirements for high-risk AI systems embedded in regulated products will apply from August 2, 2028.

Is all AI used in insurance considered high-risk under the EU AI Act?

No. Annex III specifically lists AI intended for risk assessment and pricing in relation to natural persons in life and health insurance. It does not classify every use of AI in insurance or underwriting as high-risk.

Does the updated EU AI Act timeline change Sixfold’s approach?

Sixfold’s AI governance program continues regardless of the EU AI Act timeline changes. The updated timeline gives the company more time to incorporate forthcoming standards and guidance into the program and documentation already underway.

How is Sixfold preparing for the EU AI Act?

Sixfold is preparing technical documentation aligned with Annex IV, conducting bias and fairness testing, and running regular governance reviews. Its program uses the NIST AI Risk Management Framework, the EU AI Act’s high-risk provider requirements, and international AI management frameworks as reference points.

What does the EU AI Act update mean for Sixfold customers?

Customers can continue to expect documented information about how Sixfold’s solution is developed, governed, and tested. The updated timeline gives Sixfold additional time to incorporate detailed guidance and strengthen the materials it provides as an AI provider.

Sixfold's AI Governance Program is an ongoing internal process for identifying, assessing, and mitigating the risks of deploying AI. It is organized into five focus areas with named owners, defined deliverables, and a regular cadence of reviews. 

This post explains what it covers and why we built it the way we did.

Why we established this now

Responsible AI is not a new territory for Sixfold. We have been thinking carefully about how we build and deploy AI in insurance underwriting since the day we launched in May 2023. 

The honest answer to "why now?" is this: we wanted to formalize what we were already doing, and the regulatory environment gave us both the mandate and the structure to do it well.

The reality is that AI legislation is different from most compliance frameworks insurers and vendors are familiar with. AI compliance is process-based. It is about building the organizational muscle to understand whether your systems could cause harm, and then evaluating that question consistently over time.

That kind of ongoing work requires real ownership, clear processes, and people who are explicitly accountable for staying on top of it. We also made a deliberate choice not to chase specific legislation line by line. AI laws are still in progress across jurisdictions and contested at every level. Instead, we anchored the program around established industry reference points:

1. The NIST AI Risk Management Framework (AI RMF 1.0), an industry framework that gives durable guidance for risk and oversight.
2. The EU AI Act's high-risk provider requirements (Articles 9-17), which has concrete expectations on documentation, testing, and accountability

That approach is more durable and more honest about where regulation actually is today.

From principles to practice

Sixfold has had a set of responsible AI principles since early on, we formalized and launched our first Responsible AI approach in 2024 and established and launched an updated version in 2025. What the governance program does is translate those principles into organized action.

"Principles are well and good, but without action words they're just hot air."
‍
– Noah Grosshandler, Product Manager and AI Governance Program Lead at Sixfold

The program gives us specific owners, defined deliverables, and a cadence for revisiting each area. It creates the connective tissue between what we believe and what we actually do.  And critically, it builds the structure to change over time, because the landscape will constantly move forward, our product will change, and what it means to apply those principles responsibly will change with them.

We are building toward a complete Annex IV documentation package. That is the concrete output regulators and enterprise customers ask for. The program also addresses requirements to prevent algorithmic discrimination in much of the emerging global AI legislation, especially relevant to our customers operating in the life and health space. Bias and fairness testing is part of the program, with methodology tailored to each line of business. 

How the program is structured

The program is organized around five focus areas, each with its own owners and activities:

General governance covers the program itself: keeping the right people informed, running annual retrospectives, and ensuring continuity when things change.

Risk and compliance is responsible for identifying and tracking AI-specific risks, reporting on them quarterly, and handling any ad hoc mitigation or incident response as needed.

Data and model governance focuses on the data used to develop and train our systems. This means understanding where data comes from, confirming it is used ethically, and being able to clearly explain how system behavior relates to the inputs that shaped it.

Ethics and responsible AI covers fairness, explainability, and human oversight. It is the function that asks where humans should stay in the loop, what our position is on where AI should and should not operate, and whether the way our system works is genuinely ethical, not just technically sound.

Security and privacy handles the technical infrastructure: making sure the underlying systems meet the security and privacy requirements of what we are building.

Each area has its own cadence. Risk and compliance runs quarterly reviews. The ethics function does the same, looking at new functionality and evaluating human oversight design. Between those cycles, the full group of program owners meets bi-weekly to stay coordinated.

"The different areas of focus don't need to change when a new regulation drops or a new product drops, we can slot new requirements into the existing structure."
‍
– Noah Grosshandler, Product Manager and Facilitator of the AI Governance Program

One thing worth being explicit about: having dedicated program owners does not mean this is only their responsibility. At the end of the day, the company is accountable. Every person at Sixfold is responsible for building software that is honest, transparent, and ethical. The program owners exist to maintain the expertise and the process rigor that makes that possible at scale.

What this means for our customers

We have customers in North America, EMEA, South America and Australia today. Across the globe our customers face their own compliance pressures as deployers, and we can support them with that. Many of them are being asked by their own regulators and partners to demonstrate responsible AI practices, and most are still working out what that means for them.

The governance program helps us support our customers directly: 

• When a customer receives a compliance questionnaire about their AI vendor, we will have clear and documented answers. 
• When they are trying to build their own responsible AI stance, we can share what we have learned. We are not just handing over a package of documents; we are trying to be a resource to support them build out their own AI Governance practices. 
• Our customer success team works with each customer individually, because our customers operate across different jurisdictions, different product lines, and different regulatory environments. There is no one-size-fits-all approach here.

"The company is accountable. Just because there is this program and this charter doesn't mean that it is not every single person at this company's job to make sure that what we're building is built ethically, built honestly, built transparently."
‍
– Noah Grosshandler, Product Manager and AI Governance Program Lead 

What comes next

In the near term, we are focused on achieving the right baseline: making sure all documentation is in place, all processes are codified, and we are ready for the enforcement deadlines that matter.

The longer-term goal is bigger. We want to continue to be at the forefront of what responsible AI development actually looks like in insurance, not just compliant on paper but genuinely ahead of the problem. That means working with bodies like the NAIC to share what has worked and what has not. And it means making sure the program itself stays modular and adaptable, so when new regulations drop or our product evolves, we can slot in new requirements without rebuilding from scratch.

We built the governance program because Responsible AI is a core part of Sixfold. The structure the program provides is how we make sure that commitment holds.

Questions about Sixfold's AI governance approach? Reach out to your customer success representative or get in touch with our team. 

Learn more about our commitment to being a Responsible AI organization here. 

━━━

FAQ

What is an AI Governance Program? An AI Governance Program is a structured and ongoing internal process for identifying, assessing and mitigating risks. Sixfold developed the program to mitigate the risks of deploying our AI in insurance underwriting. It is organized into five focus areas, each with named owners, defined deliverables, and a regular review cadence.

Why did Sixfold create a formal AI governance program? To formalize responsible AI practices it had been following since launch, and to meet the requirements of emerging AI legislation. AI compliance is process-based and requires ongoing evaluation.

Is the AI Governance Program required by law? Not by a single law, but most emerging AI regulations require some form of oversight program. Rather than chasing specific legislation, Sixfold anchored the program in established frameworks like NIST AI RMF, which is more durable given how frequently AI laws are still changing.

How often does Sixfold review its AI governance program? Program owners meet bi-weekly. Risk and compliance runs quarterly reviews, as does the ethics and responsible AI function. Incident response is handled ad hoc as needed.

How does the AI Governance Program help insurance customers with their own compliance? Sixfold gives customers documented answers to vendor compliance questionnaires and a framework they can learn from as they build their own responsible AI practices. Sixfold's customer success team works with each customer individually given differences in jurisdiction and product line.

Who is responsible for AI governance at Sixfold? The whole company. Named owners across the five focus areas maintain the expertise and process rigor, but every person at Sixfold is accountable for building software that is honest, transparent, and ethical.

As AI becomes more embedded in the insurance underwriting process, carriers, vendors, and regulators share a growing responsibility to ensure these systems remain fair and unbiased.

At Sixfold, our dedication to building responsible AI means regularly exploring new and thoughtful ways to evaluate fairness.¹

We sat down with Elly Millican, Responsible AI & Regulatory Research Expert, and Noah Grosshandler, Product Lead on Sixfold's Life & Health team, to discuss how Sixfold is approaching fairness testing in a new way.

Fairness As AI Systems Advance

Fairness in insurance underwriting isn’t a new concern, but testing for it in AI systems that don’t make binary decisions is.

At Sixfold, our Underwriting AI for life and health insurers don’t approve or deny applicants. Instead, it analyzes complex medical records and surface relevant information based on each insurer's unique risk appetite. This allows underwriters to work much more efficiently and focus their time on risk assessment, not document review.

“We needed to develop new methodologies for fairness testing that reflect how Sixfold works.”
‍
— Elly Millican, Responsible AI & Regulatory Research Expert

While that’s a win for underwriters, it complicates fairness testing. When your AI produces qualitative outputs such as facts and summaries, rather than scores and decisions, most traditional fairness metrics won’t work. Testing for fairness in this context requires an alternative approach.
‍

“The academic work around fairness testing is very focused on traditional predictive models, however Sixfold is doing document analysis,” explains Millican. “We needed to develop new methodologies for fairness testing that reflect how Sixfold works.”

“The academic work around fairness testing is very focused on traditional predictive models, however Sixfold is doing document analysis,” explains Millican. “We needed to develop new methodologies for fairness testing that reflect how Sixfold works.”

“Even selecting which facts to pull and highlight from medical records in the first place comes with the opportunity to introduce bias. We believe it’s our responsibility to test for and mitigate that,” Grosshandler adds.

While regulations prohibit discrimination in underwriting, they rarely spell out how to measure fairness in systems like Sixfold’s. That ambiguity has opened the door for innovation, and for Sixfold to take initiative on shaping best practices and contributing to the regulatory conversation.

A New Testing Methodology

To address the challenge of fairness testing in a system with no binary outcomes, Sixfold is developing a methodology rooted in counterfactual fairness testing. The idea is simple: hold everything constant except for a single demographic attribute and see if and how the AI’s output changes.²

“Ultimately we want to validate that medically similar cases are treated the same when their demographic attributes differ,”
‍
— Noah Grosshandler, Product Manager @Sixfold

“We start with an ‘anchor’ case and create a ‘counterfactual twin’ who is identical in every way except for one detail, like race or gender. Then we run both through our pipeline to see if the medical information that’s presented in Sixfold varies in a notable or concerning way” Millican explains.

“Ultimately we want to validate that medically similar cases are treated the same when their demographic attributes differ,” Grosshandler states.

Proof-of-Concept

For the initial proof-of-concept, the team is focused on two key dimensions of Sixfold’s Life & Health pipeline.

1. Fact Extraction Consistency
Does Sixfold extract the same facts from medically identical underwriting case records that differ only in one protected attribute?

2. Summary Framing and Content Consistency
Does Sixfold produce diagnosis summaries with equivalent clinical content and emphasis for medically identical underwriting cases?

“It’s not just about missing or added facts, sometimes it’s a shift in tone or emphasis that could change how a case is perceived,” Millican explains. “We want to be sure that if demographic details are influencing outputs, it’s only when clinically appropriate. Otherwise, we risk surfacing irrelevant information that could skew decisions.”

Expanding the Scope

While the team’s current focus is on foundational fairness markers (race and gender), the methodology is designed to evolve. Future testing will likely explore proxy variables such as ZIP codes, names, and socioeconomic indicators, which might implicitly shape model behavior.

“We want to get into cases where the demographic signal isn’t explicit, but the model might still infer something. Names, locations, insurance types, all of these could serve as proxies that unintentionally influence outcomes,” Millican elaborates.

The team is also thinking ahead to version control for prompts and model updates, ensuring fairness testing keeps pace with an evolving AI stack.

“We’re trying to define what fairness means for a new kind of AI system,” explains Millican. “One that doesn’t give a single output, but shapes what people see, read, and decide.”

Sixfold isn’t just testing for fairness in isolation, it’s aiming to contribute to a broader conversation on how LLMs should be evaluated in high-stakes contexts like insurance, healthcare, finance, and more.

That’s why Sixfold is proactively bringing this work to the attention of regulatory bodies. By doing so, we hope to support ongoing standards development in the industry and help others build responsible and transparent AI systems.

“This work isn’t just about evaluating Sixfold, it’s about setting new standards for a new category of AI." Grosshandler concludes.

“This work isn’t just about evaluating Sixfold, it’s about setting new standards for a new category of AI. Regulators are still figuring this out, so we’re taking the opportunity to contribute to the conversation and help shape how fairness is monitored in systems like ours,” Grosshandler concludes.

Positive Regulatory Feedback

When we recently walked through our testing methodology and results with a group of regulators focused on AI and data, the feedback was both thoughtful and encouraging. They didn’t shy away from the complexity, but they clearly saw the value in what we’re doing.

“The fact that it’s hard shouldn’t be a reason not to try. What you’re doing makes sense... You’re scrutinizing something that matters.” said one senior policy advisor.

“The fact that it’s hard shouldn’t be a reason not to try. What you’re doing makes sense... You’re scrutinizing something that matters.”

— Senior Policy Advisor

One of the key themes that came up during the meeting was the unique nature of generative AI, and why it demands a different kind of oversight. As one senior actuary and behavioral data scientist put it: “Large language models are more qualitative than quantitative... A lot of technical folks don’t really get qualitative. They’re used to numbers. The more you can explain how you test the language for accuracy, the more attention it will get.”

That comment really resonated. It reflects the heart of our approach, we’re not just tracking metrics. We’re evaluating how language evolves, how facts can shift, and how risk is framed and communicated depending on the inputs.

The Road Ahead

Fairness in AI isn’t a fixed destination, it’s an ongoing commitment. Sixfold’s work in developing and refining fairness and bias testing methodologies reflects that mindset.

As more organizations turn to LLMs to analyze and interpret sensitive information, the need for thoughtful, domain-specific fairness methods will only grow. At Sixfold, we’re proud to be at the forefront of that work.

Footnotes

¹While internal reviews have not surfaced evidence of systemic bias, Sixfold is committed to continuous testing and transparency to ensure that remains the case as we expand and refine our AI systems.

²To ensure accuracy, cases involving medically relevant demographic traits, like pregnancy in a gender-flipped case, are filtered out. The methodology is designed to isolate unfair influence, not obscure legitimate medical distinctions.

Today, we’re excited to introduce the first-ever AI Accuracy Validator built for insurance underwriting. 

This application provides our commerical insurance customers with a transparent and comprehensive way to evaluate Sixfold’s accuracy—reinforcing our commitment to bring reliable and trustworthy risk assessments to underwriters.

Why did we build this?

For an AI solution to truly add value in underwriting, it needs to be both efficient and accurate. Many claim to be both—but is there proof?

For an AI solution to truly add value in underwriting, it needs to be both efficient and accurate. Many claim to be both—but is there proof?

Measuring efficiency can be fairly straightforward—reducing manual work, processing submissions faster, and automating repetitive tasks all provide clear benchmarks. But accuracy? That’s a completely different challenge.

How does it work?

The Accuracy Validator compares Sixfold’s AI-generated insights to the ideal version—what an experienced underwriter at the carrier would expect. It checks for accuracy, scores the results, and provides feedback to improve alignment with human analysis.

Here is a video overview from Lana, Head of Product at Sixfold, on how the validator works:

AI that speaks Underwriter

‍‍For AI solutions built for underwriters, accuracy isn’t about finding a single “correct” answer—it’s about reasoning like an underwriter. Take a risk summary as an example, an AI-constructed risk summary shouldn’t just condense information; it should highlight the key risk factors that matter to each carrier.

But what happens if an AI summary leaves out a key risk detail? How do you measure how off it is? What do you compare it to? And when a model is updated, how do you know it’s actually improving accuracy—not just changing the output?

So we started searching for an evaluation tool that could help us answer these questions — but nothing existed.

These were the questions we asked ourselves. So we started searching for an evaluation tool that could help us answer these questions — but nothing existed. It wasn’t just that we couldn’t find the right tool—we realized the industry wasn’t even thinking about accuracy in an insurance-underwriting-specific way.

So, we built it. With this capability in place, we can continuously improve Sixfold’s output, ensuring underwriters receive factually correct, reliable, and actionable insights for every risk assessment.

Benefit #1 - Track progress over time

Evaluating AI accuracy isn’t just a one-time task—it’s about ensuring consistency and continuous improvement. With clear benchmark metrics, insurers can easily track progress and see how Sixfold’s AI aligns with their underwriting standards over time.

Accuracy benchmarks help insurers assess Sixfold’s performance during the pilot phase, ensuring it delivers value to the underwriting team before moving to full implementation.

Considering a Sixfold pilot? Accuracy benchmarks help insurers assess Sixfold’s performance during the pilot phase, ensuring it delivers value to the underwriting team before moving to full implementation. Want to keep tabs on accuracy? No problem. We offer on-demand reports to give our customers a real-time look at how well our AI is performing, whenever they need it.

Benefit #2 - Confident AI adoption

From day one, our goal has been to build an underwriting AI solution that users trust. If underwriters can’t trust Sixfold’s insights, why would they rely on them for critical decisions?

Even in low-stakes tasks, AI’s accuracy isn’t always guaranteed. Take general-purpose LLMs—they handle simple research tasks and tasks such as summarizing reports, but even then, you might find yourself second-guessing their output. They’re right sometimes—but how often? And can you tell when they’re not?

The result? More confident decisions, stronger justifications, and a clearer business case for when to quote—and when not to.

That kind of guesswork isn’t good enough for underwriting. The high-stakes decisions underwriters make every day demand high-stakes trust.

With transparent accuracy reporting, underwriters know exactly how reliable Sixfold’s insights are. The result? More confident decisions, stronger justifications, and a clearer business case for when to quote—and when not to.

Benefit #3 - Audit-ready records

To support insurers’ audit and compliance needs, we conduct regular assessments using this application — both after code updates and at scheduled intervals—to prevent model drift and ensure reliability. This process helps identify inconsistencies and flag any deviations from expected results before they impact underwriting decisions.

The Accuracy Validator generates a transparent, audit-ready log for each assessment, allowing insurers to:

✅ Verify the reasoning behind AI-generated insights and decisions.

✅ Monitor model performance over time to proactively address potential drift.

✅ Demonstrate compliance with regulatory requirements by providing clear, documented AI processes

Feedback from customers

As we’ve started to introduce this capability to insurers, the response has been overwhelmingly positive. Some have even asked if they can use it to evaluate some of their other AI applications — a very clear proof of its value from day one. Others have asked to use the Accuracy Validator outside of AI applications to monitor overall underwriting accuracy.

Another key feedback we’ve received is that no other AI solution offers this level of structured performance measurement and tracking.

Another key feedback we’ve received is that no other AI solution offers this level of structured performance measurement and tracking. Sixfold is the first to give insurers a clear way to validate AI impact and track results over time in underwriting.

Curious to learn how you can get started with Sixfold? Check out the FAQ section to learn more about our pilot program, designed to help insurers fully assess the value of Sixfold before scaling up.

Reach out with any additional questions!

In the hands of insurers, AI can drive great efficiency —safely and responsibly. We recently sat down with Matt Kelly, Data Strategy & Security expert and counsel at Debevoise & Plimpton, to explore how insurers can achieve this.

Matt has played a key role in developing Sixfold’s 2024 Responsible AI Framework. With deep expertise in AI governance, he has led a growing number of insurers through AI implementations as adoption accelerates across the insurance industry.

To support insurers in navigating the early stages of compliance evaluation, he outlined four key steps:‍

Step 1: Define the Type of Vendor

Before getting started, it’s important to define what type of AI vendor you’re dealing with. Vendors come in various forms, and each type serves a different purpose. Start by asking these key questions:

• Are they really an AI vendor at all? Practically all vendors use AI (or will do so soon) – even if only in the form of routine office productivity tools and CRM suites. The fact that a vendor uses AI does not mean they use it in a way that merits treating them as an “AI vendor.” If the vendor’s use of AI is not material to either the risk or value proposition of the service or software product being offered (as may be the case, for instance, if a vendor uses it only for internal idea generation, background research, or for logistical purposes), ask yourself whether it makes sense to treat them as an AI vendor at all.  
• Is this vendor delivering AI as a standalone product, or is it part of a broader software solution? You need to distinguish between vendors that are providing an AI system that you will interact with directly, versus those who are providing a software solution that leverages AI in a way that is removed from any end users. 
• What type of AI technology does this vendor offer? Are they providing or using machine learning models, natural language processing tools, or something else entirely? Have they built or fine-tuned any of their AI systems themselves, or are they simply built atop third-party solutions?
• How does this AI support the insurance carrier’s operations? Is it enhancing underwriting processes, improving customer service, or optimizing operational efficiency?

Pro Tip: Knowing what type of AI solution you need and what the vendor provides will set the stage for deeper evaluations. Map out a flowchart of potential vendors and their associated risks. 

Step 2: Identify the Risks Associated with the Vendor

Regulatory and compliance risks are always present when evaluating AI vendors, but it’s important to understand the specific exposures for each type of implementation. Some questions to consider are: 

• Are there specific regulations that apply? Based on your expected use of the vendor, are there likely to be specific regulations that would need to be satisfied in connection with the engagement (as would be the case, for instance, with using AI to assist with underwriting decisions in various jurisdictions)? 
• What are the data privacy risks? Does the vendor require access to sensitive information – particularly personal information or material nonpublic information – and if so, how do they protect it? Can a customer’s information easily be removed from the underlying AI or models?
• How explainable are their AI models? Are the decision-making processes clear, are they well documented, and can the outputs be explained to and understood by third parties if necessary?
• What cybersecurity protocols are in place? How does the vendor ensure that AI systems (and your data) are secure from misuse or unauthorized access?
• How will things change? What has the vendor committed to do in terms of ongoing monitoring and maintenance? How will you monitor compliance and consistency going forward?  

Pro Tip: A good approach is to create a comprehensive checklist of potential risks for evaluation. For each risk that can be addressed through contract terms, build a playbook that includes key diligence questions, preferred contract clauses, and acceptable backup options. This will help ensure all critical areas are covered and allow you to handle each risk with consistency and clarity.

Step 3: Evaluate How Best to Mitigate the Identified Risks 

Your company likely has processes in place to handle third-party risks, especially when it comes to data protection, vendor management, and quality control. However, not all risks may be covered, and they may need new or different mitigations. Start by asking:

• What existing processes already address AI vendor risks? For example, if you already have robust data privacy policies, consider whether those policies cover key AI-related risks, and if so, ensure they are incorporated into the AI vendor review process.
• Which risks remain unresolved? Identify the gaps in your current processes to identify unique residual risks – such as algorithmic biases or the need for external audits on AI models – that will require new and ongoing resource allocations.
• How can we mitigate the residual risks? Rather than relying solely on contractual provisions and commercial remedies, consider alternative methods to mitigate residual risks, including data access controls and other technical limitations. For instance, when it comes to sharing personal or other protected data, consider alternative means (including the use of anonymized, pseudonymized, or otherwise abstracted datasets) to help limit the exposure of sensitive information.

Pro Tip: You don’t always need to reinvent the wheel. Look at existing processes within your organization, such as those for data privacy, and determine if they can be adapted to cover AI-specific risks.

Step 4: Establish a Plan for Accepting and Governing Remaining Risks

Eliminating all AI vendor risks cannot be the goal. The goal must be to identify, measure, and mitigate AI vendor risks to a level that is reasonable and that can be accepted by a responsible, accountable person or committee. Keep these final considerations in mind:

• How centralized is your company’s decision-making process? Some carriers may have a centralized procurement team handling all AI vendor decisions, while others may allow business units more autonomy. Understanding this structure will guide how risks are managed.
• Who is accountable for evaluating and approving these risks? Should this decision be made by a procurement team, the business unit, or a senior executive? Larger engagements with greater risks may require involvement from higher levels of the company.
• Which risks are too significant to be accepted? In any vendor engagement, some risks may simply be unacceptable to the carrier. For example, allowing a vendor to resell policyholder information to third parties would often fall into this category. Those overseeing AI vendor risk management usually identify these types of risks instinctively, but clearly documenting them helps ensure alignment among all stakeholders, including regulators and affected parties.  

One-Process-Fits-All Doesn’t Apply 

As AI adoption grows in insurance, taking a strategic approach can help simplify review processes and prioritize efforts. These four steps provide the foundation for making informed, secure decisions from the start of your AI implementation project.

Evaluating AI vendors is a unique process for each carrier that requires clarity about the type of vendor, understanding the risks, identifying the gaps in your existing processes, and deciding how to mitigate the remaining risks moving forward. Each organization will have a unique approach based on its structure, corporate culture, and risk tolerance.

“Every insurance carrier that I’ve worked with has its own unique set of tools and rules for evaluating AI vendors, what works for one may not be the right fit for another.”

- Matt Kelly, Counsel at Debevoise & Plimpton. 

With the rise of AI solutions in the Insurance market, questions around AI regulations and compliance are increasingly at the forefront. Key questions such as “What happens when we use data in the context of AI?” and “What are the key focus areas in the new regulations?” are top of mind for both consumers and industry leaders.

To address these topics, Sixfold’s founder and CEO, Alex Schmelkin, hosted the webinar “How to Secure Your AI Compliance Team’s Approval”. Joined by industry experts Jason D. Lapham, Deputy Commissioner for P&C Insurance for the State of Colorado, and Matt Kelly, Data Strategy & Security Counsel at Debevoise & Plimpton, the discussion provided essential insights into navigating AI regulations and compliance.

Here are the key insights from the session:

AI Regulation Developments: Colorado Leads the Way in the U.S

“There’s a requirement in almost any regulatory regime to protect consumer data. But now, what happens when we start using that data in AI? Are things different?” — Alex Schmelkin

Both nationally and globally, AI regulations are being implemented. In the U.S., Colorado became the first state to pass a law and implement regulations related to AI in the insurance sector. Jason Lapham explained that the key components of this legislation revolve around two major requirements:

1. Governance and Risk Management Frameworks: Companies must establish robust frameworks to manage the risks associated with AI and predictive models.
2. Quantitative Testing: Businesses must test their AI models to ensure that outcomes generated from non-traditional data sources (e.g., external consumer data) do not lead to unfairly discriminatory results. The legislation also mandates a stakeholder process prior to adopting rules.

Initially, the focus was on life insurance, as it played a critical role in shaping the legislative process. The first regulation, implementing Colorado’s Bill 169, adopted in late 2023, addressed governance and risk management. This regulation applies to life insurers across all practices, and the Regulatory Agency received the first reports this year from companies using predictive models and external consumer data sources.

So, what’s the next move for the first-moving state in terms of AI regulations? Colorado Division of Insurance is developing a framework for quantitative testing to help insurers assess whether their models produce unfairly discriminatory outcomes. Insurers are expected to take action if their models do lead to such outcomes.

Compliance Approach: Develop Governance Programs

“When we’re discussing with clients, we say focus on the operational risk side, and it will get you largely where you need to be for most regulations out there.” — Matt Kelly

With AI regulations differing across U.S. states and globally, companies face challenges. Matt Kelly described how his team at Debevoise & Plimpton navigate these challenges by building a framework that prioritizes managing operational risk related to technology. Their approach involves asking questions such as :

• What AI is being used?
• What risks are associated with its use?
• How is the company governing or mitigating those risks?

By focusing on these questions, companies can develop strong governance programs that align with most regulatory frameworks. Kelly advises clients to center their efforts on addressing operational risks, which takes them a long way toward compliance.

The Four Pillars of AI Compliance 

Across different AI regulatory regimes, four common themes emerge:

1. Transparency and Accountability: Companies must understand and clearly explain their AI processes. Transparency is a universal requirement.
2. Ethical and Fair Usage: Organizations must ensure their AI models do not introduce bias and must be able to demonstrate fairness.
3. Consumer Protection: In all regulatory contexts, protecting consumer data is essential. With AI, this extends to ensuring models do not misuse consumer information.
4. Governance Structure: Insurance companies are responsible for ensuring that they—and any third-party model providers—comply with AI regulations. While third-party providers play a role, carriers are ultimately accountable.

Matt Kelly emphasizes that insurers can navigate these four themes successfully by establishing the right frameworks and governance structures. 

Protection vs. Innovation: Striking the Right Balance 

“We tend not to look at innovation as a risk. We see it as aligned with protecting consumers when managed correctly.” — Matt Kelly

Balancing consumer protection with innovation is crucial for insurers. When done correctly, these goals align. Matt noted that the focus should be on leveraging technology to improve services without compromising consumer rights.

One major concern in insurance is unfair discrimination, particularly in how companies categorize risks using AI and consumer data. Regulators ask whether these categorizations are justified based on coverage or risk pool considerations, or whether they are unfairly based on unrelated characteristics. Aligning these concerns with technological innovation can lead to more accurate and fair coverage decisions while ensuring compliance with regulatory standards.

Want to learn more? 

Watch the full webinar recording and download Sixfold’s Responsible AI framework for Sixfold’s approach to safe AI usage. 

‍