The Single Sentence That Changed How We Detect ERISA Violations

As a senior data scientist at Darrow, I serve as a bridge between our AI and legal teams. In this article, I’d like to share a story from my work: how a single sentence buried in a 200-page pension document turned out to be a serious ERISA violation, and how that discovery led us to build a system that can uncover hundreds of similar issues at scale.

The Raytheon Case

A few years ago, a corporation called Raytheon filed a new pension plan. At first glance, nothing appeared out of the ordinary. The 200-page document looked like any other.

But on page 111 there was a sentence stating that Raytheon would calculate benefits using a mortality table from 1971 and an interest rate of 7%. Both of these assumptions are outdated and inflated, revealing an ERISA violation that ultimately led to a $59.2 million settlement in Cruz v. Raytheon.

In this case, the attorneys detected the violation after extensive manual review. However, most attorneys do not have the time or capacity to manually review lengthy plan documents in search of rare patterns like this. That’s why uncovering these violations is treated as almost impossible.

And that’s also why my team decided it was essential to develop an AI-based system to detect these violations. 

Why Manual Review Fails

To see the scale of the problem, we first need to look at the numbers.

In the United States, there are roughly 50,000 active pension plans. If a law firm wants to detect violations in those plans, the traditional approach is to hire an ERISA expert and ask them to read pension documents all day.

However:

• Each pension plan is about 200 to 300 pages long
• A careful review of one plan takes 2 to 3 days
• One expert can read at most 100 plans per year

At that pace, even a very dedicated expert will only touch a small fraction of all active plans. Most plans will never receive a full expert review. The odds of randomly choosing the one plan with a random hidden sentence are very small. 

It was clear to me that we needed to develop a different strategy that could make this kind of large scale review realistic.

Step One: Narrow Down Relevant Plans

When I started thinking about how to solve this, I began with a simple question.

Do all pension plans deserve the same level of attention?

The answer is no.

Some plans have already been sued. Some are too small to justify the cost of a case. Some are compliant. So, the first step was to filter out the irrelevant plans.

We first developed a software to narrow our list of 50,000 pension plans down to about 7,000 that were large and active enough from a litigation perspective. But still, manually reviewing 7,000 plans manually would take years.

At this point the key question became: How can we help human experts focus only on the most relevant pages and the most promising plans?

Step Two: Use AI To Assign Signal Scores

So the next question was: how do we actually read all of this?

The answer was to build a fast, AI-based process that could scan every page of every document. At this stage, the goal wasn’t for the AI to decide whether there was definitely a violation, but to do something simpler and much more useful: give every page a signal score.

For each pension plan, the signal told us whether there were any signs of a potential violation. For plans with a signal, it pointed us to the specific pages that looked most suspicious.

This completely changed the workload. Instead of asking our legal partners to read 200 pages per plan, we could send them straight to 2–10 pages with the highest chance of containing a real problem.

Step Three: Deep Analysis and Case Ready Reports

Once the system identified the relevant plans and pages, we conducted the next stage of analysis. This required us to evaluate all flagged sections and answer two important questions:

1. Is this signal a real, legally actionable violation?
2. If so, what exactly is the evidence and how should it be presented to a lawyer?

We could then compile a report for an attorney with:

• The exact language from the plan
• The page numbers
• The explanation of why this is a violation
• The assumptions and data that matter for damages and liability

Narrowing Down 70,000 Plans to 400 in 8 Hours

From initial filtering to final results, the entire process took about eight hours. Our signal analysis flagged roughly 1,000 plans with potential violations, and a deeper review confirmed that around 400 of those plans contained actual violations. These violations were precisely the kinds of issues that appeared in the Raytheon plan.

Our Legal Intelligence team then reviewed the 400 flagged plans and found that our system had an accuracy of 98%. Only six of the 400 did not contain a true violation.

Opportunities for ERISA Litigation Attorneys

Detecting hundreds of violations only has value if attorneys can pursue them.

Our team has identified hundreds of pension plans with confirmed ERISA violations. Through a partnership with Darrow, plaintiff firms can access a pipeline of these case opportunities complete with the underlying plan language, page citations, and the key assumptions that matter for liability and damages.

Instead of spending months searching for a single viable case, ERISA-focused firms can start from a curated list of plans that already show clear indicators of unlawful assumptions. That allows legal teams to focus their time on evaluating claims, preparing complaints, and representing employees and retirees who have been underpaid.

We’re continuing to expand this detection framework and are actively partnering with ERISA litigators who want a steady stream of high-quality case opportunities.

• The Colgate ERISA Settlement & the Future of Benefit-Plan Litigation
• Strategic Approaches to Overcoming ERISA Litigation Challenges
• 3 ERISA Litigation Trends to Keep Your Eye On