From AI roadblock to breakthrough: how Wellthy transformed healthcare workflows with Tonic.ai

The challenge: unrealistic, ineffective, unstructured healthcare data

Recognized by Deloitte Technology and Inc. Magazine as one of the fastest-growing healthcare companies in America, Wellthy is revolutionizing the way families access and afford care for themselves and their loved ones. Their leading platform, offered as an employee benefit at dozens of Fortune 500 companies, helps families balance work and care responsibilities through a combination of human expertise and precision technology. 

In processing both employee and healthcare data, Wellthy handles myriad sensitive data types, including PII and PHI, which are subject to HIPAA and other data privacy regulations. Keeping that sensitive data out of their developer environments has always been mission critical, and since 2022, they’ve relied on Tonic.ai’s test data management solution Tonic Structural to de-identify their structured data for secure use in testing and development.

There was a catch, though. Within their structured datasets were columns of unstructured data: free-text messages between members and their care teams, riddled with sensitive data that also needed to be anonymized. Using Tonic Structural, Wellthy would scramble those free-text fields to safely remove all sensitive data and prevent any exposure. But for certain use cases, including genAI feature development, more realistic free-text data was key.

“A lot of the genAI features we’ve been building in recent years are oriented around the unstructured messages in our database,” explained Kevin Roche, Co-founder and CTO of Wellthy. “It was becoming difficult to develop and test these features without realistic data.”

The team attempted to generate fake messages manually or using a Faker library. But, Roche added, “They weren’t always super realistic. Obviously that led to the features not being as thoroughly tested until they reached production.” Which, as CTO, was a troublesome pain point. “How do we really know how these different features are going to perform when they hit production? We didn’t have a safe way to test with real data and people manually generating test data wasn’t cutting it.”

But as Wellthy was investing in new genAI capabilities, Tonic.ai was doing the same. In late 2023, Tonic.ai launched Tonic Textual, a platform built specifically for unstructured data de-identification and synthesis. And Roche was among the first to see it: “As Structural users, we were aware of Tonic Textual pretty early and knew we wanted to try it out.”

The solution: realistic, safe, unstructured healthcare data, refreshed daily

Wellthy’s first step in validating Tonic Textual as the solution to their needs was to put Textual to the test—on their developers’ own data. “We use Wellthy ourselves,” Roche explained, “so some of us ran Textual against our own member data and asked ourselves, ‘Would we trust all this de-identified content being exposed to everyone?’ If we trusted Textual with our own personal sensitive information, then we felt that we could trust it for others too.”

The test was a solid success: Wellthy’s team agreed that they could not identify themselves or their data in Textual’s de-identified output. And equally as important, that output still looked, felt, and behaved like real free-text data.

Shortly thereafter, Wellthy added Textual to their tech stack. “It wasn't that hard to get up and running,” Roche said. They implemented a process that applies Textual to all those columns of unstructured data in their larger database of structured data managed by Tonic Structural. “So now our test environments, including development environments, have realistic messages without any PII.” 

This synthetic free-text dataset is fed into test scripts of their genAI features for high-fidelity performance testing. With Textual embedded in their workflows, they’ve automated data refresh, as well. “Textual is literally running every night. We generate a new dump of our production database on a nightly basis. Textual de-identifies the incremental new messages that came in the previous day, and our developers always start their days with that fresh, up-to-date data set.”

Given the speed at which AI initiatives are moving, these regular data refreshes are of high value in enabling Wellthy’s features to keep pace with the latest models. “We have a feature for sentiment analysis of member messages. Even though the feature already exists, obviously new models are coming out all the time. So as new models are released, or if we want to experiment with other models, we’re able to run the models against the latest data and see how the new models perform relative to past tests.”

The results: genAI features, increased efficiency, and fewer bugs

Thanks to the realism Wellthy is able to generate in the unstructured data they de-identify and synthesize with Tonic Textual, they’ve developed and launched new genAI features that were previously unfeasible. And those features are driving pronounced results in their users’ experience, in some cases cutting down workflow inefficiencies by an astounding 50%.

One of those features is designed to assess the need for follow-up actions to be assigned to care teams after in-app messaging conversations with members. In training the AI’s model on realistic messages generated by Textual, Wellthy has dramatically improved the feature’s ability to accurately determine which messages require follow-ups. “It cut down the number of action items by tens of thousands in the past few months,” stated Roche, “Overall, it cut the total number in half. Just imagine the noise that was getting created that we're not creating anymore.”

In addition to improving the platform’s efficiency and performance, testing on more realistic data has resulted in far fewer bugs and incidents that reach production. Roche put it best: “We’ve always had development and staging environments that were pretty much replicas of production. The only difference was the data. In the past, if something went wrong in production, we knew it was because of the data. Reducing bugs was my main motivation originally to roll out Tonic Structural, and it's become even more true with Textual getting in the mix.”

Not only are their testing environments better able to replicate production, their developers are better able to understand the product they're building, too. “I’ve gotten feedback qualitatively from the team that the realistic messages help them have better context for the product itself,” shared Roche. “If they haven’t yet used Wellthy themselves, they weren't always aware of how people were using it. Seeing realistic messages was eye-opening and helps us build a better product.”

Wellthy has only just begun to roll out new AI capabilities built on Textual’s securely de-identified free-text data. More use cases and initiatives are on the horizon, including leveraging task summary data to help in automating the care summaries sent to members. Currently the summaries have to be created manually by care coordinators, but with realistic task summaries generated by Textual, Roche is looking forward to optimizing the process for Wellthy’s users.

“Without a solution for secure, realistic test data, many of our new AI features simply wouldn't have been possible,” said Roche.

Because in developing AI, data is key. And in developing healthcare solutions, data privacy is paramount. “With Tonic Textual, we can now confidently build and test these features without exposing PII, all while maintaining the rigorous privacy standards we hold ourselves accountable to as a healthcare company serving millions of families.”