Who Owns the Synthetic Patient?

The paperwork for HIPAA has always been, at its heart, a map of boundaries. It defines who is a covered entity, what constitutes protected health information (PHI), and where that information is allowed to travel. It's a tidy system of fences. But those fences are currently being walked right through by AI agents that don't just move data from point A to point B, but transform it into something else entirely.

The current debate in the HealthTech sector, which the HHS OCR is watching with its usual quiet intensity, is whether a piece of data remains "patient data" once an AI agent has touched it. If an agent takes a thousand patient records and generates a synthetic summary or a predictive health trend, who owns that output? Is it still PHI?

The press releases from the AI vendors claim these tools are mere efficiencies. They'll tell you the AI is just a sophisticated clerk, tidying up the files so doctors can spend more time with patients. That sounds lovely in a brochure. In a regulatory filing, however, "tidying up" looks suspiciously like "unauthorised processing."

The rulebook isn't designed for alchemy. It's designed for storage and transmission. When an AI agent transforms raw data into a synthetic derivative, it creates a legal grey zone. If the resulting output is sufficiently transformed to be "de-identified", it falls outside HIPAA. But if there's any way to reverse-engineer that summary back to the original patient, we're back in the area of strict liability.

There's a tendency among compliance officers to believe that using an AI agent is no different from using a spreadsheet. They argue that the tool is neutral and the provider remains the sole steward of the data.

This is a mistake.

A spreadsheet doesn't decide which parts of a patient's history are "relevant" to a summary. A spreadsheet doesn't store weights in a neural network that might later be prompted to reveal fragments of the training set. The moment an AI agent generates a new insight from PHI, the provider has ceased to be a simple steward and has become a manufacturer of synthetic data.

The second-order effect here is where it gets truly messy for the auditors. We're seeing a gap in the audit trail that would make any civil servant shudder. An auditor checks the access logs; they see the AI agent accessed the record. They check the output; they see a summary. But they can't actually see *how* the transformation happened because the logic is buried in a proprietary model.

The auditors are effectively signing off on a black box. If a regulator later finds that these "summaries" were leaked or used to train a third-party model, the auditor will be the one left holding the bag, having certified a process they couldn't actually verify.

Then we have the EU AI Act stepping in to complicate things further. Under its high-risk classification for healthcare, there are strict requirements for data governance and transparency. Article 10 isn't just a suggestion; it requires that training, validation, and testing data sets be representative and free of errors.

For those operating across the Atlantic, this means they're now juggling two different definitions of "safe" data. One focuses on the privacy of the individual (HIPAA), while the other focuses on the quality and bias of the system (EU AI Act). It's a recipe for a filing nightmare. Providers will have to maintain detailed technical documentation for these systems for north of a decade.

Who actually has to do something about this?

First, any healthcare provider using "agentic" AI needs to stop treating the software as a tool and start treating it as a data processor. This means updating Business Associate Agreements (BAAs) to explicitly cover synthetic output. If your BAA only mentions "storage and transmission", it's effectively useless against an AI that generates new content.

Second, the compliance teams need to demand "explainability" logs from their vendors. Not a marketing slide about how the model works, but a concrete record of what data went in and why a specific output was generated.

The industry will argue that this is impossible: that the nature of LLMs precludes such granularity. That's a convenient excuse for the vendors, but it won't hold up during an OCR audit. If you can't explain how the data was transformed, you can't prove it was transformed safely.

We are moving toward a situation where the "synthetic patient" exists as a legal entity of sorts, a derivative work that carries the risk of the original but none of the clear protections. The vendors will keep pushing for "innovation", which is usually code for "we haven't figured out the paperwork yet".

The regulators, however, are famously patient. They'll wait for the first massive breach involving synthetic data (likely involving millions of records) and then they'll apply the rules as they were written in the nineties.

I suspect we'll find that "de-identified" is a much smaller category than the software vendors would like us to believe. For now, the only thing that's truly certain is that someone will be paying a very large fine for a summary they didn't actually write.