AI-built tools in healthtech: the compliance risk nobody flagged

Somewhere in your healthtech company, a useful little tool got built with an AI app builder. Maybe it tracks appointment no-shows, or logs symptoms patients report, or manages a waitlist. It works, people rely on it, and it’s quietly holding health data that nobody put through a compliance review. In healthcare, that’s a bigger deal than the same mistake anywhere else.

This post covers why health data changes the risk picture, and what to check when an AI-built tool in a health setting starts handling it. It’s a practical guide, not medical or legal advice; treat anything involving patient data as a “get qualified sign-off” situation.

Why health data is a different category

GDPR treats most personal data one way and a short list of sensitive types another. Data about someone’s health is on that special list (Article 9), alongside things like biometrics and genetic data. The default rule is that you’re not allowed to process special-category data at all, unless you meet one of a narrow set of conditions, such as explicit consent or providing health care under a proper legal framework.

That flips the usual posture. For ordinary data, you find a lawful basis and proceed. For health data, you start from “no” and have to actively clear a higher bar. An AI-built tool that started as a simple tracker and drifted into logging symptoms, diagnoses, or medication has crossed into that higher-bar territory, often without anyone deciding to.

Two more points raise the stakes:

• The bar for security is higher. GDPR expects security measures appropriate to the risk. Health data is high-risk by definition, so weak access controls or an exposed key aren’t just bad practice, they’re a poor fit for the sensitivity of the data.
• A breach hurts more. A breach involving health data is more likely to require notifying both the regulator and the affected individuals, and it’s more likely to cause real harm to those people. Regulators weigh that harm when they decide on penalties.

None of this means AI-built tools are banned in healthtech. It means the “someone built it over a weekend and nobody reviewed it” pattern is far riskier here than it is for, say, an internal expense tracker.

What to check

Work through these for any AI-built tool in a health setting. Anything you can’t answer is a gap.

1. Does it actually hold health data? Be honest about scope creep. A tool that started neutral may now store symptoms, conditions, appointment reasons, or anything that reveals a health status. Even “reason for cancellation: chemo appointment” is health data.

2. What’s the lawful basis for special-category data? Ordinary consent for a mailing list isn’t enough. You need one of the Article 9 conditions, and you need to have identified it on purpose. If nobody can name it, that’s your first gap.

3. Has a DPIA been done? A Data Protection Impact Assessment is a structured risk review. For large-scale processing of health data, it’s often not optional. Even where it isn’t strictly required, it’s the right tool for spotting the risks in an unreviewed app.

4. Who can access the data, and how tightly? Health data should be on a need-to-know footing. Check the access list, check that login uses multi-factor authentication, and check that access is removed when people leave.

5. Where does the data live, and is it encrypted? Confirm the country (UK/EU residency avoids extra transfer steps) and confirm encryption in transit and at rest. An unencrypted health database is a serious finding.

6. Are there secrets exposed in the app? AI-generated apps sometimes leave API keys or database credentials in front-end code. For health data, an exposed key is close to a worst case. This is worth a specific check.

7. Is there a signed DPA with the platform? Any third party processing this data needs a proper processor contract. For health data, “probably fine” isn’t good enough.

8. Are backups tested, and where do they sit? Health records you can’t recover, or backups sitting unencrypted in the wrong country, are both problems. Confirm the restore has actually been run.

9. Who owns this tool? A named, accountable person who can reach the data and respond to a request or a breach. In healthcare especially, an orphaned tool holding patient data is a live risk.

The honest recommendation

For most internal tools, “fix the gaps in place” is a reasonable path. For a tool holding real health data, the bar is higher, and the safest move is usually to get it off consumer-grade AI hosting and onto infrastructure where you control access, residency, encryption, and backups, with the DPIA and lawful basis documented. The cost of doing that is small next to the cost of a health-data breach.

If the tool is only a prototype and hasn’t touched real patient data yet, now is the moment to sort this out, before it does.

Or, we do it for you

We move AI-built apps onto hosting where health data sits in the EU, is encrypted, has real access controls and tested backups, and comes with a signed DPA. For a tool holding health data, that’s the difference between a finding and a clean review. Start with a free audit on one tool on our for-SME page.

FAQ

Is a symptom tracker really “health data” under GDPR?

If it records anything that reveals a person’s health status, yes. That includes symptoms, conditions, medications, appointment reasons, and test results. Even indirect clues, like the reason someone cancelled a treatment appointment, can count. When in doubt, treat it as health data and apply the higher bar.

Do we always need a DPIA?

Not for everything, but large-scale or systematic processing of health data is one of the cases where a DPIA is often required, and it’s good practice even when it isn’t strictly mandatory. It’s a structured way to find the risks in a tool that was never formally reviewed. Get qualified sign-off on whether your specific case requires one.

Can we keep using the AI-built tool at all?

Often yes, if you close the gaps: a valid lawful basis, tight access, EU residency, encryption, a signed DPA, tested backups, and a named owner. What you shouldn’t do is keep running it unreviewed on consumer-grade hosting while it holds real patient data. Fix it or move it.

This is general guidance. Where do we get a real answer for our situation?

Treat this as a checklist of questions to raise, not a legal opinion. Health data sits under both GDPR and, depending on your country and setting, sector-specific rules. For anything involving real patient data, get sign-off from a qualified data protection or legal professional who knows your jurisdiction.