Clinical AI assistant that summarizes patient charts for physicians during patient handoffs
In accordance with our privacy statement.
High-risk clinical SaMD with EU AI Act conformity obligations and layered GDPR exposure at inference.
This launch sits at the intersection of three hard regulatory regimes: FDA SaMD, EU AI Act high-risk classification, and GDPR Articles 22 and 35, all triggered simultaneously by the decision to run identified patient data through the model at inference time even though training used de-identified data. That inference-time data flow is the single architectural fact that generates the most obligations, because it is the moment the system becomes a live processor of sensitive health data in a high-stakes clinical workflow with direct effects on patients. The EU AI Act adds a conformity assessment track, mandatory human oversight documentation, accuracy and robustness baselines, and Art. 50 disclosure requirements that must be built into the product before any EU hospital goes live, not after. On the US side, HIPAA BAA coverage for inference, CPRA automated decision-making obligations, and breach notification exposure under California and New York law run concurrently and require clear allocation of responsibility between the AI vendor and each pilot hospital in writing.
- 1Commission the GDPR Article 35 DPIA now, before EU hospital pilots begin, covering the full inference pipeline from EHR input through summary output, because it is the prerequisite gate for lawful processing under Articles 6, 22, and 32 and directly informs the EU AI Act technical file under Art. 16.
- 2Document the technical file required by EU AI Act Art. 16 . including model architecture, training methodology, validation results, accuracy baselines under Art. 15, and known limitations . before any EU deployment, as this is the conformity assessment foundation and the baseline for strict liability under the Revised Product Liability Directive.
- 3Confirm in writing, for each of the 12 pilot hospitals, which legal entity (hospital, IDN, or AI vendor) is the HIPAA covered entity or business associate responsible for identified patient data at inference time, and whether a BAA covers that inference use, since CPRA, SHIELD Act, and California breach notification obligations also turn on this allocation.
- 4Implement visible AI-generated disclosure on every summary output shown to physicians . a header or badge stating the summary is AI-generated . to satisfy both EU AI Act Art. 50 and CPRA ADMT notice obligations, and document physician override capability in the workflow to support the Art. 14 and GDPR Art. 22 human oversight record.
- 5Audit the de-identified training dataset to identify any copyrighted clinical materials (guidelines, drug databases, diagnostic criteria) and obtain written confirmation from hospital legal that use is licensed, in the public domain, or defensible as fair use, then incorporate that documentation into the FDA SaMD submission.
Whether the clinical summary output, given its role as the primary document physicians review during handoffs, constitutes solely automated decision-making with legal or similarly significant effects under GDPR Art. 22, which would require explicit patient consent or a Member State law basis and is not resolved by characterizing the tool as advisory.
AI laws that may apply
22 surfaced across 5 lensesGrouped by legal lens. Click any provision to see how it applies to this launch specifically.
AI-specific
5High-risk AI systems must include human oversight measures to minimize risks to health, safety or fundamental rights.
- Accuracy, robustness, security (AI Act Art.15)Settled rule, unsettled applicationVerified 2026-05-25
High-risk AI systems must achieve and maintain a high level of accuracy, robustness and cybersecurity, with continuous testing to prevent malfunctions.
- Provider obligations for high-risk AI (AI Act Art.16)Settled rule, unsettled applicationVerified 2026-05-25
Providers of high-risk AI must ensure system compliance, affix CE mark, maintain quality management and documentation, and handle logging, conformity assessment, and corrective actions.
- Disclosure of AI interaction (AI Act Art.50(1))Settled rule, unsettled applicationVerified 2026-05-25
Providers must design AI systems interacting with people so that users are informed they are interacting with AI (not a human).
- Explanation of high-risk decisions (AI Act Art.86)Settled rule, unsettled applicationVerified 2026-05-25
Individuals subject to high-risk AI decisions that significantly affect them have the right to obtain a clear, meaningful explanation of the AI system role and the main decision elements.
Privacy
9- Automated decision-making prohibition (GDPR Art.22)Settled rule, unsettled applicationVerified 2026-05-25
Data subjects have a right not to be subject to solely automated decisions (including profiling) producing legal or similarly significant effects on them.
- Data protection by design and by default (GDPR Art.25)Settled rule, unsettled applicationVerified 2026-05-25
Controllers must implement data-protection principles (e.g. minimization, pseudonymisation) into processing from the earliest design stages.
Controllers and processors must implement appropriate technical and organizational measures to secure personal data according to the risk (e.g. encryption, resiliency).
- Data Protection Impact Assessment (GDPR Art.35)Settled rule, unsettled applicationVerified 2026-05-25
Requires DPIA before processing that is likely high-risk to rights, e.g. systematic automated profiling with significant effects.
Personal data processing must fit at least one lawful basis (e.g. consent, contract performance, vital interests, public task, legitimate interest).
HIPAA requires health data to be de-identified (removing PHI identifiers) before use for secondary purposes, meaning AI training on medical data often needs de-identification or patient authorization.
- CPRA: Limit use of sensitive personal informationSettled rule, unsettled applicationVerified 2026-05-25
California consumers can direct businesses to limit use of their sensitive personal information to only what is necessary for the requested goods or services.
- CPRA: Automated decisionmaking technology (ADMT) noticeSettled rule, unsettled applicationVerified 2026-05-25
Businesses using automated decisionmaking technology to make certain decisions must provide consumers with a pre-use notice explaining the ADMT use and informing them of rights including opt-out before processing their p…
CPRA requires businesses whose processing poses significant privacy risks to conduct a documented risk assessment before initiating that processing.
Security
4CPRA mandates that covered businesses undergo annual independent cybersecurity audits assessing controls to ensure personal data protection.
Under CIRCIA, designated critical-infrastructure companies must report covered cyber incidents to CISA within 72 hours of discovery.
California Civil Code 1798.80 to 1798.84 require any person doing business in the state to disclose to California residents any data breach involving unencrypted personal information.
NY SHIELD Act requires entities holding private information to implement reasonable safeguards and notify affected NY residents of data breaches.
Liability
1The EU 2024 update to the Product Liability Directive extends strict liability to digital products including AI-based systems.
other
3Requires certain deployers of high-risk AI systems to perform a Fundamental Rights Impact Assessment (FRIA) before first use. Applies to public bodies, private entities providing public services, and deployers of high-ri…
Imposes transparency obligations on providers and deployers of AI systems. Providers must ensure persons interacting with AI systems are informed they are interacting with AI (unless obvious). Providers of generative AI…
Grants data subjects the right not to be subject to a decision based solely on automated processing (including profiling) which produces legal effects or similarly significantly affects them. Three exceptions: contract n…
Worth watching
3Provisions that may not strictly apply today but are close enough to the launch shape that they are worth keeping an eye on. No per-launch analysis is generated for these.
- Copyright and AI Training DataRecent court guidance indicates that copying copyrighted works into AI models may infringe unless clearly tran…skadden.com ↗
- NYT v. OpenAI (Training Data)The New York Times has sued OpenAI, alleging that using its copyrighted articles to train ChatGPT without perm…theverge.com ↗
- AI-Related Copyright CasesCourts are grappling with AI and IP: e.g., in Thomson Reuters v. ROSS, a judge held that output of an AI model…skadden.com ↗
Other flags
Not legal advice. Structured analysis of what a thoughtful counsel would consider given the inputs above. Does not substitute for counsel review or certify compliance.