AI governance

How EDFlow governs AI: models and providers, zero data retention and no training, data classes C0 to C3, what leaves the tenant, human approval, the medical-device boundary, enablement gates G-AI-1 to G-AI-6, the kill switch and opting a visit out.

AI features are off by default for hospital tenants and are turned on one by one, per site, after the gates below. When they're off or unavailable, the normal screens work as usual.

Principles

The facts come from the product, not the model. Patient and family answers are cards built from the same data the screens show, with at most a few connecting phrases chosen from a reviewed phrase bank. Free text from the model is never shown to patients or family.
Same identity, same view. AI tools run with the caller's own permissions and projection.
Minimum necessary to the model. Each feature declares its data class and an allow-listed context. Names are replaced by references before the model and restored after.
The model is never the last line of defence. Deterministic filters run before and checks after every call.
People take actions. Anything that changes state or reaches another person needs a person's approval. The only automatic action is the deterministic safety alert (self-harm or undismissed distress), triggered by a word list, never by the model.
Not an emergency channel. The "I feel worse" button stays on every AI screen, and distress shows the tell-staff-now card even offline.
Honest about being AI. Every AI message carries a label; machine translation carries a badge.

Intended use

EDFlow's AI helps people find and understand information about visit logistics that the product already shows, drafts plain-language text for staff to review, and summarises operational data. It does not interpret symptoms or results, give treatment advice, decide who is seen first, or estimate individual clinical outcomes. A regulatory memo confirming this non-device intended use is one of the enablement gates.

Data classes

Class	Meaning	May reach a model
C0	No patient information: catalogue, approved content, UI text, synthetic samples, public docs	Yes
C1	De-identified aggregates with small-cell suppression	Yes
C2	PHI: pseudonymised visit state (keys, enums, bucketed times, display strings)	Only after the privacy office's approval, never for confidential visits
C3	PHI: scrubbed free text or audio from a person	Only after approval, per feature, minimised and scrubbed

Never sent to any model: names, dates of birth, health card, record or visit numbers or their hashes, internal ids, tokens or codes, phones, emails, addresses, CTAS, chief complaint, room or bed labels, result content, sensitive tests or consults, or anything about a confidential visit.

Providers and retention

Requests go through Vercel AI Gateway to an allow-listed provider (Anthropic, Amazon Bedrock, Google Vertex AI), with zero data retention and prompt training disallowed on every request and team-wide. Each response's routing is verified; a violation discards the output and raises an alert.
Inference currently runs in the US. For hospital tenants, phase 1 runs only C0 and C1 features, so no patient-derived data leaves Canada. Phase 2 (C2, C3) needs the privacy office to accept US processing under zero data retention.
Web voice transcription stays off for hospital tenants until zero retention is confirmed in writing and a PHIPA flow-down agreement exists. Native voice runs on the device.
The hospital's service keeps assistant conversations for 7 days after the session by default (0 to 30); conversations with a safety event are kept for the operational window for review.

See Subprocessors.

Enablement gates

A patient- or family-facing feature can't be turned on in production until each gate is recorded:

Gate	Evidence
G-AI-1	Privacy office approval of the data flows, including the written agreement chain
G-AI-2	Regulatory memo on the non-device intended use
G-AI-3	Clinical lead sign-off of the evaluation report (critical safety suites with no failures observed)
G-AI-4	PIA addendum and TRA update
G-AI-5	Hazard log reviewed for the release
G-AI-6	Staff briefing done

Staff-only and aggregate features need G-AI-1 and G-AI-4. A platform administrator can't bypass the gates.

Controls you can use

Per-site switches for every feature, languages, budget and retention. See AI.
Kill switch: every AI entry point disappears within seconds across all sites.
Opt a visit out: staff can turn AI off for a patient's visit at the patient's or substitute decision-maker's request.
Safety review: flagged conversations are reviewed by a group outside the care team, with conflict rules and audit. See AI review.

Evaluation

Every prompt or model change runs evaluation suites (safety, scope, style, injection, family scope) before release. The showcase deploys only after the critical suites pass for the exact prompt and model versions.