We talk to about 30 practice managers a month who are evaluating AI scribes. The questions they ask are usually the wrong ones ("how accurate is it?"). The questions they should be asking are these five.
1. Where does my patient data physically reside, and who has access?
Most scribe vendors process audio in the US, even when serving Australian clinics. Some process it on-device, some don't. If the answer to 'where does it live' isn't 'in your country, in a region you specify, with access logs you can audit', that's a real risk for HIPAA / Australian Privacy Principle compliance.
2. What's your accuracy benchmark and is the methodology public?
Every vendor claims 95%+ accuracy. Some define accuracy as 'word error rate in the transcript', some as 'clinician-rated note quality on a 5-point scale'. These mean very different things. Ask for the methodology. If they won't share it, the number is marketing.
3. Does it integrate with my EHR — and if so, exactly which fields?
'EHR integration' is the most-abused term in clinical AI marketing. It often means 'you can copy and paste the output into Best Practice'. Real integration means structured fields populating without copy-paste. Ask which fields. Ask for screenshots of the workflow in their EHR connector. If they can't show them, the integration isn't real.
4. What's the contract exit term?
Many clinical-AI contracts have 12-month auto-renewing terms with 90-day cancellation windows. That's fine until it isn't. Ask: what's the notice period? Do I lose access to my historical data? What's the data-export format? A vendor confident in their product will give a month-to-month option.
5. What does the actual workflow look like beyond the scribe?
If the scribe is an island — just notes, nothing connected to care plans, screening, billing — you'll outgrow it in a year. Ask what the platform looks like after the scribe is solved. If the answer is 'nothing', be honest with yourself about whether you want to switch tools again in 12 months.
Care plans, screening, billing, patient app — connected by default.See the MedMETs platform