The risk worth watching isn't a machine taking your job. It's a trained clinician trusting an AI output they can't verify, in a tool they shouldn't be using, while their own judgment quietly erodes. All four of those are already documented.
Here is the version of the AI story that should hold a resident's attention. A clinician, mid-clinic, pastes a note into a chatbot to summarize it, signs what comes back without reading the source, and moves on. Nothing dramatic happens that day. The error shows up later, in a record nobody can reconcile, traced to an output nobody verified. That scenario is in print now, in journals you will cite.
The popular debate asks whether AI replaces physicians. That question is interesting and mostly speculative. The one with evidence behind it is narrower and more uncomfortable: practicing, licensed clinicians are using these tools without the skills to use them safely. The failures cluster into four patterns. None of them require a malfunctioning model. They require a competent doctor and a missing competency.
Knowing when to distrust an AI output is now part of clinical judgment. That skill is teachable, and residency is when you build it.
Accepting AI output as fact. Hallucinated transcripts, invented references, fabricated detail signed off without a check.
Pasting identifiable patient information into consumer tools with no business associate agreement and no control over what's retained.
Leaning on AI until the unassisted skill decays. Measured, not theoretical: detection rates drop when the tool is removed.
AI-drafted notes entering the legal chart unread. You sign it, you own it, and the original input is often already gone.
OpenAI's Whisper model powers Nabla, an ambient scribe used across more than 100 health systems and an estimated 7 million patient visits. Researchers found it inventing text that was never spoken: nonexistent medications, invented phrases, fabricated content. One University of Michigan reviewer found hallucinations in 8 of 10 transcriptions sampled. The structural problem is worse than the error rate: Nabla deletes the original audio, so there is no ground truth left to check the transcript against.
The same trap shows up in research. When researchers asked ChatGPT (GPT-3.5) to generate medical content with references, 47% of the citations were fabricated outright and only 7% were both real and accurate (Bhattacharyya et al., Cureus 2023). Newer models with live retrieval fabricate far less, so treat that figure as a warning about a behavior, not a current benchmark. The behavior is the point. A model will produce a confident, well-formatted citation that does not exist, and a tired clinician will paste it into a manuscript or a patient handout.
The competency here is plain. Verify every clinically meaningful AI output against a primary source before it leaves your hands. If the tool removes your ability to verify, that is a reason to distrust it, not a convenience.
Consumer chatbots are not covered entities. A free or personal-tier tool with no business associate agreement sits outside HIPAA's protections, and many consumer tiers may retain or train on what you submit. Pasting a note with a name, a date of birth, and a diagnosis into one of those tools is a disclosure of protected health information to a third party you have no agreement with.
I'm marking this one as reasoned rather than documented because the public record here is mostly enforcement guidance and vendor terms, not a single landmark podiatry breach I can point you to. The principle still holds. The fix is a habit: de-identify before you prompt, or use only tools your institution has vetted and signed a BAA for. When in doubt, assume the tool keeps what you give it.
This is the one with the cleanest data, and it should unsettle anyone planning a 30-year career. Budzyń and colleagues tracked 19 experienced endoscopists across four centers before and after routine AI assistance entered their colonoscopies (Lancet Gastroenterology & Hepatology, 2025). When the AI was switched off, their unassisted adenoma detection rate fell from 28.4% to 22.4%: a 6 percentage point absolute drop, a 20% relative decline, in clinicians who had each done over 2,000 procedures. The authors call it the first real-world documentation of a deskilling effect from clinical AI.
For a resident the lesson is sharper than for an attending. You are still building the baseline skill these tools assume you already have. Lean on AI before that baseline is solid and you may never form it. Read the film, the slide, the note, or the surgical plan yourself first. Then check it against the AI. The order matters.
An AI-drafted note is a draft until a clinician reviews it. The moment you sign it, it becomes the legal record, and you own every word, including the ones the model invented. The Whisper case makes the danger concrete: when the source input is deleted and the output goes unread, an error can sit in a chart for years with no way to reconstruct what actually happened in the room.
Disclosure and review are the competency. Read AI-generated documentation as critically as you'd read a co-resident's note before co-signing it. Know your institution's policy on documenting AI assistance. The signature is yours, so the accountability is too.
Four habits. Start them before they're tested by a real chart.
Every clinically meaningful AI output gets checked against a primary source. Every reference gets opened. No exceptions for time pressure, because time pressure is exactly when the error slips through.
De-identify before you prompt, or use only tools your institution has a BAA with. Treat any consumer chatbot as a public channel that keeps a copy.
Form your own read first, then compare it to the AI. Use the tool to catch what you missed, not to replace the looking. You're defending a skill you'll need for three decades.
Read AI-drafted notes as carefully as you'd read anything you co-sign. Learn your program's documentation policy. The model doesn't carry the liability. You do.
None of this is anti-AI. I use these tools daily and think the profession is behind on adopting them well. The point is that adoption without literacy is its own clinical risk, and the literacy is learnable. Build it now, while a mistake is a teaching moment instead of a malpractice question.
AI won't take your job. Using it without judgment can still cost a patient. That skill gap is yours to close, and residency is the time to do it.
High confidence (the cited facts): the Whisper/Nabla findings, the ChatGPT fabricated-reference figures, and the Budzyń deskilling numbers. All drawn from the named sources below and reported consistently across coverage.
Lower confidence (flagged in text): failure modes 2 and 4 are reasoned from HIPAA principles and documentation norms rather than a single landmark case. The reasoning is sound; the citation trail is thinner. Verify your own institution's AI and PHI policy before acting on either.