Artificial intelligence is being deployed to reduce the immense administrative burden that contributes to burnout for nearly half of all physicians. Tools that automate documentation, surface clinical evidence, and triage diagnostic images promise to free up time for patient care. But adoption is outpacing validation, and the rush to implement AI creates significant risks. These systems introduce new failure modes that can impact patient safety, data privacy, and clinical integrity.
At ZEKAI, we review AI tools independently to help working professionals separate genuine utility from hype. For clinicians exploring AI, understanding the potential pitfalls is the first step toward responsible adoption. This guide details seven of the most critical mistakes we see practices making and how to avoid them. The goal isn’t to discourage the use of AI, but to equip you to use it safely and effectively, starting with the insights from our work in the AI for healthcare professionals hub.
The short answer
The key problems with AI in healthcare involve blindly trusting AI-generated notes, which can “hallucinate” incorrect clinical facts; ignoring patient consent requirements for AI scribes, which has led to class-action lawsuits; using algorithms with inherent racial or gender bias; and misunderstanding how “free” tools use patient data. Avoiding these mistakes requires rigorous verification of AI outputs and a clear understanding of data governance.
Mistake 1: Blindly Trusting AI-Generated Notes (The Hallucination Problem)
The most urgent problem with generative AI in medicine is its capacity to “hallucinate”—producing fluent, confident, but entirely fabricated information. An AI scribe might invent a symptom the patient never reported, cite a non-existent lab value, or silently omit a critical detail from a treatment plan.
Because the output reads as plausible, these errors are easy to miss. Yet even low hallucination rates of 1-3% have profound safety implications when scaled across thousands of patient encounters. A May 2026 report from Ontario’s Auditor General reviewed 20 government-approved AI scribe vendors and found that every single one produced inaccuracies, including hallucinations, during testing.
Source: nature.com
The approximate time savings AI scribes can offer on documentation, a major driver of adoption despite accuracy risks.
This is not a theoretical risk. It’s a structural weakness of current large language models. The AI doesn’t “know” it’s wrong; it simply generates the most statistically probable text. This makes physician oversight not just a best practice, but an essential safeguard.
How to avoid this mistake:
- Verify every note. Treat every AI-generated summary as a draft that requires your explicit review and sign-off before it enters the EHR.
- Use structured prompts. When using prompt-based tools, command the AI to only use the information provided and to flag any ambiguities rather than filling them in.
- Report errors. Establish a workflow for clinicians to report inaccuracies to the vendor and your internal IT team. This helps track performance and pressure vendors to improve their models.
Summarize the following clinical encounter transcript into a SOAP note. Use ONLY the information present in the transcript. Do not infer or add any information that is not explicitly stated. If a standard SOAP section (Subjective, Objective, Assessment, Plan) has no information, write "Not discussed." If any part of the transcript is ambiguous, quote the ambiguous phrase directly and add "[Clinician to clarify]".
Mistake 2: Ignoring Patient Consent & Data Privacy (The Lawsuit Risk)
Patient privacy is a foundational ethical concern with AI, especially for “ambient” scribes that record entire clinical conversations. A series of class-action lawsuits filed in 2025 and 2026 against major health systems like Sutter Health, MemorialCare, and Sharp HealthCare demonstrate the severe legal and financial risks of deploying these tools without explicit patient consent.
These lawsuits allege that using AI scribes to record and transmit conversations to a third-party vendor without the patient’s knowledge and consent violates state and federal wiretapping laws, including the California Invasion of Privacy Act (CIPA). CIPA requires all-party consent for confidential conversations and carries statutory damages of up to $5,000 per violation. The plaintiffs allege they were unaware their sensitive medical conversations were being recorded and processed by an outside AI company.
How to avoid this mistake:
- Assume all-party consent is required. Even if your state is not a “two-party consent” state, the sensitive nature of medical conversations makes explicit consent a critical best practice. HIPAA compliance alone does not override these recording laws.
- Implement a multi-step consent process. Use a combination of clear signage in exam rooms, information in pre-visit materials, and a required verbal confirmation from the physician at the start of the encounter.
- Vet your vendor’s BAA. Ensure your Business Associate Agreement with the AI vendor clearly outlines data handling, security protocols, encryption standards, and data residency (ideally U.S.-only).
Mistake 3: Adopting Tools with Hidden Algorithmic Bias
AI models learn from the data they are trained on, and if that data reflects existing societal and historical biases, the AI will perpetuate and even amplify them. This is one of the most insidious problems with AI in healthcare, as it can systematically worsen health disparities for already marginalized populations.
The evidence for this is well-documented:
- Risk Scoring: A widely used commercial algorithm was found to underestimate the health needs of Black patients because it used healthcare cost as a proxy for illness, and less money is historically spent on Black patients with the same level of sickness.
- Dermatology: AI diagnostic tools trained primarily on data from lighter-skinned patients perform poorly when identifying skin cancers in patients with darker skin tones.
- Cardiovascular Risk: A major cardiovascular risk algorithm proved less accurate for African American patients because the training data was not sufficiently representative.
- Mental Health: Some AI models have been shown to make inferior treatment recommendations for Black mental health patients and perform worse at suicide prediction for Black patients versus White patients.
How to avoid this mistake:
- Question vendors about training data. Ask for documentation on the diversity and representativeness of the datasets used to train their models. Be wary of vendors who cannot provide this information.
- Monitor outcomes by demographic. If you deploy an AI tool, track its performance and accuracy across different patient populations in your own practice to spot potential disparities.
- Favor “explainable AI” (XAI). Choose tools that can provide some insight into how they reached a conclusion, rather than operating as an uninterpretable “black box.”
Mistake 4: Misunderstanding “Free” Tiers and Data Rights
Many AI tools market a “free” tier to encourage adoption, but these offerings often come with significant limitations or hidden costs. In healthcare, the cost is often paid with data. It is crucial to understand the terms of service for any free tool, as some may use de-identified patient data to train their models. While this may be HIPAA-compliant, it’s a governance decision your practice must make consciously.
Other “free” offerings are simply time-limited or feature-capped trials that convert to paid plans. For example, some popular AI scribes offer a short trial before requiring a monthly subscription.
Freed AI
Excellent for solo clinicians, but its pricing tiers separate key features.
Free plan; paid from $39/mo
Freed AI is a popular ambient scribe designed for small practices. Its strength is its simplicity and transparent security posture. However, as of September 2026, it does not offer a permanent free tier, only a 7-day trial. Its pricing tiers can be complex; unlimited notes require the $79/mo “Core” plan, while essential features like ICD-10 coding and EHR integration require the $104-$119/mo “Premier” plan. This model is less suitable for larger teams or those needing all features at a lower price point.
- Price from
- As of Sep 2026: $39-$119/mo
- Free tier
- 7-day free trial, no permanent free tier
How to avoid this mistake:
- Read the Terms of Service and BAA. Specifically look for clauses on data use for AI training.
- Distinguish “free trial” from “free tier.” A trial is for evaluation; a free tier is a permanent, if limited, plan. Understand which you are signing up for.
- Never upload PHI to a consumer tool. Do not use general-purpose chatbots like ChatGPT or Gemini for any task involving patient information. Use only HIPAA-compliant tools with a signed BAA.
Mistake 5: Allowing Clinical Skills to Atrophy (The De-Skilling Risk)
As AI takes over more cognitive tasks, there is a real concern that clinicians may become over-reliant on the technology, leading to a decline in critical thinking and diagnostic skills. If an AI consistently provides differential diagnoses or interprets images, the physician’s ability to perform those tasks from first principles may weaken over time.
This risk is greatest for trainees and early-career physicians who are still developing their clinical judgment. While AI can be a powerful educational tool, it should not become a replacement for foundational knowledge and reasoning. The goal is “augmented intelligence,” where AI assists the human expert, not autonomous AI that replaces them.
How to avoid this mistake:
- Use AI for confirmation, not origination. Formulate your own assessment and plan first, then use the AI as a second opinion or to check for things you might have missed.
- Frame AI use as a training tool. For residents and medical students, structure AI usage as a learning exercise. Have them compare their own notes and diagnoses to the AI’s output and critique the differences.
- Protect core diagnostic reasoning. Delegate administrative tasks to AI first (e.g., documentation, patient letters), and be more cautious about automating core clinical reasoning and decision-making.
Mistake 6: Adopting AI Without a Clear Workflow
Deploying an AI tool without integrating it thoughtfully into existing clinical workflows is a recipe for failure. Resistance from clinicians who find the tool cumbersome or disruptive can kill an implementation. If an AI scribe requires more clicks to edit its output than it would take to write a note from scratch, it will be abandoned.
The administrative burden on physicians is immense, with some spending nearly half their workday on EHRs and paperwork. AI’s primary promise is to reduce this load. If the implementation adds new burdens, it has failed.
| Risk Category | AI Scribes (e.g., Freed AI) | Diagnostic AI (e.g., Imaging) | Decision Support (e.g., CDS) |
|---|---|---|---|
| Primary Risk | Hallucination & Consent Violation | Inherent Bias & Over-reliance | Inaccurate Recommendations |
| Failure Mode | An incorrect fact enters the legal medical record. | A specific demographic receives systematically poorer diagnoses. | AI suggests an outdated or inappropriate treatment path. |
| Mitigation | Mandatory physician review of every note; explicit patient consent. | Vendor data transparency; internal performance monitoring by demographic. | Use as a reference tool, not a directive; verify against current guidelines. |
Swipe the table sideways →
How to avoid this mistake:
- Pilot the tool with a small group. Before a full rollout, test the tool with a few champion physicians to identify workflow friction points.
- Map the entire process. Go beyond the vendor’s demo. Document every step: launching the tool, gaining consent, using it during the visit, editing the output, signing the note, and billing.
- Measure the right metric. The goal isn’t just a “good note.” The goal is to reduce the total time and cognitive load spent on documentation. Measure “pajama time”—the after-hours charting that contributes heavily to burnout.
Mistake 7: Letting the Tech Overshadow the Patient
The ultimate goal of clinical AI is to give physicians more face-to-face time with their patients by automating administrative work. However, a poorly implemented tool can have the opposite effect, becoming a distraction that dehumanizes the clinical encounter.
If the physician is focused on managing a clunky app or the patient is made uncomfortable by a recording device, the technology has failed, even if it produces a perfect note. Patient perceptions of AI are mixed; while some appreciate the increased attention from their doctor, others report self-censoring or feeling anxious about being recorded.
How to avoid this mistake:
- Explain the “why” to the patient. Frame the AI scribe as a tool that allows you to give them your full attention instead of typing into a computer.
- Make it unobtrusive. Use technology that is as invisible as possible. The ideal ambient scribe is one that the physician can activate and then forget about for the rest of the visit.
- Maintain human connection. Ultimately, patients need things only humans can provide. AI can handle the data, but empathy, contextual judgment, and the therapeutic relationship remain the physician’s domain.
AI holds significant promise for healthcare, but it is not a plug-and-play solution. By anticipating these common mistakes, clinicians and health systems can navigate the risks and adopt these powerful new technologies responsibly, ensuring they serve both physicians and patients. To continue exploring specific tools and workflows, visit our AI for healthcare professionals hub.
What are the main ethical concerns with AI in healthcare?
The main ethical concerns are patient privacy and consent, algorithmic bias that can worsen health disparities, and accountability when an AI makes a mistake. Ensuring data security, training AI on diverse datasets, and maintaining human oversight are key strategies to address these issues.
Can AI be wrong in diagnosis?
Yes. While AI has shown high accuracy in specific, narrow tasks like identifying certain findings on images, it can be wrong. Errors can stem from biased training data, “hallucinations” where the model invents facts, or a lack of real-world context that a human physician provides. AI should be used to augment, not replace, clinical judgment.
What is an AI hallucination?
An AI hallucination is when a generative AI model produces information that is factually incorrect, fabricated, or nonsensical, but presents it with confidence as if it were accurate. In a clinical note, this could be an invented symptom or a wrong medication dosage, posing a significant patient safety risk.
What are the data privacy risks of AI in healthcare?
AI systems often require access to vast amounts of sensitive patient data, creating risks of data breaches, unauthorized access, or misuse. Using third-party AI vendors also introduces complexities around data ownership, security standards, and ensuring strict HIPAA compliance, including a signed Business Associate Agreement (BAA).
What is the CIPA lawsuit about?
The CIPA (California Invasion of Privacy Act) lawsuits against health systems like Sutter Health allege that they used ambient AI scribes to record confidential doctor-patient conversations without first getting consent from all parties. This is a critical issue because CIPA and similar laws in other states require all-party consent for recording, and HIPAA compliance does not override this.
How does algorithmic bias affect healthcare?
Algorithmic bias occurs when an AI model produces systematically prejudiced results against certain groups. In healthcare, this happens when AI is trained on data that lacks diversity (e.g., primarily from one race or gender). This can lead to less accurate diagnoses, incorrect risk assessments, and worse health outcomes for underrepresented populations.
Do doctors have to verify AI-generated notes?
Yes, absolutely. Given the documented risk of AI hallucinations and other inaccuracies, every AI-generated note must be treated as a draft. The standard of care requires the clinician to review, correct, and personally sign off on the final note, taking full responsibility for its content.
Paid remote work · AI for Doctors & Medical
Paid remote AI jobs in AI for Doctors & Medical
AI companies pay experts by the hour to test and improve AI. Remote and flexible. These roles are open now.
1,538 open roles · updated today
Get new roles like these by email
One short email each morning with new roles open in your country. Free. Unsubscribe any time.
Where to go next
Three routes, picked for what you just read.
Sources (41)
- ncbi.nlm.nih.gov · PMC9905201 · ncbi.nlm.nih.gov
- pneumon.org · Risks of artificial intelligence AI in medicine · pneumon.org
- cdc.gov · cdc.gov
- thehastingscenter.org · Artificial intelligence · thehastingscenter.org
- jmir.org · E59863 · jmir.org
- ama-assn.org · Burnout way down pajama time stands still · ama-assn.org
- authenticx.com · Ai bias in healthcare · authenticx.com
- prs.global · Healthcare AI mistakes · prs.global
- paubox.com · Real world examples of healthcare AI bias · paubox.com
- freed.ai · Physician burnout statistics · freed.ai
- accuray.com · Overcoming AI bias understanding identifying and mitigating
- koreascience.or.kr · JAKO202404751336264.page · koreascience.or.kr
- hitrustalliance.net · Ethics of AI in healthcare and medicine · hitrustalliance.net
- europarl.europa.eu · STOA STU(2022)729512 · europarl.europa.eu
- ncbi.nlm.nih.gov · PMC11202812 · ncbi.nlm.nih.gov
- weforum.org · Ai global health equity data gap · weforum.org
- hipaajournal.com · Lawsuit alleges AI platform illegally recorded patient clinician conversations · hipaajournal.com
- feldesman.com · Health care providers face lawsuit over use of AI note taking platform · feldesman.com
- aicerts.io · Ambient transcription privacy lawsuit sutter health · aicerts.io
- heidi.health · Physician burnout · heidi.health
- mayoclinichealthsystem.org · Can you trust AI for health advice · mayoclinichealthsystem.org
- kevinmd.com · Administrative burden is driving severe physician burnout · kevinmd.com
- executiveeducation.bryant.edu · Pros and cons of AI in healthcare · executiveeducation.bryant.edu
- evidence.care · Administrative burden on physicians · evidence.care
- twofold.com · Freed AI review · twofold.com
- remediesoflife.com · The pros and cons of AI in healthcare · remediesoflife.com
- futurism.com · Ai medical scribes hallucinating · futurism.com
- news.unchealthcare.org · Confronting the mirror reflecting on our biases through AI in health care · news.unchealthcare.org
- sps.columbia.edu · Ambient AI medical scribes efficiency gains burnout uncertainty and
- twofold.com · Freed AI for therapists · twofold.com
- kff.org · The growing use of artificial intelligence in health care and
- medschool.duke.edu · Hidden risks asking AI health advice · medschool.duke.edu
- ncbi.nlm.nih.gov · PMC11542617 · ncbi.nlm.nih.gov
- ncbi.nlm.nih.gov · PMC11508603 · ncbi.nlm.nih.gov
- healio.com · Ai scribes may have profound impact on patient care · healio.com
- vitea.com · Ai hallucinations in healthcare · vitea.com
- dopequickreads.com · Ai privacy doctor visit recordings spark lawsuit in sd county · dopequickreads.com
- intuitionlabs.ai · Suki vs nuance dax vs abridge vs freed · intuitionlabs.ai
- freed.ai · Multilingual AI scribe · freed.ai
- freed.ai · Medical dictation vs AI scribes · freed.ai
- fisherphillips.com · New class action targets healthcare AI recordings · fisherphillips.com
See Zekai first in Google
The weekly AI briefing for your profession
One weekly email: the AI changes that actually affect your profession — tools, deals, and what to do about them.




