Using medical AI as ‘autopilot’ risks deskilling of clinicians

Researchers at University College London and Moorfields Eye Hospital, London, and aviation experts at the Flight Safety Department of Lufthansa issued a press release detailing a collaborative study¹ between the two groups.

Their Perspective article, published in npj Digital Medicine, warns that treating medical artificial intelligence (AI) as an "autopilot" rather than a "digital copilot" risks a dangerous "deskilling" of healthcare professionals.

“As AI becomes increasingly integrated into healthcare services, there are important lessons that the medical profession can learn from the aviation industry, which faced widespread loss of human skills after the adoption of autopilot,” they explained in the press release.

The result of automation

The authors’ core concern is the "automation paradox,” ie, as systems become more automated, human operators lose the skills and situational awareness needed to intervene when those systems fail. The authors point to aviation’s "children of the magenta line," a generation of pilots who became so dependent on automated navigation that they struggled to fly manually. They argue that medicine is at a similar crossroads, where over-reliance on AI could erode clinical judgment.

While medicine has already adopted safety tools from aviation, such as surgical checklists and incident reporting, their article focuses specifically on how to future-proof the workforce as AI becomes embedded in daily workflows.

Key recommendations

The authors provided the following five specific recommendations that are based on flight safety.

Benchmark clinicians and monitor unaided performance

Institutions should assess real-world clinician performance without AI assistance and set minimum unaided practice requirements after AI deployment, just as pilots have to maintain manual flying skills during routine flights with ongoing monitoring to detect overreliance.

Prioritize independent reasoning in early training

For younger clinicians trained in AI-rich environments, the risk shifts from deskilling to "never skilling" or "mis-skilling." Evidence suggests learners develop shallower knowledge with AI tools than through self-directed learning. Early training should build independent reasoning before automation is introduced, allowing AI to scaffold rather than substitute skill development.

Ensure clinicians understand AI limitations

Medical schools should teach AI literacy and technical competence in using AI tools. These skills should then be maintained and enhanced through professional development so clinicians are equipped to identify AI bias and other shortcomings.

Introduce scenario-based simulation training

Mandatory simulator training that recreates AI failure scenarios should be adopted, similar to aviation practice. This should extend beyond traditional surgical simulation to include development of dedicated simulation environments for non-surgical, end-to-end clinical workflows where AI is or might be used. In addition, routine AI settings, unannounced "surprise breaks" from AI can assess a clinician’s readiness to operate safely without automation.

Cultivate operational understanding

Clinicians should have a fundamental grasp of how an AI tool arrives at a decision and know when to override it. This mirrors aviation's "golden rule": understand the automated system at all times. When that understanding is lost, reduce automation step by step until situational awareness is restored.”

A "Co-Intelligent" Partnership

The authors conclude that the goal is to create a "co-intelligent" partnership. This model combines the algorithmic speed and pattern recognition of AI with the contextual reasoning and moral accountability of a human clinician. This approach aligns with patient preferences, as surveys consistently show that patients want clinicians to lead decision-making, with AI acting as a supportive tool.

The full recommendations can be accessed in the npj Digital Medicine publication.

Reference

1. Ong AY, Merle DA, Pollreisz A, et al. Flight rules for clinical AI: lessons from aviation for human-AI collaboration in medicine. npj Digit Med. 2026;9:201. https://doi.org/10.1038/s41746-026-02410-1