You Can Argue With a Machine, Doctor

In the “Star Trek: The Original Series” episode “The Deadly Years,” Captain Kirk’s career stalls when radiation poisoning accelerates his aging, leaving him forgetful, irritable, and physically weakened. A competency hearing orders the Enterprise computer estimate his true age. When Spock asks the chief medic McCoy whether she agrees, the surgeon replies, “It’s a blasted machine, Spock! You can’t argue with a machine.”

The remark elicits nervous laughter, yet carries deeper weight: McCoy confronts a verdict that feels impersonal and definitive. While the system provides an answer, the deliberating panel must still weigh this against broader human realities the algorithm cannot encompass.

Contemporary medicine finds itself entangled similarly. Diagnostic software assists in diagnosis, prognosis, scan interpretation, risk stratification, documentation, scheduling, and even insurance adjudication. Such systems excel at speed and pattern recognition. Employed judiciously, artificial intelligence (AI) detects anomalies missed by exhausted clinicians, integrates fragmented data, and illuminates broader clinical insights.

So the real issue is not humans versus machines. The real issue is thoughtful human judgment versus handing over judgment without thinking.

A machine can identify a pattern, but it does not inherit the doctor’s moral duty. It does not truly “know” the patient as a person. It does not feel doubt, loyalty, regret, courage, or responsibility. It cannot sit with a family after harm has occurred and say, “This was my fault” or “This was unforeseeable.” Even if its advice is strong from a heuristic standpoint, the question “What should we do?” remains a human one.

“The Deadly Years” episode also warns about automation bias, our tendency to treat a computer’s result as more trustworthy than it deserves. Kirk is clearly impaired, so requesting a competency hearing is reasonable. But the computer’s guess at his biological age is not equivalent to a thorough, careful assessment of his abilities. Doctors therefore have a duty to challenge AI, not merely because it is AI, but because knee‑jerk acceptance is no more sensible than automatic distrust.

A doctor’s job is to push back when a system’s recommendation conflicts with the patient’s best interests, is based on missing or flawed data, is applied beyond its tested scope, obscures its uncertainty, perpetuates unfairness, or cannot be meaningfully questioned or checked. Doctors also need to be skeptical when a machine’s “authority” is used to hide an organization’s true goals, such as fewer admissions, shorter visits, cheaper drugs, fewer staff, or more insurance denials.

So, how can doctors know when to resist? They can ask themselves a few questions. Does this recommendation actually fit this patient? What important information was omitted or inaccurately captured? Is this tool being used for the purpose it was designed and tested for? Can I explain this result to the patient in plain language? Is there a real way to appeal or override this decision? If we follow this advice, who gains and who bears the risk if it’s wrong? Above all: Would I be willing to stand up and defend this choice in my own words, without saying, “The model told me to”? In the age of AI, that may become medicine’s version of Flip Wilson’s “The devil made me do it.”

Current guidance holds that AI can help shape a choice, but it must never serve as an excuse. The American Medical Association deliberately uses the term “augmented intelligence” to underscore that these tools should support and strengthen human thinking rather than replace it. Its principles for AI development, deployment, and use also emphasize oversight, transparency, disclosure, equity, and accountability.

The World Health Organization’s guidance on the ethics and governance of AI for health also places similar principles at the center of how these technologies should be designed and used. The National Institute of Standards and Technology’s AI Risk Management Framework, although voluntary and not specific to medicine, calls for clearly defined human roles, ongoing monitoring, and mechanisms to contest harmful AI outcomes.

These are not merely technical details. They are the basic safeguards that enable physicians to question, challenge, and resist the misuse of AI.

But “The Deadly Years” complicates matters beyond simply following a set of guidelines. McCoy’s intense loyalty to Kirk is moving, yet loyalty can slide into psychological denial. Kirk is impaired. His anger at the hearing does not restore his competence. Doctors must resist machines when machines are wrong, but they must also resist pressure from patients, coworkers, bosses, and even their own feelings when those loyalties skew their judgment. Sometimes the algorithm may be right, and the doctor may be the one who is biased. Human oversight is not inherently kind or fair.

McCoy’s outburst, then, should not be taken as simple hatred of machines. It should be heard as a demand that machines remain under human control. We must be able to question the data they use, test their outputs, show where they fall short, and say no to their advice when necessary. Just as important, patients must have a way to challenge decisions made about them when these tools are used.

“You can’t argue with a machine” is funny because it sounds like giving up. In today’s medicine, it should be a red flag. Doctors do not need to treat AI as a foe, but they need to stop it from becoming a source of authority without responsibility, a ruling with no appeal, or a shield that lets people hide from owning their choices.

The machine may perform the calculations, but the doctor still has to make the ultimate decision. When a patient’s life, dignity, or personal story is reduced to a single line of output, the doctor’s most enduring duty remains unchanged: to stand beside the patient and speak up.

This piece was adapted from the author’s book, “From Starship to Sickbed: Star Trek and the Meaning of Medicine.”

Source link