AI Has A Darker Side In Healthcare, Expert Says

TL;DR

Before you orderOffer from Amazon

Get health and wellness essentials delivered free with Prime

• Fast, free delivery on millions of items
• Prime Video, Amazon Music and more included
• Member-only deals all year

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

Researchers at the Association of Health Care Journalists Fall Summit warned that workplace AI can affect healthcare workers through monitoring, algorithmic management and task automation. They raised concerns about skill loss, patient relationships, data use and accountability, while noting that some states are considering or have passed laws addressing AI in care.

Researchers speaking at the Association of Health Care Journalists Fall Summit warned that artificial intelligence in healthcare could bring harms for workers as well as potential benefits, including loss of clinical skills, reduced control over work and strain on patient relationships. Mishal Khan of UC Berkeley’s Labor Center said risks can arise even if AI does not eliminate jobs, while a Stanford researcher described residents who struggled to interpret heart tests after losing access to AI tools during a hospital visit in Uganda.

Khan, a senior researcher in the Technology and Work Program at the University of California, Berkeley Labor Center, described three broad workplace uses of AI: electronic monitoring, algorithmic management and task automation. Monitoring can use cameras to track workers, while algorithmic systems may evaluate conversations and prompt workers on how to respond. Task automation shifts parts of a job to software.

She identified possible consequences including job loss, theft of workers’ output, deskilling, lower wages or wage theft, faster work pace and injuries, discrimination, reduced autonomy, and limits on labor organizing. In healthcare, Khan said, deskilling is a particular concern because it could affect workers’ scope of practice and ability to perform tasks when AI is unavailable.

Tina Hernandez-Boussard, a Stanford University professor of biomedical informatics, recounted a case in which medical residents sent to support a hospital cardiac ward in Uganda lost electricity and could not use AI to read electrocardiograms. The residents, she said, were not confident synthesizing the information to make recommendations. The account illustrates a concern about dependence on automated tools; it does not establish how common such difficulties are.

Clinical Skills and Worker Control

The concerns matter because healthcare decisions can depend on workers’ ability to assess information and act when digital systems fail or are unavailable. If routine tasks are increasingly delegated to AI, researchers warn that workers may have fewer opportunities to maintain specialized skills. The Uganda example raises that possibility, although the report does not provide evidence that it reflects a broad pattern across hospitals.

AI may also change how care is delivered and how workers are evaluated. Khan said automated patient handoffs could weaken relationships between staff and patients, while monitoring and algorithmic management may give employers new ways to collect data and direct work. These systems could affect privacy, autonomy and accountability, including who is responsible when an AI recommendation contributes to an error.

Khan cautioned that outcomes depend in part on workers’ influence over how the technology is introduced. Her assessment was that AI can benefit workers, but that without meaningful power in its deployment, harms may outweigh those benefits. That is a researcher’s warning, not a finding that every healthcare AI system produces those effects.

Three Ways AI Can Shape Work

Khan’s framework separates workplace AI by what it does: it can observe workers, help manage or direct their work, or take over particular tasks. The categories can overlap. For example, a system that listens to customer-service calls may both assess performance and recommend what a worker should say. Khan cited such coaching as an example of algorithmic management.

The report also described concern about ambient scribes, which record or process conversations to generate clinical notes. Khan said workers want to know what information these systems collect about them and whether it could later be used in decisions about their employment. The report did not specify the systems involved or establish how employers currently use that data.

Some government action is already in place or under consideration, according to Khan. She said Illinois and Vermont passed bills barring the replacement of mental-health workers with AI, and that a California law prohibits companies from advertising an AI chatbot as a health professional. She also described a California bill under consideration that would prevent workers from having to train their replacements, including an AI system.

“Even if they don’t lose a single job to AI, it doesn’t mean there are no harms to workers.”

— Mishal Khan, senior researcher at UC Berkeley’s Labor Center

Evidence and Liability Questions

The report does not quantify how often healthcare workers experience the harms Khan described, or how widely the cited systems are used across hospitals and clinics. The residents’ experience in Uganda is an example shared at the summit; further details about the hospital, the length of the power loss and the residents’ training were not provided.

It is also unclear what data ambient scribes collect about workers, who can access it, and whether it is used for employment decisions. Khan raised questions about liability if an AI recommendation contributes to a mistake, but the report did not describe a specific case or identify a settled answer. The scope and current status of the state measures also depend on the wording and implementation of the laws and proposed bill.

Policy and Workplace Decisions Ahead

The California proposal described by Khan remains under consideration, so its progress and final provisions are not established in the report. State laws and bills addressing AI in healthcare may offer one route for setting limits, but the article does not give implementation dates or explain how enforcement will work.

For healthcare employers and workers, the unresolved questions include how AI systems will be tested, what information they will collect, how staff can challenge automated decisions, and how clinical skills will be maintained if tools fail. Khan’s remarks call attention to those choices, but the report did not announce a new policy or a specific agreement by healthcare organizations. Further reporting would be needed to establish how prevalent the risks are and what protections are being adopted in practice.

Key Questions

What workplace AI risks did the researchers identify?

Khan described potential job loss, deskilling, lower wages, faster work pace, discrimination and reduced worker autonomy. She also raised concerns about monitoring, patient relationships and the use of data collected by workplace systems.

What happened to the medical residents in Uganda?

According to Hernandez-Boussard, residents supporting a hospital cardiac ward lost electricity and could not use AI to read EKGs. She said they lacked confidence synthesizing the information to make recommendations. The report gives no further details about the incident.

Does the report say AI is already replacing healthcare workers?

No. Khan discussed job replacement as a possible risk, and the report cited state measures aimed at limiting some forms of replacement. It does not establish how many healthcare workers, if any, have lost jobs to AI.

What laws or proposals did Khan cite?

Khan said Illinois and Vermont passed bills barring the replacement of mental-health workers with AI, and that California law bars companies from advertising an AI chatbot as a health professional. She also said a California bill concerning workers training AI replacements was under consideration.

Source: rss