-Despite his concerns, Hinton says he believes AI can greatly improve several fields, including healthcare.

The American Medical Association also sees that potential but says the guardrails needed to protect patients are not keeping pace with the technology.

At Ai4, I sat down with the AMA's CEO, Dr.

John Whyte.

I began by asking him how AI could help address healthcare workforce shortages.

(Dr.

John Whyte) I want to start off by saying AI is not going to replace doctors or nurses or pharmacists.

What I think they can do is actually help reduce that administrative burden, the amount of charting that nurses need to do, the amount of forms that I need to fill out as a doctor.

That frees up more time for health professionals to see patients, because we're not going to be able to churn out more doctors, nurses, and pharmacists in any short period of time, just given the amount of training.

That's where I think they're going to help in these workforce shortages.

-When you think about how AI is being used in healthcare, can you give our viewers some examples that may surprise them of how it's being used?

-Some exciting ways that AI is being used, it's really assisting doctors make decisions; so, in terms of checking for polyps or checking for tumors and mammograms, an AI tool that helps the radiologist or the GI doctor find more tumors.

That's what I think is really exciting.

-What keeps you up at night when you think about AI in healthcare and regulations that do not yet exist?

-Well, I don't get caught up, you know, losing sleep over that there's going to be robots taking over the healthcare system.

What I get frustrated by is the lack of trust in the healthcare system, the lack of trust in these institutions that we've had, the FDA, the CDC.

So where do patients go to get information?

We have to acknowledge that patients are using generative tools.

Before they used to use WebMD and Google.

Now they're putting into a chatbot, Hey, like, what condition do I have?

What do we know about the fidelity of that information?

How do I know it's credible?

That's what keeps me up at night, because I think there's potential harm there, too, as well as benefit.

-And there are documented examples of the harm that chatbots have caused when people are seeking medical treatment or mental health treatment.

Where do you start with regulating that?

-And the AMA has been very involved, particularly in mental health chatbots.

So let's be honest.

This is to the point, there's a shortage of mental health professionals.

So let's not think that patients aren't utilizing these tools.

They are.

But what's the guardrails?

What's the framework?

So if someone's talking about something that might give the idea that there's something called suicidal ideation, that I might harm myself, you got to escalate that up to a real health professional.

You need to make sure that it doesn't give them information as to how to harm themselves.

And there's been data and instances where that's been shown.

We need the industry, first of all, to step up and say, These are the guardrails.

We don't start with regulation.

We want industry to be good partners to actually help people get better care.

Maybe we start with anxiety.

Maybe there's other issues that we can think about, and let's evaluate them.

I do think there's a role for these tools, but we have to have the right framework.

And it's not there yet, Amber.

-Who are you talking to in the industry?

Who's doing it well, in your opinion?

-I'm going to be honest.

Right now, I think the tech industry is the tail wagging their dog.

And they're kind of saying where innovation is, and I'm saying, okay, as doctors, let's talk about what problems we're trying to solve and then let's utilize tech to help solve those problems.

That's not how it's being done.

What we need to do is to have physicians, have the medical community, and have patients working with tech from the beginning.

We want to protect patients.

This is about outcomes and patient safety as we're talking about health.

It's our most important commodity.

And what I often say to patients is, would you upload all your financial advice into some generative tool?

But patients upload their labs, their medical records.

They don't even take their name off of things.

You wouldn't do it with your financial information.

Your health is even more private.

There's no expectation of privacy on many of these tools.

How do I know they're not going to sell that information, market me products from that?

So we need some guardrails there as well.

-The AMA represents physicians across the country.

Does that mean that the AMA also supports a federal framework to regulate AI?

-What's a little tricky, Amber, is there's the practice of medicine, the practice of nursing pharmacy, which are regulated by states, and some of these tools are getting dangerously close to practicing medicine.

And AI should be augmented intelligence, helping support the doctor, not functioning autonomously, just making decisions.

States play a role there because it's the practice of medicine.

But the federal regulation also plays a role because most of these are tools or devices.

And if we have a new framework about these devices, the FDA could preempt them from being utilized if they don't meet certain evidentiary standards.

There is a company that has a pilot program in Utah where patients, if they participate in it, can ultimately have this tool that will renew prescriptions.

Right now, it's in a pilot.

But ultimately, without a physician ever looking at it, will just decide that whether or not you still need this statin or you need this antidepressant.

We don't support that at the AMA.

There's a lot of factors that go into renewing medications.

But here's also the challenge, Amber: What's the problem they're trying to solve?

Do patients not get their prescriptions renewed in a quick period of time?

They do.

Most data shows within two days.

Is it reducing cost or improving access?

That's not what this is doing.

So here's an interesting technology, but what's the problem that we're trying to solve?

Where's the physician in this?

-Dr.

Whyte did praise a Nevada law that took effect last year.

Assembly Bill 406 says counselors, psychologists, and social workers in Nevada's public schools can only use AI for administrative tasks like scheduling and managing records.