A certain danger lurks there

December 1, 2024 2:08 AM

It doesn’t seem far-fetched that Medicare or private insurance companies might eventually turn to AI therapy tools as an even cheaper way of ostensibly expanding access. Woebot’s website cites studies that show early intervention via outpatient care can lead to a reduction in mental health emergency room visits and inpatient hospitalizations. But the concern isn’t coming from a place of compassion for the seriously ill; rather, the point is to imply that widespread adoption of Woebot could yield “potential healthcare cost savings of up to $1,377 per patient, per year.” Just as mental health awareness campaigns eventually became a way for governments to justify prioritizing cheaper primary care interventions over crisis care, AI therapy may be the next step in a long tradition of cutting back care for the people who need it the most. from The Therapist in the Machine [The Baffler; ungated]

Related, and horrible: A 14-Year-Old Boy Killed Himself to Get Closer to a Chatbot. He Thought They Were In Love. [WSJ; ungated] [CW: suicide]

posted by chavenet (24 comments total)

What's really depressing about the now is that you know that if there's a new technology, it will either be used to screw people out of the resources they need to survive or, if it's any good, it will be so expensive that it's for the rich only. There's so seldom any sense of "hooray, this new development will make human life better for everyone".
posted by Frowner at 5:12 AM on December 1 [29 favorites]

There's so seldom any sense of "hooray, this new development will make human life better for everyone".

oh but it's so very often sold as that
posted by lalochezia at 5:35 AM on December 1 [12 favorites]

It doesn’t seem far-fetched that Medicare...might eventually turn to AI therapy tools as an even cheaper way of ostensibly expanding access.

The issue with Medicare and access to mental health therapy tends to lie firmly in the laps of the providers, and not with Medicare. Finding a mental health provider who accepts Medicare is a near sisyphean task, as the vast majority of mental health providers who take any insurance at all do not accept Medicare.

If Medicare turns to AI therapy as a covered option, human therapists have only themselves to blame.
posted by Thorzdad at 5:52 AM on December 1 [6 favorites]

I started to rush in and make a joke about the therapy bot saying "oh no maybe you should check your stove" but then I kept reading and somehow the things being described just kept getting dumber and more depressing. My feelings about therapy are complicated and mostly negative, but none of my complaints about it are addressed by replacing human beings with a chatbot. That feeling you sometimes get that your therapist has forgotten something crucial you said three sessions ago--or has forgotten entirely who you are? Welcome to having that feeling with every conversation! And how long until that ambient AI thing the article mentions--the one that listens in and suggests billing codes for the therapist to extract the maximum from insurance--blows up with privacy and hallucinatory issues, assigning you a diagnosis you do not have, but is now part of your medical record?

One can only hope the things are so dumb that nobody uses them. Please? One can hope that, right?
posted by mittens at 6:32 AM on December 1 [10 favorites]

Thorzdad, do you enjoy working for free? Because that is essentially what Medicare is expecting of “human therapists.” Between extremely low reimbursement rates, time-consuming, onerous paperwork that drops the hourly reimbursement rate even lower, delayed payments, missing payments, clawbacks, etc., it’s often the case that therapists are essentially working for free when they accept Medicare.

Since you’re so keen to assign blame, consider that Medicare’s entire business model is about maximizing subscribers/revenue and avoiding having to pay out for services. The point of this post is that Medicare is looking for shortcuts and cost-cutting measures, up to the point of replacing human providers with AI because it’s cheaper, despite there being no benefit to patients.

I’ll also add that, as a human therapist myself, your entitled attitude and refusal to see me as a person worthy of an income would make me really not interested in working with you.
posted by sleepingwithcats at 7:12 AM on December 1 [14 favorites]

Whether it is governments and militaries trapped in an arms race or neofuedal tech overlords looking for better slaves or even masters... the quest for AI (and the modest progress so far) is inherently about decieving people, exploiting people, and destroying people. That segments of our human community are collaborators with this evil project is disappointing but not surprising. There will always be capos. Will there be any resistance. We have already been seduced and subdued by one alien intelligence (money).
posted by No Climate - No Food, No Food - No Future. at 7:27 AM on December 1 [5 favorites]

Yep, still nothing has ever even sort of changed my mind that all "AI" should be burnt to the ground.
posted by outgrown_hobnail at 7:35 AM on December 1 [11 favorites]

Medicare’s entire business model is about maximizing subscribers

...wait, what?
posted by mittens at 7:38 AM on December 1 [7 favorites]

Porngraphy is not a true substitue for romance and intimacy, tv and movies are not a true substitute for having a social life, videogames are not a true substitute for physical activity and questing. Rockband and farmville are not susbstitutes for a career or vocation and this AI- placebo mind hacking is not a true substitute for having a therapist.

But we live in a world where "Line must go up" and all true and useful and healthy and natural things will be sacrificed to "Line must go up".

it's enough to drive one batty.
posted by No Climate - No Food, No Food - No Future. at 7:39 AM on December 1 [5 favorites]

@mittens, here’s an article about people being lured into Medicare Advantage plans only to be denied coverage: https://stateline.org/2024/10/21/1-million-patients-lose-coverage-as-insurers-hospitals-drop-medicare-advantage/#:~:text=Health%20systems%20have%20cited%20delayed,to%20drop%20Medicare%20Advantage%20plans.
posted by sleepingwithcats at 7:49 AM on December 1 [2 favorites]

One can only hope the things are so dumb that nobody uses them. Please? One can hope that, right?

I tried Woebot for a bit and came away unimpressed. Knowing it was a chatbot dampened my spirits more than if I had never tried it in the first place. "They couldn't even pay a living person to talk to me about my problems."

I hope they get the message that they really need people with real empathy, instead of Empathy Simulation Circuits, to talk to people, but I know the real lesson they'll get is "we should just not tell people it's just a chatbot." I'm certain they're going to try that, it's only a matter of time, the business model is just too juicy to them to soak up mental health monies while only having to power a server somewhere, to the extent that I'm already prepared for the backlash to the furor over the discovery, "OUR online chat help is staffed by real people, but everyone thinks they're bots!"
posted by JHarris at 8:16 AM on December 1 [6 favorites]

Well, doesn't this just fucking suck for anybody with mental problems.
posted by BlueHorse at 8:33 AM on December 1 [7 favorites]

sleepwithcats, please correct me if I am wrong, but I thought Medicare reimbursed psychotherapy at $75-110 an hour, depending on the state. That seems like a decent rate to me, and not working for free. I get that there's a lot of paperwork, and yet, all the other medical providers do the work. I am not trying to be snarky, but am honestly curious to understand the problem.
posted by birdsongster at 8:48 AM on December 1 [3 favorites]

I dunno, I thought the journaling chatbot they mentioned that acted like a supplement to therapy seemed reasonable. The founder seemed to have a healthy and realistic attitude towards AI as a supplement not a replacement. There's no substitute for human connection but having a chatbot to help me with my biweekly "homework" would be nice.

Generally you can completely dismiss anyone who suggests that AI can 1:1 replace human labor, but it can help humans scale up. I'd love it if my therapist could see a generated summary of my journals prior to our session as a way to prompt conversation and check in.

I don't want a chatbot therapist, but adding a chatbot to my overall care team would be OK with me.
posted by xthlc at 9:13 AM on December 1 [4 favorites]

Generally you can completely dismiss anyone who suggests that AI can 1:1 replace human labor, but it can help humans scale up.

With significant exceptions I think this is closer to the truth than the following:

the quest for AI (and the modest progress so far) is inherently about decieving people, exploiting people, and destroying people

Among those exceptions is using AI to replace therapists entirely. The problem is this (by the same Mefite):

we live in a world where "Line must go up"

You cannot fix a fundamentally broken system with this tool. US healthcare is broken. US mental healthcare is broken. US medical regulation is broken. US medical legislation is broken. And most fundamental of all: the US implementation of shareholder capitalism is completely, utterly broken.

And there are any number of industries where careful, mindful use of machine learning with task-specific fine-tuning could legitimately improve efficiency and overall quality of care/life/service for everyone it touches.

But because “line must go up” that just isn’t what we’re going to do.
posted by Ryvar at 9:30 AM on December 1 [4 favorites]

birdsongster, great question! Medicare reimbursement rates vary by state, in my state a 53-minute session is reimbursed on average around $80. While that sounds like a reasonable rate, it’s actually not realistic for most PP therapists to accept anything lower than $100 per session. For a full-time self-employed PP therapist, taxes suck up about 30% of your pay, plus you have to cover your own overhead, time off, and benefits - all of which sucks up approximately another 20% of your pay. Any time you spend outside of session doing paperwork is not compensated, so that $80 quickly dwindles to about $20-30. For providers in larger practice settings (like hospitals or group practices), that $80 also has to stretch to cover salaries for admin staff and overhead as well, so again the therapist is only receiving $20-30. Not to mention the claims that get denied or payments that are delayed or clawed back, which is happening with increasing frequency. Personally, I’m able to accept most insurances and offer lower-cost options to people paying out of pocket, but only because I have another full-time job with benefits (my therapy PP is a part-time side gig). There’s a misconception that therapists are making tons of money, but the unfortunate reality is that, for therapists doing full-time PP, they either have to refuse to take insurance so they can actually earn a living wage, or they have to see a minimum of 36 clients per week and never take time off, which is just not sustainable. There’s been other posts about this on the blue recently (I’m on my phone so can’t look them up/link to them), but the culprits here really are insurance companies (both public and private) that are trying to avoid paying for services while both providers and patients get screwed.
posted by sleepingwithcats at 9:37 AM on December 1 [9 favorites]

as a human therapist myself, your entitled attitude and refusal to see me as a person worthy of an income would make me really not interested in working with you.

No one sees therapists as unworthy of an income. But saying that you won’t take Medicare, or Tricare, or VA, is essentially saying you are refusing to treat certain populations for situations out of their control. If Medicare is failing to reimburse appropriately, the right way to handle that is to publicly shame them for it, or to be transparent about your pricing and the reasons why it’s necessary.
posted by corb at 10:40 AM on December 1 [5 favorites]

corb, I agree with you. My point is that, rather than blaming therapists for not wanting to settle for low wages (as thorzdad explicitly did in their post), it would be great if patients/people seeking care would instead act in solidarity with providers by also advocating for these systems to reimburse properly. I’m going to bow out, since I recognize metafilter is not for extended back and forth. I appreciate the discussion and hope people will keep in mind that these are systemic issues that require systemic solutions (and none of us should settle for AI chatbots and exploitation!).
posted by sleepingwithcats at 10:54 AM on December 1 [11 favorites]

(Medicare Advantage is kind of a egg-parasite, yes? Really not Medicare but allowed to pretend it is? Attacking it specifically might make it easier to fix Medicare. Or, more realistically, assign Advantage blame where it belongs?)
posted by clew at 12:37 PM on December 1 [3 favorites]

Correct, clew. Advantage plans act as the foot in the door for eventually privatizing Medicare. It’s corporate private-pay insurance (with all the drawbacks) on the public dime.

Medicare’s entire business model is about maximizing subscribers

Codswallop. Everyone gets Medicare. Period. There’s no maximizing subscribers involved. Are you a citizen? Yes? You get Medicare. If there’s any maximizing subscribers going on, it’s on the part of the private Advantage plans, which are optional add-ons to cover bits OG Medicare doesn’t (which doesn’t leave much, really)
posted by Thorzdad at 12:52 PM on December 1 [4 favorites]

Fair point, and I confess that I was conflating private and public insurance plans in previous posts. The problem with Medicare is that it is grossly underfunded (currently projected to run out by 2036) due to a number of factors, and, as is often the case when the U.S. has a problem, they’re looking for the cheapest shortcut they can find and AI is basically their dream come true. Unfortunately, the healthcare crisis is going to get far, far worse under the incoming U.S. administration. As a side note, I thought it was interesting that the article in the OP also gave examples of other systems like the NHS which simply don’t have enough human providers to meet the large demand/need for services.

I have something of a personal axe to grind about this, because I spent over a decade of my career - up until 2022 - working in community mental health settings that exclusively served patients on public health plans. The most I ever made was $50k (really $30k take-home pay) a year and I live in a high COL city. Meanwhile my friends in other sectors were making six figures with unlimited PTO, etc. It broke my heart to leave the public sector, but I literally could not afford to continue. The breaking point for me was when my administrative duties (for which I was not compensated) outweighed my time with my patients.

It still took me a long time to leave and I felt so guilty. I was constantly bombarded by the rhetoric that I was harming/abandoning low-income patients - by my bosses who were actually making good money, by the general public who go into an uproar whenever working people - especially those in the “helping professions” - advocate for higher pay and better working conditions, and thankfully, not really by my patients, although they were sad to see me go.

As corb wrote above, the solution here is not to brow-beat therapists into accepting low wages, but to publicly shame our public systems for failing to provide the services to which we are entitled as tax-payers and/or citizens. For those of us (myself included!) who want/hope to have human medical and mental health providers in the future, we need to organize and demand that our elected officials (whoever they may be) act to save and properly fund SS and Medicare. The full economic details of this are beyond my pay grade, but I found this article helpful/illuminating (and alarming): https://www.cato.org/blog/unsustainable-burdens-posed-so-called-medicare-social-security-trust-funds

Ok, bowing out for real this time. Thanks chavenet for a thought-provoking post!
posted by sleepingwithcats at 2:03 PM on December 1 [13 favorites]

Interesting article - though I kept waiting for it to get into privacy/security issues, and it never did. It's easy to imagine how a data breach/hacking could be devastating. Or all the unethical ways these tech companies might use the data.

That said, I agree that some of the applications - like the NHS using AI to speed up processing of new patients, for example, seem reasonable. I don't really trust private companies to do this, but AI as a tool for hospitals to supplement care, with strong privacy protections/secure data storage seems fine. A journaling app seems potentially good, provided they don't keep or have any access to your data.

But, from the article:

Viggósson is certainly right about one thing: speaking to an AI chatbot is essentially speaking to yourself.

A mentally unwell person talking to themselves, what could go wrong? The story (from the other article) of the 14 year-old who killed themselves in order to "be with" a chatbot reveals how easy it is for some people, especially sad people, to forget they are talking to a machine and not a human - which gives these chatbots far too much power for something that doesn't feel a sense of responsibility, guilt, shame, etc.
posted by coffeecat at 2:22 PM on December 1 [1 favorite]

Oooh, I just remembered the Science Fiction Precedent for all this: Frederik Pohl's Gateway (1977 Hugo winner), where a man who works as a shale oil miner wins in a lottery a medical insurance policy, which uplifts him from drone to human. I don't remember quite how we get from A to B, but he's able to make it to an alien space station where people engage in frightfully dangerous experiments to see where the ships parked there might go. He has a terrible abusive relationship with a woman, who he ends up abandoning (mostly by accident, IIRC) in a weird destination (a black hole?) and returns to become fabulously wealthy for having discovered it. Much of the book, or one of its sequels, takes the form of this guy (who is a super whiny douche) having sessions with an AI psychiatrist modeled after Freud. Not sure it did him that much good. I was a teenager when I read it, so I'm undoubtedly botching some details, here, but the point was that this guy was one of the luckiest people on Shitty Crowded Seventies Future Earth because he had access to medical care at all, let alone metal health care. Still think AI should be burned to the ground.
posted by outgrown_hobnail at 5:36 PM on December 1

(not to be that person but koch-funded cato is really not to be trusted here. their solution is providing less care so that the deficit can be cut, which is ridiculous and flies in the face of what medicare's supposed to be about.)
posted by mittens at 6:44 PM on December 1 [2 favorites]