[Discussion] Medical Quackery

This is a follow up to the thread "Medical quackery in the US upsets me very, very much". The aim of this current thread is to take up the discussion on medical quackery (widening the scope since the US isn't the only country concerned), discuss news item pertaining to it and the potential responses to address it.
The definition of medical quackery is not up for debate and includes, among others, homeopathy, vaccine skepticism, naturopathy, crystal healing, psychic healing.

polypusher wrote:

I get you Reaper, but 'Better' is not more unproven (or worse yet proven ineffective) There's no good reason to throw garbage after garbage.

I understand that people get all squicked out when somebody challenges the scientific orthodoxy.

Reaper81 wrote:

But to say, ‘Hey, here’s a thing that, at best, works half the time and MAYBE six months from now you won’t have the same or worse symptoms,’ is kind of bullsh*t.

There's a term for what you describe there. It's called "practicing medicine".

It's what you get pretty much any time you go see a doctor. "Based on what we know about your inexact diagnosis and bearing in mind the limits of what we know about the treatment, here's a thing that stands the best chance out of all our options of improving your situation."

There are no magic cure-all pills outside of videogames.

Reaper81 wrote:

Edit: I am not going to do the work for you. There are serious ethical issues with how what data is gathered from whether a given RTC is funded or not.

Once you understand that, you’ll understand why a 50% success rate is actually hot garbage.

If you’re going to make a claim against the accepted standard of care, then the onus is upon you to provide a reliable source to back up your claim. Then we can continue the conversation.

-edit-

Reaper81 wrote:
polypusher wrote:

I get you Reaper, but 'Better' is not more unproven (or worse yet proven ineffective) There's no good reason to throw garbage after garbage.

I understand that people get all squicked out when somebody challenges the scientific orthodoxy.

But science is not static. When done properly, science builds upon previous work. And if there is robust evidence that contradicts previously accepted research, then science must evolve to incorporate the new findings.

I think that most science minded people are completely fascinated by new research that suggests a paradigm shift. That’s super exciting and invigorating.

[I wasn't really contributing with this post]

"works about half the time" is better than "is proven to not work, ever" What if we were talking about cancer treatment instead of PTSD. Would you still advocate prescribing nice spells in place of a 50% effective chemotherapy/radiation regimen?

RawkGWJ wrote:
Reaper81 wrote:

Edit: I am not going to do the work for you. There are serious ethical issues with how what data is gathered from whether a given RTC is funded or not.

Once you understand that, you’ll understand why a 50% success rate is actually hot garbage.

If you’re going to make a claim against the accepted standard of care, then the onus is upon you to provide a reliable source to back up your claim. Then we can continue the conversation.

-edit-

Reaper81 wrote:
polypusher wrote:

I get you Reaper, but 'Better' is not more unproven (or worse yet proven ineffective) There's no good reason to throw garbage after garbage.

I understand that people get all squicked out when somebody challenges the scientific orthodoxy.

But science is not static. When done properly, science builds upon previous work. And if there is robust evidence that contradicts previously accepted research, then science must evolve to incorporate the new findings.

I think that most science minded people are completely fascinated by new research that suggests a paradigm shift. That’s super exciting and invigorating.

Agreed. I may (or may not depending on time and personal commitment to this specific issue) dig up a bunch of books, articles, journal articles, and research that strengthen my strongly-worded claim.

thrawn82 wrote:

"works about half the time" is better than "is proven to not work, ever" What if we were talking about cancer treatment instead of PTSD. Would you still advocate prescribing nice spells in place of a 50% effective chemotherapy/radiation regimen?

I made no such claim. Ever.

Reaper81 wrote:
thrawn82 wrote:

"works about half the time" is better than "is proven to not work, ever" What if we were talking about cancer treatment instead of PTSD. Would you still advocate prescribing nice spells in place of a 50% effective chemotherapy/radiation regimen?

I made no such claim. Ever.

Didn't you just claim that alternative medicine treatments would be a reasonable replacement for the established treatment which has a 50% efficacy? I apologize if I misread your argument.

edit: I picked aromatherapy because it is ridiculous low hanging fruit, but IS a recommended alternative medicine treatment for cancer patients.

Yes, I'm seeing Reaper suggest CBT may not be as effective as claimed. The suggestions that he believes in snake oil as a result of that view are misplaced and unfair. Everyone cool down, please.

I apologize. On rereading I did misunderstand his statements. I was reacting to the original post indicating CBT being replaced with theraputic touch at the VA and thinking he was in support of that specifically. My error.

Certis wrote:

Yes, I'm seeing Reaper suggest CBT may not be as effective as claimed. The suggestions that he believes in snake oil as a result of that view are misplaced and unfair. Everyone cool down, please.

Reaper81 wrote:
polypusher wrote:

I get you Reaper, but 'Better' is not more unproven (or worse yet proven ineffective) There's no good reason to throw garbage after garbage.

I understand that people get all squicked out when somebody challenges the scientific orthodoxy.

With a lack of a definition of what 'challenges scientific orthodoxy' means, it looks fair to assume Reaper is indicating that adding a slew of treatments that experimentally show no efficacy beyond placebo or statistical noise is ok.

If that's not the case, some clarification would be nice.

polypusher wrote:
Certis wrote:

Yes, I'm seeing Reaper suggest CBT may not be as effective as claimed. The suggestions that he believes in snake oil as a result of that view are misplaced and unfair. Everyone cool down, please.

Reaper81 wrote:
polypusher wrote:

I get you Reaper, but 'Better' is not more unproven (or worse yet proven ineffective) There's no good reason to throw garbage after garbage.

I understand that people get all squicked out when somebody challenges the scientific orthodoxy.

With a lack of a definition of what 'challenges scientific orthodoxy' means, it looks fair to assume Reaper is indicating that adding a slew of treatments that experimentally show no efficacy beyond placebo or statistical noise is ok.

If that's not the case, some clarification would be nice.

Do you agree that researcher bias exists and can influence research results?

Reaper81 wrote:
polypusher wrote:
Certis wrote:

Yes, I'm seeing Reaper suggest CBT may not be as effective as claimed. The suggestions that he believes in snake oil as a result of that view are misplaced and unfair. Everyone cool down, please.

Reaper81 wrote:
polypusher wrote:

I get you Reaper, but 'Better' is not more unproven (or worse yet proven ineffective) There's no good reason to throw garbage after garbage.

I understand that people get all squicked out when somebody challenges the scientific orthodoxy.

With a lack of a definition of what 'challenges scientific orthodoxy' means, it looks fair to assume Reaper is indicating that adding a slew of treatments that experimentally show no efficacy beyond placebo or statistical noise is ok.

If that's not the case, some clarification would be nice.

Do you agree that researcher bias exists and can influence research results?

Acknowledging research bias is a core tenet of scientific orthodoxy.

Systemically? No. On a per study basis, something like that is possible (and has been demonstrated) This is why independently replicating the outcomes of studies is so important.

Edit: actually yeah more like Jonman's answer. The recognition of bias is why study design is so careful to reduce it to as small an impact as possible, which is why double-blind is the standard and why publishing all of your results, not just the positive ones are so important, to avoid statistic 'hacking' (p-hacking)

polypusher wrote:

Systemically? No. On a per study basis, something like that is possible (and has been demonstrated) This is why independently replicating the outcomes of studies is so important.

How often are randomized clinical trials replicated? And who funds them?

Seems like you're highly skeptical of 'the establishment'. That's not inherently a bad thing, just try to be equally skeptical of the power of energy healing, needle poking and other such treatments.

Veterans deserve the highest quality treatments and should neither be experimented on nor subjected to cheap, ineffective options.

At least Homeopathy isnt on the new VA list... micro-victory there in otherwise a major defeat to reason.

Reaper81 wrote:

And who funds them?

Big Psychotherapy?

Reaper81 wrote:
polypusher wrote:

Systemically? No. On a per study basis, something like that is possible (and has been demonstrated) This is why independently replicating the outcomes of studies is so important.

How often are randomized clinical trials replicated? And who funds them?

Well for an FDA approved medical treatment, at least twice: once each during the phase 2 and 3 human trails portion of FDA approval.

Funding varies with FDA approval, often it is the company seeking to develop the treatment who pays for the approval, but it may also be done through NIH grants or funding from organizations. My direct experience is limited to one stint as a technologist conducting a clinical trial.

The set up we worked under was typical (at least according to people I've talked to in clinical trials) and structured like this (without names): Biotech Company A contracted with a third party Medical Laboratory Company B (where I was employed) to perform the actual technologist work, and the FDA provided (brutally rigorous) supervision of the entire process (seriously, these FDA folks make Sheldon Cooper look like Phillip J Fry.) SO company A pays company B to do the lab work, but company B has no stake in the results. Whether to tests are successful or not company B and the hands doing the work (me) get paid the same either way. The FDA is there every step to asses the results and make sure all the data is recorded properly and all the proper paperwork is filled out, and aside from the making sure all the i's are dotted and t's crossed they don't have a stake in the result either (the fee associated with the FDA application is not dependent on the result of the trial either)

Obviously that's a bare bones description of a very long and involved process but I hope you get the idea.

OG_slinger wrote:
Reaper81 wrote:

And who funds them?

Big Psychotherapy?

I've done some reading on the webs. I found an article about treating PTSD with CBT, and two studies. I will link them below. I don't know who funded the studies or the article, but certainly no red flags went up for me as I looked them over. It appears that the medical consensus is that CBT is substantially effective for treating PTSD.

If anybody has reliable information which contradicts this, please share it with us. Otherwise lets consider this conversation to be over.

an article

a study

another study

I don't know anything about the data supporting CBT as a treatment for PTSD. I have served as a principal investigator on over 100 clinical trials, though. I've done research with funding from pharma, NIH, Department of Defense and private donors. I've also spent the past year as a medical director for a contract research organization so I was running clinical trials from the industry side. I have pretty good insight into how the research sausage is made.

Of all of the types of studies that I've been involved in, the ones with the greatest risk of scientific misconduct were the studies funded by private donors. It was just me and my team running the study and usually just me alone analyzing the data. It would have been easy for me to exclude data points that didn't get me to the conclusion I wanted or to just frankly falsify the data. I've published trials in peer reviewed journals where I was the only one who looked at the data and drew the conclusions.

The studies where there is virtually no chance of this are the industry sponsored trials. The amount of oversight, the number of people that are double and triple checking the data to make sure there is source to support it and that there are no shenanigans going on is unbelievable. We send monitors to each site to double check every piece of data to make sure it is entered correctly and is supported by the medical record. It was all then reviewed again by me and by our data team. We then randomly audit sites and then the FDA audits high volume sites. It is ridiculously redundant, nitpicky and frankly wasteful and expensive. Fraudulent, it is not. And most oncology studies are repeated to ensure that the data can be replicated or run in tandem internationally to make sure that the results are in agreement.

I have had single institution phase I studies also funded by pharma. The only pressure I ever received was to get off my butt and write the manuscript so that the negative data showing their drug didn't work could be published.

Now the system is far from perfect. There are academics who benefit from publishing data to support the work that they are expert in. It can be difficult to find funding to conduct research for treatments that cannot be easily monetized (an issue for non-conventional interventions). But it is not even close to my experience that funding from pharma taints research results.

polypusher wrote:

Seems like you're highly skeptical of 'the establishment'. That's not inherently a bad thing, just try to be equally skeptical of the power of energy healing, needle poking and other such treatments.

Veterans deserve the highest quality treatments and should neither be experimented on nor subjected to cheap, ineffective options.

At least Homeopathy isnt on the new VA list... micro-victory there in otherwise a major defeat to reason.

Part 1.

The problem with all of this is that present evidence-based practices for treatment of PTSD are pretty bad with high drop-out and failure rates, to include CBT.

Here's a great article from JAMA that goes into a non-CBT based treatment (mindfulness and present-centered therapy) RCT that references the research I'm referring to.

Most historic evidence around PTSD is gathered from VA clinical trials (whether medically focused or not) on adults (typically males) who have experienced a very specific type of trauma. The VA, of course, is a stakeholder with a vested interest in proving that what it does is effective. More recent research is beginning to focus on broader populations outside of the military.

The two primary models for PTSD treatment (usually meaning symptom reduction) are cognitive processing therapy and prolonged exposure. Both of which are essentially CBT-based methods. And this is the thing. They're bad. The VA's own research says this. Up to 50% of the time these models won't work and over a third of people drop out before completion? There's a strong argument that these models aren't even effective or if they are, are they effective for the reasons we think they are? Hence, my previous statement of CBT for PTSD being hot garbage.

It's okay to say that they're bad. And I say this as a recipient of prolonged exposure who found it to be highly effective in treating his trauma symptoms. But, anecdotally, I'm the only guy I know who found PE effective. That anecdote is not good and not a ringing endorsement when you get down to it.

Research integrity and model fidelity are super important. Not many people who claim to utilize core CBT practices actually follow the model correctly. And it's a pretty f*cking simple model when you drill right down to it. That's not the model's fault.

So why am I so down on CBT? Well, I'm crabby today, for one. For two, the volume of research done of CBT doesn't mean CBT is great or even good. It just means there's a ton of research on CBT. The NIH's own meta-analytic study called into question the quality of much of that research. There's a far cry between a research hospital measuring changes in hippocampus volume and an undergrad psych student polling his peers about they feel about therapy. Both could be called research but likely only one actually is.

I would argue that if we, as a research and practice community, produced a similar volume of similar quality research on the efficacy of salt licks in treatment of mood disorder symptoms, we'd probably get about the same results.

The research seems to indicate that, more often than not, CBT (and most therapy) is maybe better than nothing and most likely will not hurt you. So, is that snake oil or not?

It's not, clearly. Snake oil does nothing. CBT maybe sometimes does something. It helps if you, as a recipient, believe it will help.

Part 2.

I'm not a quack. I'm not a believer in crank bullsh*t. I strongly resent the implications that I am.

I am, in fact, a therapist working towards my clinical license. I've worked primarily with adolescents in my near eight year career and I'm proud to say that I've very nearly completed my national certification for trauma-focused cognitive behavioral therapy. (I have one final supervision call for those familiar with the model.)

I'm pretty hard on a model and career that I knowingly, willingly chose to learn and hitch my wagon to. I sincerely, truly want to help people. It's not about whether I think CBT works or not. It's about whether a person who's getting CBT thinks it will work for them.

Not everybody that gets in therapy, research, or medical professions do want to help. Some people are, in fact, just in it for the money. In a few months, I'll be hopefully pulling down a pretty good salary for somebody who, to paraphrase Bessel Van Der Kolk is a bullsh*t artist who somehow gets paid to say 'And then what happened?'

We've got psychopaths like everybody else. Psychology and research are great tools to oppress and hurt people if you think about it.

Some of the research that has been done has been intellectually and academically dishonest but passed off as legit. Doc and Thrawn have great posts about how rigorous RCTs should be (and most often are). Unfortunately, this is not always the case.

It's hard to sort out the real stuff from the quackery. But the thing is is that much of therapy is borderline quackery when rigorously scrutinized. A strong effect size should be strong. Not "strong relative to a placebo," if you all follow me.

And that's often simply the standard to which most therapy models are held. It's better than nothing! Is that acceptable? I guess so because it sure seems to help people.

Anyhooooo. Have a good night, y'all.

I mean, goddamn, friends. Reap and I have had our disagreements but he’s never been anything less than honorable. I feel like ya’ll are draggin’ him for no reason here. He’s a vet saying the VA’s implementation of CBT isn’t all it’s cracked up to be. I have family in the VA who agree with him.

Why the harsh?

Reaper, I get where you're coming from. My own personal analogy was when we were trying to figure out how to get my autistic son help. ABA seemed to be the "standard" therapy and had studies showing efficacy. But after we looked into it, our impression was "seriously, this is what you call success?" As I'm sure you all know, there is a ton of nonsense and crap that people try to sell desperate parents of autistic kids. We chose a less standard approach that still had some scientific evidence behind it. Kid is overall doing pretty well today, maybe we would have ended up in the same place if we had chosen ABA, who knows.

My area of research is comparatively easy. Tumor response rates and overall survival are very objective endpoints that are very reproducible. My sense is that its a lot easier to finesse data when the endpoints are patient reported outcomes. Again, I know nothing about CBT and PTSD but I can see how it might be easier to be more skeptical of the data. Sort of like some of the quality of life studies done in cancer patients in my field.

Also, sometimes the standard of care is absolutely crap. I remember sitting at a big research meeting when they were presenting data on a new pancreatic cancer study during the plenary session. They showed the survival curves and it appeared that the new treatment was really improving survival. Then they showed the scale and it was in days of survival. On average, the new treatment improved survival by 13 days compared to the old treatment. They then said this is the new standard of care. It was a travesty. I don't give a crap that your p value reached statistical significance. That's no standard of care.

First, I want to say that I respect the fact that Reaper is formally trained. Second, I’m simply a lay person who is passionate about science and critical thinking. I have no formal training. Also, I didn’t realize that he is a veteran.

Reaper,
Thank you for your service to our country.

As unreliable as therapy-based interventions for PTSD are, unfortunately, medications are even worse.

Wait, salt licks treat mood disorders?

Damn, I better go to the local feed store and stock up so I can finally beat this social anxiety disorder!

BTW, Cognitive Behavioral Therapy is supposed to help with Major Depressive Disorder, too, but like pretty much every treatment for MDD, results vary. Cognitive Behavioral Therapy did precisely nothing for me. Nor did any SSRI, SNRI, or TCA I tried. However, a cocktail of antipsychotic and NDRI has kept my mood somewhat stable for several years.

From my perspective, there is no freakin way that any drug trial, however rigorous, could possibly identify a medication that would work for me. It took me years, with multiple psychiatrists and literally dozens of combinations of medications to find one that sort of works some of the time.

I get that medication is a numbers game, and that research tries to identify the therapy that is most helpful for the largest number of people. There is so much variability, though, that I really wish we understood more about the mechanisms of more diseases and disorders so that we could create more targeted therapies.

Mechanisms are getting understood better, at least for some things. A number of people I know seem to have gotten a significant improvement in managing their mental health after getting a genetic profile done. Apparently that can show whole classes of psychoactive drugs that will work better, worse, or differently for individuals.

Pretty amazing stuff.

But even then, it's not a panacea. One close friend is struggling really really hard right now with hir mental health, despite going the genetic profiling route to try to narrow down the choice of medications more quickly. PTSD along with related anxiety and depression issues.

Mental health is really really hard in some cases... and the health care system in the US handles it pretty terribly across the board. :l

Hypatian wrote:

Mechanisms are getting understood better, at least for some things. A number of people I know seem to have gotten a significant improvement in managing their mental health after getting a genetic profile done. Apparently that can show whole classes of psychoactive drugs that will work better, worse, or differently for individuals.

Pretty amazing stuff.

But even then, it's not a panacea. One close friend is struggling really really hard right now with hir mental health, despite going the genetic profiling route to try to narrow down the choice of medications more quickly. PTSD along with related anxiety and depression issues.

Mental health is really really hard in some cases... and the health care system in the US handles it pretty terribly across the board. :l

PTSD is a tough diagnosis to treat with medication because... well, what are you trying to treat?

Are you trying to manage the physiological responses that may or may not be present or the negative alterations in cognition?

Negative alterations in cognition don’t get a magic pill solution. These are usually the cases where conventional psychotherapeutic response is indicated.

Or are you trying to manage physiological responses? This also is tough because PTSD physiological symptomology is very broad and engages a wide variety of interconnected body systems.

Reducing severely terrifying and disturbing nightmares is, as I understand it, completely different medically than reducing mild dissociative episodes while driving a car.

There are studies arguing that the placebo effect is responsible for up to 50% of the effect of a prescribed medicine in many therapeutic situations; that is, the patients *belief* that the treatment will be efficacious (ie, trust in the doctor) has a *large* influence on outcome, even when the drug is doing its work properly in the body. What strikes me here is that CBT is a *behavioral* therapy, requiring the patient to change. (Placebos can also be ineffective in treatment of acute problems, and can even cause adverse reactions, so the main takeaway is that the placebo effect is real, and constitutes treatment when used, and even when it's a built-in component of treatment, as with patient confidence in the treatment.)

I see a *ton* of skepticism in my peers and many others about the effectiveness of any kind of psychological or psychiatric treatment. My naive suspicion is that that kind of skepticism is driving negative results in counseling situations. The belief that the treatment on offer will not do the job could be devastating to the outcome.