Return to the Herbalism Forum | Post a Follow-Up

Limits of so-called 'evidence based medicine'

Posted by apollog (My Page) on
Thu, May 7, 09 at 17:19

I just came across an interesting study on the limits of so called "evidence based medicine" which has relevancy here. Note that "evidence based medicine" is quite different than merely relying on scientific evidence; it has it's own set of rules and standards. A summary of the case:

A patient is prescribed an antibiotic after getting some dental work done. A pharmacist makes an error, and instead of dispensing the amoxicillin, gave the patient the antidepressant doxepin. The man starts taking the doxepin at a dose that is four times the maximum daily dose (which would have been a normal dose of antibiotic). After taking the overdose of the wrong medicine for several days, the toxic effects of the antidepressant are so bad that the patient stops taking it, and the error is discovered. The patient develops glaucoma and neurological damage, which is blamed on the medicine.

There were no studies showing the effects on humans of a massive overdose of the antidepressant (probably hard to recruit people for such a study). The doctors representing the injured man looked to a variety of scientific studies about doxepin toxicity, and presented enough evidence linking doxepin to glaucoma so that the man wins a lawsuit against the pharmacy.

Of course, the pharmacy appealed the case - they admitted that they goofed on the prescription, but though it was unfair that they pay for damages related to the glaucoma. Their argument was that there were no "evidence based medicine" to prove that doxepin can cause glaucoma - that is, there were not multiple, large controlled studies where humans had been given large doses and then developed glaucoma. Without such studies, the pharmacy company said, there was only conjecture. The fact that a drug might cause glaucoma in animals was dismissed as irrelevant. The fact that the drug in humans has an effect on the optic nerve that is consistent with developing glaucoma is interesting, but again, not proof of anything. The fact that the warning that came with the drug mentioned an increase in eye pressure as a possible temporary side effect was not even seen as proof of anything - the fact that normal dose of doxepin causes a temporary increase in eye pressure is not proof that an overdose of doxepin would cause the pressure to go so high that it caused glaucoma. The pharmacy argued that it would be wrong to put all of these fragments of evidence together and claim that it proved that an overdose of doxepin causes glaucoma.

The judge agrees with this argument, reverses the decision, and orders a new trial. In the new trial, evidence that is not 'evidence-based' is strictly limited.

One obvious problem here is that for certain type of questions, we cannot look to 'evidence based medicine' for an answer, because that type of study hasn't, won't, or can't be done! And while questions that get asked a lot (do statins lower cholesterol) are likely to be answered quite well by such "evidence based" methods, other questions that are asked rarely simply don't get the resources needed for such large studies.

One of the regular posters here uses the same tactics as the pharmacy company - routinely claiming if there aren't multiple, large scale trials on humans, we aren't certain of anything. It's B.S.

Not only is this ridiculous on its face, it also cuts both ways. If we demand the same "evidence-based medicine" proof for everything, we cannot say that Hydroxycut actually causes problems ... no need for a recall, because we don't have large placebo-controlled studies that actually show that Hydroxycut really is dangerous. Sure, there are 23 cases of liver damage linked to hydroxycut, but if millions of people took placebo, how many would develop liver problems in the course of year?? Sure, there are animal studies linking one of the ingredients in Hydroxycut to liver damage, but according to the standards of evidence based medicine, that doesn't prove anything - we don't really know that hydroxycut causes liver damage ... it is only by making a leap of inference that the animal studies are remotely relevant.

Follow-Up Postings:

The limits of a bad argument

If you set up a straw man argument, it can be fun to tear it down. It is however a meaningless victory.

Since the statements above are directed at me, perhaps apollog can show where exactly I've said that every single question in medicine must be answered by large-scale clinical trials. Since I haven't, let's move on to other points raised.

The case as described regarding doxepin seems to be further evidence that courts can do all kinds of nutty things. It seems evident that if a drug is known to have a tendency to cause a certain side effect, giving a whopping overdose of that drug through pharmacy error that results in that effect being especially severe, doesn't require separate studies to show the effects of massive overdosage (see ethical limitations discussed below).

Now take the argument that serious side effects can't be linked to the use of certain supplements, since they haven't been demonstrated in big clinical trials. This is nonsensical for multiple reasons.

First, the companies that market supplements and herbal remedies take full advantage of current laws that make it unnecessary (and unwise from a marketing standpoint) to do sizable clinical trials on their products, which can be labeled as supplements instead of the drugs they really are. There is no adequate premarketing phase during which side effects and toxicity can be evaluated, so we're generally dependent on voluntary reporting after the product has been in use for awhile (in the case of Hydroxycut, it took two years for the death associated with the product to be reported to the FDA). Is apollog saying that the Hydroxycut people should be let off the hook because they didn't do the work in the first place that might have discovered this problem? (not all serious side effects are found during clinical trials as we know, but some pharmaceuticals never make it to market for this reason, and others are given warning labels or come with listings of potential side effects, so that patients can be monitored for possible problems).

Once dangerous side effects are reported for a supplement, there's then a strong ethical prohibition against recruiting patients for a large-scale study, assuming someone's willing to pony up the money for it. It's unethical to put people at risk, especially for a product with dubious efficacy.

In Hydroxycut's case*, the company is stressing that millions supposedly took the product without the problems reported in the 23 cases that have gained attention (this of course overlooks the strong likelihood that many cases were never reported and/or were never connected with use of the supplement(s)). One could argue that shattering of liver function in people who took Hydroxycut products, leading to fatality or needing liver transplants could be a mere coincidence (even though other hepatotoxic agents could not be blamed in these cases). I wouldn't take such a chance, with this or any other product that offers unproven or slim benefits and even a small chance of devastating injury (other herbs like comfrey and kava fall into this category).

So if apollog is really offended by the idea of Hydroxycut (or any other supplement) being taken off the market because of reports of severe side effects, the logical thing would be for him to support changes in current law (DSHEA), making it mandatory for supplements to undergo clinical trials like other drugs, in order to establish their safety and efficacy.

*is there some reason the initial post required a separate thread from the one already opened about Hydroxycut?

RE: Limits of so-called 'evidence based medicine'

Eric, you totally missed Appolog's point.

He was not defending Hydroxycut.

Not in the least.


He was making a point regarding the limits of evidence based medicine, and using the Hydroxycut case as an example of why EBM is not always sufficient to handle a situation.

RE: Limits of so-called 'evidence based medicine'

I think you missed the point of my post, which directly addressed the anti-EBM argument. :)

RE: Limits of so-called 'evidence based medicine'

My point is that we don't need 100% certainty to make a decision - and in the case of hydroxycut, applying EBM would be ridiculous and potentially dangerous. In this case, (ordinary) science and (ordinary) medicine beat so called "evidence based medicine" hands down. We can decide to stay away from a supplement even if there is not absolute proof that it is dangerous; on the flip side, we can say that there is good evidence that an herb is useful for something, even if there is not absolute certainty according to EBM methodologies.

EBM is scale dependent - it can be very useful when there is lots of research. EBM is very poor at answering questions where there is limited research - it must resort to the conclusion of 'unproven' for most questions. That is a limitation when dealing with most of modern medicine. That is a limitation when dealing with most of herbalism.

    If every clinical decision was supported by solid, irrefutable evidence, there would be little controversy in clinical practice. Appropriate treatment decisions would be obvious. In reality, treatment decisions are based on imperfect evidence. Physicians receive bits of data from a variety of sources, including study results, expert opinions, and experience. These data must be combined and applied to a specific situation.

    Ideally, every decision should be based on the highest standard of evidence available - the results of a randomized trial. However, very few informative random trials are undertaken. They are expensive and may be impractical, with delayed results. Further, the results of a randomized trial apply to the average patient in a study; they may not apply to the particular patient for whom a treatment decision is being made. ...

    Entirely rational clinical decision making is not achievable. Subjectivity is unavoidable. (source)

Objectivity and EBM can be great - when it is applied to the right questions. To pretend that EBM is a universal tool to apply to every question in every field of health and medicine is wrong. To accept the notion that everything in herbalism is 'unproven' until EBM validates it is a fallacy ... we can (and must) rely on various types of scientific evidence that EBM might not be entirely comfortable with to answer important questions.

RE: Limits of so-called 'evidence based medicine'

"EBM is very poor at answering questions where there is limited research - it must resort to the conclusion of 'unproven' for most questions. That is a limitation when dealing with most of modern medicine. That is a limitation when dealing with most of herbalism. "

You're mistaken - and again I think this stems from a belief that EBM is founded entirely on the outcome of large-scale clinical trials. Not so.

"Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research. By individual clinical expertise we mean the proficiency and judgment that individual clinicians acquire through clinical experience and clinical practice. Increased expertise is reflected in many ways, but especially in more effective and efficient diagnosis and in the more thoughtful identification and compassionate use of individual patients' predicaments, rights, and preferences in making clinical decisions about their care. "

One more example - you can't expect that advanced cardiac life support (emergency resuscitation) can be subjected to randomized clinical trials - it would be inhumane and unethical to separate coding patients into no treatment and treatment groups to judge outcomes. But ACLS protocols can be judged by standards of EBM, including scrutiny of clinical practices in the field.

Even if research is limited, quality clinical experience (as opposed to commercial testimonials from unknown sources) can be used to help formulate evidence-based practice. Good, reliable clinical experience is difficult to come by in herbalism, given that so many sources are quite old and a large variety of amateur and "professional" practitioners are involved, plus reporting systems are primitive at best.

Subjectivity is indeed unavoidable - but it is far more of a problem in herbalism than in mainstream medicine.

"To pretend that EBM is a universal tool to apply to every question in every field of health and medicine is wrong."

It is an excellent tool, the best we have to promote effective medicine, and it can in fact be used to recommend the best practices in every field. A key is not to confuse it with randomized clinical trials - which are not synonymous with EBM.

RE: Limits of so-called 'evidence based medicine'

>> Even if research is limited, quality clinical experience (as opposed to commercial testimonials from unknown sources) can be used to help formulate evidence-based practice.

I agree that commercial testimonials are inherently suspect, commonly inflated or fabricated, and not worth anything. Where we disagree is on the value of the traditional use of herbs, the clinical experience of herbalists, personal experiences, the use of animal studies, the use of in vitro studies, and the ability to make deductions and inferences using the body of knowledge derived from science.

By way of example: If an herb has historical and clinical uses to treat anxiety, if it also is shown to reduce anxiety in mice, if it is also shown to be a potent GABA agonist in experiments with nerve tissue, and if I take it and feel quite relaxed and my palpitations cease, then my perspective as a scientist is that there is good evidence that the herb in fact does reduce anxiety. But ask the EBM people what can be said of that herb and anxiety, and they will say it is unproven, there is no good evidence, more research is needed, etc etc. Why? Because the EBM community is used to dealing with a different scale of research, their methodology discounts or rejects a variety of evidence that may be quite valuable, and they are often disconnected from the actual phenomena they 'study' with EBM. (side note: that attitude reminds me of the American idea that kids coming out of a business school are better qualified to run any type of business than people who have worked in the business for years - the special universal skills of the biz school allow them to manage any type of business activity, and they are superior to other skills or training.)

In theory, the herbal example above might fit into Level D of the UK's implementation of EBM: ... based on physiology, bench research or first principles." But in practice, what we see in the literature (and what you engage in on this board) is a rejection of anything on this level as "worthless" and "unproven." In fact, you have frequently ridiculed such bench research with statements like "humans are not rats." But of course, science can only be conducted by humans, and humans are often biased.

RE: Limits of so-called 'evidence based medicine'

Can anyone tell me how this is relevent to me or herbalism?
PS please don't use any reference to medical stuff as this is a herbalism forum and i for one aint intrested even if the question was asked you have done runed it into the ground dug it up and made it smell like donkey poo!!!! LOL!!!! ;0

RE: Limits of so-called 'evidence based medicine'

If an herb has historical and clinical uses to treat anxiety, if it also is shown to reduce anxiety in mice, if it is also shown to be a potent GABA agonist in experiments with nerve tissue, and if I take it and feel quite relaxed and my palpitations cease, then my perspective as a scientist is that there is good evidence that the herb in fact does reduce anxiety."

No, that's not the perspective of a scientist. It's the opinion of an herb user who wants to believe that a remedy works. That's why you'll never see a scientific study close with remarks by the authors saying "well, our evidence is only preliminary, but since this stuff worked for us we think that no further research is necessary." They'd be laughed out of their profession.

There is no "EBM community" at odds with the scientific and medical community. The people who are most interested in optimizing medical care and making it as rational and effective as possible naturally are drawn to these principles, and work closely with physicians and hospitals to promote evidence-based care.

When it comes to herbalism, there are basically three sorts of users (conveniently, all three are represented in this thread). First there are those who don't care about scientific validation of herbs (or at least profess this opinion). There are others who strongly believe in the value of good scientific evidence for the safety and efficacy of herbs before they'll use them, especially for more than minor conditions or as substitutes for mainstream therapies.

There's also a large group of people who, like apollog, seem to want it both ways. They'll call attention to scientific studies that appear to support their position. But when it's noted that those studies are irrelevant to the topic at hand, limited in important ways and more evidence is needed, they'll fall back on testimonials, point fingers at other types of medical practice (the tu quoque fallacy), or deny the legitimacy of scientific inquiry that invalidates their arguments. This latter form of denial can be summarized as "Your science can never adequately measure my brand of woo."

This is what has people devoted to sound scientific inquiry and rational medicine shaking their heads. They want advocates of all forms of health care to follow the rules by which the value of care can be established, including valid research and disciplined clinical observation. Unfortunately there are still self-styled advocates of herbalism who think the rules apply only when convenient.


RE: Limits of so-called 'evidence based medicine'

Goshen this forum is for the discussion of herbalism, the use of herbs for medicinal purposes. If you don't see the connection between medicinal uses and medical literature on the medical problems and medical uses of herbs then I just don't know what to say to you.

RE: Limits of so-called 'evidence based medicine'

There are many "herbalists" who are very happy to rely on historical use, personal use, and some studies done to pick and use their medicine. What they don't want, are government regulations turning every plant and herb into a "drug" to be studied "scientifically" and then become something else they can market. Why is it when an herb has a possible side effect, there is a big blow-up about it being unsafe? At any time, I can turn on the tv or pick up a magazine, and see an add for the newest medicine, approved quite quickly by pharmacutical companies and the FDA.....after their trials and science based medicine and blah blah blah. Then we get to the truth: The approved medicines have a list of "possible" side effects that are much worse than the symptom they are suppose to help! Please , the day I take a pill for indigestion that can cause kidney or liver failure or blindness or whatever, is the day I become the mindless, unthinking, unquestioning, consumer. I'll take good old fashioned herbs any day before this so-called "science." So, eric darling, what class am I in?

RE: Limits of so-called 'evidence based medicine'

I am sure there are many pharmaceutical companies that would love to not have their products labeled "drugs" and not have to "prove" that they work or "scientifically" study their side effects.

RE: Limits of so-called 'evidence based medicine'

Yes, and they would still make billions!

RE: Limits of so-called 'evidence based medicine'

Oh they would make many billions more if we let them out from under the scrutiny, the point is that we shouldn't. If you make medical claims you get medical scrutiny, and when they don't bad things happen.

RE: Limits of so-called 'evidence based medicine'

Scrutiny? Yeah right. Everyone knows they have the FDA in their pockets. Ever work in a dr.s office? The drug dealers , um i mean drug reps, come in, buy lunch, hand out goodies, and samples. Then when you go to see the dr., you get the samples. We are daily bombarded with ads for a new drug to treat a symptom. That's my complaint with western medicine. They get rich treating symptoms, never addressing the root/cause of the problem. It's all about money, not well-being and total health. Understand what I'm saying?

RE: Limits of so-called 'evidence based medicine'

You are arguing for more scrutiny for the more scrutinized end of things, and less scrutiny for the less scrutinized end of things. Take a few days to think about your argument, then reply.

Western medicine treats the root cause when ever we can, the problem is that the root cause is often hard to get at (a misspelling in the genome, a patient who just wont stop eating cupcakes, a lung full of asbestos).

That gambit about only treating symptoms just doesn't hold water. Why is my grandmother on blood thinners? She experiences no symptoms at all when she is off of them, and is actually happier because she doesn't bruise so easily, however the clotting, which is something she cannot feel happening, will kill her, so she stays on the meds (she was born with blue baby syndrome, something that is 100% fatal in every case if not treated by western medicine). That is a clear case of western medicine not conforming to your preconcieved notions.

RE: Limits of so-called 'evidence based medicine'

"That's my complaint with western medicine. They get rich treating symptoms, never addressing the root/cause of the problem. It's all about money, not well-being and total health. Understand what I'm saying?"

Well understood - it's a common argument among alt med enthusiasts. It's also dead wrong.

It's amazing how those who preach alternative remedies seem to think they invented the concept of avoiding health problems through proper diet, exercise etc. - when these have been tenets of mainstream preventative care for a very long time. And while mainstream care is discovering the actual "root causes" of many diseases (largely through genetic/molecular studies), alternative practitioners are typically stuck in a rut blaming various things (Candida, "acid pH", parasites etc.) that have nothing to do with root causes of major chronic illness but sell a lot of books and supplements.

As to pooh-poohing of sympatomatic relief - try that argument on someone with rheumatoid arthritis or another highly painful/crippling disorder who can live a relatively normal life with the aid of modern drugs.

By the way, I'd happily support major restrictions or even a ban on TV marketing of prescription drugs, right along with a ban on the zillions of ads I see for supplements that claim to cure/prevent disease.

RE: Limits of so-called 'evidence based medicine'

Well, I have accomplished something: I've got eric and brendan directing their fire at me! No, you guys totally missed my point: if it's meds or supplements, it's all about making money for a lot of people involved. I don't say that western medicine is "bad", it just needs an overhaul, much like our government. Too much green paper ruling peoples lives. My grandmother's sister suffered from polio, and to this day walks with a terrible limp. Vaccines eradicated that dx, and I don't worry about my kids getting it. No, when it comes to people that practice herbalism, I don't think their should be more scrutiny. If there are studies done on herbs, great. However, as I've stated before, I don't use herbal supplements as they are not close to the natural form and less powerful medicinally. Yes, my argument may seem like a contradiction to you, brendan, because you have missed my point. It's about greed, and people's motives for making meds, supplements, latest diet pill, whatever. Do you not agree that there are too many new medicines put out every month? Don't you think some things should be tested further, and given more time? Oh, I know this goes back to your "think about your argument" statement. My point is, herbs have been used for centuries. These new medicines have not. Some of them cost a LOT. But you know this. I can grow some herbs in my yard for the price of seeds and my efforts, and make a little medicine if I like. What's wrong with that? I'm not promising cures, or selling them to make lots of money. As for ads, my husband and i were driving along and heard one on the radio. It was something for runny nose or allergies or something, but as we listened, possible side effects included:loss of taste, nosebleed, headache, and dizziness, and something else. We started laughing. I'd rather have a runny nose! Lol. If you can't see where I'm coming from, that's fine. I see we are on different pages, but do not think I reject western medicine and science. I have a degree in science, although it's veterinary medicine. I can't waste my time arguing with you anymore. Can't we just agree to disagree on certain things?

 o Post a Follow-Up

Please Note: Only registered members are able to post messages to this forum.

    If you are a member, please log in.

    If you aren't yet a member, join now!

Return to the Herbalism Forum

Information about Posting

  • You must be logged in to post a message. Once you are logged in, a posting window will appear at the bottom of the messages. If you are not a member, please register for an account.
  • Please review our Rules of Play before posting.
  • Posting is a two-step process. Once you have composed your message, you will be taken to the preview page. You will then have a chance to review your post, make changes and upload photos.
  • After posting your message, you may need to refresh the forum page in order to see it.
  • Before posting copyrighted material, please read about Copyright and Fair Use.
  • We have a strict no-advertising policy!
  • If you would like to practice posting or uploading photos, please visit our Test forum.
  • If you need assistance, please Contact Us and we will be happy to help.

Learn more about in-text links on this page here