Hundreds of thousands of knee arthroscopic meniscus repairs are done. At a cost of $10,000 per surgery. Now let me tell you about the FIDELITY trial follow-up at 10 years.
There is a massive issue with that study from my perspective... There's no true control group. No group that got no surgery (not real or sham). The placebo effect is powerful and active in both the real and sham arms. For all we know, the real surgery group, though they did worse than the sham group, might outperform the "true" control group that got neither!!!
The sham surgery arm is not a genuine control, they got the same placebo dose as the real surgery arm. This has not shown that no surgery is better than surgery; only that sham surgery is better than real surgery. This What it has shown is that if real surgery is better outcome wise than no surgery, the mechanisms are placebo etc. (I haven't looked up the evidence on this.)
What is problematic is that we don't offer patients placebo (sham) surgery. Ethically it would be a minefield. So patients are never faced with choosing between placebo or placebo + active treatment, their choice is between placebo + active treatment or no treatment (or whatever therapies are available). So it may very well be that the pure placebo (sham surgery) > placebo + active treatment (surgery) > no treatment. In which case, given no-one offers pure placebo surgery, real surgery is worth it for the placebo benefits it offers, even if the surgery itself doesn't provide a benefit, it works solely via a placebo (and miscellaneous related effects).
Have I made sense? It just seems a gaping hole in the research, and doesn't answer the question that matters to patients.
Pain-related disability in the manifesting knee can, sometimes, be due to actual tears, rips and gaps, but really in most instances the storyline is more complicated.
We know that in part because there are loads of older folks walking and running and if you look at their knee MRI's they look terrible. I have no cartilage buffering the inside of my right patella, medial aspect. When I do sports, sometimes I notice it's a bit painful, but then I forget it or it just goes away.
This reflects one anatomic reality and one-story about pain. Anatomically, pain in a joint is heavily influenced by the load and offloading of the "symptom" joint based on the strength or weakness or imbalance of everything around but that joint. A knee takes a beating MORE when the articulation of leg and hip is not well-controlled, when core muscles and gluteal muscles can't keep you upright and even keeled .
The second part of the story is that pain is always, ultimately, announced and refined and reinforced in the brain. Your knee doesn't tell you "I'm in agony". Your brain does. We doctors can't assuredly stop all brains from announcing catastrophe (thought some will try). But in reality many patients will find ways to deamplify the concern they feel about the achy joint when they realize they have compelling things they want to do. So much power here lies in strengthening joints around the ailing one, and upholding people as they write the story of their lives.
Great post by John Mandrola — and I LOVE the title = "Bravery and Humility NEEDED to do proper medical science!”. Negative studies are often NOT published (sadly all-too-often because there is NO money in it for those sponsoring the costs of the study) ==> which is why BRAVERY is needed to get these studies published in the literature!
KEY Point that I would want to know from the FIDELITY investigators is WHY subjects underwent arthroscopic partial meniscectomy in the first place? How severe were the symptoms? It would be one thing if the indication for arthroscopy was an acute injury and a meniscal tear was confirmed as the probable cause — vs uncertainty as to the cause of the meniscal tear with this lesion more or less “incidentally” discovered on arthroscopy (ie, How many of us “live with” similar meniscal tears that don’t truly impact on our quality of life?).
Botom Line — The concept Dr. Mandrola highlights is an IMPORTANT one that is all-too-often ignored by study sponsors whose principal goal is financial gain. His discussion is “Sensible Medicine” at its BEST!
Fascinating that actual surgery can’t even beat simulated surgery on purely subjective outcomes, in this case involving meniscus tears.
I wonder how many surgeries have actually been subjected to the rigors of properly placebo controlled RCT.
However, I also wonder (just like Nordicc for colonoscopy) whether this will change practice.
For this, I must turn once again to Upton Sinclair: "It is difficult to get a man to understand something, when his salary depends on his not understanding it,"
Thank you for nicely disposing of the phony "ethics" objection to testing therapeutic regimens and procedures that are felt to be beneficial because they make sense and/or have traditionally been accepted without some sort of scientific confirmation.
I thought you were talking about the bravery needed by the speakers reporting on the evidence in pediatric gender medicine weekend before last, at the 2026 Pediatric Academic Societies meeting in Boston.
Two protesters violently rushed the stage and fought as they were dragged out, interrupting the speaker, an expert from the (now closed) largest pediatric gender clinic in the world, in the UK. That speaker (Dr. Anna Hutchinson) is now leading the UK trainings, incorporating the comprehensive evidence review by Dr. Cass (which included input from over 1000 stakeholders, 7 new systematic reviews, etc.). To add to this, several in the audience (embarrassingly!) clapped for the violent protesters.
Ben Ryan reported on the attempts (encouraged by a blog quoting and/or making several incorrect claims) to prevent the panel from even happening. He has some footage of the attacks as well.
The panel reported on facts: How so little is known about the natural history of gender dysphoria, the inability to know for whom it will persist, and how little is known about whether those who receive the current medical protocols improve rather than end up harmed, long term.
Most US clinicians don't know about what is happening in this field-- the ASPS position statement from February is a good start, if one does not want to read the longer peer reviewed HHS report.
Thank you for a wonderful perspective. I began training in orthopedics at Emory in 1974 during which time it was acknowledged that the meniscus was a load sharing device and therefore a cushion or pad, which certainly makes sense.
At that time, so many years ago, I wondered, how could removal of any of it benefit a patient. In spite of this, the treatment of a torn meniscus years ago was to remove all of it since it was “diseased and not healthy“ this obviously made no sense but it’s what we did. Today in sports Medicine the main objective is to retain any repairable meniscus tissue and only remove that which is causing impingement or mechanical issues, unresponsive to nonoperative management.
With no meniscus and increased load with obesity, we now have a flood of patients whose only remaining option is knee replacement.
Thanks so much for sharing this article. I became aware of the original study while reading "Ending Medical Reversal"--truly a wonderful and eye opening book. I teach an undergraduate class on Psychology of Aging--it is taken by students from psychology, as well as many premed and pre nursing students, and I have always included the original study in the section on Research Methods. I also teach about the importance of different kinds of control groups in studying psychological intervention, including attention control groups. I also include a great study from Oncology Nursing Forum, 2011, with Anita Catlin as lead author, comparing standard care, sham Reiki, and actual Reiki in enhancing comfort during chemotherapy infusion. Both sham and actual Reiki were found to have similar benefits compared to standard care.
I will now add the 10 year followup to my summer version of the class.
It is very important to help students see the value of skepticism and putting every intervention to the test.
It defies belief that he proposes that he expects confidence in his opinions/research when he doesn't know what a meniscus is.
Otherwise, I am almost convinced that I should accept this post puts together some interesting and important results of published experimental placebo controlled studies. How were they cherry picked?
When I was a medical student, the great Francis Moore quoted Osler who said, the patient with empyema needs an optimistic surgeon and six inches of cold steel. Forgive any imprecision in my paraphrase or misquotation.
I suspect his comment was tongue-in-cheek. I also suspect you'd already decided he was wrong regardless of what he wrote, as evidenced by your question about cherry picking, which isn't reasonable when the study methodology is described in the article.
And I'm struggling to understand the relevance of your quote from Osler via Moore. How does treatment of empyema relate to this article? Even if there is some relevance that I'm missing, this seems to be a classic appeal to authority and thus a weak argument. History shows us that even the most eminent physicians (Galen, for example) are later proved wrong about many things as scientific understanding progresses. I'd suggest putting more store in empirical evidence than the bon mots of great men.
I've had one meniscus repair (in 1983 when I was 21, and it lasted 7 years before tearing again) and two meniscus arthroscopies (1990 on the same knee, and 1995 on the other knee). I had no choice because my knee was locked every time.
I knew that I would be susceptible to arthritis, since I'd had my surgeries so young, but what was I to do? I did gymnastics and then jazz dance, and was pretty abusive to my joints. I've also had a hip replacement and a supraspinatus/biceps repair (TMI, but whatever) more recently.
I've done PT off and on for 14 years; highly recommend it. I still have my original knees. I've had fluid drained and a steroid shot in the "bad" knee, and of course, I've modified my activities (much less running/jumping), but all in all, I'm still quite active.
In my case, athroscopies were a good thing, but I understand every case is unique.
Interesting food for thought about ethical study designs here. Would love to see a thorough meta-analysis on the insights gained from placebo/sham controlled studies if any are out there (or soon-to-be out there).
To be honest I'm conflicted (and incidentally the new results are no different from the older one I seem to remember being reported in Vinay and Adam's book on Medical Reversal, or perhaps in a previous blog on either sensible Medicine or Vinay's blog). Here's the thing. An appropriately conducted RCT, including surgery vs sham surgery, is clearly the gold standard. That being said, there are several issues with the results of this particular RCT (for meniscus injuries/tears) as well as others (including various cardiac procedures): (1) The results relate to an average patient; however, there is no such thing as an average patient; every patient is an individual. (2) The results are very operator dependent (something that the current RCT probably didn't look at; however, everybody knows that some surgeons are simply better than others and get better results with less pain. The more of a particular surgery a surgeon does, the better the end result usually. (3) If one has a situation where some individuals would benefit from the arthroscopy and meniscus clean-up while others would not or experience a net negative, the end result would be an RCT that showed no benefit. However, had the cases been selected more carefully so that only people who would really benefit from surgery actually had the real surgery, the result may well have been different. That's where the "art" in the "art of medicine" comes into play.
That's my scepticism about evidence based medicine. My migraines don't respond to the standard first line evidence based medicine, but they do respond reliably to a non evidence based therapy.
And... I have 3 cats. One adores catnip, rolling her head in it, one jumps back and won't have a bar of it, the third ignores it. One positive, one negative, one nonevent. If this was a trial, the conclusion would be that catnip has no effect, because a positive, a negative and a neutral cancels out to a big fat zero - no effect. Yet they all three consistently display different responses. Only one out of three truly has a nothing response. But if this was a RCT, catnip would have no effect on cats.
I was taught that RCTs only ever tell you about groups of people, they only ever tell you what's likely to happen, they can never predict the response of the individual patient in front of me... because of all the reasons you gave among others.
What you're describing is the fact that there may be circumstances under which a treatment "works" - and that's true, but what's also true is that we have no evidence that it works. So if you try it and it works, we are clueless as to whether it worked because of the placebo effect etc or not. Given the fact that for heaps of our interventions, the main reason they "work" is via a placebo effect etc (antidepressants for depression I'm looking at you) maybe it doesn't really matter what the mechanism is?
These are common criticisms of any evidence that questions a surgical intervention: some patients will still benefit, some surgeons are better than others, more research is (always) needed, medicine is an art, surgeons should carry on with the existing practice until the perfect trial is conducted. Another popular one is 'we've got much better / technology has advanced since this research was conducted.'
Similar criticisms can be levelled at drug RCTs. Perhaps a drug that has been shown to be no more effective than a placebo might help an unspecified subset of patients who have a particular genetic make-up or manifestation of the condition? Perhaps more skilled diagnosticians prescribe it more effectively? Why not leave it to doctors to prescribe the drug if they 'feel' it might work?
Arguments like these place an extremely high burden of proof on research that challenges current practice while relying on unsupported hypotheses to justify the status quo. This might be reasonable if new research was seeking to overturn practices that were originally based on strong evidence, but not when they were based on hypotheses regarding plausible mechanisms. In these cases, if a well-constructed RCT shows that surgery has no benefits over sham surgery, the onus should be on those who argue for the status quo to produce evidence to support their hypotheses.
I completely disagree with you and my impression is that you're blinded by RCTs. If you are looking as to whether a particular drug has an effect an RCT is perfectly fine, although as I'll illustrate further down even there there are subtleties that one has to consider. But surgery tends to be a whole different issue because so much is operator and judgement dependent.
For example, in the case of arthroscopy and meniscus surgery, judgement and skill is absolutely critical. Shave too much off and you're guaranteed to end up with degenerative changes. Do too little and there is no benefit. So the line between a successful surgery (in terms of outcome) and an unsuccessful one (with no benefit or a negative effect) is actually rather narrow. So just blindly doing an RCT with sham surgery as your control arm is not that helpful in telling you anything that you don't already know: namely, patient selection and really accurate assessment (radiological) before surgery is critical, and the surgeon is absolutely critical in terms of outcome. Did the study from Finland parse the results in terms of individual surgeons, and the answer would be no. So the bottom line is that this study tells one absolutely nothing that one doesn't already know.
Similarly with stenting for coronary artery disease. Sure the RCTs indicate no benefit in the absence of a heart attack. But is this too narrow a view. After all, if an individual has generalized atheromatous vascular disease, then it is evident that if your stent one blockage, another blockage will simply occur upstream or downstream, so no benefit. But for an individual who does not have generalized atheromatous cardiovascular disease but a very specific localized issue with blockage, then stenting should be of benefit.
Or consider the use of statins. RCTs indicate that statins are basically useless in the absence of a prior cardiovascular event but are very useful post-cardiovascular event. But has anybody asked the following question: if one could determine diagnostically and with certainty which individuals were going to have heart attacks and which were not, would statins have a beneficial effect if given before the heart attack in those individuals who you knew were going to have one. And I'm not advocating for statin use here willy nilly but simply saying they should be used with great care and not given out like M&Ms.
The bottom line is that, at least as far as most surgeries are concerned, it doesn't matter how well constructed a trial may be in terms of statistics if it doesn't take into account (a) correct judgement on the part of the physician as to whether to operate or not, and (b) the quality of the surgeon where skill and judgement are absolutely critical. And lastly, it is important to understand that RCTs apply to an average patient who never exists as each person is an individual.
A further example to consider is the use of drugs to treat rare diseases, something that has been made a great fuss of recently as a result of FDA denials (which I don't necessarily disagree with). The problem is that a drug may be useful in a very small subset of cases but not in general and that's something that an RCT will not tell one. All it tells one is that on average drug X may have no significant effect. But the reality is that in some patients the drug may have a positive effect, in others a negative effect and in a thir group a null effect. The key is to be able to ascertain which set of patients will benefit and which wont. And perhaps, for many drugs, nobody benefits.
I appreciate you taking the time to set out your arguments in detail.
I don't disagree with the principles or aims you set out. I agree that constructing RCTs in these ways could deliver significant benefits; I don't think anyone could reasonably disagree that producing statistically meaningful results at these more detailed levels would add considerable value. However, I think there are three major problems in practice.
The first is the practicality and expense of constructing such trials. The approach you suggest would require sufficiently large sample sizes for each surgeon (or at least quality category) and for each patient category, as well as independent expert assessments of the quality of each surgery (and possibly of the pre-surgery radiological report). The time and resources required to undertake such trials would mean that far fewer trials could be done, which would lead to delayed-but-detailed findings in a few fields and none in others.
The second is that even if such trials could be undertaken, translating them into practice would require accurate mechanisms to categorise surgeon quality. For example, suppose this RCT had found that surgery only delivered significantly better results than sham surgery when they were assessed as upper-quartile for quality. How would this translate into practice, given that many surgeons (if not most) are likely to regard themselves as high performers? Some form of objective ongoing quality assessment mechanism would be needed, which would add more costs and/or come up against statistical limits for lower-volume surgeons.
The third is that while we demand these ideal (but costly and lengthy) trials, we continue to practice in ways that ignore what RCTs based on averages tell us: in this case, that the average patient treated by the average surgeon does not seem to benefit from the surgery. Instead, we maintain the status quo based on plausible but unsubstantiated hypotheses that (a) patients benefit when treated by surgeons who can accurately assess their condition and deliver high-quality surgery, (b) surgeons can accurately assess whether or not they are one of these high-performers, and (c) they will refrain from practicing this surgery if they conclude that they are only an average performer. This seems to me to be an unduly optimistic view to take.
In summary, I think I agree with you in theory but not in practice.
That's not quite what I'm saying. What I'm saying is that RCTs are probably not suitable to surgical procedures where a huge amount of judgement on the day is required. For arthroscopic meniscal surgery (cleaning up, shaving the meniscus), it doesn't matter how good the surgeon may be, because the line between doing too little and doing too much is very fine. Further, patient selection is absolutely critical in terms of achieving excellent outcomes. You can't simply use this type of procedure willy-nilly like handing out M&Ms. The same goes with coronary artery stents. Here the surgery/intervention is not that hard, but the prior workup has to be really thorough to truly assess who will benefit and wont.
So my basic view is that RCTs are extremely difficult to do rigt with surgical procedures. Far easier with drugs where one is assigned to take either the new pill or the placebo/standard of care pill. One still has to do these properly. For example, when it comes to chemotherapy, and especially second line/rescue therapy one cannot compare the drug against a sugar pill but rather one has to compare against the standard of care in the US (not in some 3rd world country).
The bottom line is that RCTs are certainly the gold standard but they are not the be all and end all. It's also worth bearing in mind that one only needs an RCT for something that is marginal. As an example, when penicillin was being tested in the 1940s by Florey et al. there was no need for an RCT as the results were so dramatic.
Prior meniscus repair in both knees. Could not be happier with the results even 15-20 years out. Never had an issue again. Zero knee problems since. The type of tear largely correlates with repair need and success.
At this late date it seems strange to me that pharmaceuticals must demonstrate safety and effectiveness for approval but surgery apparently does not.
No one promised me that I'd increase my cycling prowess after a mitral valve repair. True, I haven't progressed to HF; neither has my brother, who had the procedure 15 years before I did. Still, both of us were asymptomatic pre op, and both felt worse post-op.
There is a massive issue with that study from my perspective... There's no true control group. No group that got no surgery (not real or sham). The placebo effect is powerful and active in both the real and sham arms. For all we know, the real surgery group, though they did worse than the sham group, might outperform the "true" control group that got neither!!!
The sham surgery arm is not a genuine control, they got the same placebo dose as the real surgery arm. This has not shown that no surgery is better than surgery; only that sham surgery is better than real surgery. This What it has shown is that if real surgery is better outcome wise than no surgery, the mechanisms are placebo etc. (I haven't looked up the evidence on this.)
What is problematic is that we don't offer patients placebo (sham) surgery. Ethically it would be a minefield. So patients are never faced with choosing between placebo or placebo + active treatment, their choice is between placebo + active treatment or no treatment (or whatever therapies are available). So it may very well be that the pure placebo (sham surgery) > placebo + active treatment (surgery) > no treatment. In which case, given no-one offers pure placebo surgery, real surgery is worth it for the placebo benefits it offers, even if the surgery itself doesn't provide a benefit, it works solely via a placebo (and miscellaneous related effects).
Have I made sense? It just seems a gaping hole in the research, and doesn't answer the question that matters to patients.
Pain-related disability in the manifesting knee can, sometimes, be due to actual tears, rips and gaps, but really in most instances the storyline is more complicated.
We know that in part because there are loads of older folks walking and running and if you look at their knee MRI's they look terrible. I have no cartilage buffering the inside of my right patella, medial aspect. When I do sports, sometimes I notice it's a bit painful, but then I forget it or it just goes away.
This reflects one anatomic reality and one-story about pain. Anatomically, pain in a joint is heavily influenced by the load and offloading of the "symptom" joint based on the strength or weakness or imbalance of everything around but that joint. A knee takes a beating MORE when the articulation of leg and hip is not well-controlled, when core muscles and gluteal muscles can't keep you upright and even keeled .
The second part of the story is that pain is always, ultimately, announced and refined and reinforced in the brain. Your knee doesn't tell you "I'm in agony". Your brain does. We doctors can't assuredly stop all brains from announcing catastrophe (thought some will try). But in reality many patients will find ways to deamplify the concern they feel about the achy joint when they realize they have compelling things they want to do. So much power here lies in strengthening joints around the ailing one, and upholding people as they write the story of their lives.
Great post by John Mandrola — and I LOVE the title = "Bravery and Humility NEEDED to do proper medical science!”. Negative studies are often NOT published (sadly all-too-often because there is NO money in it for those sponsoring the costs of the study) ==> which is why BRAVERY is needed to get these studies published in the literature!
KEY Point that I would want to know from the FIDELITY investigators is WHY subjects underwent arthroscopic partial meniscectomy in the first place? How severe were the symptoms? It would be one thing if the indication for arthroscopy was an acute injury and a meniscal tear was confirmed as the probable cause — vs uncertainty as to the cause of the meniscal tear with this lesion more or less “incidentally” discovered on arthroscopy (ie, How many of us “live with” similar meniscal tears that don’t truly impact on our quality of life?).
Botom Line — The concept Dr. Mandrola highlights is an IMPORTANT one that is all-too-often ignored by study sponsors whose principal goal is financial gain. His discussion is “Sensible Medicine” at its BEST!
Fascinating that actual surgery can’t even beat simulated surgery on purely subjective outcomes, in this case involving meniscus tears.
I wonder how many surgeries have actually been subjected to the rigors of properly placebo controlled RCT.
However, I also wonder (just like Nordicc for colonoscopy) whether this will change practice.
For this, I must turn once again to Upton Sinclair: "It is difficult to get a man to understand something, when his salary depends on his not understanding it,"
Thank you for nicely disposing of the phony "ethics" objection to testing therapeutic regimens and procedures that are felt to be beneficial because they make sense and/or have traditionally been accepted without some sort of scientific confirmation.
I would like to see a study with a real versus placebo hospital administrator. Those results would probably be………….
I thought you were talking about the bravery needed by the speakers reporting on the evidence in pediatric gender medicine weekend before last, at the 2026 Pediatric Academic Societies meeting in Boston.
Two protesters violently rushed the stage and fought as they were dragged out, interrupting the speaker, an expert from the (now closed) largest pediatric gender clinic in the world, in the UK. That speaker (Dr. Anna Hutchinson) is now leading the UK trainings, incorporating the comprehensive evidence review by Dr. Cass (which included input from over 1000 stakeholders, 7 new systematic reviews, etc.). To add to this, several in the audience (embarrassingly!) clapped for the violent protesters.
Ben Ryan reported on the attempts (encouraged by a blog quoting and/or making several incorrect claims) to prevent the panel from even happening. He has some footage of the attacks as well.
The panel reported on facts: How so little is known about the natural history of gender dysphoria, the inability to know for whom it will persist, and how little is known about whether those who receive the current medical protocols improve rather than end up harmed, long term.
Most US clinicians don't know about what is happening in this field-- the ASPS position statement from February is a good start, if one does not want to read the longer peer reviewed HHS report.
https://www.plasticsurgery.org/documents/health-policy/positions/2026-gender-surgery-children-adolescents.pdf
https://opa.hhs.gov/gender-dysphoria-report
Bravery is needed!
Thank you for a wonderful perspective. I began training in orthopedics at Emory in 1974 during which time it was acknowledged that the meniscus was a load sharing device and therefore a cushion or pad, which certainly makes sense.
At that time, so many years ago, I wondered, how could removal of any of it benefit a patient. In spite of this, the treatment of a torn meniscus years ago was to remove all of it since it was “diseased and not healthy“ this obviously made no sense but it’s what we did. Today in sports Medicine the main objective is to retain any repairable meniscus tissue and only remove that which is causing impingement or mechanical issues, unresponsive to nonoperative management.
With no meniscus and increased load with obesity, we now have a flood of patients whose only remaining option is knee replacement.
THANK YOU for providing your invaluable perspective as one trained in Orthopedics!
Thanks so much for sharing this article. I became aware of the original study while reading "Ending Medical Reversal"--truly a wonderful and eye opening book. I teach an undergraduate class on Psychology of Aging--it is taken by students from psychology, as well as many premed and pre nursing students, and I have always included the original study in the section on Research Methods. I also teach about the importance of different kinds of control groups in studying psychological intervention, including attention control groups. I also include a great study from Oncology Nursing Forum, 2011, with Anita Catlin as lead author, comparing standard care, sham Reiki, and actual Reiki in enhancing comfort during chemotherapy infusion. Both sham and actual Reiki were found to have similar benefits compared to standard care.
I will now add the 10 year followup to my summer version of the class.
It is very important to help students see the value of skepticism and putting every intervention to the test.
It defies belief that he proposes that he expects confidence in his opinions/research when he doesn't know what a meniscus is.
Otherwise, I am almost convinced that I should accept this post puts together some interesting and important results of published experimental placebo controlled studies. How were they cherry picked?
When I was a medical student, the great Francis Moore quoted Osler who said, the patient with empyema needs an optimistic surgeon and six inches of cold steel. Forgive any imprecision in my paraphrase or misquotation.
I suspect his comment was tongue-in-cheek. I also suspect you'd already decided he was wrong regardless of what he wrote, as evidenced by your question about cherry picking, which isn't reasonable when the study methodology is described in the article.
And I'm struggling to understand the relevance of your quote from Osler via Moore. How does treatment of empyema relate to this article? Even if there is some relevance that I'm missing, this seems to be a classic appeal to authority and thus a weak argument. History shows us that even the most eminent physicians (Galen, for example) are later proved wrong about many things as scientific understanding progresses. I'd suggest putting more store in empirical evidence than the bon mots of great men.
Could we please have similar trials of red light therapy, UV therapy, PEMF, and combinations of all three?
I've had one meniscus repair (in 1983 when I was 21, and it lasted 7 years before tearing again) and two meniscus arthroscopies (1990 on the same knee, and 1995 on the other knee). I had no choice because my knee was locked every time.
I knew that I would be susceptible to arthritis, since I'd had my surgeries so young, but what was I to do? I did gymnastics and then jazz dance, and was pretty abusive to my joints. I've also had a hip replacement and a supraspinatus/biceps repair (TMI, but whatever) more recently.
I've done PT off and on for 14 years; highly recommend it. I still have my original knees. I've had fluid drained and a steroid shot in the "bad" knee, and of course, I've modified my activities (much less running/jumping), but all in all, I'm still quite active.
In my case, athroscopies were a good thing, but I understand every case is unique.
Interesting food for thought about ethical study designs here. Would love to see a thorough meta-analysis on the insights gained from placebo/sham controlled studies if any are out there (or soon-to-be out there).
To be honest I'm conflicted (and incidentally the new results are no different from the older one I seem to remember being reported in Vinay and Adam's book on Medical Reversal, or perhaps in a previous blog on either sensible Medicine or Vinay's blog). Here's the thing. An appropriately conducted RCT, including surgery vs sham surgery, is clearly the gold standard. That being said, there are several issues with the results of this particular RCT (for meniscus injuries/tears) as well as others (including various cardiac procedures): (1) The results relate to an average patient; however, there is no such thing as an average patient; every patient is an individual. (2) The results are very operator dependent (something that the current RCT probably didn't look at; however, everybody knows that some surgeons are simply better than others and get better results with less pain. The more of a particular surgery a surgeon does, the better the end result usually. (3) If one has a situation where some individuals would benefit from the arthroscopy and meniscus clean-up while others would not or experience a net negative, the end result would be an RCT that showed no benefit. However, had the cases been selected more carefully so that only people who would really benefit from surgery actually had the real surgery, the result may well have been different. That's where the "art" in the "art of medicine" comes into play.
That's my scepticism about evidence based medicine. My migraines don't respond to the standard first line evidence based medicine, but they do respond reliably to a non evidence based therapy.
And... I have 3 cats. One adores catnip, rolling her head in it, one jumps back and won't have a bar of it, the third ignores it. One positive, one negative, one nonevent. If this was a trial, the conclusion would be that catnip has no effect, because a positive, a negative and a neutral cancels out to a big fat zero - no effect. Yet they all three consistently display different responses. Only one out of three truly has a nothing response. But if this was a RCT, catnip would have no effect on cats.
I was taught that RCTs only ever tell you about groups of people, they only ever tell you what's likely to happen, they can never predict the response of the individual patient in front of me... because of all the reasons you gave among others.
What you're describing is the fact that there may be circumstances under which a treatment "works" - and that's true, but what's also true is that we have no evidence that it works. So if you try it and it works, we are clueless as to whether it worked because of the placebo effect etc or not. Given the fact that for heaps of our interventions, the main reason they "work" is via a placebo effect etc (antidepressants for depression I'm looking at you) maybe it doesn't really matter what the mechanism is?
These are common criticisms of any evidence that questions a surgical intervention: some patients will still benefit, some surgeons are better than others, more research is (always) needed, medicine is an art, surgeons should carry on with the existing practice until the perfect trial is conducted. Another popular one is 'we've got much better / technology has advanced since this research was conducted.'
Similar criticisms can be levelled at drug RCTs. Perhaps a drug that has been shown to be no more effective than a placebo might help an unspecified subset of patients who have a particular genetic make-up or manifestation of the condition? Perhaps more skilled diagnosticians prescribe it more effectively? Why not leave it to doctors to prescribe the drug if they 'feel' it might work?
Arguments like these place an extremely high burden of proof on research that challenges current practice while relying on unsupported hypotheses to justify the status quo. This might be reasonable if new research was seeking to overturn practices that were originally based on strong evidence, but not when they were based on hypotheses regarding plausible mechanisms. In these cases, if a well-constructed RCT shows that surgery has no benefits over sham surgery, the onus should be on those who argue for the status quo to produce evidence to support their hypotheses.
I completely disagree with you and my impression is that you're blinded by RCTs. If you are looking as to whether a particular drug has an effect an RCT is perfectly fine, although as I'll illustrate further down even there there are subtleties that one has to consider. But surgery tends to be a whole different issue because so much is operator and judgement dependent.
For example, in the case of arthroscopy and meniscus surgery, judgement and skill is absolutely critical. Shave too much off and you're guaranteed to end up with degenerative changes. Do too little and there is no benefit. So the line between a successful surgery (in terms of outcome) and an unsuccessful one (with no benefit or a negative effect) is actually rather narrow. So just blindly doing an RCT with sham surgery as your control arm is not that helpful in telling you anything that you don't already know: namely, patient selection and really accurate assessment (radiological) before surgery is critical, and the surgeon is absolutely critical in terms of outcome. Did the study from Finland parse the results in terms of individual surgeons, and the answer would be no. So the bottom line is that this study tells one absolutely nothing that one doesn't already know.
Similarly with stenting for coronary artery disease. Sure the RCTs indicate no benefit in the absence of a heart attack. But is this too narrow a view. After all, if an individual has generalized atheromatous vascular disease, then it is evident that if your stent one blockage, another blockage will simply occur upstream or downstream, so no benefit. But for an individual who does not have generalized atheromatous cardiovascular disease but a very specific localized issue with blockage, then stenting should be of benefit.
Or consider the use of statins. RCTs indicate that statins are basically useless in the absence of a prior cardiovascular event but are very useful post-cardiovascular event. But has anybody asked the following question: if one could determine diagnostically and with certainty which individuals were going to have heart attacks and which were not, would statins have a beneficial effect if given before the heart attack in those individuals who you knew were going to have one. And I'm not advocating for statin use here willy nilly but simply saying they should be used with great care and not given out like M&Ms.
The bottom line is that, at least as far as most surgeries are concerned, it doesn't matter how well constructed a trial may be in terms of statistics if it doesn't take into account (a) correct judgement on the part of the physician as to whether to operate or not, and (b) the quality of the surgeon where skill and judgement are absolutely critical. And lastly, it is important to understand that RCTs apply to an average patient who never exists as each person is an individual.
A further example to consider is the use of drugs to treat rare diseases, something that has been made a great fuss of recently as a result of FDA denials (which I don't necessarily disagree with). The problem is that a drug may be useful in a very small subset of cases but not in general and that's something that an RCT will not tell one. All it tells one is that on average drug X may have no significant effect. But the reality is that in some patients the drug may have a positive effect, in others a negative effect and in a thir group a null effect. The key is to be able to ascertain which set of patients will benefit and which wont. And perhaps, for many drugs, nobody benefits.
I appreciate you taking the time to set out your arguments in detail.
I don't disagree with the principles or aims you set out. I agree that constructing RCTs in these ways could deliver significant benefits; I don't think anyone could reasonably disagree that producing statistically meaningful results at these more detailed levels would add considerable value. However, I think there are three major problems in practice.
The first is the practicality and expense of constructing such trials. The approach you suggest would require sufficiently large sample sizes for each surgeon (or at least quality category) and for each patient category, as well as independent expert assessments of the quality of each surgery (and possibly of the pre-surgery radiological report). The time and resources required to undertake such trials would mean that far fewer trials could be done, which would lead to delayed-but-detailed findings in a few fields and none in others.
The second is that even if such trials could be undertaken, translating them into practice would require accurate mechanisms to categorise surgeon quality. For example, suppose this RCT had found that surgery only delivered significantly better results than sham surgery when they were assessed as upper-quartile for quality. How would this translate into practice, given that many surgeons (if not most) are likely to regard themselves as high performers? Some form of objective ongoing quality assessment mechanism would be needed, which would add more costs and/or come up against statistical limits for lower-volume surgeons.
The third is that while we demand these ideal (but costly and lengthy) trials, we continue to practice in ways that ignore what RCTs based on averages tell us: in this case, that the average patient treated by the average surgeon does not seem to benefit from the surgery. Instead, we maintain the status quo based on plausible but unsubstantiated hypotheses that (a) patients benefit when treated by surgeons who can accurately assess their condition and deliver high-quality surgery, (b) surgeons can accurately assess whether or not they are one of these high-performers, and (c) they will refrain from practicing this surgery if they conclude that they are only an average performer. This seems to me to be an unduly optimistic view to take.
In summary, I think I agree with you in theory but not in practice.
That's not quite what I'm saying. What I'm saying is that RCTs are probably not suitable to surgical procedures where a huge amount of judgement on the day is required. For arthroscopic meniscal surgery (cleaning up, shaving the meniscus), it doesn't matter how good the surgeon may be, because the line between doing too little and doing too much is very fine. Further, patient selection is absolutely critical in terms of achieving excellent outcomes. You can't simply use this type of procedure willy-nilly like handing out M&Ms. The same goes with coronary artery stents. Here the surgery/intervention is not that hard, but the prior workup has to be really thorough to truly assess who will benefit and wont.
So my basic view is that RCTs are extremely difficult to do rigt with surgical procedures. Far easier with drugs where one is assigned to take either the new pill or the placebo/standard of care pill. One still has to do these properly. For example, when it comes to chemotherapy, and especially second line/rescue therapy one cannot compare the drug against a sugar pill but rather one has to compare against the standard of care in the US (not in some 3rd world country).
The bottom line is that RCTs are certainly the gold standard but they are not the be all and end all. It's also worth bearing in mind that one only needs an RCT for something that is marginal. As an example, when penicillin was being tested in the 1940s by Florey et al. there was no need for an RCT as the results were so dramatic.
Excellent comment!
Prior meniscus repair in both knees. Could not be happier with the results even 15-20 years out. Never had an issue again. Zero knee problems since. The type of tear largely correlates with repair need and success.
At this late date it seems strange to me that pharmaceuticals must demonstrate safety and effectiveness for approval but surgery apparently does not.
No one promised me that I'd increase my cycling prowess after a mitral valve repair. True, I haven't progressed to HF; neither has my brother, who had the procedure 15 years before I did. Still, both of us were asymptomatic pre op, and both felt worse post-op.