I am not really sure how this is a major drawback. As others have commented , it makes little difference. If they had had a control group with zero intervention, the differences would have been even more impressive! In my opinion, it is better to have a control group with some intervention so it cancels out some of the motivational bias. There is very little motivation for participants to stay in a 2-year study or even perform better in cognitive post-testing when they know that they missed out on the intervention they were excited to be in the study for.
Also, the control group here has "minimal contact" (6 sessions in 2 years) which is probably nothing when compared to the intervention group, which has one of the most intensive regimens I have seen for a lifestyle RCT!
The inclusion criteria were sedentary (per the Modified Telephone Assessment of Physical Activity: less than 60 minutes per week at moderate intensity). So they are sedentary based on their self-reports, which is how PA guidelines are based on and used. So I don't see a major issue there.
One point I would make is cognitive tests based on computer/paper tests are just surrogate measures This doesn't mean anything meaningful to people. The question people care about is can they function better in their everyday life or is their quality of life better now. That should be the primary outcome here.
Risk factor analysis has an almost unbroken history of failure as an avenue of research to discover causation of disease. And that is with diseases that are fairly well defined and produce findings that can be measured with some degree of accuracy. Linking diseases with "lifestyle" factors that are logistically impossible to measure in any reliable way is, to quote a famous philosopher, nonsense upon stilts. To apply any of this to a disorder like dementia is stretching medical and scientific credibility well beyond any acceptable limits. Of course, great careers and pharmaceutical fortunes have been built upon these shaky foundations and will, no doubt, continue to do so as long as people take these sorts of things seriously.
These would be fair points if the trial had been negative, but taken with a positive trial, they simply mean that the trial likely *understates* the magnitude of the effect. The first criticism (a "control" which is still a treatment) will serve to make the two arms more similar to each other, thereby reducing effect size and decreasing power (making it more likely to get a negative result). The second criticism (that the groups were already healthy at baseline) would also serve to minimize effect size, decreasing power and making it more likely to get a negative result.
If your point is that we can't use the estimated effect sizes here to generalize to a sicker population (and in reality the effect sizes, and therefore cost effectiveness, are even better than this trial makes it seem) then that is a fair point. But if you're asking us to throw the entire study out, then I think you're missing the fact that these methodological critiques are less pertinent to a positive study
I think that is totally fair and I've since come to appreciate that the results are still positive despite these concerns. But we already know that exercise and a good diet is good for brain health. What is not known is how best to design an intervention to improve these things in at-risk older adults. This trial misses an opportunity to help answer that question because of both flaws. No control means we can't know what exactly improved cognition. You could also imagine that what works for motivated healthy individuals could be different than unmotivated, unhealthy individuals. Thanks for the reply
Instructive - thanks. On a related but separate note... I'm wondering why blarcamesine is not getting more attention in AD. It has shown clear benefit to mild AD pts and is a small molecule with almost no side effects. Is it because it does not follow the amyloid dogma or is it because the results are not strong enough. To my eye, they are the same or better than the biogen drug.
With regards to the lack of a control group. Would there not be enough data on the expected rates of dimentia that you could be comparing to the poulation as a whole?
For a study purportedly investigating patients who are sedentary and with poor diet…the enrolled subjects did not seem very sedentary or eating particularly poorly…even at study outset. So the results would not seem generalizable to the study authors’ intended population.
And the participants also seemed very motivated to adhere to the program…which is also a trait that may lack generalizability, esp without a true control group as you’ve noted.
Wouldn't the two stated flaws most likely each cause underestimation of the relative benefit? Though even still it would be unclear how much benefit is just from socializing. Perhaps the control group should just be pure socializing.
I agree that both flaws make it difficult to have a positive trial. But both flaws together completely undermine the stated goal of the trial which was to test the efficacy of a structured lifestyle intervention in older at-risk adults. The study does not contain at-risk adults and the efficacy cannot be attributed to the structured intervention.
Thank you for the article.
I am not really sure how this is a major drawback. As others have commented , it makes little difference. If they had had a control group with zero intervention, the differences would have been even more impressive! In my opinion, it is better to have a control group with some intervention so it cancels out some of the motivational bias. There is very little motivation for participants to stay in a 2-year study or even perform better in cognitive post-testing when they know that they missed out on the intervention they were excited to be in the study for.
Also, the control group here has "minimal contact" (6 sessions in 2 years) which is probably nothing when compared to the intervention group, which has one of the most intensive regimens I have seen for a lifestyle RCT!
The inclusion criteria were sedentary (per the Modified Telephone Assessment of Physical Activity: less than 60 minutes per week at moderate intensity). So they are sedentary based on their self-reports, which is how PA guidelines are based on and used. So I don't see a major issue there.
One point I would make is cognitive tests based on computer/paper tests are just surrogate measures This doesn't mean anything meaningful to people. The question people care about is can they function better in their everyday life or is their quality of life better now. That should be the primary outcome here.
Risk factor analysis has an almost unbroken history of failure as an avenue of research to discover causation of disease. And that is with diseases that are fairly well defined and produce findings that can be measured with some degree of accuracy. Linking diseases with "lifestyle" factors that are logistically impossible to measure in any reliable way is, to quote a famous philosopher, nonsense upon stilts. To apply any of this to a disorder like dementia is stretching medical and scientific credibility well beyond any acceptable limits. Of course, great careers and pharmaceutical fortunes have been built upon these shaky foundations and will, no doubt, continue to do so as long as people take these sorts of things seriously.
Maybe POINTER was positive but pointless?
These would be fair points if the trial had been negative, but taken with a positive trial, they simply mean that the trial likely *understates* the magnitude of the effect. The first criticism (a "control" which is still a treatment) will serve to make the two arms more similar to each other, thereby reducing effect size and decreasing power (making it more likely to get a negative result). The second criticism (that the groups were already healthy at baseline) would also serve to minimize effect size, decreasing power and making it more likely to get a negative result.
If your point is that we can't use the estimated effect sizes here to generalize to a sicker population (and in reality the effect sizes, and therefore cost effectiveness, are even better than this trial makes it seem) then that is a fair point. But if you're asking us to throw the entire study out, then I think you're missing the fact that these methodological critiques are less pertinent to a positive study
I think that is totally fair and I've since come to appreciate that the results are still positive despite these concerns. But we already know that exercise and a good diet is good for brain health. What is not known is how best to design an intervention to improve these things in at-risk older adults. This trial misses an opportunity to help answer that question because of both flaws. No control means we can't know what exactly improved cognition. You could also imagine that what works for motivated healthy individuals could be different than unmotivated, unhealthy individuals. Thanks for the reply
Instructive - thanks. On a related but separate note... I'm wondering why blarcamesine is not getting more attention in AD. It has shown clear benefit to mild AD pts and is a small molecule with almost no side effects. Is it because it does not follow the amyloid dogma or is it because the results are not strong enough. To my eye, they are the same or better than the biogen drug.
With regards to the lack of a control group. Would there not be enough data on the expected rates of dimentia that you could be comparing to the poulation as a whole?
For a study purportedly investigating patients who are sedentary and with poor diet…the enrolled subjects did not seem very sedentary or eating particularly poorly…even at study outset. So the results would not seem generalizable to the study authors’ intended population.
And the participants also seemed very motivated to adhere to the program…which is also a trait that may lack generalizability, esp without a true control group as you’ve noted.
Wouldn't the two stated flaws most likely each cause underestimation of the relative benefit? Though even still it would be unclear how much benefit is just from socializing. Perhaps the control group should just be pure socializing.
I agree that both flaws make it difficult to have a positive trial. But both flaws together completely undermine the stated goal of the trial which was to test the efficacy of a structured lifestyle intervention in older at-risk adults. The study does not contain at-risk adults and the efficacy cannot be attributed to the structured intervention.