This post introduces an occasional series, available to our paid subscribers, aimed at helping people improve their ability to read and interpret the medical literature. Improving Your Critical Appraisal Skills will be made up of “chapters” dealing with how to read articles about various study types. These chapters are adapted from my former course, Critical Appraisal of the Landmark Literature. That course was designed for senior medical students and was meant to improve their skills in using the medical literature to solve clinical problems and prepare them to learn from the cases and assimilate new knowledge from journal articles.
Improving Your Critical Appraisal Skills is meant for everyone: people who have just started reading the literature and those who are truly pros; people in the medical field or those currently on the receiving end of healthcare; and, especially, people just interested in understanding medical information. For readers without much experience, I hope the posts serve as a primer. For readers with more advanced skills, I hope they will be a refresher, a resource, and maybe present a new way of looking at a familiar topic. My primary goal is to help people provide and receive better care. A secondary, and more pithy, goal: given a statement, be able to identify the type of evidence that might support it and recognize potential shortcomings. This should enable you to argue with someone about the quality of the evidence given a statement but knowing nothing else. (Use this power cautiously!)
I’d love it if some of you will choose to submit additions to what I post, either to be included in the original post or posted as a supplement. Maybe this will turn into The Martian.
Some of the posts will be a bit longer than usual Sensible Medicine posts. I see them as posts that might be read over multiple settings or bookmarked and returned to.
There are many great resources for learning to read the medical literature. I will draw from many of them. My favorite, and the one that formed the basis of the course handouts 25 years ago is JAMA’s Users’ Guides, collected in the book Users' Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice. For people with no experience reading medical studies, I’d recommend starting with these.1 Another JAMA series that is an excellent resource, for those with more advanced knowledge, is the JAMA Guide to Statistics and Methods. I also hope that readers will be inspired to expand on what I present. I’ll be happy to post other people’s takes on the topics.
Evidence based medicine was originally, and most famously, defined as the process of integrating clinical experience and expertise with the best available evidence from systematic research. Practicing evidence-based medicine requires a strong foundation of medical knowledge, the ability to ask well-formulated questions, analyze evidence in the medical literature, and incorporate the answers to your questions into clinical decision making.
Mariana Barosa published this figure to describe EBM in her Sensible Medicine post.
Improving Your Critical Appraisal Skills will not be teaching EBM. It will peel off one aspect, critical appraisal, and aim to improve your skills there. I think this is the easiest aspect of EBM. The synthesis of evidence with biomedical knowledge, clinical experience, and patient preference is the true challenge to practicing good medicine and to making good decisions about your own healthcare.
So, let’s begin. The initial post, The Randomized Control Trial, will be in your inbox tomorrow.
Incorporating patient experience and preferences into the paradigm of EBM represents a significant challenge. Similarly, critically evaluating publications that take into account this element is crucial but also presents its own difficulties.
In this context, it is essential to recognize that EBM seeks to integrate the best available scientific evidence with clinical experience and patient values and preferences to make informed and personalized decisions in healthcare. However, achieving this integration effectively involves overcoming various obstacles.
On one hand, including patient experience and preferences requires a more holistic approach to data collection and analysis, which is often not adequately addressed in traditional research. Additionally, variability in individual preferences and values can hinder the generalization of study results to clinical practice.
On the other hand, critically evaluating publications in this context is challenged by the need to identify and assess the quality of evidence related to patient experience and preferences. This involves considering the validity and relevance of studies, as well as how qualitative and quantitative aspects of this information have been integrated into the interpretation of results.
Translating to English:
I am excited to start reading to them.
Dayami Martinez
There is a "CliffsNotes" approach to understanding a medical study/journal report. It is useful to read the Methods section in order to see how the patients were selected and how the end points are defined. Then go right to the Results section and look at the raw data and the percentage differences in the incidences for each end point. Ignore the statistical analysis with "p" values, confidence intervals, etc. Then use your common sense to decide whether the differences are of any practical significance. Always keep in mind the law of small numbers: small sample size and/or low incidence of end points renders differences in incidence far less reliable. Most people have an intuitive awareness of the law of large numbers but often fail to understand the opposite. This shortcut can give one an immediate sense of whether the study is worth the time and effort to analyze it in more detail.