Kent Bottles: David Eddy Delivers Controversial Reinertsen Lecture About Future of Evidence-based Medicine
At ICSI, we pride ourselves on our evidence-based medicine guidelines. Starting in 1993 under the guidance of President Gordon Mosser, ICSI has developed a global reputation for excellence in providing guidelines that physicians follow and that help improve the care of patients. We proudly check the stats from www.icsi.org that show ICSI guidelines are being downloaded and used by clinicians from all over the world. We believe we know guidelines, and fervently believe in the work we do.
And yet at times, I personally have had nagging doubts about the whole guideline business. Our excellent guidelines almost always concentrate on best practices for treating one disease like diabetes. But patients with diabetes often come in with multiple conditions. Our guidelines by design tell us to treat patients with blood pressures of 145/80 and to not treat those with blood pressures of 139/80. But we all can identify patients with the latter reading who smoke, eat poorly, do not exercise, and have a family history of heart attacks and strokes. And we encounter patients with the higher blood pressure reading who have none of those risk factors. Does it make sense that our guidelines tell us to treat the lower risk patient and to not treat the higher risk patient? After all, the whole idea of guidelines is to prevent heart attacks and strokes.
When I asked Dr. David Eddy to deliver the James L. Reinertsen Lecture in Bloomington, MN, on November 19, I did not think that he would challenge us to think about the future of ICSI and of evidence-based medicine. I didn’t believe we would attract a crowd of more than 300 people, nor did I anticipate that some would leave the lecture angry that their conventional approach to treating patients had been forcefully called into question.
David Eddy has been controversial for a long time. At dinner after the lecture, he told me he went into a CT surgery residency at Stanford because in the 1970’s heart surgery was the “biggest hill I could climb,” it was the time of the first heart transplant in South Africa, and Dr. Norman Shumway at Stanford was famous. However, Eddy soon realized that doctors were “making important decisions without real scientific proof and in a way that I would not call rational.” Deciding that he wanted to bring logic and scientific rigor to the medical decision-making process, Eddy completed a PhD in applied mathematics at Stanford. The field of medicine would never be the same.
How can someone who states, “The problem is that we don’t know what we are doing,” not be controversial (http://www.businessweek.com/magazine/content/06_22/b3986001.htm)? Eddy became famous for showing that annual chest X-rays were not useful in screening for lung cancer, that preventing women from having a vaginal delivery if they had had a previous C-section was based on the recommendation of one doctor, and that Pap smear screening for cervical cancer did not have to be done every year.
In this year’s Reinertsen Lecture, Eddy, who is credited with coining the term “evidence-based guidelines,” presented an evidence-based medicine lecture in three parts: the past, the present, and the future.
For the past, Eddy recalled the time in the 1990s when President George Herbert Walker Bush was being evaluated for treatment of increased ocular pressure that could lead to glaucoma and blindness. The 1989 textbook said that the treatment cut-off is 30 mm Hg, but Eddy presented a slide with the actual studies and the results were all over the place. I remember the controversy among ophthalmologists when he presented a similar analysis years ago in JAMA. During that time, Dr. Eddy said he “felt like Salman Rushdie.” Stanford ophthalmologist Kuldev Singh stated: “Dr. Eddy challenged the community to prove that we actually had evidence. He did a service by stimulating clinical trials” (http://www.businessweek.com/magazine/content/06_22/b3986001.htm).
He also told the ICSI audience about an expert panel studying the problem of leakage of silicone breast implants; when Dr. Eddy asked the assembled experts to predict the likelihood of a patient experiencing such a rupture of the implant, the results ranged from 0% to 100%.
In addressing the present state of evidence-based medicine, Dr. Eddy outlined the same concerns that I listed in the second paragraph of this blog.
The real excitement of the evening occurred when Dr. Eddy presented his latest approach to the problem of lack of evidence for what we do clinically. With funding from Kaiser, Eddy founded a company called Archimedes to develop a SimCity-like world where researchers conduct trials of virtual patients to figure out the best treatments for real patients. With the expertise of a particle physicist named Len Schlessinger, the model writes mathematical equations to describe the complex interactions of physiology (the death of heart muscle after a heart attack and the effect of aspirin on the muscle) and what we know about the development of disease in different people (those with certain genes, those who smoke, exercise, and/or who meditate, etc). (Jennifer Kahn, The Body Synthetic, Wired, December 2009).
A health plan or a state like Minnesota could use this approach to develop personalized treatment plans that would identify the patients who could most benefit from a given medical treatment or intervention. Such an approach could also solve some of the problems we run into by regarding the clinical trial as the gold standard of medical research. Clinical trials are so expensive and so time-consuming that Eddy realized that they would never fully meet the needs of a system like Medicare which is scheduled to go bankrupt in eight years. If the Archimedes simulation approach works, we could perhaps have third party payers only reimburse for treatments that actually work and are based on science.
Eddy compared the computer simulation approach to the clinical trials approach by trying to predict the outcomes of the seven-year Collaborative Atorvastatin Diabetes Study (Cards). Of the four principal findings of the Cards trial, Archimedes had predicted two right, a third within the margin of error, and a fourth just below the margin of error. Eddy estimates that the computer simulation took a few months and cost 200th the cost of the Cards trial (Jennifer Kahn, The Body Synthetic, Wired, December 2009).
At the Reinertsen lecture, one physician asked why he should rely on computer simulation any more than he should rely on weather forecasts that often don’t accurately predict rain or snow in Minnesota. Zeke Emanuel, the bioethicist and President Obama health policy advisor, calls Archimedes “sophisticated but speculative. When you see the demo, there’s a certain ‘wow’ factor. And the fact that it has been able to predict some clinical trials is intriguing. But most of us would want to say, ‘OK – let’s try it on this problem, which isn’t one that you picked personally.’ Like any good presenter, presumably the results that Eddy shows are selective.”
David Nathan, director of Diabetes at Massachusetts General Hospital, says, “All the calculations happen inside a black box. And that’s a problem because there’s no way to tell whether the model’s underlying assumptions are right” (Jennifer Kahn, The Body Synthetic, Wired, December 2009).
When asked about these reservations in an email exchange with me, Dr. Eddy wrote: “Specifically, we did not hand pick the trials that we used to validate the model. They were picked for us by an independent panel convened by the American Diabetes Association. They supervised the entire exercise. All of the results, not just a few hand picked by us, were peer reviewed and published in Diabetes Care. So Zeke Emanuel was wrong. And as for David Nathan’s comment about black box, I’m not sure that he has ever been to one of our technical presentations. In any event, on request, we provide a detailed description of how the model works including how we derive equations. Anyone who is seriously interested in working with us and wants to know the details of the inside of the model can just ask and we will set up the meetings.”
It is also worth noting that many experts are impressed by the Archimedes model. Mark Roberts, chief of the Section of Decision Sciences and head of the Clinical Systems Modeling program at the University of Pittsburgh, has stated, “I’ve spent probably 25 years of my academic life trying to understand how to make these kinds of decision models be more clinically realistic. But David is so far ahead of anyone else in the field – it really is amazing” (Jennifer Kahn, The Body Synthetic, Wired, December 2009). Dr. William Herman, director of the Michigan Diabetes Research & Training Center, has a competing computer model, but he says, “Dr. Eddy is one of my heroes. He’s sort of the father of health economics”(http://www.businessweek.com/magazine/content/06_22/b3986001.htm).
At the Reinertsen lecture, Eddy admitted that any model will, like the weatherman, make mistakes. But he also noted that the real question is not whether or not the modeling is perfect, it is whether the model comes up with answers that are better than what we use today.
Eddy gives those of us in the evidence-based medicine world a lot to think about. By making us at ICSI question how we develop guidelines, he is challenging us to make sure that we stay on the cutting edge. He is also proving the point made by Tim Ferriss in a recent blog entitled “The Benefits of Pissing People Off” (http://www.fourhourworkweek.com/blog/2009/11/25/the-benefits-of-pissing-people-off/).
Ferriss references a Scott Boras mentor saying, “If you are really effective at what you do, 95% of the things said about you will be negative. Keep your head on straight, don’t get emotional, take the heat, and just make sure your clients are smiling.”
Colin Powell on leadership makes a similar point: “Trying to get everyone to like you is a sign of mediocrity: you’ll avoid the tough decisions, you’ll avoid confronting the people who need to be confronted, and you’ll avoid offering differential rewards based on differential performance because some people might get upset. Ironically, by procrastinating on the difficult choices, by not trying to get anyone mad, and by treating everyone equally ‘nicely’ regardless of their contributions, you’ll simply ensure that the only people you’ll wind up angering are the most creative and productive people in the organization.”
Entry filed under: Evidence-Based Medicine. Tags: .