Last week former FDA commissioner Dr. Robert Califf spoke in New York City at a conference co-organized by New York University and the New York Academy of Sciences, with funding from Johnson & Johnson. He gave a fascinating talk with an unexciting title: “Finding the Right Balance in Learning about Therapies.”
Califf recently began working with Google Alphabet’s life sciences company, Verily. On its website, Verily states, credibly given its relationship with Google, its “mission is to make the world's health data useful so that people enjoy healthier lives.” Califf said he’ll split his time between Verily and a new Center for Health Data Science at Duke University.
In a blog post last week, Califf reflected on the confluence of increased computing power and analytical capability which creates “a potential so enormous as to be almost beyond comprehension.” He highlighted disparities: “Our country is experiencing an unprecedented divergence of health outcomes that mirrors gaps in wealth and education…wealthy, highly educated people are benefiting from information that allows them to lead longer, more functional lives, while others are suffering.”
Califf hopes the data center at Duke will move beyond “academic analyses” to generate knowledge and tools for patients or consumers, and for health systems, to improve outcomes all around. This ambitious plan—essentially to use real-world data for developing tools to inform medical decisions—relates to what Califf spoke on at the NYU-NYAS conference.
Early in his keynote, Califf considered the issue of balancing patients’ need and want for speed and access to new drugs, vs. the need for evidence to support use of those drugs. “If acceleration is our only goal, we can just take all the useless stuff and throw it out there and hope for the best.” We also have to get it right, he said.
Califf considered whether randomized clinical trials are necessary, and the role of observational studies. “The answer is not either or, the answer is some combination,” he said, referring to those two broad types of data collection. “In the end, most people want to know is the risks and benefits of my treatment.” People especially want to know comparative risks and benefits, and if there are several options, how those stack up against each other.
“We can’t answer every question from an RCT,” Califf said. “There are times when observational analyses are quite reliable.” Statistical methods are improving almost daily, and sources of high-quality data are expanding. He referred to a “somewhat friendly battle” going on between traditional biostatisticians and others who cut information from machine learning, i.e. observational data analysis.
Califf showed a PowerPoint slide titled “My View,” and articulated key points. “The clinical trials enterprise has gone awry,” he said. “It’s become unnecessarily expensive, cumbersome and arcane,” he said. “It’s inarguable that the system has become so costly and onerous that most of the important questions go unasked.” Because clinical trials fail to generate needed answers, investigators try alternative approaches to collecting data, sometimes “out of desperation.”
Califf ventured into philosophy. There’s a schism between those who think that research should be part of clinical practice, and those who believe it should be separate. In his view, most questions can be answered by studies performed in context of real-world practice. Doctors have a fiduciary relationship to the well-being of the individual patient, he considered. He clarified what this means: that doctors’ actions always align with the patient’s interest (or should).
However, the current reality is that doctors increasingly belong to healthcare teams and provider networks. Doctors’ employers—healthcare systems—constrain options. They limit what doctors can say or advise. “To whom is the fiduciary relationship?” Califf asked. “Doctors are heavily conflicted between patients and the institutions they work for,” he said. “The rosy view that doctors and patients are discussing all options and making the best decisions flies in the face of all evidence.”
Too often, patients don’t receive optimal care. Many practice guidelines recommendations aren’t based on high-quality evidence, Califf said. In addition, there’s “highly significant” variation in practice; doctors in different locations make different recommendations for the same problem. Variations in practice, and recommendations unsupported by evidence, inevitably cause harm.
“Medicine is rapidly falling behind business in the accrual of high-quality evidence,” Califf said, mentioning his new gig at Verily. He went on to expound on ways by which information could be gleaned, at comparatively low cost, from real-world data. You can listen to the full talk here.
Some of his later points emphasized the value of data-sharing. “The secret sauce is the patients,” he said. Because it’s they who might drive investigators and health systems to stop hoarding data. He referred to PCORNet (the national Patient-Centered Clinical Research Network), which encompasses information from 122 million people, approximately a third of the U.S. population, through a network of networked electronic health records, from which huge amounts of medical information might be gleaned.
Right now, health data are being observed by businesses and networks, to assure that payments are correct, i.e. for financial purposes. That same data, as it gets better and better, could be used to improve health outcomes, Califf said. It’s possible to insert randomization into health systems for study of particular questions. These kinds of observations, with constant monitoring and interrogation of real-world data, is written into the 21st Century Cures Act, he said.
Real world evidence is not the opposite of RCTs, Califf said. There are two dimensions to consider, he explained: One is where you get the information, if it’s in the real world, such as electronic records, or data skimmed from questions people might ask “Alexa” in their homes, for instance. The other dimension is the method of study, i.e. how the investigator draws conclusions. The method might include randomization, or not.
When you do a Google search, you’re typically participating in at least 10 randomized trials, Califf said. Google figures out what’s better, what works, through constant analysis of consumers’ decisions and outcomes. This is being taught to MBA students. “But out medical students are pretty clueless about it because they’re learning other things,” he said. His point was that doctors could gain vast amounts of knowledge about medications, doses and outcomes for all kinds of medical conditions from information that already exists, apart from “trials,” in patients’ electronic health records and elsewhere.
There will always be a need for some randomized trials, Califf said. But the number and size of those trials can be reduced, to focus on questions that can’t be answered by large observational systems that sort through most pressing issues including, for instance, questions of dosing that are rarely addressed by RCTs. He mentioned how despite so many years of research and trials in cardiology, the optimal dose of aspirin–325mg or 81mg (“baby aspirin”) remains a mystery. In his new position, he plans to study that persisting question that affects millions.
In healthcare, cultural beliefs, social mores and academic goals tend to hinder progress, Califf said. “The New Einsteins Will be Scientists Who Share,” headlined a slide. “We need to get over attitudes that delay sharing, in a way that’s respectful,” he said. “The number one culprit is academic centers.” Hubris is a cause of data hoarding, he said in response to a question.
Someone asked about privacy concerns, with so much data-sharing and observation of patients’ records. “I struggle with HIPPA,” Califf said. Some concerns are legitimate. But “90% of it is like hiding under your mother’s skirt.” Perhaps he meant that fear—and being too hesitant, not wanting “to look”—play an inordinate role in doctors not seeing, and not knowing, medical information that’s already out there.
The next morning, Dr. Janet Woodcock, who currently directs the FDA’s Center for Drug Evaluation and Research, gave a fantastic talk on innovative clinical trial designs. I will write on that, and more on the conference, soon.
