clot-busting drugs to monoclonal antibodies to convalescent plasma donated by patients who have recovered—not to mention a growing number of vaccines. But with speed borne of desperation comes risk and confusion—of trials too small to yield answers, of treatments overhyped, and of uncertainty about how to design the best studies possible.The novel coronavirus has upended the world of clinical research, with scientists under pressure to identify effective treatments for COVID-19 and vaccines to prevent new infections. More than 22,000 papers on the virus have been published this year, and more than 2000 trials are underway. They are testing everything from
Science spoke with three clinical trial experts about this unprecedented period for modern medical research.
Kathryn Edwards is a pediatric infectious disease specialist at Vanderbilt University Medical Center and scientific director of the Vanderbilt Vaccine Research Program. She has investigated the safety and efficacy of many vaccines in adults and children and is helping monitor the safety of trials of vaccines for COVID-19. Anthony Gordon is a critical care physician at Imperial College London and St. Mary’s Hospital. He is the U.K. chief investigator of the REMAP-CAP trial, an international effort to test five treatment strategies in critically ill COVID-19 patients. The trial is adaptive, meaning that over time, as evidence builds from the trial, more patients are allocated to therapies that appear most promising. Neil Schluger is chief of the division of pulmonary, allergy, and critical care medicine at Columbia University Medical Center. He recently published an analysis of hydroxychloroquine use in COVID-19 patients treated at the hospital.
Science’s conversation with them has been edited for brevity and clarity.
Q: Is there anything special about running clinical trials during a pandemic?
Anthony Gordon: What’s different for us is the speed at which we’ve been working. I’ve run trials before. The largest I’ve run is 35 centers. Here we were given extra funding by our big U.K. funder, the NIHR [National Institute for Health Research]. They said to me, “We want you to move fast and go to every ICU [intensive care unit] in the country.” We haven’t managed every ICU but we’ve gone to 130 so far. It shows what can be done when there’s a will, and a lot of bureaucracy can be done away with. It’s vital to deliver answers. Some of me thinks if only we could do this more in clinical research, we would get a lot more answers a lot more quickly.
Neil Schluger: Institutional review boards, which evaluate ethics and other aspects of proposed clinical trials, started meeting essentially every day. Trials got approved quite quickly. But New York had the sad distinction of being the epicenter for a time. The first month or 6 weeks, we were just trying to keep the hospital functioning and take care of the patients. It was difficult to think about getting clinical trials going. We had to really reinvent what the hospital was every day for several weeks running.
In the U.S., I think what you’ve had is a tremendous duplication of clinical trials. I don’t think we were as ready to go with networks of hospitals. Many clinical trials at individual sites are going to be underpowered. There are probably 50 hospitals studying convalescent plasma. No one’s going to wind up enrolling enough patients in any one of those studies to be able to say anything.
A.G.: If I can come back on that, that coordinated approach was definitely something that the Department of Health and Social Care and the NIHR in the U.K. took. They said, “We don’t want duplication.” It led to some unhappy investigators who were told their work wasn’t going to be prioritized. For convalescent plasma, there are two trials—one for patients as they come through the door of the hospital and one for the critically ill.
In the U.K., most non–COVID-19 clinical research stopped. The surgical trials, the cardiology trials, all paused. Those researchers were reallocated to deal with COVID research. That allowed us to roll out to so many sites.
Kathryn Edwards: One of the challenges for vaccine trials has been we don’t totally understand the immune response or the inflammatory response. We don’t know what a good protective immune response might be.
What is our definition of success? Are we going to try to prevent infection, which we all know is harder to do than to prevent disease? How are we going to define disease? Will it be more severe disease, or will it be mild disease?
It generally takes about 10 years to get a vaccine. We’re trying to do this in 12 months.
Q: One thing I’m hearing you all say is, “We’re trying to do fast science, but also good science.” That’s obviously difficult. Should we be rethinking how we design trials and generate evidence?
N.S.: We shouldn’t lower our standards for scientific evidence. Many physicians felt that they were seeing syndromes in COVID-19 that they understood, and so they knew what the treatment should be. And they were unwilling, I think, initially to put patients into clinical trials. Experience has shown us that that turned out to be a mistake. We’ve learned the hard way. How many hundreds of thousands of people received hydroxychloroquine or chloroquine? And now we have evidence that those don’t seem to do much.
When you have a disease for which there is no treatment, a placebo-controlled trial is the most efficient first step. We should be able to take advantage of advances in thinking about trial design, but it’s no time to go soft on the rigor.
A.G.: I totally agree. People said, “Surely you’ve got to have an antiviral. It’s a virus and it’s killing people.” I would say, “No, we don’t know any of them work.” But what I do think you can still do is be adaptive. Adaptive trial designs maintain rigor.
We can’t necessarily do things in the traditional way, A versus B, get to the end, then make a decision. We’ve got to be able to move more quickly. That is about adapting but still maintaining rigorous science.
Within our REMAP-CAP trial, we are testing multiple interventions. We also built adaptive randomization into the trial: We started with equal randomization to all treatment arms, but if something is having a better outcome according to predefined statistical rules, the computer program would weight randomization in favor of the drugs that were looking better. Patients will more likely get the better treatment as the trial goes along.
K.E.: As scientists, there are so many pressures and there are so many people who don’t really respect what we have to say. That makes it terribly imperative that what we do is really, really good—that we have our research endpoints clearly defined, and that they’re important endpoints, not just one day less in the hospital.
We have to do our best science to make sure that we answer the questions as definitively as possible. But sometimes it’s very hard to do that.
N.S.: The volume that’s been published is just amazing. Some of the posted preprints about clinical research have turned into very good publications, but many of them haven’t. But they get picked up everywhere, reported, and tweeted about, and spread around social media.
If you type the word COVID into PubMed, you get more than 22,000 citations. They can’t all be good articles. Journals have rushed a lot of things into print and probably have lowered their standards to a certain degree. Scientists themselves have overhyped their research.
Q: When you talk about overhyping, are there examples you’re thinking of?
K.E.: With the Moderna vaccine trial, there was a press release that the vaccine was very well tolerated. And then the same day I hear that one of the vaccine recipients had fevers to 103°[F] and passed out.
A.G.: It’s not a specific example, because there are multiples. For me, it’s people publishing what I would call a small case series. They have patients they’ve treated and they say, “We compared them to a matched controlled population that was treated at a different time.” I would say it’s an uncontrolled study and it’s being passed off as a clinical trial and published in higher impact journals than this would normally have been published in, and therefore attracts the headlines. A couple of papers have been withdrawn.
But when you’re talking to patients and families, they say, “But I read last week this drug works so why aren’t you just giving it to all of us?”
Q: The trial registry ClinicalTrials.gov lists 2000 COVID-19 trials. That’s mind-bending. On the one hand, that’s far too many. On the other, what the United Kingdom has done is pick just a few treatments to test and leave out a lot of potential approaches. How do we decide how many strategies to test?
K.E.: We need to work together as scientists to really understand the range of questions that need to be addressed. But we need to not do it in such a dilute way that none of us get any answers. Maybe we focus on several important questions in several important areas.
A.G.: It is better to provide a few clear answers than no answers to lots of things. If we all do our own little thing, we’ll have investigated everything and we’ll still have no answers.
N.S.: Coming out of SARS [severe acute respiratory syndrome, in 2002–03] and Ebola [in 2013–16], lessons were learned, including the importance of a very coordinated research approach that can be put into place quickly. I think the lack of a strong central coordinated effort in the United States is something we’ll have to grapple with coming out of this.
K.E.: I want to mention a bit about the international scene. I have been impressed with CEPI, the Coalition for Epidemic Preparedness Innovations, which has begun to look at a number of COVID vaccines in developing countries. There’s a common protocol for clinical studies. CEPI is beginning those trials of COVID-19 vaccines very soon. That has been better than what we saw with clinical trials during the West African Ebola outbreak.
Q: Another area where it seems there could be change is in racial disparities in clinical research. Trials typically include few people of color, and the benefits of trials often don’t flow to them either. Black Americans and people of African descent in other countries are being hit disproportionately by COVID-19. Should trials focus on recruiting those patients? How do we ensure they’re not left out of clinical research?
K.E.: We need to make sure that the people we train as clinical investigators and translational scientists are from groups that that have been traditionally underrepresented. When young investigators look like the people they’re enrolling, it really clicks for those study volunteers, and they are happy to participate. They might relate more to those scientists than an old gray-headed lady like me.
N.S.: Interestingly, for us in New York, the disparities of who enrolled in COVID-19 trials were less of an issue than usual. Our hospital serves a poor neighborhood, and it was full of people from the neighborhood. That’s who got enrolled in the trials we were doing.
I also think it goes beyond the United States. This a global effort. I’ve been working in Ethiopia for the past 9 years, and the research capability there and in many other countries in the global south is pretty thin. That’s a huge gap that needs to be filled. We also will need to account for people in those parts of the world when these new therapies are developed. How are drugs and vaccines going to get distributed? The benefits of research can’t just flow to the resource-rich settings.
Q: I’ve been thinking about how we judge efficacy in these trials of vaccines or treatments. People are desperate for something that works. What if you have an antiviral like remdesivir, which shortens hospitalization by 4 days? What if we end up with a vaccine that protects people for 6 months?
K.E.: Those issues are going to be really, really tough to address. There will be pressures on regulators. Will you allow a vaccine that’s 30% effective to be licensed and given to people? It’s really important to set up the trials with endpoints that are meaningful.
A.G.: The other thing that’s important is the patient’s voice. Right now, many of us are trying to prevent people dying. But quality of life is something we need to assess as patients recover. It’s harder in COVID time, but I think still important.
Q: Dr. Schluger, you were a young doctor in New York City during the early days of the HIV epidemic, when physicians had little to offer. Here, too, physicians are overwhelmed and may throw everything at their patients to try to help. What happens if a treatment becomes standard before being fully vetted? What’s fair to expect of physicians on the front lines right now?
N.S.: The history of medicine is littered with good intentions that didn’t work out well. But when our hospital was overwhelmed with COVID patients, it was very difficult to just give supportive care. People were dying at such an incredible rate. So I understand the impulse: Don’t just stand there, do something. But there’s no substitute for well-done trials.
Many hospitals here in the states put forth guidelines that, if they didn’t encourage use of hydroxychloroquine, they certainly allowed it. We now have a fair amount of pretty good evidence that hydroxychloroquine doesn’t seem to do much.
A.G.: Our chief medical officer in the U.K. put out advice that they did not want patients treated with novel therapies outside of trials. You couldn’t get hydroxychloroquine unless you were in a trial.
Q: It feels like it’s been a lifetime, but we’re only 5 months into this pandemic. We surely have a long way to go. What can clinical researchers do better?
K.E.: We need to make sure that that there will be funding to do the science. There should not be undue political pressure on the regulators and the scientists and the public health officials to make them deviate from good science.
N.S.: The scientific community also has to do a better job of communicating the importance of research to the general public. Coronavirus is not something that can be insulted or intimidated or humiliated into disappearing. I’ve been disappointed in the organized scientific community in the United States for its lack of willingness to really stand up. Professional organizations and societies haven’t been very vocal at all.
We should take advantage of what, at least in the states in many places, is a relative lull in cases right now. We should be getting together and saying, “OK, here’s the state of knowledge, what are our priorities?” And then figure out how to organize networks that are ready to go to answer these questions. There’s been relatively little done in mildly ill patients. Maybe a lot can be learned there.
A.G.: I’m not an epidemiologist, but from my simple reading, there’s always a second wave. Let’s use this time to prepare for what is going to come.