Failure to Protect?
Juan Celedón, a respected pulmonary researcher at the University of Pittsburgh, wanted to address an urgent national problem. Severe asthma attacks send hundreds of thousands of U.S. children to the hospital every year. For decades, researchers have suspected extra vitamin D—essential for bone growth and healthy development, and also an immune modulator in children and adults—might help them. In 2016, Celedón and colleagues launched a major trial to test that premise.
With $4.3 million in funding from the National Heart, Lung, and Blood Institute (NHLBI) and support from a vitamin company and a drug firm, they enrolled asthmatic kids who had low or deficient levels of vitamin D—many from urban, minority communities; most were Black. Half of the 400 planned participants would receive a daily high-dose vitamin D supplement for about 1 year. The other half would serve as controls. The researchers also made a decision that cast a shadow over the trial—and inflamed a controversy confronting many other trials of similar design. Instead of treating the randomly chosen control group with a more modest dose of vitamin D—as many medical authorities advise for children with a deficiency of the vitamin—the researchers chose to give them a placebo.
When Seattle physician Bruce Davidson, a pulmonary specialist who has studied vitamin D and asthma, heard about the “Vit-D-Kids” trial, he was stunned. The children, as asthmatics treated with corticosteroids, already faced possible bone health problems and diminished growth, and any vitamin D deficiency would place them at greater risk for fractures. Yet Davidson discovered that informed consent forms posted online failed to inform parents of enrolled children about those dangers.
Davidson, who had worked on a comparable vitamin D trial that rejected a placebo arm as unethical, repeatedly voiced his concerns about Vit-D-Kids to the scientists running it, institutional review boards (IRBs) that approved it, and NHLBI. But trial researchers called the risks minimal. The placebo was justified because vitamin D testing is not routine, they argued. If not for the trial, the kids’ vitamin deficiency probably wouldn’t have been detected, so they were no worse off in the study.
Davidson’s objections drew some media attention during the trial and led to small changes to its enrollment criteria and consent forms, but Vit-D-Kids pressed ahead. It wasn’t a success. Trial enrollment grew to nearly 200 children but was halted early for “futility” in 2019 after a data safety monitoring board (DSMB) concluded during an interim review of results that vitamin D supplementation had failed to prevent asthma attacks. Yet the researchers kept an unspecified number of kids, even if very deficient in the vitamin, on a placebo for up to six more months—to ensure “an orderly closeout,” James Kiley, director of NHLBI’s Division of Lung Diseases, later told a U.S. politician who asked about the decision.
“That approach was stunning and callous,” Davidson says. And possibly harmful. At least nine kids, across both arms of the trial, broke bones during the trial—nearly double the number expected among asthmatic children over a comparable period. The fractures were neither disclosed as possible adverse events when the study was published in JAMA last year nor noted in another public summary of the trial results.
A Science investigation of Vit-D-Kids reviewed thousands of pages of trial protocols and consent forms; previously undisclosed DSMB deliberations; emails from the trial’s principal investigator, NHLBI officials, and JAMA editors; and letters between National Institutes of Health (NIH) officials and a concerned member of Congress. Those documents and others reveal new aspects of the trial that concern asthma researchers like Davidson, medical ethicists, and specialists in clinical trial design who reviewed the materials at Science’s request.
“The advantages to society of this trial, because of the poor design, were likely none. And the risks did outweigh the benefits,” says Charles Natanson, a physician and expert on trial design at the NIH Clinical Center. “This trial did not, in my opinion, meet the standards set forth in The Belmont Report,” which in 1979 established ethical guidelines, adopted by the U.S. government, for protecting human subjects. Keeping children on a placebo after the trial was stopped for futility stood out as an “unconscionable” error, adds Ruth Macklin, a medical ethicist at Albert Einstein College of Medicine.
Davidson and others suggest the study’s focus on minority children—which Kiley called “appropriate” in a statement to Science—only elevates their concerns about using a placebo. Jill Fisher of the University of North Carolina, Chapel Hill, who wrote a book on racial inequality in clinical trials, says Vit-D-Kids ignored the ethical imperative to treat children at known risk from vitamin D deficiency because of inadequate diet, poverty, and a lack of Sun exposure in inner cities. “We shouldn’t say, ‘It’s unfortunate that there are these health and nutritional disparities, but it serves the interests of science to have placebo-controlled trials,’” she says. It is “structural racism” to scientifically exploit such inequalities, Fisher adds.
Kiley and Celedón declined to be interviewed but provided statements after being sent a list of questions. The trial, its protocol, and consent forms “underwent rigorous review before and after it was funded,” Celedón wrote in a brief note emailed to Science, adding: “All regulatory bodies, including a [DSMB] and [IRBs] at seven pediatric hospitals, stated that the trial was both safe and ethical.” In NHLBI’s statement to Science, Kiley wrote, “The highest priority was to keep every child in the study safe.”
Vit-D-Kids could easily be dismissed as a controversial outlier, but Natanson and others suggest it instead exemplifies the growing number of studies in humans that inappropriately reject control groups receiving “usual care”—current best practice treatments used by doctors. In a hunt for compelling results, many researchers favor using sharply divergent treatment arms in a trial. But such extreme comparisons mean doctors can’t learn whether a new treatment is better or worse than usual care, says Natanson, who has analyzed the issue in trials involving critically ill patients. He has found that many such trials mislabel one or more arms as “usual care,” sometimes endangering participants and misinforming physicians, which he calls “a big problem.”
MORE THAN A DECADE AGO, NIH supported an ambitious trial that foreshadowed the usual-care issue in Vit-D-Kids. The study’s researchers meant to solve a life-and-death medical conundrum affecting premature infants: They depend on supplemental oxygen to stay alive, but too much can cause blindness. The trial aimed to identify an oxygen level that would save lives with the fewest side effects.
It assigned more than 1300 preemies to be maintained at a relatively low blood oxygen range (85% to 89%) or a higher range (91% to 95%). Most consent forms said either range represented “usual” or “standard” care and that babies in both groups had the same likelihood of dying. Prominent supporters, including NIH Director Francis Collins, argued that both infant groups met the standard of care as practiced at trial sites.
But in a 2016 analysis in PLOS ONE, whose authors included Natanson, researchers examined other trials of oxygen management in preemies and concluded the bottom of the trial’s low-oxygen range was not considered usual care in multiple countries. The trial departed from usual care in another respect, not noted on the consent forms: By design, the oxygen monitors displayed inaccurate readings to prevent caregivers from knowing a baby’s study group.
Scores of bioethicists and clinicians—and the federal Office for Human Research Protections (OHRP)—said the informed consent forms inadequately described the risks. About 20% of babies in the low-oxygen range died, compared with 16% in the high-oxygen group.
Several other U.S. trials (see sidebar, below) also became magnets for criticism that they violated usual-care safeguards seen as crucial, even if sometimes complex to define. Each trial had eminent backers who said accepted practices can be ambiguous, fueling divisive debates. “Usual care in Seattle may differ from usual care in San Antonio. This applies particularly to uses of technology and high-cost interventions,” said Edward Campion, an editor at The New England Journal of Medicine when it published the infant oxygen study, at a public hearing.
Those issues came to a head again in Vit-D-Kids. Vitamin D supplementation has long been contentious. It is said to be a remedy for diabetes, cancer, heart disease, and other ailments, but clinical trials often failed to show such benefits, particularly for the high-dose pills touted by the supplement industry. How much vitamin D a growing child needs also is debated, but most authorities say kids need levels in the blood of 20 to 29 nanograms per milliliter (ng/ml) to minimize risks of bone fractures and impaired immune response, and to protect lifelong skeletal health. Guidelines from the American Academy of Pediatrics and Pediatric Endocrine Society call anything below that threshold “deficient” or “insufficient” and recommend supplements. The Vit- D-Kids protocol also cites an Institute of Medicine report that agrees with those assessments. And NIH labels 20 ng/ml “inadequate” and below 12 ng/ml “deficient.”
With sites at big-city hospitals, Vit-D-Kids originally recruited asthmatic kids, ages 6 to 16, who had vitamin D levels between 10 and 29 ng/ml. Many kids were below 20 ng/ml—levels the study itself, in its protocol, deemed “deficient” and inadequate for musculoskeletal health. Yet that protocol, posted publicly online and included with the JAMA publication, also described the risk of leaving those children untreated as “no greater than encountered in daily life by healthy community-dwelling children.” But the kids participating in the study, afflicted with asthma and receiving powerful steroid drugs to treat it, were far from healthy.
In his statement to Science, Kiley defended Vit-D-Kids by saying vitamin D–deficient children were excluded in favor of those with “vitamin D levels in the low to low-normal range, who normally would not be treated with supplemental vitamin D.” He cited a 2016 global consensus report on rickets, a condition linked to vitamin D deficiency that causes soft bones and bow legs. Yet even the rickets report defined vitamin D levels of 12 to 20 ng/ml as “insufficient.”
Vit-D-Kids compared inadequate treatment and overtreatment, according to Natanson. Kids in the treatment arm were given daily vitamin D supplements of 4000 international units, or seven times their recommended daily allowance, and some reached serum levels above 100 ng/ml. NIH guidelines say anything above 50 ng/ml is potentially hazardous; studies have suggested such levels encourage some cancers and cardiovascular problems or increase risk of death overall.
The trial protocol noted the high-dose supplement was tested against a placebo to avoid a “false negative” outcome. “They wanted to maximize their chances of finding a difference,” says physician Michael Carome, a former top regulator at OHRP who directs health research for the consumer advocacy group Public Citizen.
Science also reviewed versions of the informed consent forms used by Vit-D-Kids, some of which Davidson acquired through a Freedom of Information Act (FOIA) request. Experts in trial design say those forms stressed potential benefits over harms and were overly complex and confusing.
For example, instead of discussing vitamin D deficiency, the forms used the more benign-sounding term “low vitamin D,” says Columbia University cardiologist Raymond Givens, who studies institutional racism in medicine and medical publishing. Ethicist Harriet Washington, whose book Medical Apartheid discusses clinical experiments on Black Americans, also notes parents often misunderstand essential research terms.
The informed consent forms for Vit-D-Kids called rickets, not bone fractures, the primary risk for children who received placebos. But rickets affects infants and very young children—far younger than those enrolled in Vit-D-Kids. The consent form should have clearly stated that any child in the placebo group with insufficient vitamin D would face a higher risk of broken bones, says Boston University medical ethicist George Annas. But if such an explicit concern had been noted, he suspects few parents would have signed the consent form because “it almost sounds like child abuse.”
In his statement, Kiley wrote to Science that the fracture risk from too little vitamin D didn’t apply to the trial’s kids because they used only inhaled steroids, not injected or oral forms, which in adults cause bone to demineralize. But up to 14 children on the placebo took systemic steroids at least twice during the trial—enough to increase the fracture risk, according to an authoritative asthma study.
Science also reviewed versions of the informed consent forms used by Vit-D-Kids, some of which Davidson acquired through a Freedom of Information Act (FOIA) request. Experts in trial design say those forms stressed potential benefits over harms and were overly complex and confusing.
For example, instead of discussing vitamin D deficiency, the forms used the more benign-sounding term “low vitamin D,” says Columbia University cardiologist Raymond Givens, who studies institutional racism in medicine and medical publishing. Ethicist Harriet Washington, whose book Medical Apartheid discusses clinical experiments on Black Americans, also notes parents often misunderstand essential research terms.
The informed consent forms for Vit-D-Kids called rickets, not bone fractures, the primary risk for children who received placebos. But rickets affects infants and very young children—far younger than those enrolled in Vit-D-Kids. The consent form should have clearly stated that any child in the placebo group with insufficient vitamin D would face a higher risk of broken bones, says Boston University medical ethicist George Annas. But if such an explicit concern had been noted, he suspects few parents would have signed the consent form because “it almost sounds like child abuse.”
In his statement, Kiley wrote to Science that the fracture risk from too little vitamin D didn’t apply to the trial’s kids because they used only inhaled steroids, not injected or oral forms, which in adults cause bone to demineralize. But up to 14 children on the placebo took systemic steroids at least twice during the trial—enough to increase the fracture risk, according to an authoritative asthma study.
AFTER LEARNING MANY DETAILS of Vit-D-Kids, Davidson in August 2017 sought advice from Frank Greer, an emeritus professor of pediatrics at the University of Wisconsin, Madison, and member of the Vit-D-Kids DSMB. In an email Davidson gave to Science, Greer expressed alarm about giving a placebo and concerns about possible adverse effects in the high-dose group. (NHLBI recused Greer from discussions of the matter after learning of Davidson’s communication with him.)
Davidson then complained to NHLBI officials. “[Davidson] makes very valid points and … [the investigators] need to address this in a very substantive way,” Kiley subsequently wrote to colleagues in an email Davidson obtained via a FOIA request. “I am inclined to put a clinical hold on this study if we cannot get [a DSMB] review done next week.”
After months of deliberation, the DSMB in early 2018 approved changes to the trial, excluding any future enrollees with vitamin D levels below 14 ng/ml, compared with the prior cutoff of 10 ng/ml, and adding new wording to the consent form. The revised version said, “The risk to bone health is unclear if the vitamin D level is 14–19 ng/ml. However, many doctors would treat with vitamin D at this level.” The board acted “out of an abundance of caution,” Kiley wrote in his recent statement.
Carome calls the revisions a tacit acknowledgment that the consent form used to recruit many of the kids “failed to disclose important information that parents would have needed to make a fully informed decision.” He says the new form continued to obfuscate risk by implying that treating vitamin D deficiency was a gray area. Moreover, parents were never told what their children’s specific vitamin D levels were, either at enrollment or later in the trial. Care decisions, including whether to supplement a kid’s vitamin D on their own, were effectively out of their hands.
WHEN JAMA PUBLISHED THE RESULTS of Vit-D-Kids in August 2020, it looked like just another failed vitamin supplement trial. The placebo group and treatment groups experienced about the same number of adverse events—mainly asthma-triggered hospitalizations or emergency department visits, according to the paper. JAMA had in 2018 rejected a commentary Davidson submitted criticizing the trial’s use of a placebo arm; after the study’s publication, he sent a letter to the editor outlining that concern and questions about the trial’s racial mix. JAMA rejected that as well.
JAMA’s editors declined interviews, but wrote in a statement they were “aware” of Davidson’s concerns and that the study was designed to “ensure the safety” of participants. They noted the paper reported that trial investigators stopped giving placebos to several kids, and referred them to an endocrinologist, after their vitamin D levels dropped below the study’s minimum requirement.
But Celedón and colleagues did not report in the JAMA paper, or in results posted to ClinicalTrials.gov, that kids in the trial broke bones. Kiley disclosed that at least nine fractures had occurred only when Representative Lloyd Doggett (D–TX), who chairs an NIH oversight panel and had been contacted by Davidson, asked about the trial.
Five fractures had occurred among kids given vitamin D and four in the placebo group, which is nearly twice the rate expected for asthma sufferers in that age group. But Kiley told Doggett that the trial’s oversight board found no safety issues. (In an interview, DSMB chair Dennis Ownby of Augusta University said his panel was told that the rate of fractures was very low, but does not recall independently verifying that information.) Kiley refused to share details of those fractures with Doggett for unspecified “reasons of patient privacy and scientific integrity.”
Davidson, Natanson, and other trial design experts Science contacted were troubled that Vit-D-Kids failed to report the fractures and their details publicly. Although the comparable number of fractures in each arm might suggest the placebo group was not at greater risk, they note the breaks are impossible to interpret without information on their severity and precipitating events, such as contact sports versus low-stress activities.
“Letting kids spend 48 weeks as low as 10 ng/ml—and doing this primarily to minority kids while warning them about irrelevant rickets instead, then covering up bone fractures—is awful,” Davidson says. “Those kids with the lower vitamin D levels need to be located, carefully assessed, and treated if necessary. They need explanations and oversight for a while to minimize their future risk.”
THE MAKEUP OF THE 192 TRIAL PARTICIPANTS, which included 100 Black kids, intensified the debate. Kiley in his statement defends the trial’s demographics, noting that Black children “are twice as likely as white children to be affected by asthma.” Yet even with higher asthma rates, Black children were vastly overrepresented in the trial, in comparison with their numbers at study sites like Pittsburgh and Boston.
That concerns Givens and others. “[If ] most of [a trial’s] subjects will be nonwhite, and a large proportion low-income and perhaps lacking advanced education among parents, there is a need for heightened attention to the ethics and appropriateness of the trial,” he says, including intensive community outreach before recruitment in poor or minority populations.
Macklin calls it “surprising—if not appalling—that IRBs in these major U.S. medical centers are willing to approve studies inwhich disadvantaged children are randomly assigned to a placebo group, justified by the argument that they are not being made worse off than they would be if not enrolled in the study.” Annas compares Vit-D-Kids to “no-care as usual care” trials in resource-poor nations, such as NIH-funded trials in African nations in the 1990s. In pregnant women, the antiviral zidovudine, also known as AZT, was tested against a placebo to see whether it blocked mother-to-child HIV transmission. The drug was already the standard of care during pregnancy for HIV-infected women elsewhere.
Washington said she was dismayed to learn the trial protocol and consent forms prominently discussed analyzing kids’ genes for clues connecting low vitamin D to asthma but glossed over obvious inner-city risk factors such as air pollution, stress, and poverty, which could also lead to vitamin deficiency and asthma. The hunt for genetic explanations above social linkages “reinforces the belief that … biological dimorphism drives a lot of illnesses,” she says. “It’s unethical. It’s deeply illogical. And it fits a racist pattern.”
Kiley and Celedón declined to comment on that issue. Ownby says he also sees asthma as a disease “primarily of lower socioeconomic status,” closely linked to disadvantages experienced by Black people and other people of color and to the social issues Washington notes. But he adds that “to try to look at them all at once just isn’t within the scope of most NIH grant studies. There’s just not enough money.” Studying genetic questions, particularly for those most affected by asthma, is also important and ethical, Ownby says.
To Carome, Vit-D-Kids offers new evidence that, overall, “our IRB system is broken.” He doubts that any hospital panel that green-lit the study asked how its own physicians would normally treat asthmatic children deficient in vitamin D. None would fail to give baseline supplements, he says. It’s a sign that IRBs tend to uncritically accept NIH funding as a stamp of approval, Carome adds.
Critics of Vit-D-Kids say the study, though an extreme case, also points to troubling trends amid an explosion of comparative effectiveness research (CER) trials, which examine the benefits and harms of treatments. U.S. funding for CER clinical trials and related trial support rocketed from an annual average of about $34 million between 2009 and 2011 to $284 million since then—largely because of the government-created nonprofit Patient-Centered Outcomes Research Institute (PCORI), which specializes in assessing treatments side by side.
Meanwhile, criticisms of such studies have mounted. In 2018 Public Citizen challenged what it called “reckless” flaws in an NIH-backed study of treatments for septic shock, a life-threatening effect of infection. The group said the trial randomly assigned subjects to combinations of fluids and drugs in ways that departed sharply from usual care tailored to each patient’s condition. In a public statement, Carome called patients in the ongoing trial “unwitting guinea pigs in a physiology experiment that will not advance medical care for sepsis and likely will harm many.”
Trial organizers challenged that verdict but last year OHRP and NHLBI forced changes in the trial’s protocol to allow individualized care—improvements Public Citizen commended but called insufficient. (At the time, NIH barred Natanson and another NIH scientist from commenting on the sepsis trial.)
Natanson recently analyzed CER trials of critically ill people published over 1 year in three “high-impact journals with a reputation for a rigorous review process—the best clinical trial journals in the country.” The yet-unpublished study examined roughly 50 trials, identifying those that improperly excluded a usual-care arm. He estimates that “up to half of the [CER] trials done in critically ill subjects have this problem.” A huge proportion of trials of nonemergency interventions, like Vit-D-Kids, also excludes a usual-care arm, including more than 70% of PCORI’s CER trials. Natanson says scientific results, patient safety, and the informed consent process all suffer when a usual-care comparator is absent. But he acknowledges that trial funders and IRBs struggle when no bright line differentiates experimental interventions from usual care. Making those distinctions can require laborious observational studies and surveys.
PCORI itself says usual-care comparators are often important or necessary. But given frequent challenges in defining standard practice, the group actively discourages usual-care arms—“unless they represent legitimate and coherent clinical options.” That’s an abdication, Natanson says. “It’s much easier to say, ‘I have two ideas, two strategies.’ … It’s much more difficult to say, ‘What is usual care? How is it practiced? How can I design the trial so that … at least one arm is receiving usual care?’”
A real-world benchmark can be essential to evidence-based medicine—whether a trial examines oxygen levels for preemies or vitamin D for asthmatic kids, Natanson adds. “It’s just common sense. Why study two things inside of a trial that nobody does outside of the trial?”
This story was supported by the Science Fund for Investigative Reporting.
Charles Piller is a contributing correspondent based in Oakland, California.
Published by AAAS, the world’s largest general science membership society, the Science family of journals continues to set the standard for original research and invaluable content. So when you become a AAAS member, you’re not just getting a magazine about science—you’re becoming part of the community that’s driving scientific progress. Get the very best research from around the world delivered right to you weekly.
Become a AAAS Member today and receive:
- Breaking news from Science
- Weekly Policy Alerts impacting STEM
- Exclusive member-focused content
- Unlimited access to our online Member Community
- Discounts through partner organizations like Apple, Avis, and Subaru
- And much more!
Your membership also helps AAAS educate and communicate with policymakers at local, state, and national levels, bringing evidence-based research to bear on critical issues such as climate change, STEM education, and public funding of science research.