Build a Better Study
Traditionally, sponsors of clinical trials have set forth all aspects of the trial before the study begins in a clinical trial protocol. These traditional protocols outline in extensive detail the study objectives, design, methodologies, and more. Once the study begins, no deviations from the protocol can be made. The primary reason for such rigid protocols is to ensure the statistical integrity of the data collected and otherwise remove biases from the trial results. The trade-off is that sponsors are unable to respond to any indications during the trial that the protocol is flawed. This means they are unable terminate the study and start a new one and are forced to spend significant time and money on a study with a very low probability of success.
In light of these problems, adaptive trial design has become a popular alternative. This approach allows studies to adapt mid-stream based on data collected early in the trial. For example, the protocol may be designed to allow the sponsors to adjust the dosage of a drug that is not showing efficacy for the early patients, or by increasing the number of patients studied if early data suggest it would lead to a more statistically successful final study result. By changing the protocol while the study is in progress, based on what is commonly called "interim analysis", researchers can maximize the usefulness of the study.
In connection with the growing interest in adaptive trial design, the FDA issued draft guidelines for sponsors considering using these techniques. In these guidelines, the FDA recognizes the potential power of adaptive trial design to make clinical trials more efficient and more informative, and even encourages sponsors to gain experience with adaptive trial design with exploratory studies.
Sponsors are responding to the encouragement of the FDA and the industry, and the results of the first-generation of adaptive design clinical trials are beginning to appear: data from the first successful adaptive clinical trial was published in August 2010 and more are expected to follow.
The current limitations
The major caveat of adaptive trial design is that the study’s statistical integrity must be maintained. The main concern with any type of adaptive trial is introducing biases that ultimately nullify the study results. Any adaptive technique that would allow researchers to review unblinded data is particularly suspect.
The magnitude of this concern is underscored by the recommendations in the FDA guidelines. As the current guidelines are drafted, the only type of trial adaptations that the FDA considers acceptable are ones that are planned when the study is designed, before the data are examined unblinded by any personnel involved in planning the revision. Unplanned revisions that are made after an unblinded, interim analysis raise major concerns about study integrity.
The requirement that trial adaptations be planned in advance limits the usefulness of adaptive trial design because it requires sponsors to anticipate what pieces of interim data will be useful. This creates issues because, as noted in the FDA guidelines, "adequate knowledge regarding all the important parameters needed for planning study design may not be present at the time the study is designed." If all possible adaptations must be established prospectively, sponsors will still be unable to efficiently respond to early and unexpected patient data that indicates the clinical trial protocol should be changed.
The key to overcoming limitations in adaptive trial design lies in next-generation clinical data tools that allow advanced interim analysis while still preserving study integrity. For example, as part of the study design process, researchers could designate the software tool that will be used to conduct the interim analysis and specify what limitations the software tool must impose on the data being reviewed. Examples include limiting data to aggregated data with no additional granularity; maintaining the blindedness of the data; or not revealing factors that may lead to bias, such as certain patient characteristics. In this way, researchers could ensure that biases that may jeopardize the statistical success of the trial are not introduced, while still preserving the flexibility to examine interim data in a robust manner and efficiently respond to unexpected results.
Before such an approach can be fully implemented from a technology perspective, a great deal of further investigation into adaptive trial design is required. Some of these insights can only be gained through experience and experimentation; the establishment of industry-accepted methods for adaptive trial design are likely a number of years off. Still, today’s investment means safer and more effective treatments tomorrow.
The path ahead
The path to establishing truly adaptive clinical trial design lies in a close collaboration between researchers and the makers of clinical technology tools. As researchers develop methods of ensuring statistical integrity and introducing dynamic protocol changes without increasing bias, technology providers should focus their efforts on data capture systems that can more easily adapt to changes in the study. These systems should support more thorough metadata descriptions of the data that is stored. The review systems can then connect to all relevant data sources and understand the underlying information enough to prevent biasing of the data. At each stage of development, technology providers should consult with researchers to confirm that the technology meets the ultimate goal of improving the effectiveness of adaptive trial design.
None of new approaches are easy, but the potential benefits of creating them should well exceed the cost and effort of implementing the new systems. We are well on the road to developing a framework for technology-assisted advanced adaptive trials, but only a continued push throughout the industry will make it a reality.
About the Author
Rick Morrison is co-founder and CEO of Comprehend Systems. Before starting the company, he spent more than a decade writing software for clinical trials, including tools that are now used by the FDA and top pharma.