The Absence of Errors That Matter: The Value of RBM in QbD

Halloran Consulting Group, in collaboration with Boston Scientific (BSC), recently hosted a Clinical Operations Breakfast – a community of clinical operations professionals that meet monthly over a virtual breakfast to share common challenges and solutions  – to discuss “The New Risk-Based Monitoring: The Path Ahead and Ideas for Implementation” for the conduct of clinical studies. The COVID-19 pandemic has forced the hands of sponsors to begin immediate incorporation of risk-based monitoring (RBM) into clinical studies and a “new normal” is on the horizon. The BSC team shared their journey to implementation and how companies of all sizes can take steps to accelerate the use of RBM into their studies. With lively engagement from participants, the ultimate question to be answered is “How much data do you need to see to be comfortable that the study data is accurate?” 

Why RBM and Quality by Design Matters (QbD)

Quality by Design in clinical trials is defined as the absence of errors that matter. The use of RBM, specifically, in clinical trials allows operations teams to focus their efforts on monitoring the activities of the study that are crucial to the study outcomes. In today’s clinical research world, trial data is entered in real-time. Gone are the days of sponsors getting a first look at data from an annual report after manual entry from paper Case Report Forms. With the implementation of Electronic Data Capture (EDC), data collected at sites is quickly available for centralized aggregate data review. This tactic, along with focused source data review, has been shown to produce higher quality data at a more efficient cost, but the industry has been slow to redefine monitoring from a traditional 100% SDV process to a more risk-based approach.

Boston Scientific Journey

About three years ago, the team at BSC began to examine the traditional model for monitoring and asked how they could evolve to a new model using risk-based processes. The organization, like many biotech companies, was shifting focus in their trials toward a culture of quality by design, which influenced the implementation of RBM. Much like in today’s COVID world, the question was “what is the least amount of data we need to advance the trial while maintaining patient safety and data integrity?”

BSC built a team composed of experienced operations professionals to utilize existing technology in trial conduct. The initial decision to shift from the comfort zone of traditional monitoring to RBM, and the subsequent commitment to stick with it and “lean in” was the biggest hurdle to overcome, but it was the buy-in from top down that drove successful implementation.

Where to Start?

Not all companies are the size of nor have the resources to create an entire department dedicated to the implementation of RBM that a larger company like BSC has. Many of the attendees at the breakfast asked how to start – and this is a common question we hear in the community – especially when so many companies are already moving at full speed with limited resources. There isn’t time to pause to redesign operational structure, so companies are required to adopt changes while in motion. COVID has created an opportunity to accelerate strictly out of need. Whether the motive is born strictly out of need or planning was already in the works, advice for implementation of RBM was much the same:

• Engage stakeholders:
  • The management and operations of a clinical trial involves many different stakeholders.  Communication to various groups should be targeted and clear. To the executive team, the focus may be on cost optimization and avoidance of risk. Conversely, communication with monitors may be focused on how the new RMB approach will require key critical thinking and analysis in addition to traditional site management that may be done remotely, which could provide increased quality of life due to reduction of time on the road.
  • Setting clear expectations and defining ownership of tasks with vendors, like CROs, during implementation will ensure roles and responsibilities are understood from the outset. Even prioritization of maturity and experience during site qualification and selection can help bring success. 
  • Across all groups, setting the realistic expectation that refining the monitoring process is a calculated change that may cause some immediate disruption is important, but change will provide long-term efficiencies and higher quality data. It’s also important to set the precedent that there will be bumps in the road, but “stick with it,” as opposed to seeing those bumps as an invitation to revert to the old ways.
• Incremental change:
  • It is not expected that sponsors will make an immediate shift from a traditional monitoring process to a risk-based approach overnight. Sponsors are encouraged to start with a smaller, incremental adoption of one or more of the principles of RBM.
  • Operations teams can then showcase lessons learned as evidence of success, and in effect, this will convince stakeholders to stick with this approach. This could be as simple as engaging new but already existing technology and tools that would align with planned or active protocols studies like eSource, eConsent, central monitoring, direct data capture, or EMR access. Or more complex changes like transitioning interim monitoring visit(IMV) schedules to be dependent on metrics such as completed patient visits instead of relying on a preset IMV schedule of every 6 weeks, for example. CRAs will then be on site when there is an appropriate volume of data to be verified. 
• Incorporate Risk Management
  • Start with a robust risk assessment to identify and define risk. This includes defining critical processes and critical data (primary and secondary endpoints, safety, and other critical variables) and identifying the thresholds for when action is needed. These standards are then used to inform study plans across functional areas to define when action should be taken and what specifically that action should be. Simply put: Set the standard, make sure it’s documented, and ensure the standards are followed. 
• Rethinking monitoring and monitor training
  • Sponsors need broader ‘thinking about monitoring’ and monitor training. Monitoring is no longer just on-site, “boots on the ground” SDV, but entails more active data review between visits. With robust training and onboarding, monitors will be called upon to evaluate quality of data and trending analysis during centralized review that requires a shift in core competencies to include more critical and strategic thinking. On-site visits will be more focused on other tasks like drug accountability, addressing enrollment challenges, assessing protocol compliance, and building relationships instead of 100% SDV of every data point. 
• Start early in the development process
  • Where possible, setting the stage early in the design process allows for protocol development to be streamlined and focused on what matters most for a specific trial.  Focusing on the initial question of “How much data do you need to see to be comfortable that the study data is accurate?” can help make clear which study activities are essential to patient safety and credible study results and eliminating nonessential activities.

Challenges

Implementation of RBM is not without challenges. As it takes a shift in mindset within companies, it also requires a broader shift in the thinking across the industry. Standard practices are comfortable and to some, RBM feels like giving up control of data so engaging across functional areas has been slow.  Likewise, ensuring vendors are aligned can also be a challenge. Setting clear ownership and expectations up front can help alleviate some of the concerns. For example, getting input from all stakeholders to inform the risk assessment is critical. Using the RACT as a measurement tool to then inform study plans across all functional areas will provide all stakeholders documented details of the work to be done, how it will be measured, and the prescribed escalation.

Another challenge for sponsors is determining how to measure and document SDR since it is less clear cut than traditional 1:1 SDV. Documentation of SDV that is focused on a smaller number of data points related to safety and/or endpoints can be easily captured as it has been with traditional monitoring.  However, the question remains how to most efficiently capture source data review to avoid additional inefficiencies like documenting every page that has been reviewed into CTMS, for example.

Concerns exist about implementation of RBM at ex-US sites considering GDPR and privacy concerns. Overall, there needs to be further engagement across the industry in various regions with prohibitive guidelines to determine how to move forward. COVID has allowed for some relaxation of guidelines like remote review of EMR in some countries but it remains to be seen if these will be long-lasting changes.

Finally, training particularly at the monitor level needs to be revised. Additional competencies of critical thinking and data analysis to evaluate and assess quality need to become a part of job descriptions for monitors. However, ensuring employees that their jobs are not becoming obsolete and providing robust training and onboarding will be key to maintaining staffing and preventing turnover.

The Value of RBM

Where will companies find the value of implementing robust risk-based monitoring? The change in processes related to RBM and utilization of technology can impact time, quality, and cost. In the long-term, those impacts can improve patient outcomes, bring new treatments to patients faster, and ease burden on study sites.

• Time:
  • RBM can help address inefficiencies in how monitors spend their time on site. While off-site doing central review, monitors can utilize tools like edit checks and data listings to track trends or outliers to do aggregate data reviews remotely. As a result, on-site visits are shorter if they’re only evaluating critical variables, and the time the monitors have on site can be used for tasks that can’t be done remotely like drug accountability or visiting the labs. Therefore, resources on-site can be more focused on reducing risk through addressing deviations, assess protocol compliance, and any necessary retraining. 
• Quality:
  • Implementing RBM and central monitoring allows sponsors to get a full view of data as opposed to a variable-by-variable comparison to source. Many would argue it is a more robust way to ensure the data is clean, accurate and correct since the purpose is to detect outliers and review trends. By not relying solely on on-site monitoring visits, ongoing central review allows the operations team to analyze trends, assess outliers, and use the trends to respond and direct next steps. Compliance issues can be identified and resolved in real-time instead of waiting for the next on-site visit – a major benefit
• Cost:
  • The impact on cost should be viewed more in terms of cost optimization than direct cost savings. Change to process may involve some upfront costs like shifting to new resources and technology but can be balanced with reduced on-site monitoring costs due to expected less in-person time on site. Keep in mind, this isn’t less costly; but rather, you’re shifting your expenses for a long-term, sustainable process.

RBM is an evolving process, sped along by the need to quickly adapt due to COVID. Prior to the pandemic, less than half of the participants in the breakfast had implemented RBM strategies at all while another third were only in the planning implementation. However, during COVID, most if not all had begun using technologies such as file sharing, video conferencing, and access to EMR to facilitate monitoring remotely. COVID has amplified the need for the shift and change is beginning. The opportunity to adapt and accelerate change, step-by-step, is now to ensure the industry keeps moving forward.