biochemist INDIA (3 nov. 011): When the Centers for Medicare and Medicaid Services (CMS) finalized the Clinical Laboratory Improvement Amendments regulations in 2003, many in the lab community expressed dissatisfaction with what was perceived as ambiguous and unscientific guidance on how to conduct quality control (QC). While the regulations set basic requirements for testing external QC materials, most laboratories found they needed to go above and beyond these standards to avoid quality problems. In the 8 years since the agency published the final regulation, exactly how often labs need to perform external QC and other quality checks has been widely debated. Quality tools like Six Sigma, Lean, and others abound, but so far, a comprehensive approach to QC that suits regulators and a majority of the laboratory community has not emerged.
Now the Clinical and Laboratory Standards Institute (CLSI) has published a long-awaited guideline that aims to fill this gap, enabling labs to customize QC to match both changing technology and the uniqueness of each lab. However, in what form CMS and other accrediting organizations will adopt or endorse it remains to be seen.
Published in October, the new guideline, EP-23, Laboratory Quality Control Based on Risk Management, translates the time-honored concept of risk management used in manufacturing, defense, aerospace, and other industries into the language of the clinical lab. Risk management has been standard for engineering and other professions for decades, and an application for laboratories is long overdue, said James Nichols, PhD, chair of the CLSI subcommittee that developed the guideline.
“We’re all using devices in different ways with different types of staff. And it’s becoming even more complicated the more laboratories decentralize testing to nursing units or physician offices,” Nichols said. “Risk management can help labs build a custom QC plan for each device and test that strikes the optimum balance of built-in internal monitoring systems with external QC, as well as all of the other processes that we have to reduce our risk of errors.” Nichols is a professor of pathology at Tufts University School of Medicine and director of clinical chemistry at Baystate Health in Springfield, Mass.
Risk in the Lab
Whether a lab produces 500 or 5 million results a year, the lab director bears ultimate responsibility for the accuracy and timeliness of each test result. Yet even as advanced instruments, information technology, and automation have enabled labs to accomplish more with less, the complexity of the lab as a web of systems within systems continues to swell, and the greater volume and power of lab testing means the stakes are even higher if a serious error occurs.
CLSI designed EP-23 to help labs create thoughtful, deliberate QC plans that tackle the many sources of error that other approaches do not always account for. In fact, laboratorians already engage in ad hoc risk management every day, according to Nichols. “Risk management is essentially the process of sitting down and saying, what can go wrong when I perform a test, and what do I do to prevent that error from happening? We as laboratorians are actually trained to think in that manner, and we do that every day,” he said. “With risk management in EP-23, we’re giving this a name and putting it together in a formal plan—for example, documenting why I run liquid QC, and when I do based on specific risks in my lab. Is liquid QC really telling me everything I need to know about what could go wrong with the system, or do I need other types of control processes to fill in those holes and improve my confidence?”
The heart of risk management is balancing potential sources of error, referred to as hazards, with the right controls. To help labs do this, EP-23 explains how risk management applies to the lab as a continuous process of risk assessment, control, monitoring, and failure investigation. Most of what will be new to laboratorians falls under the risk assessment phase, which includes tools for identifying hazards and estimating and evaluating the risk (See Figure 1,below).
Figure 1
Risk management aims to identify sources of error and match up these hazards with appropriate controls. It begins with creating a map of each lab process, such as performing a particular test, then considering how the process could fail at each step. Risk estimation follows, which includes assessing the likelihood of a given failure, as well as the severity of patient harm resulting from the failure. Third, risk evaluation compares the results of risk estimation to the lab’s risk controls. Risk monitoring and failure investigation complete the loop. Source: CLSI EP-23A. |
EP-23 encourages laboratorians to think globally and consider information on risk from many sources, for example: the environment of the lab, staff competence, internal and external evaluation/verification data, clinical application of tests, and the severity of patient harm that would result from an error. The idea is to make decisions about quality control based on greater depth and breadth of information.
As a primer on risk management, EP-23 can help laboratorians take a more comprehensive view of their operations that builds on the real-world consequences and sources of errors, according to Greg Cooper, CLS, MHA, CQA, who worked on the CLSI subcommittee responsible for EP-23 and also is chair of the organization’s Consensus Committee on Evaluation Protocols. “This is written by laboratorians for laboratorians—it’s not just something extracted from documents for another industry,” he said. “The lab is going to gain a better understanding of how it’s really operating and where its potential failures might come from, and what it needs to do to prevent those failures in the future. Labs will also better use their resources because QC is going to be focused where it needs to be focused.” Formerly a quality control and education expert for Bio-Rad Laboratories Quality Systems Division, Cooper now runs his own consulting practice.
Doing the Right QC
Although in some cases, risk management could enable labs to limit their dependence on external QC, EP-23 is about doing the right QC, not less, Cooper emphasized. “EP-23 is about right-sizing the QC for the test,” he said. “This gets labs thinking about all of the conditions, activities, and processes that they control that could potentially create a hazardous situation and cause harm to a patient. It’s not only about the device failing and not catching it.”
In effect, EP-23 targets the vacuum left after stakeholders could not agree on in-depth, evidence-based QC requirements for the 2003 CLIA final regulations, which now carry minimum requirements held over from older technology. The requirement for testing two levels of liquid QC every day a test is run comes from the days when labs ran just a few batches of patient samples a day, Nichols explained. “With the new, more automated analyzers, there is no longer batch analysis and patient samples are analyzed continuously,” he said. “So now the question is, do we hold those samples until the next QC run, or do we run QC continuously, every 10, 20, or 50 samples and release results in small batches? These operational considerations lead to turnaround time issues, cost issues, and resource issues. By analyzing the risk of errors and the mitigations in place, EP-23 can help provide a framework for explaining the reasoning behind the lab’s decision-making.”
With a focus on the analytical phase of testing, EP-23 zeroes in on how to make such decisions about QC at a time when manufacturers advertise a litany of internal monitoring systems and other quality checks built into new instruments. These high-tech features minimize the potential for certain errors, in many cases duplicating external QC. For other potential errors, external QC is not redundant, leaving the lab to weigh all the potential causes of error against the options for checking quality. Risk management includes process mapping that lets laboratories examine all processes, how these processes relate or interact with one another, and potential sources of error so that the lab has a complete picture of its unique situation when preparing its customized QC plan (See Figure 2, below).
Figure 2
A tool used in the hazard identification phase of risk management, a fishbone diagram identifies the possible causes of errors—called failure modes—and their effects. This example from EP-23 lists failures that could lead to incorrect test results. Source: CLSI EP-23A. |
A customized plan means that each test on each system can get the particular QC treatment it needs, noted Cooper. “Frankly, there are some tests—and everyone in the lab knows which ones those are—that are rock-solid performers that never, ever change. So, do you really need to do external QC every day for those? Probably not. But there are many other tests that are not quite so rock-solid in their performance, and they’re sensitive to reagent lot changes and other variables. For these tests, there is a need to have more intense scrutiny of your QC.” EP-23 also prompts the laboratorian to look at factors outside of the analytical phase that might inform the type and intensity of QC, Cooper said. For example, how the test is used clinically: an incorrect triglyceride would not carry the same risk as a false-negative HIV result.
Nichols emphasized that risk management would rarely mean complete reliance on an instrument’s internal monitoring systems, however. “A device may have an electronic check that looks at the device itself and tells us that it’s functioning to specification, but that doesn’t tell you anything about the liquid reagents that are read by that device,” he said. “That’s where we have to consider what other types of controls we have.”
A customized QC plan based on risk management can also help the lab justify elements of its budget. Testing external liquid QC means consuming reagents and takes staff time, Nichols noted. “More and more, hospital administrators are asking us why we have so much non-productive testing, and everyone is thinking about limited resources in terms of staff and money,” he said. “With risk management, you’ve documented why you run external QC or perform other controls when you do, and how it addresses your particular risks.”
Beyond the Analytical Phase
Despite EP-23’s focus on the analytical phase of testing, risk management could also do a lot to help labs deal with pre-analytical and post-analytical errors, stressed Jan Krouwer, PhD, an in vitro diagnostics industry consultant who has worked with test and instrument manufacturers on risk management. As a standard in engineering, manufacturers have employed risk management for decades, but the concept has broad application outside that realm. “It could work for labs. In fact, it’s an essential part of how you would manage quality in a laboratory, because there are a lot of errors in a lab that are really dealt with best by risk management,” he said. “In particular, all of the pre-analytical and post-analytical errors, you’re not going to assess them very effectively by the usual evaluation means of method comparison and imprecision studies. But they are amenable to risk management.”
Risk management can help labs understand and manage the limitations of traditional external QC, according to Krouwer. “Essentially, external QC is just looking at differences from the target. But, if you’re asking about how all of the steps in your process could go wrong, and seeing whether or not you have effective means of preventing each item from going wrong, that’s quite a different process, and one that could be quite useful for laboratories.”
However, Krouwer warned that the well-established techniques in risk management need to be followed closely to achieve the maximum benefit. “This is one of the potential problems with risk management. Everyone does risk management to some degree; it’s in everyone’s life. For example, when do I change lanes on the freeway? Formal risk management comprises a set of techniques that’s not hard to learn. But to do the technique informally is not as valuable,” he said. “If risk management is implemented informally, you can miss things.”
Hospitals accredited by the Joint Commission must conduct a risk assessment of one area of their operations every 18 months. The organization does not track which operational area hospitals choose, and could not say which, if any, hospitals had chosen the lab. Krouwer’s experience with hospital risk management serves as a cautionary tale, though. He wrote a software application for hospitals to help them fulfill this requirement, but hospitals struggled with taking the time and effort the risk management process required even with the aid of the software, he said. “My sense was that they wanted something much quicker where they could just check off boxes,” he said.
A New Option for Regulators?
While quality experts in the lab community have mulled the concept of risk management for a long time, the current push that resulted in EP-23 began in 2005 when CMS convened its “QC For the Future” meeting that brought together professional associations, industry representatives, and other regulatory agencies to tackle the gaps left by the CLIA final rule.
In the original 1992 version, CLIA called for the FDA to review manufacturers’ claims about the built-in QC features of their devices, with labs responsible for following manufacturers’ instructions. When FDA could not meet its end of the bargain due to resource constraints, CMS had to regroup. The result was the basic 2-levels-a-day default for external QC, and the call for lab directors to develop their own quality plan that went beyond the minimum and that took into account the lab’s particular needs and environment.
CMS now plans to evaluate EP-23, in concert with other lab accreditors, to consider making risk management a part of CLIA’s interpretive guidelines. “We engaged in a partnership with CLSI because they use a consensus approach to their documents. We felt it would be good to work with an organization like CLSI so that people would not assume that we were sitting in our ivory tower and making up policy on quality control,” said Judith Yost, MA, MT (ASCP), director of CMS’s Division of Laboratory Services. “We have also provided, among the other government agencies involved in CLIA, representation on the subcommittee that developed the document.”
Yost pointed out that the top 10 deficiencies chart that CMS has tracked over the years always features QC problems. Although the number of labs cited by CLIA for QC failures has declined in recent years, other accreditors like the College of American Pathologists (CAP) and the Joint Commission rank QC issues at the top of their deficiencies. “Problems with quality control do not seem to be going away, and that’s why we feel that whatever we do next is so crucial,” Yost said. “That’s why we will be looking at EP-23 so closely.”
Great shyam. Keep it up.
ReplyDelete