Pa Patient Saf Advis 2012 Mar;9(1):23-6.
Pennsylvania: On the CUSP of Measuring Infection Prevention Culture
Critical Care; Infectious Diseases
Expand All Collapse All
Author

James Davis, BSN, RN, CCRN, CIC
Senior Infection Prevention Analyst
Pennsylvania Patient Safety Authority

Abstract

The Comprehensive Unit-based Safety Program (CUSP) works to establish a safety climate through the establishment of a unit-based culture of safety. The second progress report on the national On the CUSP: Stop Blood Stream Infection project states that the pre- and post-CUSP implementation safety culture survey showed little change upon comparison. Instead, CUSP’s success has been measured by a surrogate outcome (infection rate). When central venous catheters are in use, safe culture is evident in direct measurement of compliance with best practices, as well as in device utilization ratio (DUR). The implementation of CUSP in cohort 2 in Pennsylvania units has resulted in improved compliance with best practices and an 8% decrease in DUR from baseline.

Introduction

The Comprehensive Unit-based Safety Program (CUSP) is a structured, strategic framework with the intent of improving the culture of patient safety.1 The CUSP methodology is flexible as it can be applied to many patient safety issues. For the purpose of this article, Pennsylvania Patient Safety Authority analysts have chosen to focus on cohort 2 of the On the CUSP: Stop Blood Stream Infection (BSI) project in Pennsylvania. The second progress report on the national On the CUSP: Stop BSI project exhibits results from the pre- and post-CUSP implementation Hospital Survey on Patient Safety (HSOPS). HSOPS was administered as part of the project and showed little change pre- and postimplementation.2 Interested in whether CUSP implementation improved safety culture in participating Pennsylvania critical care units, Authority analysts queried National Healthcare Safety Network (NHSN) event data reported by Pennsylvania hospitals in order to determine compliance with best practices related to CUSP implementation. Authority analysts found an increase in best-practice compliance possibly related to improved safety culture in the CUSP group.

Background

The intensive care unit project of the Michigan Health and Hospital Association’s Keystone Center for Patient Safety and Quality (MHA Keystone Center) achieved impressive results with clinical interventions that mainly focused on central venous catheter (CVC) care. The MHA Keystone Center project recommended evidence-based procedures for CVC insertion and daily goal sheets. The MHA Keystone Center study also implemented a comprehensive program (CUSP) that sought to improve the culture of safety in the units where data was collected.3 Zhang et al. have defined safety culture asthe enduring value and priority placed on worker and public safety by everyone in every group at every level of an organization. It refers to the extent to which individuals and groups will commit to personal responsibility for safety [and] strive to actively learn, adapt and modify (both individual and organizational) behavior based on lessons learned from mistakes.4 Safety climate is defined as “the temporal state measure of safety culture, subject to commonalities among individual perceptions of the organization. [Safety climate] is therefore situationally based, refers to the perceived state of safety at a particular place at a particular time, is relatively unstable, and subject to change depending on the features of the current environment or prevailing conditions.”4 Climate refers to environmental influence on culture, and culture is the behavior of the individual within the climate. CUSP’s intent is to have the clinicians learn from mistakes, thereby improving the culture of safety.1 The CUSP manual states that “culture is a major focus [of CUSP] because it represents a set of shared attitudes, values, goals, practices, and behaviors that make one unit distinct from another.”1 Furthermore, Bandura observed that “what people [clinicians] think, believe, and feel affects how they [clinicians] behave.”5

Although the CUSP project may not have been able to statistically prove—as measured by the questionnaire—that a culture of safety had been caused by CUSP, the CUSP group did demonstrate a reduction in central line-associated bloodstream infection (CLABSI) rates after CUSP was implemented, which may allude to the presence of improved safety culture. If CUSP methodology has influenced how participants think, believe, and feel about safety culture, participant behavior toward compliance with best practices related to CVC care would change. Can measurement of compliance with best-practice data reflect the prevalence of safety culture?

Methods

The Authority collects responses to compliance questions in custom data fields through NHSN that can be used to gauge compliance in Pennsylvania hospitals. Cohort 2 of the On the CUSP: Stop BSI project was instituted by the Hospital and Health System Association of Pennsylvania (HAP), the Health Research and Educational Trust (HRET), the Johns Hopkins University Quality and Safety Research Group, and MHA Keystone Center. Cohort 2 was chosen for analysis related to the availability of a complete pre- and post-CUSP implementation data set that was inclusive of the Authority’s custom data fields. The Authority’s data pull from NHSN was conducted on September 20, 2011. Baseline, or preimplementation, data query started September 2008 and continued through August 2009. CUSP cohort 2 data query started in September 2009, and the data through June 2011 was analyzed. September 2009 marks the beginning of the postimplementation period. The tables break out baseline (pre-September 2009) and postimplementation (post-September 2009) data for both groups despite the non-CUSP group having had no direct intervention related to official CUSP participation in cohort 2. The best-practice compliance question fields included in the analysis are part of the NHSN CLABSI event report. The responses to the questions have been normalized and reflected as percentages for comparison between non-CUSP critical care units and CUSP cohort 2 critical care units.

CVC insertion is a quick procedure performed by a group of providers that adhere to the culture of their practice. Maintenance of the line occurs over many hours to months and involves a host of individuals (e.g., nurses, physicians, caregivers, patients, families), all of whom have a culture in regard to causing or preventing the development of CLABSI.6 Therefore, given the distinct differences between insertion and maintenance, it should be understood that each phase of CVC life would possess its own unique climate based solely on the culture of the individuals involved in a CVC phase. Noting the distinct differences between insertion and maintenance phases, Tables 1 and 2 speak to the culture associated with insertion. The ability to determine CVC necessity (see Table 3) would be indicative of a quality-based system for tracking and surveillance of the line post-insertion, transcending into the maintenance phase of CVC culture. Culture (behavioral choices) associated with compliance with best practices would then be measured by the answers associated with each compliance question. To fully evaluate CUSP’s impact through a traditional outcome metric, Authority analysts also examined the pre- and postimplementation device utilization ratios (DUR) of each group.

 

Table 1. Maximal Barriers on Insertion ​ ​ ​ ​
 Non-CuspCusp Cohort 2
 BaselinePost-
Implementation
BaselinePost-
Implementation
Yes58.6%67.2%81.4%91.5%
No1.7%1.3%0.7%0.6%
Unknown39.4%31.1%17.9%7.9%
Blank0.3%0.5%0.0%0.0%

 

Table 2. Chlorhexidine Prep Preinsertion ​ ​    
 Non-CuspCusp Cohort 2
 BaselinePost-
Implementation
BaselinePost-
Implementation
Yes57.2%64.8%80.0%84.8%
No3.6%2.7%0.0%0.6%
Unknown38.3%31.0%17.9%7.9%
Contra-
indicated
0.6%0.9%2.1%6.7%
Blank0.3%0.6%0.0%0.0%

 

Table 3. Daily Review of Central Venous Catheter Necessity ​ ​ ​  
 Non-CuspCusp Cohort 2
 BaselinePost-
Implementation
BaselinePost-
Implementation
Yes53.4%57.6%72.1%79.3%
No8.0%7.4%8.6%12.2%
Unknown38.3%34.3%19.3%8.5%
Blank0.3%0.7%0.0%0.0%


Tables 1, 2, and 3 show the responses to the individual questions that target compliance with best practices. Overall, CUSP cohort 2 units outperformed non-CUSP units in percent compliant with best practices both pre- and post-CUSP implementation. Note the decrease in “blank” and “unknown” responses and the difference between the “no” and “yes” fields post-implementation in the CUSP group, indicating better surveillance of compliance metrics, possibly due to CUSP climate. Furthermore, the CUSP intervention took demonstrated high-performers (as noted by preimplementation percentages) and pushed compliance even higher.

Table 4 refers to the DUR of non-CUSP units and DUR of CUSP cohort 2. Central line-days divided by patient-days equals DUR. When considering the lower DUR in the CUSP group and the results represented in Table 3, it would appear anecdotally that removal of unneeded CVCs is a high priority in the CUSP group. If compliance with best-practice culture (removal of unnecessary CVCs) is in fact due to CUSP, there should be a significant difference of differences in proportions between the groups.

Table 4. Device Utilization Ratio (DUR) ​ ​ ​ ​
 Non-CuspCusp Cohort 2
 BaselinePost-ImplementationBaselinePost-Implementation
DUR (95% CI)0.383 (0.382-0.384)0.378 (0.377-0.379)0.344 (0.341-0.346)0.311 (0.309-0.313)


The DUR for the CUSP hospitals decreased from 34.4% to 31.1% (before CUSP implementation versus after implementation), which is a decrease of 3.3 percentage points. Some of this decrease may not have been due directly to CUSP, but may have in part been general cultural changes that were also experienced at non-CUSP hospitals. To estimate this impact, analysts found that in non-CUSP hospitals, in the exact same time period, the DUR decreased from 38.3% to 37.8%, which is a decrease of only a half of a percentage point. Therefore, subtracting the general decrease of 0.5 percentage points from the raw 3.3 percentage points yields an estimated impact of CUSP of 2.8 percentage points. Analysts formally tested this 2.8% and found it to be statistically significant using the Gaussian method described by Wallis.7 The decrease of 2.8 percentage points in DUR represents an 8% decrease in DUR from the baseline level of 34.4%.

Discussion

The relationship between clinical change (compliance with best practices) and culture is noted in the second CUSP progress report as “clinical changes require and reinforce changes in safety culture.”2 The importance of linking culture and quality improvement is stressed in the CUSP manual through the statement: “culture and quality improvement need to be linked.”1 If clinical change is evident and positive, safety culture would be reinforced; therefore, one should observe positive compliance (behavior) and better outcome rates would follow. Benner has observed that experts behave with the future in mind, and that they consider likely possibilities based on current data.8 Kunkel and Nagasawa note that “present circumstances provide information about probable future events based on past experiences, and thus serve as signals for present behavior.”9

The challenge for those who wish to improve compliance will be to set the circumstance (climate) that delivers current data to bedside experts about their behaviors (culture) in regard to best practice in order to achieve desired future events (outcome). CLABSI prevention is at a unique juncture; many facilities can report CLABSI rates at or approaching zero for individual units. How can one have situational awareness, deliver current data, and intervene when the metric of monitoring outcome (infection rates) fails to be sensitive enough to address cultural drift? If compliance with best practices is monitored by way of methods like statistical process control (SPC), infection preventionists can identify behaviors that fail to comply with best practices based on process signals, allowing the experts to be informed of potential issues ahead of time instead of waiting for an infection rate increase in order to investigate system defects.

At the unit level, compliance data should be collected at regular intervals. Systems can be designed to use snapshot data depicted in SPC format of carefully chosen best-practice metrics in order to gauge safety culture. In addition, systems can be designed to make it extremely difficult for those who function within them to misstep; however, if someone does misstep, others within the system may then, because of established climate, discourage behavior that is not part of a safe climate. If there is widespread cultural deviance or normalization of deviance, one would expect to see compliance data signals. When the data signals for a particular metric, action can be taken in order to correct the culture associated with the defect, normalizing compliance. Wiemken noted that “through adequate data collection and critical analysis of control charts, the infection preventionist can detect aberrant data early, which allows for prompt intervention and mitigation of any poor outcomes.”10

Following similar methodology, Harpel et al. decreased the incidence of CLABSI by redesigning the traditional intravenous team into a vascular resource team (VRT) whose duties included weekly audits of best practices related to CVC maintenance.11 Bedside nurses were educated by the VRT in central line maintenance techniques. In addition, bedside nurses and managers were provided with regular feedback on compliance audit data. When best-practice compliance defects signaled via SPC, the infection preventionist and VRT would investigate issues and mentor staff nurses in performing high-quality, evidence-based vascular access care. Harpel et al. found that “compliance with central line dressing changes rose from 68% to more than 90%.” The authors also wrote: “This collaborative approach between the bedside nurses and the vascular resource team [which allowed for the optimum balance of resources] led to a hospital-wide reduction in CLABSI.” The VRT is an example of a system that was created to identify behavior, nourish a climate that supports competence, encourage cultural compliance with best practices, and achieve reductions in CLABSI.

Conclusion

System Framework

CUSP is a valuable framework providing a climate for the development of cultures that are centered in, and supportive of, compliance with best practices, which is essential for the well-being of all patients. CUSP is the centerpiece of a patient safety foundation that is essential for an effective infection prevention program. Aligning CUSP methodology, current evidence-based guidelines, and dedicated administrative support helps to set the foundation of an effective program. Foundations, however, need to be built upon for the structure to be useful. When epidemiological and compliance measurement are tracked through SPC, it is possible to have current telemetry of culture, which will enable activities for clinical change providing for actionable defect mitigation. Continuous mapping of behavioral data from compliance telemetry allows for balanced resources, thereby influencing and supporting the climate of best future practice and effectively deflecting CLABSI from patients. According to Streed, “elimination of HAIs [healthcare-acquired infections] requires this constant investment of resources in terms of enquiry, action, vigilance, and ownership strategies to increase sustainability.”12

Rely on Outcome Metrics or Process Control?

Control of process, or lack thereof, leads to an outcome. There is value in the monitoring of data points aimed at evaluation of best practices. In the current climate of infection prevention, outcome has been the traditional measurement of success. Proactive intervention using process control data as telemetry of culture—rather than the traditional method of reacting to outcome (infection) rates—can possibly hold the key to less CLABSI. Streed states that “outcome measurement is at best a surrogate indicator of process adherence, and that effective process control leads to predictable outcomes.”12 Outcomes need to be predictable in order to know the risk to the patient in the designed systems. The future of infection prevention will rely on the preventionist’s ability to measure processes, predict outcomes, and control processes with appropriate interventions that focus on improving the culture of compliance.

Acknowledgments

Edward Finley, BS, Pennsylvania Patient Safety Authority, contributed to data acquisition and validity in this article. Jonathan R. Treadwell, PhD, ECRI Institute, consulted on and contributed to statistical testing in this article.

Notes

  1. Johns Hopkins Quality and Safety Research Group, Michigan Health and Hospital Association Keystone Center for Patient Safety and Quality. On the CUSP: stop BSI. Comprehensive unit-based safety program manual. Agency for Healthcare Research and Quality (AHRQ). 2009 Apr. Also available: http://www.onthecuspstophai.org/wpcontent/uploads/BSImanualsTools/CUSPimplementation/CUSP_Manual.doc.
  2. Health Research and Educational Trust, Johns Hopkins University Quality and Safety Research Group, Michigan Health and Hospital Association Keystone Center for Patient Safety and Quality. Eliminating CLABSI: a national patient safety imperative. second progress report on the national on the CUSP: stop BSI project [online]. 2011 Sep [cited 2011 Nov 2]. Available from Internet: http://www.ahrq.gov/qual/clabsiupdate/clabsiupdate.pdf.
  3. Provost P, Needham D, Berenholtz S,et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med 2006 Dec 28;355(26):2725-32.
  4. Zhang H, Wiegmann DA, von Thaden TL, et al. Safety culture: a concept in chaos? Proceedings of the 46th Annual Meeting of the Human Factors and Ergonomics Society; 2002 Santa Monica (CA). Also available: http://www.humanfactors.illinois.edu/Reports&PapersPDFs/humfac02/zhawiegvonshamithf02.pdf.
  5. Bandura A. Social foundations of thought and action: a social cognitive theory. Englewood Cliffs (NJ): Prentice Hall; 1986:25.
  6. Davis J. Central-line-associated bloodstream infection: comprehensive, data-driven prevention. Pa Patient Saf Advis [online] 2011 Sep [cited 2011 Nov 23]. Available from Internet: http://patientsafetyauthority.org/ADVISORIES/AdvisoryLibrary/2011/sep8(3)/Pages/100.aspx.
  7. Wallis S. Comparing x2 tests for separability: interval estimation for the difference between a pair of differences between two proportions [online]. 2011 Aug [cited 2012 Jan 5]. Available from Internet: http://www.ucl.ac.uk/english-usage/staff/sean/resources/comparing-x2-tests.pdf.
  8. Benner P. From novice to expert, excellence and power in clinical nursing practice. Menlo Park (CA): Addison-Wesley Publishing Company; 1984:104.
  9. Kunkel JH, Nagasawa RH. A behavioral model of man: propositions and implications. Am Sociol Rev 1973 Oct;38(5): 530-43.
  10. Wiemken T. Statistical process control. Chapter 6. In: APIC text of infection control and epidemiology, 3rd edition. Washington (DC): Association for Professionals in Infection Control and Epidemiology; 2009:6.1-6.10.
  11. Harpel J, Reilly T, Hendler C, et al. Transforming a team: a key to reducing central line-associated bloodstream infections. Abstract at: SHEA Fifth Decennial Conference on Healthcare-Associated Infections; 2010. Abstract No. 458.
  12. Streed SA. Metrics and management: two unresolved problems on the pathway to health care-associated infection elimination. Am J Infect Control 2011 Oct;39(8):678-84.
PSA LOGO

The Pennsylvania Patient Safety Advisory may be reprinted and distributed without restriction, provided it is printed or distributed in its entirety and without alteration. Individual articles may be reprinted in their entirety and without alteration, provided the source is clearly attributed.

Current and previous issues are available online at http://patientsafety.pa.gov.

©2019 Pennsylvania Patient Safety Authority