Pa Patient Saf Advis 2011 Dec;8(4):138-43.
Gap Assessment of Hospitals’ Adoption of the Just Culture Principles
Authors

Denise M. Barger, BA, CPHRM, CPHQ, HEM
Patient Safety Liaison
Southeast Region—Delaware Valley South

William Marella, MBA
Program Director
Franchesca J. Charney, RN, BS, MSHA, CPHRM, CPHQ, CPSO, FASHRM
Director of Educational Programs

Pennsylvania Patient Safety Authority

Background

Patient safety experts recognize that a healthcare organization’s culture exerts a critical influence on staff response to patient safety issues, as well as on staff members involved in those events.1,2 Willingness to report both actual and potential adverse events can be a strong indicator of the organization’s attitude toward patient safety generally and a key to its perception and treatment of staff involved in adverse events. Organizations with strong safety cultures have robust reporting mechanisms to identify risks and effective systems for evaluating causes and taking action to address process weaknesses. They develop and reinforce the perception among staff that reporting is accepted, expected, and nonpunitive. In the alternative “blame and shame” environment, adverse events go unreported and process failures are not identified, causes go unaddressed, and the cycle of event-blame-punishment is often repeated to the detriment of patients and healthcare staff alike.

Many healthcare organizations have made the transition from the punitive culture that dominated thinking in the years before the Institute of Medicine’s landmark report To Err Is Human to the nonpunitive stance that many hospitals now take toward reporting. Some hospitals, such as those in the Department of Veterans Affairs system, have evolved to a view that does not punish individuals for reporting or committing human errors and mistakes and instead choose to identify the reasons for the error and to change the underlying process that either caused or contributed to the error.3 Recognizing the inevitability of human error while still holding staff accountable for individual actions is the cornerstone of the just culture approach that has been articulated by Outcome Engineering president David Marx. Outcome Engineering has developed an algorithm for assessing the role of human behavior in individual events, as well as a methodology for evaluating an organization’s culture and commitment to a just culture that can be used by hospitals and other facilities.4

In the Pennsylvania Patient Safety Authority’s 2007 survey of Pennsylvania healthcare organizations reported in the Authority’s 2008 annual report, facilities were asked whether their internal policies and procedures related to adverse event reporting incorporated just culture principles.1 Statewide, 118 hospitals and 82 other facilities (including ambulatory surgical facilities [ASFs] and birthing centers) responded to the survey. The majority of hospitals that responded (70%) reported some level of implementation of a just culture, and 59% reported that the just culture model was fully implemented hospital-wide. The remaining 30% of hospitals reported that the principles were not yet implemented. Similar results were found among responding ASFs and other facilities, with 72% reporting some or partial implementation and 28% having not adopted the just culture approach.

The Authority wanted to ensure that Pennsylvania facilities fully understood the tenets of a just culture and sought out Outcome Engineering to discuss the survey results. That discussion was the genesis for the Pennsylvania Just Culture Project. The project, which began in spring 2010 and concluded with a report of gap survey results in February 2011, tested whether Pennsylvania facilities have more verbal commitment to just culture than is codified in facility policies and reflected in facility handling of adverse events and staff error. Indeed, during the process of identifying hospitals interested in participating in the project, many shared the view that they might have overstated the extent of their implementation.

Methods

The Authority partnered with Outcome Engineering in early 2010 to assess the degree to which a just culture was implemented in Pennsylvania. The Authority challenged Outcome Engineering to devise a method that could be used on a larger scale and that would protect hospitals’ confidentiality while assessing their adherence to a just culture’s essential tenets. Meanwhile, the Authority invited all Pennsylvania hospitals to consider participating in the project and, ultimately, 10 hospitals volunteered. Hospital patient safety officers (PSOs) were engaged to lead the assessment effort in each facility and the gap analysis and be the conduit for communicating results. The hospitals represented different regions of the commonwealth and ranged from large urban hospitals to smaller community hospitals.

The just culture self-assessment tool developed by Outcome Engineering was based on two existing survey tools: a gap analysis and a benchmarking survey. Outcome Engineering developed the tool specifically for this project to address concerns that traditional methods of assessing a hospital’s culture might breach confidentiality and required a significant time commitment. The just culture self-assessment tool comprises two parts:

Part 1 measures organizational culture through 13 questions about organizational policies, adverse event investigations, and human resources actions. This section is completed by the PSO after reviewing a representative sample of documents related to these three areas.

Part 2 measures the perceptions of leaders about the organization’s culture through 20 questions about critical behavioral markers, such as system design, coaching, reporting, responses to human error, responses to reckless behavior, severity bias, equity, and transparency. This section is completed by 10 to 15 leaders within each organization. Recommended respondents include the chief nursing and medical officers, PSOs, and directors or managers of human resources, quality assurance, and risk management departments. (No information identifying individuals by name, position, or title was collected.)

Participating hospitals received their survey forms in January 2011 and were given three weeks to complete the survey tasks. The surveys were then reviewed and scored by Outcome Engineering working with Authority staff. Results were tabulated and presented to the participating hospitals in late February 2011. After completing the self-assessment, each hospital received a confidential report presenting its results compared with the average results of the other deidentified participating facilities. This report explained the significance of each attribute examined in the tool and provided guidance on how to improve attributes on which the facility scored low.

The survey tool is available to Pennsylvania PSOs on the Authority’s secure PassKey website. The Authority’s regional patient safety liaisons can assist facilities in the use and scoring of the assessment.

Results

The two parts of the tool were scored separately. Part 1, which evaluated the hospitals policies and practices, contained elements that could produce a maximum score of 22 points for each hospital. Only one of the participating hospitals scored well—it earned 20 points, indicating compliance with key just culture tenets in policies, human resources practices, and investigation documentation. Two hospitals met the required adherence on approximately 50% of the scored items, while the majority of hospitals (seven) met just culture expectations on fewer than 50% of the elements.  Six hospitals scored below 5 of a possible 22 points. All 10 participating hospitals as a group scored 62 of a total 220 points (see Table 1).

Table 1. Part 1: Patient Safety Officer Assessment of Just Culture Principles Based on Document Review ​ ​ ​ ​ ​


Just Culture Attribute


Question
Hospitals
Answering
"Yes"
Hospitals
Answering
"No"

Question Weight
Score Across All 10 Hospitals
Policies
A just culture organization avoids using certain terms that can be misunderstood or not aligned with the principles of the model. Are the following terms used in your disciplinary policies: negligent, careless, criminal conduct, egregious? (Reverse worded: "No" preferred) 4616
A just culture organization makes a distinction between values supportive discussions
with employees to influence behavior and those conversations that are intended as steps in a disciplinary process.
Do your organization's human resource policies distinguish between coaching and counseling? 1911
A just culture organization defines the three manageable behaviors: human error, at-risk behavior, and reckless behavior.*Do your organization's policies define the following behaviors: human error, at-risk behavior, reckless behavior?1933
A just culture organization expects justifiable breaches of policies and procedures to
occur and provides clear examples.
Do your organization's policies provide clear examples of justifiable violations of policies and procedures? 4614
A just culture organization emphasizes the need to improve system design while
simultaneously managing human behavior.
Do your organization's policies emphasize both system design and the management of employee behavior? 2824
Event Investigations
A just culture organization investigates and explains the causes of human errors. Does your event reporting system require explanations for each human error identified?1922
A just culture organization investigates and explains the causes of at-risk behaviors
and procedural deviations.
Does your event reporting system require explanations for each at-risk behavior and/or procedural deviation identified?2824
Human Resource Actions
A just culture organization recognizes and avoids the severity bias. It is the quality
of the choice involved in the behavior that determines the level of response to an employee, not the actual harm that results.
Does your organization's disciplinary response to employees consistently depend on the quality of the choices involved in their behaviors, irrespective of actual harm that occurs? 3739
Does evidence suggest that your organization's employees have not been disciplined for human errors, unless reckless choices were contributory?8218
Does evidence suggest that your organization consistently takes disciplinary action with employees who have made a reckless choice?6416
A just culture organization consoles an employee who makes a human error and examines both the quality of the choices involved in the behavior as well as the design of the system around the employee. Do managers in your organization consistently console employees who make human errors and examine both the choices involved and the system designed around the employee? 3713
A just culture organization coaches an employee who makes an at-risk behavioral choice and examines both the incentives for the employee's choice and the design of the system around the employee. Do managers in your organization consistently coach employees who make an at-risk behavioral choice and also examine the incentives for the employee's choice and the design of the system around the employee? 2836
A just culture organization places an employee on notice of disciplinary action when repetitive human errors or repetitive at-risk behaviors are present and not caused
by system performance shaping factors and not correctable through changes in work choices, remedial education, or coaching.
Do managers in your organization consistently place employees on notice of disciplinary action when repetitive human errors or repetitive at-risk behaviors are present and not caused by system performance shaping factors and not correctable through changes in work choices, remedial education, or coaching?6416

* Human error—an inadvertent action; inadvertently doing other than what should have been done; a slip, lapse, or mistake
    At-risk behavior—a behavioral choice that increases risk where risk is not recognized or that is mistakenly believed to be justified
   Reckless behavior—a behavioral choice to consciously disregard a substantial and unjustifiable risk

† The severity bias is present when the severity of the actual outcome influences how we think about the person involved or how we respond to them if we have  managerial authority over them. In other words, the level of actual harm determines whether discipline or punishment is used.

 

While none of the just culture principles was consistently present across all 10 hospitals, elements most widely adopted included not disciplining employees for human errors in the absence of reckless choices, taking disciplinary action with employees who have made reckless choices, and placing employees on notice of disciplinary action when repetitive human errors or repetitive at-risk behaviors are present and not caused by system performance shaping factors and not correctable through changes in work choices, remedial education, or coaching. However, only one hospital reported that their policies define human error, at-risk behavior, or reckless behavior. If staff do not understand the distinctions between these types of errors—or the distinctions management makes among them—human resources actions may appear arbitrary. Other principles of the just culture model that were not widely adopted were distinguishing between discussions aimed at coaching versus counseling, requiring explanations in event reporting systems for human errors and at-risk behaviors, and emphasizing both system design and management of employee behavior.

Part 2 involved a series of 20 statements to be evaluated by key leaders in the organization to assess their perception of the organizational culture. For each statement, response categories were presented on a five-point Likert scale ranging from “strongly disagree” (-2) to “strongly agree” (2) with a neutral value (0) for “neither agree nor disagree.” Most statements were worded positively, with “strongly agree” being the preferred response. Those statements worded negatively were scored in reverse. The maximum number of points available for each hospital was 40. Results of this section are presented in Table 2.

Table 2. Part 2: Survey of Hospital Leaders ​ ​ ​


Evaluation Statements
Lowest Hospital ScoreHighest
Hospital Score
Average
Hospital Score
1.   Managers in this organization discipline employees who make mistakes that might impact patient safety.*-0.790.09-0.43
2.   When a safety concern is reported, the way we work is changed to make things safer.0.721.761.20
3.   If employees are doing something unsafe, their managers will talk to them and explain a safer way to behave or work.0.71.381.06
4.   If employees are doing something unsafe, their coworkers will talk to them and explain a safer way to behave or work. -0.220.840.26
5.   Managers in this organization treat all employees and staff, regardless of their position in the hospital, fairly after an event involving harm to a patient.0.01.381.01
6.   Over the past 12 months, this organization has reduced its number of safety events resulting in harm to patients.0.01.60.90
7.   Employees and staff at this organization are reporting things they see that could impact the safety of the patients.0.01.30.96
8.   This organization looks into "close calls"—things that could have harmed the patients but did not—to understand the underlying causes.0.01.821.26
9.   Physicians are less likely than other staff to be disciplined in similar circumstances.*-1.110.69-0.40
10. Managers in this organization talk to employees and staff about adverse events and lessons learned.0.531.310.95
11. Managers in this organization discipline employees and staff who intentionally endanger safety, whether or not harm occurs. 0.81.431.16
12. Managers in this organization address safety events only if a patient is seriously harmed.*-0.171.30.75
13. Employees will report their own mistakes that could have resulted in patient harm. -0.110.710.31
14. Employees will report their own mistakes that did result in patient harm.0.211.170.83
15. Occasionally our core organizational values will be in conflict.-0.450.80.15
16. Some patient safety events are 100% preventable.*,†-1.75-0.5-1.33
17. Our employees know they will be consoled if they make a human error.-0.510.24
18. Our employees know they will be coached if they engage in at-risk behavior (e.g., taking shortcuts).0.010.54
19. Our employees know they will be disciplined for reckless behavior regardless of whether harm results.0.51.230.96
20. There is never an acceptable reason for an employee to violate patient safety policies and procedures.*-1.540.17-0.82
Score Sum (maximum possible = 40)   9.56

* Reverse worded and reverse scored, so that a higher score is always indicative of higher just culture alignment.

† The project team determined after survey administration that this question, on which hospitals scored the lowest, would have been better worded to read, "Some human errors are 100% preventable." The just culture model incorporates the notion that humans are fallible and will always make errors. Systems should be improved so that they are resistant to such errors without resulting in patient harm.


The maximum score achievable was 40. No participating hospital scored well on these elements of the survey. The average score for all participating hospitals was 9.56 or only 24% of the 40 possible points. Six of the 10 participating hospitals scored slightly higher than the project average; 4 had scores significantly lower than average. The highest score from any hospital was 15.41, while the lowest was 3.19 of the total 40 possible points.

Based on the average scores, the aspects of their organizational culture hospital leaders rated most positively were investigating “close calls” to understand the underlying causes, changing work practices to improve safety when concerns are reported, and disciplining employees who intentionally endanger safety regardless of whether harm resulted. The negative aspects of organizational culture included disciplining employees for mistakes, treating physicians more leniently than other staff in similar circumstances, and believing there is never an acceptable reason for employees to violate safety policies and procedures.

This last item, while seeming to hold employees accountable for willful violations of procedure, is problematic because it encourages following the rules for their own sake even in situations in which the rules do not serve the greater good or when rules conflict with one another. For example, if a patient were falling, the correct action is to prevent the fall, even if this means ignoring normal hand hygiene protocols. The item on which participating hospitals scored lowest was the belief that some patient safety events are 100% preventable, which fails to acknowledge that human errors are inevitable.

Conclusion

The results supported the Authority’s perception that Pennsylvania hospitals may have overestimated the degree to which the hospital is in alignment with core principles of the just culture approach. Some hospitals’ scores revealed significant gaps in multiple just culture elements, while others can focus on a few key points to strengthen a solid foundation. The self-assessment results along with the suggested improvement strategies provided by Outcome Engineering can help participating hospitals’ PSOs identify their organizations’ weaknesses and set a plan for working with hospital leaders to improve their culture. Despite these gaps, the 10 hospitals in this project voluntarily chose to participate, in part because the PSOs recognized that their culture was not as aligned with the just culture principles as it could be. These hospitals are to be commended for their willingness to be self-critical and to focus on their shortcomings. These are among the defining characteristics of high-reliability organizations.

Overall, the results of this project suggest that work remains to be done to bring Pennsylvania hospitals and other healthcare facilities closer to achieving a just culture in healthcare. While this project focused on a small sample of the state’s hospitals, the results suggest that facilities may overestimate their implementation of key principles of the just culture model. With reporting a crucial feature of a culture of patient safety and the need to focus on process design rather than human error to reduce adverse events for patients, the results suggests that there is room to improve staff awareness of the value of reporting, the need to focus on system process redesign, and the nature and cause of human error.

Notes

  1. Pennsylvania Patient Safety Authority. 2008 annual report [online]. 2009 Apr [cited 2011 Jul 5]. Available from Internet: http://patientsafetyauthority.org/PatientSafetyAuthority/Documents/annual_report_2008.pdf.
  2. Agency for Healthcare Research and Quality. Hospital survey on patient safety culture: 2009 comparative database report [online]. [cited 2011 Jul 5]. Available from Internet: http://www.ahrq.gov/qual/hospsurvey09/hosp09ch1.htm.
  3. Marx D. The just culture community [online]. [cited 2011 Jul 5]. Available from Internet: http://www.outcome-eng.com/justculture.html.
  4. The just culture algorithm™ [online]. [cited 2011 Jul 5]. Available from Internet: http://www.justculture.org/algorithm.aspx.
PSA LOGO

The Pennsylvania Patient Safety Advisory may be reprinted and distributed without restriction, provided it is printed or distributed in its entirety and without alteration. Individual articles may be reprinted in their entirety and without alteration, provided the source is clearly attributed.

Current and previous issues are available online at http://patientsafety.pa.gov.

©2019 Pennsylvania Patient Safety Authority