Pa Patient Saf Advis 2009 Dec;6(4):122-5.
Using Administrative Data from Pennsylvania Hospitals to Monitor Patient Safety
Infectious Diseases; Surgery
Expand All Collapse All

Introduction

Since the Pennsylvania Patient Safety Authority was established, the most challenging question asked of its staff has been whether healthcare in Pennsylvania is becoming safer. This question is not unique to Pennsylvania, nor is it unique to the United States. Experts in patient safety are forced to admit that while progress has been made since the 1999 publication of the Institute of Medicine’s To Err Is Human, improving patient safety is a journey that is just beginning.

The ultimate measures of safety are the number of lives saved or the number of injuries prevented, but these measures are notoriously difficult to estimate reliably in a cost-effective way. The sources of data typically reviewed for evidence of improvement are all imperfect. Adverse event reports are subject to underreporting and variation in interpretation of reporting requirements. Survey data on structural or process measures, as presented in the Authority’s 2008 annual report, is subject to response bias, the selective memory of the respondent, and many other biases inherent in all survey research. Even retrospective expert review of medical charts, often used as the gold standard in research on adverse events, is subject to the validity of the decision rules used by the reviewers and the quality of the documentation in the patient records.

While all these sources of data are imperfect, each can provide a unique perspective on the safety and resilience of the healthcare system. While each source on its own is too flawed to rely on in isolation, when taken together they can paint a richer portrait of the problems faced in patient safety and whether there is progress in resolving them.

Another source of information readily available to all hospitals is uniform administrative data used in billing. Under contract to the Agency for Healthcare Research and Quality (AHRQ), researchers from Stanford University and the University of California developed the Patient Safety Indicators (PSIs) as a tool to identify potentially preventable adverse events related to hospitalization. These indicators are based on records that hospitals complete on all inpatient discharges. While administrative systems were not designed to identify adverse events, by screening patients’ diagnoses and what services they received, the PSIs identify by inference patients who may have suffered selected adverse events.

As with other sources of patient safety information, administrative data is subject to technical limitations. These include variations in coding practices at different institutions and by different individuals, errors in coding, and the quality of the underlying medical records on which the administrative data is based. Refer to the section “Technical Notes and Limitations” for further detail.

The PSIs that can be used at a state or regional level (referred to as the “area level” indicators) are as follows:

  • Accidental Puncture or Laceration
  • Foreign Body Left during Procedure
  • Iatrogenic Pneumothorax (i.e., collapsed lung)
  • Postoperative Hemorrhage or Hematoma (i.e., bleeding)
  • Postoperative Wound Dehiscence (i.e., rupturing of the suture line following surgery)
  • Selected Infections due to Medical Care (primarily related to intravenous lines and catheters)
  • Transfusion Reaction (due to blood incompatibility)

These PSIs provide one window into the safety of Pennsylvania hospitals, and over time one hopes to see these rates decline, suggesting that safety is improving. Because of differences between the PSI definitions and how reportable events are defined under Pennsylvania’s MCARE (Medical Care Availability and Reduction of Error) Act of 2002, direct comparisons with the reports submitted to the Authority are not appropriate. What the PSIs provide is an independent source of information about patient safety. Use of multiple data sources can help ensure greater confidence in potential trends; changes observed in any single source of data are more likely to be genuine and meaningful when corroborated by changes observed in other independent sources.

Overall, the evidence for improvement in these PSIs over the past several years is mixed and uncertain. Some PSIs, such as Transfusion Reaction and Postoperative Wound Dehiscence, seem to have declined, suggesting a move in the right direction. Yet others, such as Selected Infections due to Medical Care and Postoperative Hemorrhage or Hematoma, seem to be trending upward. However, all linear trend lines that were fit to these indicators failed tests for statistical significance, leaving no convincing evidence that the apparent trends in the data are due to anything other than chance.* The Figure presents the rates of these complications from 2002 through 2007.


Figure. Patient Safety Indicators for Pennsylvania Hospitals, Rates per 100,000 Population (2002 to 2007)

Figure. Patient Safety Indicators for Pennsylvania Hospitals, Rates per 100,000 Population (2002 to 2007)

 
Even if the apparent declines in some of these complication rates were statistically significant, the improvement would be only moderate (though encouraging). Table 1 shows the PSIs with the percent change between 2002 and 2007 and with the number of cases avoided or added based on the percent change over this period. Those with the greatest percent change are not necessarily those in which the most improvement would have occurred. For example, a decline in the rate of Transfusion Reactions per 100,000 population from 0.08 in 2002 to 0.02 in 2007 represents a 76% decline and 3 injuries avoided. In comparison, cases of Accidental Puncture or Laceration, which occur more frequently, declined about 5% from 2002 to 2007, but this equates to 72 cases avoided.

Table 1. Change in Patient Safety Indicator Rates ​ ​ ​
Patient Safety Indicator % Change In Observed Rate (2002 To 2007) Linear Trend Slope* Number Of Cases Avoided/Added (2007)
Accidental Puncture or Laceration -4.8-0.1572 avoided
Foreign Body Left during Procedure -9.3-0.0210 avoided
Iatrogenic Pneumothorax -0.80.0420 added
Postoperative Hemorrhage or Hematoma +0.70.1678 added
Postoperative Wound Dehiscence -9.3-0.0522 avoided
Selected Infections due to Medical Care +2.90.34163 added
Transfusion Reaction -75.6-0.013 avoided

* A Student t-test was performed on the observed slope for each indicator, all of which were found to be not statistically significant at the α = 0.05 level.

Refers to the difference between the number of cases predicted for 2007 based on the linear trend lines shown in the Figure and the number that would have been predicted for 2007 if 2002 predicted rates had stayed constant (i.e., if linear trend lines were flat).

 

Data from Pennsylvania is on a par with the most recent national data available (see Table 2). While the observed rates in Pennsylvania for most PSIs are slightly higher than national estimates, hospital discharge coding practices vary between hospitals and between states. Therefore, tracking changes in the same set of institutions over time is more meaningful than making comparisons between hospitals or geographic regions.

Table 2. Patient Safety Indicators, Comparison of Pennsylvania Observed Rates and National Estimated Rates ​ ​
Patient Safety Indicator Pennsylvania Observed Rate Per 100,000 (2007)* National Estimated Rate Per 100,000 (2006)
1. Accidental Puncture or Laceration 57.3648.08
2. Foreign Body Left during Procedure 1.531.53
3. Iatrogenic Pneumothorax 10.618.09
4. Postoperative Hemorrhage or Hematoma 18.9316.11
5. Postoperative Wound Dehiscence 3.182.48
6. Selected Infections due to Medical Care 37.6129.82
7. Transfusion Reaction 0.020.06

* The Pennsylvania observed rate is the actual number of cases meeting the Patient Safety Indicator inclusion criteria divided by the Pennsylvania population as published in the U.S. Census; it is not risk-adjusted for differences between the Pennsylvania and U.S. populations.

National rates from: Agency for Healthcare Research and Quality (AHRQ). (1-3; 5-7) HCUPnet, Healthcare Cost and Utilization Project, QI summary tables [online]. [cited 2009 Aug 5]. Available from Internet: http://hcupnet.ahrq.gov; (4) PSI comparative data for area indicators, ver. 3.1 [online]. 2007 Mar 12 [cited 2009 Mar 30]. Available from Internet: http://www.qualityindicators.ahrq.gov/downloads/psi/psi_area_comparative_v31.pdf.

Based on 2004 data; 2006 data unavailable for this indicator.

 

_______________
* For each indicator, a linear trend line was fit to the data, and a Student t-test was performed on the slope of each trend line, testing the hypothesis that the slope was different from 0 at the
α = 0.05 level.

Technical Notes and Limitations

The observed rates of complications presented here are subject to the limitations inherent in all hospital discharge data. The primary concern is with the accuracy of discharge-based diagnosis coding. Errors made in individual institutions’ discharge abstraction may bias the rates calculated using those data sources. As with any source of patient safety data, it is not possible to identify all relevant adverse events without some false positives and false negatives.

It is not possible to distinguish, in this data, cases that represent preventable adverse events from those representing adverse events that are not preventable. Likewise, it is not possible to distinguish cases that represent medical errors from cases in which no error occurred. For these reasons, it is not expected that the number of potential adverse events identified in the PSIs would equal the number of reports submitted to the Authority during the same time period. The statutory definition of events reportable to the Authority requires healthcare providers to assess whether adverse events were unanticipated, whether they require additional healthcare services, and whether they compromise patient safety.

These rates do not take into account the “Present on Admission” (or POA) indicator, which identifies in each patient’s discharge abstract the diagnosis codes that were present when the patient was admitted to the hospital. While hospitals were required to report this indicator starting in October 2008, it is not yet included in publicly available discharge data. Of the area-level indicators, the POA indicator is used only as an exclusion criterion for Selected Infections due to Medical Care, and it would not affect calculations of the other indicators.

The Authority calculated the rates of the seven area-level PSIs using data provided by the Pennsylvania Health Care Cost Containment Council (PHC4). The rates were calculated for the years 2002 through 2007, the most recent full year for which data was publicly available. Rates were calculated using AHRQ’s Quality Indicators software, Windows version 3.2a (Rockville, Maryland). For more information about the AHRQ Quality Indicators, visit http://www.qualityindicators.ahrq.gov.

PHC4 is an independent state agency responsible for addressing the problem of escalating health costs, ensuring the quality of healthcare, and increasing access to healthcare for all citizens regardless of ability to pay. PHC4 has provided data to the Authority in an effort to further PHC4’s mission of educating the public and containing healthcare costs in Pennsylvania. PHC4, including its agents and staff, has made no representation, guarantee, or warranty, express or implied, that the financial, patient, payer, and physician-specific data provided to the Authority is error-free, or that the use of the data will avoid differences of opinion or interpretation. This analysis was not prepared by PHC4. This analysis was done by the Authority. PHC4, including its agents and staff, bears no responsibility or liability for the results of the analysis, which are solely the opinion of the Authority.

Supplemental Material

Using the PSIs in Your Hospital

The Agency for Healthcare Research and Quality’s Quality Indicators software tool, which includes the Patient Safety Indicators (PSIs), is distributed free of charge. The software can be used to help hospitals identify potential adverse events that might need further study. The software programs can be applied to any hospital inpatient administrative data. This data is readily available and relatively inexpensive to use.

In addition to the seven area-level PSIs discussed in this article, additional measures valid for use at the level of individual institutions are available, including:

  • Complications of Anesthesia (PSI 1)
  • Death in Low-Mortality DRGs (PSI 2)
  • Decubitus Ulcer (PSI 3)
  • Failure to Rescue (PSI 4)
  • Foreign Body Left during Procedure (PSI 5)
  • Iatrogenic Pneumothorax (PSI 6)
  • Selected Infections due to Medical Care (PSI 7)
  • Postoperative Hip Fracture (PSI 8)
  • Postoperative Hemorrhage or Hematoma (PSI 9)
  • Postoperative Physiologic and Metabolic Derangements (PSI 10)
  • Postoperative Respiratory Failure (PSI 11)
  • Postoperative Pulmonary Embolism or Deep Vein Thrombosis (PSI 12)
  • Postoperative Sepsis (PSI 13)
  • Postoperative Wound Dehiscence in Abdominopelvic Surgical Patients (PSI 14)
  • Accidental Puncture or Laceration (PSI 15)
  • Transfusion Reaction (PSI 16)
  • Birth Trauma—Injury to Neonate (PSI 17)
  • Obstetric Trauma—Vaginal Delivery with Instrument (PSI 18)
  • Obstetric Trauma—Vaginal Delivery without Instrument (PSI 19)
  • Obstetric Trauma—Cesarean Delivery (PSI 20)

The software is available in SAS® and Microsoft Windows® formats. User guides and technical documentation are available. Visit    http://www.qualityindicators.ahrq.gov/software.htm.

PSA LOGO

The Pennsylvania Patient Safety Advisory may be reprinted and distributed without restriction, provided it is printed or distributed in its entirety and without alteration. Individual articles may be reprinted in their entirety and without alteration, provided the source is clearly attributed.

Current and previous issues are available online at http://patientsafety.pa.gov.

©2019 Pennsylvania Patient Safety Authority