Continuous Quality Improvement

CQI and Performance Enhancement in State Substance Abuse Treatment Systems
DeltaMetrics is a national research and consulting organization dedicated to the measurement and improvement of treatment effectiveness in the behavioral health care field.  A continuing focus of the organization is the design and implementation of systems and methodologies to measure the outcomes of treatment for a variety of populations, in different types of treatment.

Beginning in the late 1990s, DeltaMetrics began advancing this central focus beyond one-time measurement and evaluation.  Through its carefully developed understanding of how treatment systems are organized and an appreciation of the important clinical aspects of treatment, it became clear that by using treatment process data, patient demographic and historical data, financial data, outcomes data and other quantitative measurements, DeltaMetrics could assist its clients in establishing programs of Continuous Quality Improvement (CQI) consisting of a series of “Performance Enhancement” (PE) initiatives.

A program of CQI is essentially one that transforms an organization into a learning organization by creating a framework for continuous comparison of experiences against internal and external standards.

The CQI Initiative
At the core of a CQI strategy is a feedback loop through which empirical data are systematically collected and interpreted so that they specifically suggest or guide the strategic application of PE interventions.  After adopting a particular intervention, its effectiveness is evaluated and the strategy refined, leading to additional performance enhancement opportunities.  The result is a systematic and continuous process of quality improvement.  The major elements of a successful CQI initiative are:

1. Collection of a core set of data elements that can identify and guide PE opportunities in a treatment system.  In order to implement CQI, an organization must collect data that enables it to measure performance in terms of items such as:

Patient Characteristics

  • Prognostic Indicators
  • Demographics
  • Diagnosis 
  • Severity

Process Measures

  • Type and quantity of treatment services provided
  • Length of stay

Financial Measures

  • Cost of services
  • Cost offsets (e.g. reduction in crime)

Outcome Measures (at discharge and follow-up)

  • Substance usage
  • Social functioning
  • Economic functioning

Staff Measures

  • Staff type
  • Staff credentials
  • Administrative procedures

Because a CQI strategy that relies on recurring assessment of patient outcomes at follow-up as a primary data source is ideal, but very expensive, DeltaMetrics is employing less costly approaches to CQI that can be just as effective.  One such approach uses performance indicators (PIs), or data elements routinely collected within the system for which there is evidence of some value in predicting patient outcomes.  For example, there is substantial evidence to suggest that retention in substance abuse treatment is a good predictor of patient outcome.  Consequently, the percentage of patients retained in treatment for the intended period of time (e.g., 1 month) may be employed as one PI.  It is important to monitor and reevaluate PIs from time to time to make certain that they retain value as surrogates for patient outcomes.  Small, focused, and more intensive studies on selected samples of patients can serve that purpose, aid in the interpretation of PIs and more accurately define the impact of systems changes on outcomes.
2. Analyze and interpret core data to uncover actionable targets for quality improvement and implement interventions.  When data on a  delivery system, its clinician practices, or its provider characteristics are linked to client outcomes, analysis can uncover actionable targets for quality improvement.  Such targets could include the adoption of more effective treatment practices, or the need to increase relevant clinical skills.  Analysis of PIs, administrative procedures, and other provider characteristics should suggest actions on the part of the providers to improve care.  Such actions include:

  • Adopting procedures to minimize wait time and thus increase treatment accessibility;
  • Developing and using a referral network of providers to address a broader spectrum of a client’s multiple needs as part of their course of care;
  • Developing a series of standardized practices such as screening for co-morbid disorders;
  • Formalizing treatment planning with routine and monitored reevaluations of treatment plans on a regular basis. 

The simple table below illustrates how data generated under a CQI system can be used to develop performance-enhancing treatment interventions in a substance abuse treatment system:  

Diagnosis: 123.4
Treatment Modality: 1
Period: 1/1/00 to 2/28/00

 Provider ID

 Number Of Cases

FT Employed at 6 months post discharge (%)

PT Employed at 6 months post discharge (%)

Not Employed at 6 months post discharge (%)

  Average Cost Of Treatment ($000)































One intervention suggested by the data in this example is a hands-on training program for clinicians to improve employment status outcomes at six months post discharge.  Such a training program could draw upon empirically-based treatment protocols, such as those that are available at no cost from the Center for Substance Abuse Treatment and other federal agencies, or from the research community. 

However, another form of intervention may be suggested if the question is asked "What does this data tell us about who is our best provider from the point of view of achieving good employment status outcomes at 6 months post discharge for the lowest cost?"  If we get the same result consistently, then that provider may have something to offer by way of training to the other providers that might enhance across-the-board performance on this measure.

It must be emphasized, however, that differences at baseline make comparisons between different groups of patients in a  system difficult to interpret. Thus, in the above example, as in any PE intervention, case mix adjustment strategies must be used to control for biases that may exist in the analyses because of initial patient differences. Case-mix adjusted outcomes data can also be used to identify targets for performance improvement through qualitative analytic strategies, such as benchmarking.  As illustrated in the above example, providers that perform in the top and bottom quintiles on an outcomes dimension can be identified and their practices and characteristics compared through in-depth semi-structured interviews.  Differences between the two groups of performers can reveal programmatic elements or clinical practices that would serve as a model for those who may need to modify their practices in order to improve patient outcomes.

3. Initiate a continuous feedback process so that each intervention helps to modify or inform subsequent plans, actions and data collection.  By design, a CQI strategy is an ongoing and dynamic enterprise. The approach is to define the appropriate measures, monitor them on a continuous basis, and evaluate them for changes over time, and whether the changes are process- or content-based.  In the previous example, an intervention consisting of peer-directed, best practices workshops on improving employment outcomes might be pilot tested with a subset of clinicians and the results measured through a targeted outcomes evaluation.  Empirical-based modifications to the intervention might then be applied system-wide and be periodically evaluated. 

The application of one intervention may also yield additional targets for improvement.  Application of an intervention to improve employment outcomes, for example, may yield data showing that the skill sets of clinicians differ from the needs of their clients.  For example, a provider organization that has experienced a recent increase in users of a particular substance might not be trained in the most current empirical-based therapeutic strategies for treating its abuse.  Similarly, other changes in client characteristics might be better addressed with new skills.  Consequently, after identifying the skills of providers and the characteristics of clients, the need for more appropriate clinical interventions could be identified and strategies disseminated. 

To learn more about CQI programs for your systems of care, contact Richard Weiss, DeltaMetrics Director of Research and Evaluation, at 215-399-0988.

 ~ About DeltaMetrics ~ Services ~ Clients & Projects ~ Careers ~ Contacting DeltaMetrics ~ Home ~

Copyright © 2000~2012 DeltaMetrics
600 Public Ledger Building ~ 150 South Independence Mall West
Philadelphia, PA 19106-3475
Phone 215-399-0988 ~ Fax 215-399-0989 ~ Contact via Internet