uSPEQ® GREW OUT OF WORK ON PERFORMANCE INDICATORS
VOICES OF PERSONS SERVED, PROVIDERS, AND PAYERS GUIDED DEVELOPMENT
uSPEQ arose from a decade of extensive work on performance indicators conducted by CARF International, an independent accreditor of human service providers. In developing uSPEQ, CARF was guided by the voices of organizations, payers, and, most importantly, persons served by human service providers.
1996-1998: CARF convened two national leadership panels, a working conference for performance indicators, and an advisory council. Each venue involved stakeholders from across the fields and program areas CARF accredits. Although the objectives of each leadership panel varied, CARF's overall goal was to seek advice on developing performance indicators to be applied in the human service field as part of CARF's activities to carry out its strategic outcomes initiative. These activities culminated in the publication of the Performance Indicators for Rehabilitation Programs (PDF).
CARF developed and first administered (in private industry) the original employee survey upon which the uSPEQ Employee Climate Survey is based.
1999-2000: CARF conducted consumer forums in Washington, D.C., and Boston in collaboration with Boston University's Center for Psychiatric Rehabilitation. Findings from these forums led to the identification of indicators highlighted as most important by consumers. The "draft keepers," as they came to be known, represented a more compact set of indicators that now revealed striking commonalities across interest groups of many consumers, providers, payers, and other stakeholders.
2002: CARF renewed its performance indicators efforts, beginning with a review of developments and literature in the field. In this phase, CARF aligned its indicators with major national and international conceptual frameworks, including Healthy People 2010, Disability and Secondary Conditions, Chapter 6; World Health Organization/International Classification of Functioning, Disability and Health; Institute of Medicine quality framework domains; CARF business practices standards; and CARF's ends policies. A crosswalk between CARF's performance indicators and performance measurement vendors and systems was also completed.
2003: A performance indicators workgroup composed of consumers, experts, and researchers met to review and refine the work accomplished the previous year. Following this meeting, two rounds of consensus-developing discussions, using a modified Delphi process, were conducted to recommend a set of potential subjective items to create a crosscutting consumer questionnaire. The ensuing draft of the consumer questionnaire contained both demographic questions and survey items on which the consumer would rate his or her experience with services received and the outcomes he or she attained.
2004-2005: Providers in the human service field participated in pilot testing of the draft of the consumer questionnaire, by then named uSPEQ. The pilot results were compiled in the uSPEQ Consumer Experience Survey development and technical report .
2006: CARF introduced subscriptions to a full range of uSPEQ Consumer Experience Survey data services.
2006-2007: Following nine consecutive years of administration with only minor revisions, the content of the uSPEQ Employee Climate Survey was refined through extensive literature review and research on factors associated with and components of employee satisfaction. A variety of organizations participated in a pilot test of the survey. The pilot results were compiled in the uSPEQ Employee Climate Survey development and technical report .
2007: CARF introduced subscriptions to a full range of uSPEQ Employee Climate Survey data services.
2008: uSPEQ Consumer Experience Survey underwent another round of data analysis in an effort to reduce the length of the Tier 1 universal items and, at the same time, to validate the psychometric properties of the refined questionnaire. Both qualitative and quantitative analyses were conducted, resulting in uSPEQ 2.0. The new version features a refined set of 20 universal items and an enriched set of 105 optional items presented in modules corresponding to service fields. The results were compiled in the uSPEQ Consumer Experience Survey development and technical report .
2009: The first uSPEQ Consumer Experience Survey Benchmark Report was released to uSPEQ subscriber organizations. External benchmarks can be used to compare organizational results to the results of other organizations in the same industry or region. The ability to understand how one organization's results compare to those of others can help in identifying areas for performance improvement and marketing efforts. The uSPEQ benchmark report incorporated survey data from over 300 programs/surveys with more than 23,000 responses. In addition to overall benchmarks, survey results were presented by service area (aging service, behavioral health [adults], child and youth services, community services, employment services, medical rehabilitation, and other service area).
2011: The first uSPEQ Employee Climate Survey Benchmark Report was released to uSPEQ subscribers. More than 17,000 responses from more 100 organizations/surveys were included in the report. Survey results were presented by service area, ownership, organization size, as well as by respondent demographics. In addition to summary statistics and frequency distributions, the report displayed the percentile rankings for Percent Positives and Percent Strongly Agree (Top Box) for each of the survey items.
2012: The 2009-2011 uSPEQ Consumer Survey Benchmark Report was released to its subscribers. The second edition incorporated more than 65,000 responses from 400 programs/surveys. In addition to summary statistics and frequency distributions on the overall patterns, two measures were presented with percentile rankings for Percent Positives and Percent Strongly Agree (Top Box) for each survey item. Survey results were also presented by service area, including aging service, behavioral health, child and youth services, community services, employment services, medical rehabilitation, and other service area. A profile of key consumer demographics was generated for each service area.
2013: The 2010-2012 uSPEQ Employee Climate Survey Benchmark Report was released to the participating organizations. It incorporated employee surveys from more than 70 organizations/surveys with more than 15,000 responses.
2013: The uSPEQ Online Reporting Portal was developed using IBM/COGNOS technology to support uSPEQ reporting. The key features of the system include:
- Generation of standard uSPEQ reports.
- Enhanced comparisons of an organization's survey data.
- Benchmarking comparisons.
2014: First uSPEQ consumer survey report was prepared in Chinese.
2014: The 2011-2013 uSPEQ Consumer Survey Benchmark Report was released to its subscribers. The third edition incorporated more than 84,000 responses from nearly 2,000 programs/surveys.
ONGOING RESEARCH AND DEVELOPMENT
Ongoing data analysis will continue to evaluate, and if needed, refine existing questionnaires. uSPEQ also has a long-range research agenda to develop additional stakeholder surveys to assist organizations in their quality and performance improvement efforts.