Brief Treatment and Crisis Intervention Advance Access originally published online on November 26, 2008
Brief Treatment and Crisis Intervention 2008 8(4):304-312; doi:10.1093/brief-treatment/mhn017
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Development of Clinical Ratings for Crisis Assessment In Community Mental Health
From the Psychology Department Woodland Centers, 1125 6th Street SE, PO Box 787, Willmar, MN 56201
Contact author: E. Bonynge, Woodland Centers, 1125 6th Street SE, PO Box 787, Willmar, MN 56201. E-mail: ceo{at}woodlandcenters.com.
Few measurements have been developed to aid clinicians in mental health crisis assessment. Authors developed clinical dimensions from both experienced, licensed professionals working in crisis and salient characteristics of individuals presented to a crisis service located in a community mental health center. Five criteria, combined with a standard assessment protocol, were rated by 15 licensed professionals. These five criteria were tested as plausible dimensions for future measurement development. Clinical ratings by licensed professionals on 618 episodes of crisis over an 18-month period were analyzed to determine the latent structure and predictive potential of triage dispositions. The five criteria and a composite total were predictive of differential triage dispositions. An exploratory factor analysis indicated a two-factor model, which we termed "self-destructive mood" and "perturbation." Future development of this model and future refinement of this measure are discussed.
KEY WORDS: crisis measurement, psychometric characteristics, community mental health
The mental health service delivery system has evidenced a great deal of interest in crisis response development (commonly referred to as psychiatric emergency service). A proliferation of crisis services has taken place spurred by the de-institutionalization of the 1960s, reduced mental health bed availability, and interest in alternative cost efficiencies. Such services expanded particularly since 1970 (Fitzgerald, 1996; Kates et al., 1996; Roberts, 2000).
Unfortunately, empirical efforts have not guided this proliferation. Recent reviews of the literature have identified methodological limitations that are pervasive in the crisis response literature (Brown, 2005; Ferris, Schulman, & Williams, 2001). The theme of these limitations highlights a need for better measurement, outcome, and definition. DeClercq (1999) in his review of the emergency psychiatry literature concluded that one of the areas of emphasis should be greater refinement of assessment.
The empirical literature indicates that crisis intervention essentially lacks meaningful measurement. We have no way of classifying or describing crisis intervention episodes. There exists no reasonable explication of any construct we call crisis intervention. "Crisis Intervention" is loosely referenced without any explicit understanding of the very phenomenon we are responding to (Munizza et al. 1993). For this reason, authors will be limiting their scope to what we term "mental health" crisis intervention. Regardless of whatever term we use, we do so with the understanding that we are referencing a continuum of crisis response while attempting to quantify, by certain criteria, a mental health crisis intervention.
This lack of classification, measurement, and outcome demonstration in the area of mental health crisis intervention has been borne out most recently in the evaluation of mobile crisis services. Geller, Fisher, and McDermitt (1995) found that providers do not collect data with any degree of serious attention to classification and measurement of their services. This renders the demonstration of outcomes almost impossible. Ferris et al. (2001) allude to these same issues as severe methodological challenges in evaluating crisis response.
Nonetheless, the few efforts at quantifying mental health crisis episodes have generally followed two strategies: brief self-report inventories and clinical ratings by trained individuals usually following some developed protocol. Self-report measurements comprise a number of suicide inventories. A number of inventories have been developed (Brown, 2001; Lewis & Roberts, 2001). Although suicidal risk is an important aspect of mental health crisis intervention, it is but one component area for assessment. Assessment of individuals in a state of crisis is also replete with a number of impression management issues and demands upon psychologically distressed individuals. Response sets within self-report questionnaires are well documented (Cronbach, 1950). Such response issues are particularly important when working with people in high states of distress. The testability of individuals is often an issue in a crisis state. It becomes difficult to insist upon clients filling out questionnaires. Use of self-report instruments often results in generalized or unidimensional representations of discomfort, aspects which are readily apparent to clinicians (Bonynge, 1993).
The alternative is to rely upon clinical ratings by trained personnel. This strategy does provide the distinct advantage of limiting demands upon the individual in a crisis state.
Two research endeavors exist, which have followed this strategy. The first is by Bengelsdorf, Levy, Emerson, and Barile (1984). They developed the Crisis Triage Rating Scale (CTRS) based upon clinical ratings of three dimensions on a scale from 1 to 5: dangerousness (1 = most dangerous to self or others, 5 = least dangerous), support system (1 = poor, 5 = available and able), and ability to cooperate (1 = unable or refuses, 5 = willing and able). Use of this instrument was based upon experience with 300 participants in a preliminary study and 122 participants prospectively at 6 months beyond initial rating on the CTRS. A cutoff score to differentiate inpatient versus outpatient interventions was suggested. Further validation of a cutoff threshold using a sample of 500 individuals in a mental health crisis was later pursued by Turner and Turner (1991). They altered the recommended cutoff score slightly for prediction of inpatient versus outpatient intervention. Use of this instrument in crisis contexts has since been very limited with only one published document (Links, Enynan, Ball, Barr, & Rourke, 2005), which incorporated a rating derived from the CTRS in a cohort of 65 subjects with severe and persistent mental illness. Analyses were limited to descriptive statistics due to limitations of the sample.
The second major undertaking utilizing clinical ratings was by Myer, Williams, Ottens, and Schmidt (1992). They developed the Triage Assessment Form that assessed the types and severity of crises experienced by college students. Ratings are made on a Likert-type scale (1—no impairment to 10—severe impairment) of affective, behavioral, and cognitive status. This has since evolved into the Triage Assessment System (Myer & Conte, 2006). Watters (1997) has published limited reliability and validity of this measure to date. The Triage Assessment System is now being implemented as a model to guide an assessment of a crisis but is not a proven measure in a clinical sample.
The Suicide Assessment Checklist (Rogers, 1994), Scale for Suicide Ideation (Beck, Kovacs, & Weissman, 1979), and the Modified Scale for Suicide Ideation (Miller, Norman, Bishop, & Dow, 1986) round out the major clinician-rated measurement strategies. All are one-dimensional and lack ecological validity to a number of crisis contexts that require a greater breadth of assessment.
A mental health crisis measurement inclusive of suicide potential yet broad enough to assess additional factors is currently lacking. Although a variety of measurements designed for suicide potential have been developed (Brown, 2001), crisis intervention assessment goes beyond the assessment of suicide risk. As mental health crisis services have evolved, greater specificity in predicting triage alternatives is needed. Fewer people are triaged to inpatient settings with more alternative outpatient options becoming available.
Mental health crisis assessment is hard pressed to develop without a firm measurement base. Measurement of mental health crisis response is crucial for comparison of contexts, samples, and determining outcome. Authors set out to explore the domain of mental health crisis interventions within the context of a long-standing crisis center in a Community Mental Health Center (CMHC). One of our goals was to be as least intrusive to individuals in a crisis and consequently we based our exploration upon ratings by trained, experienced clinicians. Exploratory factor analysis (EFA) was used to understand the latent structure of clinical dimensions. Authors hoped to determine the predictive ability of clinically derived criteria to differentiate referral options using discriminant analysis. We considered this an initial step toward developing a measurement, which we could potentially build upon and one that would eventually provide practical guidance for clinicians.
 |
Method |
|---|
 TOP Method ResultsDiscussionReferences |
|---|
Setting
The setting of this research was a CMHC. A detailed description of the CMHC and crisis response service is contained elsewhere (Bonynge, Lee, & Thurber, 2005). Briefly, the CMHC is a comprehensive provider in a rural setting. The Crisis Center is a 24/7 centralized, mental health emergency center staffed with licensed mental health professionals, psychiatrists, nurses, and practitioners under professional supervision. A variety of crisis services are offered: (a) hot-line, (b) urgent care, (c) mobile crisis, (d) short-term residential beds, and (e) walk-in counseling. Mental health professional oversight was 24/7 both on-site and via telephone. The crisis hot-line rarely has any direct professional involvement except in situations of high acuity of harm (which is very seldom). Most individuals (74%) are assessed and evaluated directly by professionals. The focus of this investigation involved the face-to-face contact of mental health professionals with individuals both with and without a mental health disorder presenting voluntarily, or involuntarily referred by social services, the local medical community, or law enforcement to a community mental health crisis service.
Assessment Protocol
The assessment protocol by the professional involved a review of several critical areas followed by a mental status examination, a Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) five axis diagnosis, immediate treatment recommendations, and longer term treatment recommendations. The critical areas reviewed by the professional consisted of current stressors, current functioning, mental health symptomatology, risk factors for suicide or aggression, status of support network, cultural considerations, and client strengths. Clinicians also had considerable collateral information from referral sources and family, which was incorporated into the clinical picture.
Professionals then completed ratings on five key variables. These five variables were based upon a consensus by our clinicians directly working in the crisis unit and were identified as salient characteristics of persons worthy of consideration in assessing individuals in crisis. These variables formed an operational definition of crisis requiring the presence of three or more variables (Bonynge, Lee, & Thurber, 2005). These variables were as follows: (1) danger to self, (2) danger to others, (3) functional decline, (4) confusion, and (5) depression. Each dimension was rated on a Likert type scale (reference) from 0 (not present) to 4 (extreme). This created five variable scores plus a total score. Clinicians then made referrals to one or more of six options: (1) inpatient psychiatric or inpatient substance abuse treatment, (2) partial hospital that usually involved use of a crisis stabilization bed with an average length of stay (ALOS) of 3.8 days (standard deviation [SD] = 2.5), (3) short-term stabilization bed with ALOS of 2.13 days (SD = 1.6), (4) short-term stabilization bed with referral to an intermediate residential setting, and (5) outpatient mental health/substance abuse services within or outside the CMHC. After initial assessment and ratings, a crisis team made final determinations as to appropriate treatment setting for those who were not directly referred to the outpatient option five. Ratings and recommendations made by the mental health professional were generally in agreement with a crisis team in their final determinations.
The clinical raters consisted of Licensed Doctoral Psychologists (5), Licensed Masters level prepared Psychologists (3), Licensed Master level Social Workers (4), and Licensed Marriage and Family Therapists (3). A total of 15 licensed professionals rendered ratings on 618 episodes of crisis. Average licensed experience was 13.53 (range 3 to 25 years) and SD 6.98. Average experience in behavioral crisis assessment was 11.93 years with SD 7.27 (range 3 to 28 years).
Subjects and Episodes
Ratings were completed on consecutive crisis episodes requiring professional assessment beginning August of 2004 through February 2006. A total of 618 episodes were rated: 245 male episodes on 185 subjects and 373 female episodes on 233 subjects. Mean age of all subjects was 36.35 (SD = 12.48). Mean age for males was 36.62 (SD = 12.51) and mean age of females was 36.47 (SD = 12.47). There was no significant difference in age between male and female subjects, t (614) = .546, p = .585. The majority of this sample was single (57.6%), with 13% married, 22% divorced, 4% separated, 1.5% widowed, and 2% with missing data. The mean education for males was 12.42 (SD = 1.7) and mean education for females was 12.63 (SD = 1.7). This was a rural, mid-western community sample where 96% were Caucasian and 3% Hispanic descent.
Only licensed mental health professionals took part in the ratings. Approximately 26% of crisis episodes were not evaluated by a mental health professional. Nurses and practitioners under supervision of professionals addressed the least acute of crisis episodes, which typically involved basic counseling. If any serious question was encountered with respect to acuity, mental health professionals were called upon to complete a face-to-face assessment. The professional breakdown of clinical raters was 387 episodes by clinical social workers, 106 by doctoral psychologists, 66 by master level psychologists, and 59 by licensed marriage and family therapists. There was no significant difference between the four groupings of professionals and the crisis team decisions on the five referral dispositions (
2 = 6.58, 12 degrees of freedom [df], p = .884). This would indicate that there was no bias involving a profession or referral disposition.
Data Analysis
Professional ratings were collected over a 15-month period. Intercorrelations of the five clinical criteria, age, and sex were calculated using Pearson rs for continuous variables and Kendall's Tau-b for the categorical variable sex. Analysis of variance (ANOVA) and post-hoc comparisons of the five criteria and an additional composite total score by the five referral options (disposition) were conducted to determine if any separation of groupings based upon referral dispositions was apparent. A Welch variance weighted correction was used due to unequal sample sizes as well as post-hoc Tamhane comparisons for the same reason. EFA procedures were conducted on the five clinical ratings by professionals. Discriminant analytic procedures were used to explore the predictive validity of the five clinical criteria as well as a total score composite in the prediction of treatment options.
|
Results |
|---|
|
TOPMethodResultsDiscussionReferences |
|---|
Table 1 depicts the correlation matrix of the five criteria, age, and sex. Out of 21 correlations, 3 were significant at the .05 level of significance and 11 were significant at or greater than the .01 level of significance according to a two-tailed distribution.
|
The five clinical criteria are correlated in essentially expected ways. Depression and Danger to Self are highly correlated as are Confusion and Functional Decline. Age is positively related to Functional Decline and Confusion. The magnitudes of these correlations were moderate. These were encouraging for further analysis. The point biserial negative correlation of Sex with Danger to Others reflects a disproportionate number of males coded as 1 and rated at some degree as dangerous. Clinical criteria and composite total score means and SDs by disposition along with Welch's variance weighted ANOVA's and post-hoc comparisons are depicted in Table 2. The disposition of individuals to various referral options is also available within this table. All univariate Fs were significant beyond a .000 significance level. Post-hoc comparisons were Tamhane p < .05. Inpatient ratings were highest, whereas Outpatient ratings were lowest. This finding was encouraging and also an initial indication of salient group validity of our model. Partial Hospital Program (PHP) appears to be referred individuals that have more severe levels of Depression, have less Confusion, and are less Dangerous to Self. Intensive Residential Treatment (IRT) referrals were as Dangerous to Others, with equal levels of Functional Decline and equally Depressed as the Inpatient group. Outpatient referrals were clearly lowest on all criteria. Total Score demonstrated a decline with progressive intensity of disposition. These results suggested fair to strong discriminatory power of the five clinical criteria and composite total score. We followed up with discriminant analyses of referral options based upon the clinical criteria and the composite total score (Table 3). Prediction was essentially robust until inclusion of the Stabilization Bed option. Differences between this option and the Inpatient, PHP, and IRT options were small and therefore accounted for significant decreases in prediction. Consistently, Total Score was as accurate as the five criteria in prediction of treatment options.
|
|
We subjected the clinical criteria to an EFA to explicate the underlying constructs of the clinical criteria. An unweighted least squares factor analytic procedure was employed utilizing an oblique rotation. An oblique rotation was appropriate due to the intercorrelations of the five clinical criteria noted in Table 1. Unweighted least squares was appropriate due to the Danger to Others criterion. This criterion revealed an atypical distribution issue. We noted 66.2% of responses with no Danger to Others observed. In this setting, most individuals do not present with an aggressive component. Nevertheless, when Danger to Others was observed, it demonstrated itself to be highly predictive in dispositional decisions. The appropriateness of the correlation matrix for factor analysis is measured principally by the Kaiser-Meyer-Olkin measure of sampling adequacy and Bartlett test of sphericity. The latter was significant (
2 = 358.92, 10 df, p = .000), suggesting that the variables were not independent of one another. See Table 1 indicating moderate correlations of the variables. The Kaiser-Meyer-Olkin measure was .571. This is heavily influenced by the number of variables in an analysis, which in this instance, were relatively few. Authors proceeded with this methodology as a preliminary examination for developing facets of crisis with development of items at a later date. Nevertheless, the obtained outcome was acceptable (Dzubian & Shirkey, 1974) for proceeding with a factor analysis procedure. Factor analysis, utilizing an unweighted least squares procedure with Quartimax, oblique rotation of the five criteria, extracted two factors with eigenvalues exceeding 1. Examination of factor 1 comprised the criteria of Danger to Self and Depression with rotated factor loadings of .55 and .73, respectively. The second factor comprised the remaining three criteria of Functional Decline, Confusion, and Danger to Others. These three criteria produced rotated factor loadings of .60, .65, and .29, respectively. The final estimate of total communality of both factors was 43.1% total variance. The Danger to Others factor loading was borderline acceptable in absolute terms.
The EFA resulted in the extraction of two oblique factors. The correlation between the factors indicates that the factors are correlated with a general factor that might be termed "perturbation," similar to a concept coined by Schneidman (1996).
|
Discussion |
|---|
|
TOPMethodResultsDiscussionReferences |
|---|
One of our major undertakings was to establish whether ratings completed by professionals according to a standard protocol could predict referral dispositions. This could be useful as a triage measurement in crisis situations. Although reliability is an incipient issue at this stage of measurement development, we did compute Theta, which is an estimate of internal consistency reliability, and obtained a .68 estimate with only five items, which is more than adequate and will only increase with additional variables and items in the future. The potential of these criteria with raters who are nurses or practitioners is yet to be explored.
Predictions of treatment disposition based upon the five clinical criteria as well as total score were robust for a number of groupings. The robust predictions at the Inpatient and Outpatient levels of care indicate a fundamental level of validity for our criteria. Prediction becomes more complex when the solitary Stabilization Bed disposition is compared to other dispositions (Table 3). Distinctions between the aggressive interventions of Inpatient, PHP, and IRT options are more interwoven and difficult to distinguish from a Stabilization Bed status. Very likely there are additional or correlated variables not accounted for by our criteria, which need further study. Clients in the Stabilization Bed are often just as disturbed on the five clinical criteria as those referred to Inpatient, PHP, and IRT but do stabilize within a 3-day time frame. IRT and Inpatient are particularly difficult to discern (Table 2). Further discriminating variables between Inpatient, PHP, IRT, and Stabilization may be difficult to define. Another avenue to pursue is further analysis of what characteristics of clients lead to quick stabilization without more aggressive interventions such as Inpatient, PHP, and IRT. Most receiving Stabilization are referred back to traditional outpatient treatments. The observation of a fine line between Aggressive Intervention and Stabilization will require further attention in developing future measurement. These criteria were helpful in determining triage, but a number of other factors do enter into disposition such as insurance coverage and cooperation.
The EFA that we conducted may be one of the first within a community sample of clients in crisis. Two dimensions were identified and labeled Self-destructive Mood and Perturbation. In practice then, a total score across all criteria is justified as well as the computation of separate factor scores. These two factors have both a theoretical and practical import. It is conceivable to approach a crisis situation with the two dimensions of Self-destructive Mood and Perturbation as guides to clinical judgment in formulating treatment dispositions. These two factors are also a basis upon which to refine a measurement of crisis situations. Future plans are for additional dimensions and improving reliability with generation of items for each dimension.
We believe these results establish that professional raters, following an established protocol, and estimating salient variables, can produce meaningful ratings that can be used for purposes of crisis assessment. Future development of crisis measurement will involve the inclusion of additional variables, item refinement, and increased reliability. By establishing a model and establishing its utility for discriminating referrals, authors now have the opportunity to build upon these five dimensions. However, the generalizability of these results is limited based upon a single setting and a predominantly Caucasian sample. Further development of these dimensions will involve establishing four items (or anchors) for each dimension that will hopefully result in a more useful tool for both professionals and practitioners working with individuals in a behavioral crisis and more detailed analysis of other factors in addition to scales and items that contribute to level of care decision.
Mental health crisis response is likely on a continuum from mild to severe. Hopefully, we may some day establish measurement that quantifies mental health crises so that comparisons between samples, settings, and crisis models can be more precise.
|
Acknowledgments |
|---|
Conflict of interest: None declared.
|
References |
|---|
|
TOPMethodResultsDiscussionReferences |
|---|
-
Beck AT, Kovacs M, Weissman A. Assessment of suicidal ideation: The scale for suicide ideation. Journal of Consulting and Clinical Psychology (1979) 47:343–352.[CrossRef][ISI][Medline][find it @ Stanford]
Bengelsdorf H, Levy LE, Emerson RL, Barile FA. A crisis triage rating scale: Brief dispositional assessment of patients as risk for hospitalization. Journal of Nervous and Mental Disease (1984) 172(7):424–430.[ISI][Medline][find it @ Stanford]
Bonynge ER. Unidimensionality of SCL-90-R scales in adult and adolescent crisis samples. Journal of Clinical Psychology (1993) 49:212–215.[CrossRef][ISI][Medline][find it @ Stanford]
Bonynge ER, Lee RG, Thurber S. A profile of mental health crisis response in a rural setting. Community Mental Health Journal (2005) 41:675–685.[CrossRef][ISI][Medline][find it @ Stanford]
Brown GK. A review of suicide assessment measures for intervention research with adults and older adults. (2001) Retrieved January 16, 2007, from http://www.nimh.nih.gov/suicideresearch/adultsuicide.pdf.
Brown JF. Psychiatric emergency services: A review of the literature and proposed research agenda. Psychiatric Quarterly (2005) 76(2):139–165.[CrossRef][ISI][Medline][find it @ Stanford]
Cronbach LJ. Further evidence on response sets and test design. Educational and Psychological Measurement (1950) 10:3–29.[CrossRef][find it @ Stanford]
DeClercq M. Reasons for increase in psychiatric emergencies. In: Emergency psychiatry in a changing world—de Clercq M, Andreoli M, Lamar S, et al, eds. (1999) New York: Elsevier. 10–21.
Dzubian CD, Shirkey EC. When is a correlation matrix appropriate for factor analysis? Psychological Bulletin (1974) 81:358–361.[CrossRef][ISI][find it @ Stanford]
Ferris LE, Schulman KL, Williams JI. Methodological challenges in evaluating mobile crisis psychiatric programs. Canadian Journal of Program Evaluation. (2001) 16(2):27–40.[find it @ Stanford]
Fitzgerald M. Structuring psychiatric emergency services for smaller communities in response to managed care. Psychiatric Services (1996) 47:233–234.
Geller JL, Fisher JH, McDermeit M. A national survey of mobile crisis services and their evaluation. Psychiatric Services (1995) 46:893–897.
Kates N, Eaman S, Sontone J, Didemus C, Steiner M, Craven M. An integrated regional emergency psychiatry service. General Hospital Psychiatry (1996) 18:251–256.[CrossRef][ISI][Medline][find it @ Stanford]
Lewis S, Roberts A. Crisis assessment tools: The good, the bad, and the available. Brief Treatment and Crisis Intervention (2001) 1(1):17–28.[Abstract]
Links SP, Eynan R, Ball JS, Barr A, Rourke S. Crisis occurrence and resolution in patients with severe and persistent mental illness. Crisis (2005) 26(4):160–169.[ISI][Medline][find it @ Stanford]
Miller IW, Norman WH, Bishop SB, Dow MG. The modified scale for suicide ideation: Reliability and validity. Journal of Consulting and Clinical Psychology (1986) 54:724–725.[CrossRef][ISI][Medline][find it @ Stanford]
Myer AR, Conte C. Assessment for crisis intervention. Journal of Clinical Psychology (2006) 62(8):959–970.[CrossRef][ISI][Medline][find it @ Stanford]
Myer RA, Williams RC, Ottens AJ, Schmidt AE. Crisis assessment: A three-dimensional model for triage. Journal of Mental Health Counseling (1992) 14:137–148.[find it @ Stanford]
Munizza C, Furlan PM, d'Elia A, D'Onofrio MR, Leggero P, Punzo F, et al. Emergency psychiatry: A review of the literature. Acta Psychiatrica Scandinavica (1993) 88(Suppl. 374):1–51.[ISI][Medline][find it @ Stanford]
Roberts AR. An overview of crisis theory and crisis intervention. In: Crisis intervention handbook: Assessment, treatment, and research—Roberts AR, ed. (2000) 2nd ed. New York: Oxford University Press. 3–30.
Rogers JR. Validity analysis of the Suicide Risk Assessment Checklist (1994) Akron, OH: University of Akron.
Schneidman E. The suicidal mind (1996) New York: Oxford University Press.
Turner PM, Turner TJ. Validation of the crisis triage rating scale for psychiatric emergencies. Canadian Journal of Psychiatry (1991) 36(9):651–654.[ISI][Medline][find it @ Stanford]
Watters D. A study of the reliability and validity of the triage assessment scale. Unpublished doctoral dissertation (1997) University of Memphis: Memphis, TN.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||