TABLE 1. Challenges to the Validation of Structured Risk Assessment Tools
| Reliability |
Insufficient evidence to indicate adequate interrater reliability in studies using expert raters. |
|
Poor implementation fidelity may compromise the reliability of instruments in practice. |
| Validity |
Poor predictive accuracy, with little variance in maltreatment recurrence accounted for by risk assessment items. |
|
Some evidence that criterion validity is better if classification accuracy is used as the criterion. However, there is significant variability across instruments. |
|
Mixed evidence for the construct validity of risk assessment instruments with regard to the relationship between risk level and service patterns or intensity. |
|
Poor construct validity of individual risk items. Elements requiring subjective evaluation or more extensive clinical observation are identified as particularly problematic. |
Approaches to validation
|
The definition of any future maltreatment as the hazard to predict identifies the average effect of these factors across all forms of maltreatment and levels of severity. Subgroup variability may compromise validation studies.
|
|
