 |
|
Prediction of Probable Alzheimer Disease in Patients With Symptoms Suggestive of Memory Impairment
Value of the Mini-Mental State Examination
Mary C. Tierney, PhD;
John P. Szalai, PhD;
Earl Dunn, MD;
Daphne Geslani;
Ian McDowell, PhD
Arch Fam Med. 2000;9:527-532.
ABSTRACT
| |
Background The Mini-Mental State Examination (MMSE) is a widely used diagnostic tool for dementia. Its use as a predictive indicator of probable Alzheimer disease (AD) has not been established.
Objectives To determine the accuracy of the MMSE in predicting emergent AD in a sample of patients who were referred because of symptoms suggestive of memory problems and to determine whether an abbreviated version of the MMSE could be developed that would be as accurate as the full MMSE in predicting emergent AD.
Design Inception cohort of participants with symptoms suggestive of memory impairment by their family physicians were given baseline assessments, including MMSE. After 2 years, the participants' conditions were diagnosed following the standard criterion for AD. Diagnosticians were blind to baseline scores.
Setting and Participants One hundred eighty-three community-residing participants were referred by their family physicians to a university teaching hospital research investigation. After baseline screening, 165 participants were included in the study who did not have dementia and had no identifiable cause for memory impairment. After 2 years, 29 participants met criteria for AD, 98 did not develop dementia, 18 developed vascular lesions or non-AD dementia, and 20 did not return.
Main Outcome Measure Diagnostic classification of AD or no evidence of dementia.
Results Logistic regression model was significant. At a cutoff score of 24 or less, sensitivity was 31%; specificity, 96%; with a likelihood ratio of 7.75. A reduced model of 2 subtests was identified with a sensitivity of 41%; specificity, 98%; with a likelihood ratio of 20.70.
Conclusions Results suggest that the full or abbreviated MMSE is useful in predicting emergent AD in patients with postive test results. However, it is not recommended for use as a screening or diagnostic instrument since a negative test result did not rule out emergent AD. It is recommended as a tool to identify those needing closer monitoring.
INTRODUCTION
THE ADVENT of new medications that may be effective in slowing the decline of Alzheimer disease (AD) in its early stages has made it imperative that we distinguish early cognitive changes that will presage AD from those that are benign. In our previous work, we reported that 2 neuropsychological tests and 2 demographic variables predicted accurately the emergence of AD within 2 years.1 These 2 tests, however, require at least 20 minutes to administer as well as training in their proper administration. As a result, it is unlikely that they will be used routinely by busy primary care physicians. Accordingly, we addressed the accuracy of the Mini-Mental State Examination (MMSE) in predicting emergent AD in patients with symptoms suggestive of memory impairment by their family physicians. We were interested in the MMSE because it is widely used as a diagnostic and screening instrument for dementia that requires approximately 5 to 10 minutes to administer with simple instructions. We wanted to determine whether the MMSE was useful as a predictive test in evidence-based clinical practice. In keeping with the principles outlined by Jaeschke et al,2-3 the MMSE must be independent of the diagnostic process, the sample must include an appropriate spectrum of patients to whom the test will be applied in clinical practice, and data must be provided to calculate sensitivity, specificity, and likelihood ratios (LR).
Previous population-based studies in which the MMSE has been kept separate from the diagnostic process4-5 have reported that the MMSE is useful in predicting incident dementia. However, these studies have not provided data to calculate the sensitivity or specificity of the MMSE alone or in combination with other tests. In the one study that provided these data,6 the best cutoff score was 25 or less to identify test-positive cases of AD. However, this cross-sectional comparison of normal elderly persons and patients with AD does not inform us about the predictive validity of the MMSE for emergent AD in an appropriate spectrum of people likely to be encountered in a clinic.
Therefore, our purpose was to determine the accuracy of the MMSE in predicting emergent AD in a sample of patients who were referred to the study because of symptoms suggestive of memory problems. In this investigation, the MMSE score that was obtained at baseline was unavailable to the diagnosticians at year 2 and, therefore, it was independent of this diagnostic process. A secondary purpose was to determine whether an abbreviated version of the MMSE could be developed that would be as accurate as the full MMSE in predicting emergent AD. Items on the MMSE are allocated to several different subtests, representing the cognitive domains of orientation, attention, memory, and language. Previous studies have shown the usefulness of abbreviated versions.4, 7 If the predictive accuracy of an abbreviated MMSE could be demonstrated, it might become suitable for routine use as a predictive tool.
PARTICIPANTS AND METHODS
PARTICIPANTS
Family physicians in Toronto, Ontario, were asked to refer patients who had symptoms suggestive of memory impairment to a research investigation being conducted at a University of Toronto teaching hospital. Letters were sent to all physicians describing the purpose of the study and the inclusion and exclusion criteria. Specifically, physicians were asked to refer patients who had not been diagnosed with a dementing illness (1) if there were complaints of a memory deficit from either the patients or from someone who knew them well, but no objective deficits were found in the clinical interview, or in employment or social situations; or (2) if they had gotten lost; coworkers noticed poor performance; they showed word- and name-finding deficits, and/or a concentration deficit. Physicians were asked to exclude patients who had a history of heavy alcohol abuse, stroke, hypoxia, brain tumors, known neurological disorder, eg, Parkinson disease, or brain trauma. Because of the nature of the cognitive testing involved in the study, physicians were also asked to refer only those who were fluent in English and were not deaf or blind. Informed consent was obtained from all participants when they arrived for their initial study visit. These individuals were assessed as part of a larger investigation examining the predictive accuracy of several cognitive, behavioral, and genetic indexes in the diagnosis of AD.1, 8-10 This project was approved by the Research Ethics Board of Sunnybrook and Women's College Health Sciences Centre.
One hundred eighty-three participants were seen by 1 of 4 experienced geriatricians for a formal baseline diagnostic assessment. Figure 1 shows a flow diagram describing the number of participants referred to the study and those meeting inclusion and exclusion criteria for the 2-year duration. During this examination, the MMSE was administered and standard questions were posed to patients and their informants (a relative or close friend) about their functioning in their normal environment. A workup consisting of a thorough physical examination, computed tomographic scan, single-photon emission computed tomographic scan, and laboratory tests including hematological, renal, hepatic, and metabolic function tests, was conducted to rule out alternative causes of memory impairment. This followed the guidelines of the Workgroup of the National Institute of Neurological and Communicative Disorders and Stroke–Alzheimer's Disease and Related Disorders Association,11 which is the reference standard for a diagnosis of AD. These criteria exclude chronic alcohol or other drug abuse, chronic infections, stroke, hypoxia, metabolic disorders, nutritional disorders, intracranial mass lesions, psychoses, brain trauma, or other neurological diseases including Parkinson and Huntington diseases. Six patients were excluded on the basis of 1 or more of these exclusionary criteria.
|
|
|
|
Participant flow diagram. AD indicates Alzheimer disease; MMSEB, Mini-Mental State Examination score at baseline.
|
|
|
Next, patients were administered a diagnostic neuropsychological test battery by a psychometrist who was unaware of the physician's diagnosis. This assessment consisted of the Wechsler Memory Scale Information and Orientation subtests12; Wechsler Memory Scale–Revised Visual Reproduction13: Immediate and Delayed Recall; California Verbal Learning Test14; odd or even items of the Boston Naming Test15; Controlled Oral Word Association Test16 (letters /p/, /r/, /w/); Category Fluency (animal names); Wechsler Adult Intelligence Scale–Revised: Digit Span, Similarities, and Digit Symbol subtests17; Read Perceptual Closure Test18; Finger Tapping Test19; and Tokens Test.20
After formulating their own diagnostic judgments independently, the experienced board-certified neuropsychologist met with the geriatrician to decide whether the patient met the criteria for dementia using the Diagnostic and Statistical Manual of Mental Disorders, Revised Third Edition (DSM-III-R).21 Twelve participants met criteria for dementia at this baseline assessment and were excluded from the study, leaving 165 participants eligible for the study at baseline.
After 2 years, participants received another medical and neuropsychological diagnostic assessment similar to the baseline assessment. Neither the geriatrician nor the neuropsychologist had access to participants' previous diagnostic assessments including the MMSE scores at baseline, thus ensuring the independence of the predictor test (MMSE) and the criterion standard (diagnosis). If the participant was diagnosed as having dementia or found to have new clinical evidence of a neurological condition other than AD by either diagnostician, a radiological and laboratory workup was conducted as previously described. Participants who developed AD and those who continued to show no evidence of dementia or those who did not develop any new neurological conditions, ie, the nondemented group (ND), were included in the subsequent analyses. Excluded participants were those who met criteria for other forms of dementia or those who developed new neurological conditions. Figure 1 shows those ineligible for follow-up after the 2-year reassessment.
MINI-MENTAL STATE EXAMINATION
We followed the instructions for administration and scoring of the MMSE as outlined by Folstein et al.22 Six subtest scores were obtained from the MMSE: Orientation to Time (ie, year, season, date, day, and month); Orientation to Place (ie, name of the hospital, province, city, floor of the hospital, country); Registration (ie, repeating the name of 3 objects immediately after the examiner has given them); Attention and Calculation (ie, serial sevens or spelling world backwards); Recall (ie, recalling the names of the 3 objects previously given—this was always administered after Attention and Calculation); and Language (ie, naming, sentence repetition, following a command, reading and obeying the command, writing a sentence, and copying a design).
STATISTICAL ANALYSES
Comparisons using the t test were made between the probable AD and ND patient groups on baseline demographic and neuropsychological test scores. The usefulness of the MMSE in predicting probable AD was examined with logistic regression analyses.23 Specifically, we examined the accuracy of the MMSE at enrollment in the study in classifying who developed probable AD (n=29) and who remained ND (n=98) after 2 years. We also conducted parallel analyses examining the classification accuracy of the MMSE with all of the participants who returned for their year 2 assessment, regardless of eligibility. These parallel analyses were conducted to examine the utility of the MMSE in predicting who developed AD (probable or possible) compared with who did not develop AD after 2 years. The full group was subdivided as follows: (1) the combined participants with probable AD (n=29) and possible AD (n=2) for a total of 31, and (2) the combined group of ND participants (n=98) and participants with non-AD neurological conditions (n=16) for a total of 114. In all parallel comparisons, the same pattern of statistical findings emerged as found with the probable AD vs ND comparisons. Therefore, these results are not presented separately in the "Results" section.
To develop a reduced regression model, we entered the 6 subtest scores of the MMSE into a stepwise logistic regression; backward elimination was used with a P .05 for removal of a variable. Before conducting the logistic regression, we examined the intercorrelation matrix derived from the 6 subtest scores for multicollinearity. No variables showed correlations of 0.80 or more and, therefore, none were eliminated.
To separate the individual contribution of the logistic regression model with the full MMSE and with the reduced MMSE, we compared the -2 log LRs of these 2 models. The term "-2 log LR" is a quantity generated by the logistic regression procedure, and it is directly proportional to the contribution of the predictors in separating the groups. It is convertible to a  2 value and allows a direct statistical comparison of predictive models of different complexity.23
The LR of a positive test, which indicates the rate of increase from pretest to posttest odds for a disease for a participant who tested positively for AD, was calculated using the equation:
and the LR of a negative test was calculated using the equation:
RESULTS
PARTICIPANT CHARACTERISTICS
Of the 165 participants eligible to return for their year 2 assessment, 29 met both DSM-III-R criteria for dementia and Workgroup of the National Institute of Neurological and Communicative Disorders and Stroke–Alzheimer's Disease and Related Disorders Association criteria for probable AD. Ninety-eight participants neither met the DSM-III-R criteria for dementia nor did they show any other cause for their memory loss (eg, stroke). They were included in the ND group. The remaining 38 participants (20 did not return and 18 developed exclusionary diagnoses) who were excluded from these analyses are described in Figure 1. Baseline MMSE scores are also provided in Figure 1, to demonstrate that those who did not return or were excluded from subsequent analyses after year 2 assessments were similar on this measure to those who were included.
The mean years of education and age for participants with probable AD and those participants who were ND at enrollment to the study are given in Table 1. Also, to characterize the baseline performance of this group of participants, their scores on 2 of the tests in the diagnostic neuropsychological battery are provided in Table 1. No significant differences were found for age, t125=1.69, P=.09 or for education, t125=0.70, P=.47 between the groups. There were significant differences between the groups for the MMSE, t37.6=4.30, P<.001, California Verbal Learning Test: Delayed Recall subtest, t87.6=13.24, P<.0001, and Wechsler Adult Intelligence Scale–Revised: Similarities subtest, t36.9=4.27, P<.001. Recall, however, that no participant met criteria for dementia at baseline based on thorough neuropsychological and medical assessments.
|
|
|
|
Table 1. Baseline Characteristics of the 2 Groups of Participants*
|
|
|
PREDICTIVE USE OF THE MMSE
Total scores on the MMSE at baseline for participants with probable AD and those who were ND were entered into a logistic regression analysis to examine the accuracy of these scores in predicting who would develop AD 2 years later. The regression model was significant, 21=20.55, P<.001. Table 2 lists the sensitivity, specificity, LR+, and LR- for a range of MMSE scores. We determined that the best cutoff score to rule in emergent AD was 24 or less because at this value only 4% of those who did not develop dementia were labeled incorrectly. A lower cutoff score would include 10% fewer participants with emergent AD, and a high cutoff would increase the number of mislabeled participants who were ND. The LR+ for this cutoff (7.75) would produce a moderate change from pretest to posttest likelihood of AD in a participant,3 ie, the pretest odds for AD would increase almost 8-fold for a positively testing participant. The LR- (0.72) indicates that a negative test score for AD on the MMSE does not alter substantially the pretest odds for AD.
|
|
|
|
Table 2. Sensitivity, Specificity, and Positive (LR+) and Negative (LR-) Likelihood Ratios for a Range of Cutoff Scores on the Mini-Mental State Examination (MMSE)*
|
|
|
Next, we wished to identify the briefest number of MMSE subtests that could accurately predict the development of AD. The reduced model based on the backward elimination procedure consisted of the subtests of Delayed Recall and Orientation to Place, and was significant, 22=38.85, P<.001. We compared this reduced model with the full model with all 6 subtests and found that the full model did not increase the precision of the reduced model significantly, 23=2.82, P=.42. The overall accuracy of the reduced model was 85%, the sensitivity was 41.4%, and the specificity was 98%. The LR+ of this reduced model was 20.70, which represents a large change in pretest to posttest probability of the disease, ie, the pretest odds for AD increases 20-fold for a positively testing participant. The LR- is 0.60 indicating that a negative test score for AD does not alter substantially the pretest odds for AD.
DIAGNOSTIC PREDICTION USING THE REDUCED MMSE MODEL
Because the abbreviated model generated a large LR+, we have provided additional information regarding its application in the clinic. Table 3 provides a convenient means of determining participants' probabilities of developing AD based on their scores on the Delayed Recall and Orientation to Place subtests. To illustrate the usage of Table 3, let us assume a predicted probability of 0.70 or more is chosen by a clinician as an indication for further assessment or closer surveillance. If a patient receives a score of 5 on the Orientation to Place subtest and a score of 0 on the Delayed Recall subtest, the predicted probability of AD would be 0.74, warranting closer attention to that patient. If, however, a patient receives a score of 5 on the Orientation to Place subtest and a score of 2 on the Delayed Recall subtest, the predicted probability of AD would be 0.22, which is too low to warrant further investigation.
|
|
|
|
Table 3. Predicted Probabilities of Alzheimer Disease Based on Scores on 2 MMSE Subtests: Delayed Recall and Orientation to Place*
|
|
|
COMMENT
The results of this study indicate that the MMSE and our abbreviated version may be useful in identifying patients who require closer monitoring because of their risk for developing AD. For a patient who tests positively for AD (ie, a score 24), the pretest odds for AD increased almost 8-fold for the full MMSE and 20-fold for our abbreviated version, suggesting that both the full and abbreviated MMSE are informative. Clinicians who use the abbreviated version may estimate the predicted probability of emergent AD (using the convenient reference table, Table 3), and decide whether further assessment is necessary for their patients. Our finding that the abbreviated version was as accurate as the full test may encourage busy physicians and other health care providers who may not have a great deal of time or familiarity with the assessment of dementia and cognitive impairment to use this test. The abbreviated version found in this study is consistent with findings from previous longitudinal4 and cross-sectional studies7 that examined the use of abbreviated versions of the MMSE in the identification of AD. However, although the abbreviated version consists of only the Delayed Recall and Orientation to Place subtests, the examiner must also administer the intervening subtests of Registration and Attention. The total time required to administer this abbreviated MMSE is approximately 5 minutes.
We found that an MMSE cutoff score of 24 or less identified participants who tested positively for emergent AD. In contrast, a previous cross-sectional study comparing patients with AD with normal control subjects found that the best cutoff score was 25 or less.6 The difference between the 2 studies may be owing to the nature of the different samples being compared. Classification accuracy is more likely to be obtained at higher cutoff scores when comparing patients with AD and normal controls because normal-functioning individuals are more likely to obtain homogeneously higher scores on the MMSE. In this study, comparisons were made between a more heterogeneous group of ND individuals referred for symptoms suggestive of memory impairment who later did or did not develop AD. Because our sample is more likely to represent the spectrum of patients seen in clinical practice by family physicians, our findings are more applicable to the clinic.
The MMSE, however, has important limitations. Our results indicate that the pretest odds for AD are not substantially altered for patients who test negative for AD. Thus, this test is not recommended for use as a screening or predictive diagnostic instrument since a negative test result did not effectively rule out emergent AD in this sample. Furthermore, as a predictive instrument for AD, the MMSE does not perform as well as neuropsychological tests. Previously, we reported that a brief index1 of 2 neuropsychological tests predicted emergent AD in a similar sample with a sensitivity of 76%, specificity of 94%, and a LR of 11.7. While the neuropsychological index is brief compared with a comprehensive neuropsychological battery, it requires approximately 20 minutes to administer as well as training in its proper administration. Unfortunately, both of these factors militate against its routine use in older primary care patients with symptoms suggestive of memory loss.
CONCLUSIONS
Our findings indicate that the MMSE is useful as a brief instrument that will accurately predict emergent AD over a 2-year period in individuals who test positively, but not in those who test negatively. It would be ideal if a brief, and easily administered, screening instrument with high sensitivity and specificity could be developed. Further studies examining the predictive accuracy of screening instruments are required to address this issue. However, it may be impossible to obtain greater precision with brief screening instruments. Tests of greater complexity, requiring more time and skill to administer, and measuring a broader range of function, may be needed to more accurately predict the development of AD in people with symptoms suggestive of memory impairment.
AUTHOR INFORMATION
Accepted for publication November 15, 1999.
This research was supported by the Ontario Ministry of Health, Toronto (Dr Tierney).
We acknowledge the contributions of neuropsychologist W. Gary Snow, PhD, and geriatricians Rory H. Fisher, MB, and Grant Nadon, MD.
Corresponding author: Mary C. Tierney, PhD, Geriatric Research, A438, Sunnybrook and Women's College Health Sciences Centre, University of Toronto, 2075 Bayview Ave, Toronto, Ontario, Canada, M4N 3M5 (e-mail: mary.tierney{at}swchsc.on.ca.).
From Geriatric Research and the Department of Family and Community Medicine (Drs Tierney and Dunn and Ms Geslani), Clinical Epidemiology and Health Services Research (Dr Szalai), Sunnybrook and Women's College Health Sciences Centre, University of Toronto, Toronto, Ontario; and the Department of Epidemiology University of Ottawa, Ottawa, Ontario (Dr McDowell).
REFERENCES
| |
1. Tierney MC, Szalai JP, Snow WG, et al. Prediction of probable Alzheimer's disease in memory-impaired patients: a prospective longitudinal study. Neurology. 1996;46:661-665.
FREE FULL TEXT
2. Jaeschke R, Guyatt G, Sackett DL. Users' guides to the medical literature, III: how to use an article about a diagnostic test, A: are the results of the study valid? JAMA. 1994;271:389-391.
FREE FULL TEXT
3. Jaeschke R, Guyatt G, Sackett DL. Users' guides to the medical literature, III: how to use an article about a diagnostic test, B: what are the results and will they help me in caring for my patients? JAMA. 1994;271:703-707.
FREE FULL TEXT
4. Small B, Vitanen M, Bachman L. Mini-Mental State Examination item scores as predictors of Alzheimer's disease: incident data from the Kungsholmen project, Stockholm. J Gerontol A Biol Sci Med Sci. 1997;52:M299-M304.
5. Brayne C, Best N, Muir M, Richards S, Gill C. Five-year incidence and prediction of dementia and cognitive decline in a population sample of women aged 70-79 at baseline. Int J Geriatr Psychiatry. 1997;12:1107-1118.
FULL TEXT
|
ISI
| PUBMED
6. Monsch A, Foldi N, Ermimi-Funfschilling D, et al. Improving the diagnostic accuracy of the Mini-Mental State Examination. Acta Neurol Scand. 1995;92:145-150.
ISI
| PUBMED
7. Galasko D, Klauber M, Hofstetter R, Salmon D, Lasker B, Thal L. The Mini-Mental State Examination in the early diagnosis of Alzheimer's disease. Arch Neurol. 1990;47:49-52.
FREE FULL TEXT
8. Tierney MC, Boyle E, Lam R. Does depression in memory-impaired elders predict probable Alzheimer's disease? Aging Mental Health. 1999;3:88-93.
FULL TEXT
|
ISI
9. Tierney MC, Szalai JP, Snow WG, Fisher RH. The prediction of Alzheimer's disease: the role of patient and informant perceptions of cognitive deficits. Arch Neurol. 1996;53:423-427.
FREE FULL TEXT
10. Tierney MC, Szalai JP, Snow WG, et al. A prospective study of the clinical utility of ApoE genotype in the prediction of outcome in patients with memory impairment. Neurology. 1996;46:149-154.
FREE FULL TEXT
11. McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer's disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer's disease. Neurology. 1984;34:939-944.
FREE FULL TEXT
12. Wechsler D, Stone CP. Wechsler Memory Scale. New York, NY: Psychological Corp; 1973.
13. Wechsler D. Wechsler Memory Scale–Revised. San Antonio: Psychological Corp, 1987.
14. Delis D, Massman P, Butters N, Salmon D. Profiles of demented and amnesic patients on the California Verbal Learning Test: implications for the assessment of memory disorders. Psychol Assess. 1991;3:19-26.
15. Kaplan E, Goodglass H, Weintraub S. The Boston Naming Test. Philadelphia, Pa: Lea & Febiger; 1983.
16. Spreen O, Benton AL. Neurosensory Center Comprehensive Examination for Aphasia. Victoria, British Columbia: Neuropsychology Laboratory; 1969.
17. Wechsler D. Wechsler Adult Intelligence Scale–Revised. New York, NY: Psychological Corp; 1981.
18. Read DE. Age-related changes in performance on a visual-closure test. J Clin Exp Neuropsychol. 1988;10:451-466.
ISI
| PUBMED
19. Reitan RM. Manual of Administration of Neuropsychological Test Batteries for Adults and Children. Tucson, Ariz: Neuropsychology Laboratories; 1977.
20. Benton A, Hamsher K. Multilingual Aphasia Examination. 2nd ed. Iowa City, Iowa: AJA Associates Inc; 1983.
21. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Revised Third Edition. Washington, DC: American Psychiatric Association; 1987.
22. Folstein MF, Folstein SE, McHugh PR. "Mini-Mental State" Examination: a practical method for grading the cognitive status of patients for the clinician. J Psychiatr Res. 1975;12:189-198.
FULL TEXT
|
ISI
| PUBMED
23. Hosmer D, Lemeshow SF. Applied Logistic Regression. New York, NY: John Wiley & Sons; 1989.
RELATED ARTICLE
The Archives of Family Medicine Continuing Medical Education Program
Arch Fam Med. 2000;9(6):551-552.
FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
Neurocognitive testing supports a broader concept of mild cognitive impairment
Gualtieri and Johnson
AM J ALZHEIMERS DIS OTHER DEMEN 2005;20:359-366.
ABSTRACT
Cognitive tests that best discriminate between presymptomatic AD and those who remain nondemented
Tierney et al.
Neurology 2001;57:163-164.
FULL TEXT
|
