 |
|
Variation in the Diagnosis of Upper Respiratory Tract Infections and Otitis Media in an Urgent Medical Care Practice
Joseph L. Lyon, MD, MPH;
Arden Ashton, MD;
Bryan Turner, MD;
Michael Magill, MD
Arch Fam Med. 1998;7:249-254.
ABSTRACT
Background Variation among hospitalized medical conditions have been reported previously, but there is little information on variation among physicians for medical conditions that constitute a substantial part of ambulatory care.
Objective To measure variation in the diagnosis of 2 common medical conditions, otitis media (OM) and upper respiratory tract infections (URIs) in an urgent care practice.
Design Estimate the variation in the incidence of new diagnoses of OM and URIs among 19 physicians working at 2 urgent care clinics between January 1, 1995, and March 31, 1995. Patients are seen in order of arrival, and assignment to each physician is random.
Participants and Setting Two urgent care clinics staffed by the same 19 physicians. During the 3-month study period, 21259 patients were seen at the 2 clinics; of these, 1839 (8.65%) received a diagnosis of first time OM; and 8020 (37.73%), of an URI.
Main Outcome Measure Incidence of new diagnoses of OM and URI estimated as a proportion of all diagnoses for each individual physician.
Results There was substantial variation between physicians in the diagnosis of OM, ranging from a low of 4.2% to a high of 21.8%. There was less variation in the diagnosis of URI (31.7%-48.4%). Some physicians with a low incidence of OM or URI diagnoses had increased the proportion of diagnoses in the other diagnostic category. For OM the variation was greatest for children younger than age 5 years, but substantial variation was also found in adults. For URIs the variation was more uniform across all age groups. The variation was not explained by type of specialty training or years in medical practice.
Conclusions There is substantial variation between physicians in their diagnosis of OM and URI in an urgent care setting. This variation has implications for cost of diagnosing and treating these conditions, the training of physicians, and our understanding of the treatment of OM.
INTRODUCTION
UPPER respiratory tract infections (URIs) accounted for 27.5 million visits to physicians in the United States in 1991 or 10.8% of all physician office visits in the United States.1 Otitis media (OM) is one of the most common reasons for visits to physicians for children aged younger than 1 year to 4 years in the United States and accounted for an additional 2.7 million office visits.1 These 2 conditions cost billions of dollars to diagnose and treat.
While substantial variation has been reported between small geographic areas in disease occurrence as measured by hospitalization,2-3 to our knowledge, variation in the diagnosis of common medical conditions such as URIs and OM has received little attention,4-5 and no studies were found that quantified variation in diagnoses among physicians.
The diagnostic criteria for URIs are often subjective and based on symptoms alone (such as persistent productive cough and facial pain) and therefore are presumed to vary substantially among physicians. The diagnosis of acute OM is made by observing inflammatory changes in the tympanic membrane. Textbooks contain standard descriptions and pictures of normal and abnormal tympanic membranes. Little is known about how these are applied by individual physicians, but the diagnosis is believed to be straightforward and subject to little variation. Some pediatricians have tried to standardize the detection of serous OM and found substantial variation.5
Because so little is known about the variation among physicians in the diagnoses of these 2 common medical conditions, we examined the diagnoses of URIs and OM in a single urgent care medical practice among a group of primary care physicians with differing levels of clinical experience and training.
PARTICIPANTS AND METHODS
SETTING
The data were obtained from 2 urgent care facilities. The clinics are 3.2 km apart and staffed by the same physicians who rotate between the 2 clinics. Between 60 and 140 new patients are seen daily at each clinic. Services are provided 7 days a week, 363 days a year (closing for Thanksgiving and Christmas). During the 3 study months (January through March 1995), each clinic served an average of 91.4 patients daily. On average, the physicians saw 3.4 patients per hour.
Each clinic is open 14 hours a day and during winter months is staffed as follows. A single physician sees all patients from 8 AM to noon. From noon until 10 PM there are 2 physicians on duty. Patients are seen in order of arrival with the exception of potentially life-threatening problems. After the patient is in the examining room, his or her chart is placed on the nurse's station counter in the order in which this patient was signed into the clinic. Physicians do not accept appointments and the decision of which physician sees a patient is made by taking the next chart from the counter. Patient selection by a physician because of presenting problem or severity of illness is not a factor. The number of patients per hour allows little time for consultation between physicians, and there is no consultation on simple diagnoses such as URIs and OM.
DIAGNOSES
A computerized record is initiated for each patient when he or she enters the clinic, and this record is used for follow-up visits. For each visit a diagnosis must be made and recorded on the record. The diagnosis is then entered into the computerized record. This information is then used to code the visit for billing purposes using International Classification of Diseases, Ninth Revision6 codes. New physicians are instructed on what has to be written on the record so that an International Classification of Diseases code can be assigned. The diagnoses used in this study were obtained by searching all diagnoses for all patients seen during the study period and selecting those of OMwith or without effusion, sinusitis, pharyngitis, tonsillitis, bronchitis, and URI unspecified. The former group was called OM and the latter group URI for this study. For the small number of patients where OM and URI were present together, the case was assigned to the OM category.
PHYSICIANS
During the study period, there were 19 physicians working in the 2 clinics; 8 had full-time status and 11 part-time. Of the 8 full-time physicians, 2 had completed a family practice residency, 4 had completed an internal medicine residency, 1 had completed 3 years of a radiation oncology residency, and 1 had completed 2 years of a pediatrics residency.
Of the 11 part-time physicians, 2 were in an ear, nose, and throat surgery residency (second and third years), 1 in an anesthesia residency (third year), 3 in an internal medicine residency (all third year), 1 in a urology residency (third year), and 4 in a family practice residency (all third year).
The 11 part-time physicians provide mainly evening (6-10 PM) and weekend coverage. The proportion of patients with injuries and children younger than age 5 years is higher during these times compared with daytime hours, so the part-time physicians' diagnoses were analyzed separately. (Table 1 gives the proportion of diagnoses by full- and part-time physicians.)
|
|
|
|
Table 1. Patients Seen From January Through March 1995 in Urgent Care Clinics by Treating Physician, Age, and Diagnoses
|
|
|
DATA ANALYSIS
The data are presented as proportions, means of proportions, and differences from means and ratios. Since the purpose of the study was to determine if variation in diagnosing OM was present, no formal tests of statistical significance were done. The data were also age adjusted by the direct method7 to the distribution of all clinic patients for each physician to remove any potential confounding introduced by differences in age distribution between each physician.
The diagnosis of URI or OM by each physician is expressed as the proportion of these diagnoses among all patients seen by each physician during the time. For each physician, the relationship between the diagnosis of URI and OM was also examined by combining the 2 diagnostic groups and expressing this as a proportion of all patients.
Because the diagnosis of OM decreases markedly with advancing age, the proportion of patients with URI and OM was examined by differing age groups, using single-year age groups for those younger than 5 years and broader age categories thereafter.
RESULTS
During the study period there were 21259 visits to the 2 clinics (Table 1). Upper respiratory tract infections constituted 8020 (37.7%) of all patient diagnoses during the study and OM an additional 1839 (8.7%). The total number of patients seen with URIs was 8020. For individual physician's diagnosis, the number of patients given the diagnosis ranged from 75 for a part-time physician to 1582 for a full-time physician. During the study period, the total number of patients receiving a diagnosis of OM was 1839 and from 18 to 131 individual physicians gave the diagnosis.
Variation in the proportion of cases in which the condition was diagnosed as OM or URI by each physician is shown as a percentage (Table 2). The physician with the lowest proportion (hereafter, low physician) was used as a referent and this percentage was divided into the percentage of the other physicians. This ratio provides a convenient measure of variation between each physician by expressing the range between the physician with the highest proportion (hereafter, high physician) and low physician as a single number. The variation for OM was 176% for full-time physicians and 271% for part-time physicians. Ninety-five percent confidence intervals were calculated for the high and low physicians and there was no overlap between the confidence intervals, suggesting a statistically significant difference between the physicians at the extreme of the range. (Data not shown.)
|
|
|
|
Table 2. Percentage of Patients Diagnosed With Otitis Media and URIs* by Physician Employment Status From January Through March 1995
|
|
|
Because the diagnoses of URI and OM change with increasing age, the variation among physicians was examined by single year of age younger than age 5 and then by broader age groups thereafter. The results of these analyses are shown in Figure 1 and Figure 2 for full-time physicians only. The data for OM by age group along with a ratio measure of the variation is shown in Table 3. Because of the difficulty in examining children younger than 1 year, we expected that the greatest variation would occur in this age group, but an equally large variation was also observed in those aged 19 and older, an age group where the examination is much less challenging (Table 3).
|
|
|
|
Figure 1. Otitis media as a percentage of all diagnoses by full-time physicians from January through March 1995.
|
|
|
|
|
|
|
Figure 2. Upper respiratory tract infection (URI) as a percentage of all diagnoses by full-time phycians from January through March 1995.
|
|
|
|
|
|
|
Table 3. Variation in the Diagnosis of Otitis Media by Age*
|
|
|
The effect of patient age on physician variability in diagnosis was less for URIs than for OM (Figure 1 and Figure 2 and Table 2). There was a 2.8-fold variation of the diagnosis of OM among full-time physicians, but only a 1.4-fold variation for the diagnosis of URIs. For individual age groups, the variation between physicians was much greater. For example, for full-time physicians there was a greater than 5-fold variation for children younger than 1 year. The difference was not as dramatic thereafter, but for the oldest age category (patients 19 and older) the variation increased again.
There was no consistent pattern of variability by years of medical training, years in medical practice, or by subspecialty training (data not shown). Physicians trained in primary care residencies (family practice, pediatrics, and internal medicine) showed as much variation as those with training in the surgical subspecialties. One of the full-time physicians had 2 years in a pediatrics residency and had the highest proportion of diagnoses of OM among the full-time physicians. The lowest proportion of OM diagnoses was generated by a board-certified family practice physician.
Among the part-time physicians were 2 residents in ear, nose, and throat surgery. These 2 physicians were compared with each other and to the overall group. It was assumed that specialty training in ear, nose, and throat surgery would decrease the diagnostic variability. There was a 2-fold variation between the 2 ear,nose, and throat residents with one resident falling below the group mean and the other above it.
Because differences in the patients' ages might introduce bias into the study, we also adjusted the proportion by age to determine if this factor might explain the variation seen. There was less than a 5% change in the unadjusted to the age-adjusted diagnostic percentage for any physician, suggesting that no physician was seeing more patients in one age group compared with his or her colleagues, and that age differences between patients was not an explanation for the variation we observed among physicians. (Data not presented.)
However, since OM was a much smaller fraction of all patient visits compared with URIs (1839 [8.7] vs 8020 [37.7]), a ratio measure might be biased by the larger proportion of cases with the diagnosis of URI. For this reason we also examined the range between the low and the high physicians for these 2 diagnoses. The range for OM was 15.9% for full-time physicians and 20.6% for part-time physicians. For URIs the range was 13.7% for full-time physicians, and 13% for part-time physicians, suggesting that the greater variability in the diagnosis of OM compared with URIs identified by the ratio measures is present also when a second measure of variation is used.
Since many patients with OM present with URI symptoms (a runny nose, coughs, etc), it is possible that physicians might classify a variable percentage of these patients as having OM, but classify some with a possible OM as having an URI. If this were so, then a physician with a low proportion of OM diagnosis would compensate with a high proportion of URI diagnosis and the sum of the 2 would be close to the group mean. We examined this possibility by rank ordering the full- and part-time physicians by their proportion of diagnoses listed as OM, URI, and the 2 combined. Those with the lowest proportion of diagnoses were ranked number 1, the next highest as 2, and so on. We then examined the difference between the rank ordering for the 2 diagnoses to determine if physicians were "hedging their bet" by listing URI because of uncertainty over the diagnosis of OM (Table 3).
For full-time physicians the greatest difference in rankings of OM compared with URIs would be 7 (8 - 1) and for the part-time physicians 10 (11 - 1). For example, a physician with a difference of 7 between the 2 rankings might have ranked eighth in the diagnosis of URIs and first in the diagnosis of an OM. Such a physician would have assigned patients called OM by colleagues to the URI category (Table 4).
|
|
|
|
Table 4. Rank-Order Score for Physicians for URI,* Otitis Media, and URI + Otitis Media
|
|
|
For the full-time physicians, 2 of 8 had a score of 6, suggesting substantial hedging on the diagnosis of an OM. Surprisingly, 1 of the 8 full-time physicians had a score of - 7 suggesting the reverse behavior, overdiagnosis of OM and underdiagnosis of URIs.
Among the 11 part-time physicians, 4 had a difference of 7 or greater, but 2 of the 4 scores were negative replicating the same pattern seen among the full-time physicians, ie, a high proportion of OM being offset by a lower proportion of URIs.
COMMENT
The differences reported by this study suggest a substantial and likely undesirable variation in the diagnosis of OM within a single physician group with different postgraduate training. There was less variation in the diagnosis of URIs, and the variation found may be within the range expected by chance.
There was no direct test of the accuracy of the diagnosis for either condition, but 2 of the full-time physicians had used a portable tympanometer to confirm their diagnosis of an OM on a similar study done the year before this study. The percentage of their patients with OM as the primary diagnosis was 11% lower than the full-time physician average (7.2% compared with 8.1%). The tympanometer measures the compliance of the eardrum and is better at detecting OM with an effusion than acute OM.
All of the physicians in this study were graduates of US medical schools, though their level of postgraduate training varied from 1 to 5 years. Among the full-time physicians, the type or amount of postgraduate training had little effect on the variability in the diagnosis. The range of variability in the diagnosis of OM among those full-time physicians who were board eligible or certified in internal medicine or family practice was 2.24% to 12.74%.
While it is possible that some of the physicians saw more patients with OM than others, it is highly unlikely. The assignment of patients to each physician is not biased by presenting complaint, age, or severity of illness. Physicians rotate between the 2 clinics and all physicians work day and evening shifts, and weekday and weekend shifts. Seventy percent of the time 2 physicians work together at the same time. In some instances, 2 physicians with a low percentage of OM diagnoses worked together with no evidence of a shift in their percentage of OM diagnoses, while at other times 2 physicians with a high percentage of OM diagnoses worked together and again there was no evidence of a shift in the percentage of OM diagnosis.
The variation in the diagnosis of OM and URI appears to be real and not an artifact of measurement or the result of an unknown confounder. Some physicians with a low percentage of giving a diagnosis of OM may compensate by giving the patient a diagnosis of URI and treating with antibiotics, while physicians with a high percentage of giving a diagnosis of OM may decrease their diagnosis of URIs. Surprisingly, the range of variation for OM was slightly greater among full-time physicians with substantially more experience than among part-time physicians suggesting that once established, a pattern of diagnosis persists and is not modified by experience. The diagnosis of an URI, which is more dependent on the patient's symptoms than on physical findings (sinusitis, bronchitis), showed less variability than the diagnosis of OM.
It was not possible to calculate true sensitivity and specificity for this study because of the lack of "criterion standard" diagnosis for each patient. However, if the group mean value is close to the correct diagnosis, then the range around that value provides some idea of the underdiagnosis and overdiagnosis of URIs and OM.
We were not able to find another study of variability among physicians in the diagnoses of these 2 common conditions. It can be argued that the study did not examine the diagnosis within a pediatric practice where the prevalence of both conditions is much higher. The assumption is that high prevalence will reduce variability; but, the strength of this study is the ability to examine these conditions across multiple physicians with varying experience and training. The failure to find an association with the type of residency training and the length of time in practice suggests that physicians establish implicit criteria for the diagnosis of both conditions early in their medical training. Perhaps because both OM and URIs are so common, no effort is made to validate these criteria with other colleagues. We are also not aware of any training in medical school or residency that provides standardization of these diagnoses.
There are no cost estimates available for the diagnosis and treatment of OM, but the cost is likely to be in the order of several hundred million dollars per year. Our findings suggest that there is substantial variation in the diagnosis of this disease, and that this variability is greatest among those at highest risk, ie, children younger than 5 years.
There are several implications of our findings. Medical educators have assumed that the diagnosis of common outpatient conditions is learned quickly and requires no standardization among students. This assumption may not be correct. Efforts at continuous quality improvement, primarily in hospitals, have demonstrated that undesirable variability is associated with higher costs and fewer satisfactory outcomes.8 We believe that cost savings may be seen if undesirable variation in the diagnosis of OM and URI is decreased.
Many clinical trials, especially those that measure actual use of antibiotics for the treatment of OM in the past, may have been biased by this undesirable variation. Variation of this magnitude could suggest a high cure rate if a significant majority of the patients never had the disease in the first place. Physicians in several European countries do not treat OM initially with antibiotics, only intervening if symptoms persist after 3 to 5 days. Some of the success of this treatment policy may be based on the overdiagnosis of the disease.
The data from this study suggest undesirable variation in the diagnosis of 2 common medical conditions. Further studies are needed to determine what training is necessary to reduce the undesirable variation.
AUTHOR INFORMATION
Accepted for publication August 6, 1997.
Reprints: Joseph L. Lyon, MD, MPH, Department of Family and Preventive Medicine, 50 N Medical Dr, Salt Lake City, UT 84132.
From the Department of Family and Preventive Medicine (Drs Lyon and Magill), University of Utah School of Medicine; Wasatch Physicians Service Incorporated and Intermountain Health Care Corporation (Drs Ashton and Turner), Salt Lake City, Utah.
REFERENCES
1. Schappert SM. National Ambulatory Medical Care Survey, 1991 Summary: National Center for Health Statistics. Vital Health Stat 13. 1994;116:1-110.
2. Perrin JM, Homer CJ, Berwick DM, Woolf AD, Freeman JL, Wennberg JE. Variations in rates of hospitalization of children in three urban communities. N Engl J Med. 1989;320:1183-1187.
WEB OF SCIENCE
| PUBMED
3. Wennberg JE, Freeman JL, Shelton RM, Bubolz TA. Hospital use and mortality among Medicare beneficiaries in Boston and New Haven. N Engl J Med. 1989;321:1168-1173.
WEB OF SCIENCE
| PUBMED
4. Keller RB, Soule DN, Wennberg JE, Hanley DF. Dealing with geographic variations in the use of hospitals:the experience of Maine Medical Assessment Foundation Orthopedic Study Group. J Bone Joint Surg Am. 1990;72:1286-1293.
PUBMED
5. Roark R, Petrofski J, Berson E, Berman S. Practice variation among pediatrics and family physicians in the management of otitis media. Arch Pediatr Adolesc Med. 1995;149:839-844.
FREE FULL TEXT
6. Manual of the International Statistical Classification of Diseases, Injuries, and Causes of Death, Volumes I and II (International Classification of Diseases, Ninth Revision [ICD-9]. Geneva, Switzerland: World Health Organization; 1975.
7. Rothman KJ. Modern Epidemiology. Boston, Mass: Little Brown & Co; 1986:41-50.
8. James BC. What is a TURP? controlling variation in the performance of clinical processes. In: Blumenthal DC, Schreck AC, eds. Improving Clinical Practice: Total Quality Managment and the Physician. San Francisco, Calif: Jossey-Bass;1995:167-202.
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
Pediatric Residents' Clinical Diagnostic Accuracy of Otitis Media
Steinbach et al.
Pediatrics 2002;109:993-998.
ABSTRACT
| FULL TEXT
Pediatric Resident Training in the Diagnosis and Treatment of Acute Otitis Media
Steinbach and Sectish
Pediatrics 2002;109:404-408.
ABSTRACT
| FULL TEXT
|
