The purpose of this study was to determine whether the awareness and inclusion of secondary sonographic signs of appendicitis, in combination with a structured evaluation as part of engagement and training for sonographers, improved appendix visualization rates and reduced equivocal findings in children with suspected acute appendicitis.
This was a prospective study of 230 children at a tertiary children’s hospital in Australia referred for an ultrasound examination of suspected appendicitis. The ultrasound findings, radiology reports, histology, clinical results, and follow-up were collated. Secondary signs were used as an additional assessment of the likelihood of disease where possible, even in the absence of an identified appendix.
The implementation of a structured evaluation as part of sonographer engagement and training resulted in a 28% improvement in appendix visualization (68.7%) compared with a prior retrospective study in a similar population (40.7%). The diagnostic accuracy was 91.7%, with likelihood ratios suggesting a meaningful influence of the pre-test probability of appendicitis in children studied (positive likelihood ratio, 11.22; negative likelihood ratio, 0.09.). Only 7.8% of the findings were equivocal. A binary 6-mm diameter cut-off did not account for equivocal cases, particularly lymphoid hyperplasia.
Acute appendicitis is the most common cause of emergent surgery. Its documented incidence is roughly 100 admissions per 100,000 person-years in Eastern Europe and North America, and is higher in Western Europe and Oceania . While there are regional and institutional variations, cases that present with a high clinical suspicion often proceed directly to treatment. Management typically involves surgical removal of the appendix. More recently, conservative treatment of cases of uncomplicated appendicitis with antibiotic therapy has been associated with a higher rate of major complications than primary appendectomy . Medical imaging plays an important role in patients who present with an atypical clinical presentation or a relatively low clinical probability of appendicitis. International differences exist in diagnostic imaging pathways for suspected appendicitis, with computed tomography (CT), ultrasound, and more recently, magnetic resonance imaging shown to be of use . It is widely accepted that, in children, ultrasound should be the first-line imaging modality, as it avoids the radiation burden of CT .
Appendiceal sonography has a wide range of reported accuracy, which is contingent upon the person performing the examination. In Australia, sonographers perform most appendiceal ultrasound examinations, usually in conjunction with a supervising radiologist, whilst in many European and Asian settings, scans are performed by radiologists themselves. Regardless of the person performing the scan, visualizing the appendix is problematic when investigating possible cases of appendicitis with ultrasound. Graded compression, first described by Puylaert in 1986 , is the most widely used sonographic technique for locating the appendix. In this method, a linear transducer is used to displace overlying bowel gas to reveal the appendix . In an Australian study in children using a tightly-curved array transducer, 91.7% of appendices were seen . Combining these principles, Lee et al. in South Korea  used different transducer shapes, varied the patient position, and extended the examination time up to 35 minutes, yielding a 99% appendix visualization rate. Studies demonstrating such high visualization rates show the usefulness of appendiceal ultrasound when time, training, and resources are available; however, in many clinical environments, visualization rates are commonly between 25% and 50% [8,9].
Making a definitive ultrasound diagnosis in suspected appendicitis relies on complete visualization of the appendix. When identified on ultrasound, an appendix is traditionally considered inflamed when it is greater than 6 mm in diameter, lacks compressibility, has wall hyperaemia, and has an appendicolith present [10,11]. A binary 6-mm cut-off criterion has been shown to result in a number of false-positive findings, leading to the proposal of a new diameter-based categorisation, with <6 mm being normal; 6-8 mm equivocal, and >8 mm positive for appendicitis . When the appendix is not visualized on ultrasound, or when the sonographic criteria for acute appendicitis are not met, secondary or ancillary signs of appendicitis can be a useful adjunct. The most useful of these is the presence of peri-appendiceal echogenic mesenteric fat (Fig. 1) . Other secondary signs may include free fluid, an appendicolith, phlegmon/abscess, prominent lymph nodes, and bowel dilatation/ileus (Fig. 2) .
The objectives of this study were to determine whether awareness of secondary sonographic signs of appendicitis, in combination with technical training for sonographers, improved appendix visualization rates and subsequently reduced equivocal findings in children with suspected acute appendicitis.
Materials and Methods
This prospective study included patients up to 16 years of age, who were referred for ultrasound evaluation of suspected appendicitis or undifferentiated right iliac fossa pain, between December 2015 and April 2017 in an Australian tertiary children’s hospital. A sample size was of 138 patients was required to achieve a sensitivity of at least 90%, with 80% power (ß) and α<0.05 . This was adjusted to 230 patients, as we estimated a 50% improvement in the visualization rate (to 60%) due to the training provided as part of the study, compared to the 40% visualization rate of our prior study in the same population . Changes in the value of ultrasound examinations between the current and prior study were evaluated through a comparison of diagnostic accuracy calculations, primarily the change in the likelihood ratio, which would alter the post-test probability and therefore the value of the examination for a referring clinician. Examinations performed after-hours were not included in the study to limit costs through overtime. Children's Health Queensland Human Research and Ethics Committee approval was obtained (HREC/15/QRCH/125). Written informed consent was provided by the parent or guardian of all children enrolled. All examinations were performed using a Philips Epiq 7G ultrasound platform (Philips Healthcare, Bothwell, WA, USA).
A single customised pre-set was used across machines and transducers (L12-5, L18-5, C8-5, C9-2, and C5-1) with the colour Doppler scale set at 6.5 cm/sec for uniformity in the evaluation of hyperaemia. The L12-5 and C8-5 transducers were primarily used, although sonographers could also use higher- or lower-frequency transducers if indicated based on the patient’s body habitus. A sonographer worksheet was developed based on variables identified in a prior retrospective study . This worksheet incorporated traditional appendiceal criteria, secondary signs of appendicitis, and clinical history. The sonographic signs evaluated as potential secondary signs were peri-appendiceal mesentery appearance (more echogenic than the contralateral side), free fluid (present or not), bowel dilatation (fluid-filled hypoperistaltic or aperistaltic small bowel), prominent lymph nodes (a single node greater than 10 mm in the short axis, or a cluster of greater than 5 nodes with short axes greater than 5 mm), bladder debris, and presence of an abscess/ phlegmon (complex intraperitoneal collection).
The sonographers performing the examinations were trained on the research protocol and the suggested scanning technique, including the various transducers (as listed above), patient positions (supine and left-lateral decubitus), graded compression and posterior-manual compression , and acoustic windows (urinary bladder and iliac vessels). The clinical experience of sonographers was diverse, from a student in their final year of training to some with decades of dedicated pediatric experience. Full-time-equivalency (FTE) also varied from full-time to 1 day per week. The decision to include all available sonographers was made to better reflect employment status and experience levels in most practices. The radiologists reporting the examinations included in the study were also made aware of the protocol. The study imposed no formal or structured changes to the radiology report, which was considered to be the ultimate finding of the ultrasound examination.
The completed worksheets were collected and electronically collated. Clinical information was retrieved from the patients’ electronic medical records, including the surgical report, histology findings, C-reactive protein (CRP) level, white blood cell (WBC) count, weight, height, age, duration of symptoms, and any differential diagnosis. The histology report, where available, was the reference standard for the final diagnosis. Patients who did not undergo appendectomy or re-present within 1 month were considered not to have suffered from acute appendicitis. Ultrasound findings were allocated into one of five diagnostic categories integrating the presence of secondary signs and the three diameter-based classifications (Table 1).
Statistical analysis was performed using IBM SPSS Statistics ver. 22 (IBM Corp., Armonk, NY, USA). Categorical data were evaluated using descriptive statistics, sensitivity, specificity, accuracy, and likelihood ratios. The diagnostic accuracy analyses required condensing the five diagnostic categories into binary outcomes (Table 1). This was performed using three methods to better permit comparisons with studies that may have treated equivocal studies in different manners by excluding the equivocal categories altogether (method 1); collapsing the probable positive and negative results into the positive and negative findings, respectively, and excluding the remaining equivocal findings (method 2); and considering the remaining equivocal examinations as negative findings (method 3). A receiver operating characteristic (ROC) curve was used to analyse the area under the curve (AUC) for appendix diameter, WBC count, and CRP. As WBC count can vary with age, a binary variable for elevated WBC count was calculated based on the patient’s age and relevant upper reference limit . The chi-square test was used to assess visualization rates and employment status (<0.4, 0.4-0.7, or 0.8-1.0 FTE) and years of experience (<5, 5-10, or >10 years), with post hoc cell-wise residual analysis used to identify significance between groups . Logistic regression analysis was used to assess the associations of the age of the patient, length of symptoms, and body mass index for age percentile (BMI-FAP) with the visibility of their appendix. Logistic regression was also used to analyse the relationships among the secondary signs of appendicitis, traditional ultrasound criteria and the presence of acute appendicitis. The threshold for statistical significance was set at P<0.05.
A total of 230 imaging studies were included during the 16-month period, with 109 in female patients (47.4%) and 121 in male patients (52.6%). The patients’ age ranged from 21 months to 16 years, with a mean of 9 years and 10 months (standard deviation [SD], 3 years and 5 months). The appendix was identified on ultrasound in 158 cases (68.7%). The incidence of appendicitis was 19.1%, with 63 instances of alternative diagnoses, the most common being mesenteric adenitis (n=36) (Table 2). There were 20 negative appendectomies performed (8.7%) (Table 3). These negative appendectomies included four cases of parasite infestation, six cases of lymphoid hyperplasia (including four false-positives), a case of ovarian torsion, a perforated Meckel diverticulum, and an omental infarct that was suspected pre-operatively on ultrasound.
Diagnostic accuracy calculations were conducted using the three aforementioned methods of classifying findings, and also including only cases where the appendix was visualized on ultrasound (Table 4). A logistic regression model analysing the effects of pain duration, BMI-FAP, and the patient’s age on the likelihood of visualizing the appendix showed statistically non-significant results (χ2 (3)=7.393, P=0.060), explaining 6% (Nagelkerke R2 ) of the variance in appendix visualization and correctly classifying 68% of cases. Increasing age was associated with a decreased likelihood of appendix visualization. Overall, the visualization rate in this study was markedly improved from our prior study and compared favourably to other recent pediatric studies (Table 5).
The mean uncompressed appendiceal diameter for normal appendices was 5.0 mm (SD, 1.4 mm; 95% confidence interval [CI], 4.8 to 5.4), and 9.2 mm for inflamed appendices (SD, 2.9 mm; 95% CI, 8.1 to 10.4) (Fig. 3). When compression could be applied to appendices during diameter measurements, the mean was reduced: normal, 3.9 mm (SD, 1.5 mm; 95% CI, 3.7 to 4.3) and inflamed, 8.6 mm (SD, 2.9 mm; 95% CI, 7.4 to 9.7). When the appendix diameter measurements were analyzed with a ROC curve, there was little difference between non-compressed measurements (AUC, 0.964; 95% CI, 0.902 to 0.985; P<0.001) and those made with compression of the appendix (AUC, 0.950; 95% CI, 0.937 to 0.990; P<0.001), with optimal cut-offs of 6.8 mm and 5.8 mm respectively (Fig. 4). Using the DeLong test, the difference between the area under the ROC curves of the compressed and uncompressed appendix diameters of 0.006 was not significant (P=0.670) . A logistic regression model analysing the effects of secondary sonographic signs (mesentery, nodes, bowel dilatation, free fluid, bladder debris, and abscess) and traditional ultrasound criteria (compressed diameter, compression, hyperaemia, or appendicolith) on the likelihood of a child having acute appendicitis yielded statistically significant results (χ2 (10)=98.179, P<0.001), explaining 85.2% (Nagelkerke R2 ) of the variance in acute appendicitis and correctly classifying 96.7% of cases. Lack of echogenic mesentery and appendiceal wall hyperaemia were associated with a decreased likelihood of appendicitis.
Analysis of common pathology results ordered for suspected appendicitis revealed that WBC count (AUC, 0.757; 95% CI, 0.680 to 0.834; P<0.001) and CRP levels (AUC, 0.701; 95% CI, 0.613 to 0.789; P<0.001) were potentially useful indicators of appendicitis (Fig. 4). An optimal WBC count cut-off of 10.55×109 /L was identified in the ROC analysis. The Fisher exact test revealed a significant relationship between having appendicitis and a WBC count above the upper reference limit for the patient’s age (P<0.001) .
Of the secondary signs evaluated, only echogenic mesentery appeared to be an independent predictor of the presence or absence of appendicitis: sensitivity, 95.5%; specificity, 90.3%; positive predictive value (PPV), 70%; and negative predictive value (NPV), 98.8%. While the PPV of the remaining secondary signs was poor, they did demonstrate some potential value for negative prediction, as follows: free fluid: PPV, 24.2% and NPV, 84.1%; prominent lymph nodes: PPV, 13.6% and NPV, 74.3%; and dilated bowel loops: PPV, 33.3% and NPV, 82.3%.
Fifteen sonographers with varying levels of experience were involved in the study examinations, including seven with more than 10 years of experience (47%); five with 5-10 years (33%); and three with fewer than 5 years, including a student (20%). Full-time employment equivalency also varied: five sonographers worked more than 8 days per fortnight (≥0.8 FTE, 33%); six between 4 and 7 days per fortnight (0.4-0.7 FTE, 40%); and the remainder worked 3 days per fortnight or less (≤0.3 FTE, 27%). The visualization rates based on the clinical experience of sonographers performing the examinations were 77% for those with 5-10 years of experience (n=104), 73% for those with fewer than 5 years (n=37), and 59% for those with over 10 years (n=89). The visualization rate according to the full-time employment status of sonographers was highest amongst those working ≥0.8 FTE (81%, n=124). Staff who worked 0.4-0.7 FTE had a rate of 64% (n=42), whilst those who worked ≤0.3 FTE had the lowest rate of 50% (n=64). Sonographer experience (χ2 (2)=7.232, P=0.027) and FTE (χ2 (2)=17.942, P<0.01) were both associated with appendix visualization. Post hoc cell-wise residual analysis revealed that those working greater than 0.8 FTE (P<0.001) were more likely to detect the appendix, while those working less than 0.3 FTE (P<0.001) or with more than 10 years of experience (P=0.007) were significantly less likely to successfully visualize it.
Complete visualization of the appendix during an ultrasound examination is ideal, but this is often not achieved, leading to a high proportion of what are often considered non-diagnostic or equivocal studies . This prospective study was designed to ascertain whether engagement of sonographers through technical training and improved awareness of the secondary signs of appendicitis would lead to improvements in appendix visualization and diagnostic accuracy. The results of this study clearly show that better awareness and recording of secondary sonographic signs by sonographers improved the post-test probability and reduced the proportion of equivocal findings. The presence of secondary signs was recorded regardless of whether the appendix was visualized, permitting probable radiological findings to be conveyed for what would otherwise be non-diagnostic studies in the event of not identifying the appendix.
There was a marked improvement in appendix visualization between this study and our previous retrospective investigation in a similar patient population (Table 5) . Whilst this was likely due to training in appendiceal ultrasound techniques for the sonographers involved, we also noted a cultural shift in terms of an enhanced expectation that the appendix should be located during every examination, which was previously recognized by Cundy et al.  as an important factor in their study, which reported an excellent visualization rate. An acknowledged source of bias is that the sonographers were not blinded to the study objectives. Our improved visualization rate is in contrast with another recent study, which sought to implement a more structured ultrasound report including secondary signs, and reported a decrease in visualization from before (43.9%) to after the intervention (32.7%) . Interestingly, that study did not detail any involvement of the sonographers performing the examinations in the quality improvement process. The engagement of staff performing these examinations, be they sonographers or doctors, appears crucial in achieving high visualization rates.
Of our false-positive findings, two had normal histology, one had a perforated Meckel diverticulum that the surgical team felt clinically warranted laparoscopy, and eight cases did not proceed to surgery. The remainder of the false-positive cases were histologically proven lymphoid hyperplasia with the presence of one or more secondary signs (appendicolith, echogenic mesentery, free fluid, or hyperaemia) and a wide range of increased appendiceal diameters noted on ultrasound. A previous study demonstrated that it was possible to differentiate lymphoid hyperplasia on ultrasound from acute appendicitis by identifying a thickened, hypoechoic lamina propria in a moderately enlarged appendix capable of resisting compression .
This prospective study was designed to overcome some of the limitations of a prior baseline retrospective study in the same patient population, particularly the lack of documentation of secondary signs and the availability of some clinical data, such as height and weight . A limitation of this study was the lack of pathological evidence in children that did not undergo appendectomy, which may not have captured cases of spontaneously resolving appendicitis. Another limitation was the potential for children to be lost to follow-up if they did not re-present at our institution; however, this was considered unlikely, as our institution is the only tertiary pediatric hospital in the region.
Studies with excellent visualization rates tend to have higher patient volumes, providing more opportunity to develop and maintain the skills necessary to achieve such results [6,7,25,26]. Our findings mirrored this effect of patient volume through differences in visualization rates based on sonographer full-time employment equivalency. When we considered only definitive radiological diagnoses in this study (161 of 230 patients), sensitivity and specificity were very high (100% and 96.8%, respectively). Excluding only the 7.9% (n=18) of cases that were equivocal and integrating the presence of secondary signs into what would otherwise be probable radiological findings maintained a high degree of accuracy (sensitivity, 97.6%; specificity, 91.9%) with an appendix visualization rate of 68.7%. To make a meaningful comparison of the diagnostic accuracy of this study and our prior one, we compared the examinations where the appendix was visualized, revealing an increase in the positive likelihood ratio from 1.63 to 9.23, with little difference in the negative likelihood ratio, which changed from 0.055 to 0.056 .
As children with suspected appendicitis do not always present to a tertiary pediatric health facility, there is a need to engage with centres providing non-dedicated pediatric ultrasound services that have lower imaging study volumes. It is also sometimes not feasible to perform ultrasound. If ultrasound cannot be performed in a way that affects pre-test probability, there may be a temptation to perform CT for further clarification or to proceed to surgery, both of which have inherent risks. Whilst Partain et al. [20,23] saw the visualization rate diminish after implementing a structured radiology report with integrated secondary signs, they found that admission and CT use were nonetheless reduced, as secondary signs provided guidance to clinicians in cases that would otherwise be equivocal and of little value.
Different academic curricula and the nature of sonography training in some centres with low pediatric or emergent caseloads can lead to varied levels of knowledge or confidence in pediatric appendiceal ultrasound. The results of this study provide clear evidence that engagement of sonographers through training in an adaptive technique to locate the appendix can improve appendix visualization. Better awareness and documentation of secondary signs of appendicitis can reduce the number of truly equivocal findings, especially in practices with a lower visualization rate. This approach can better equip radiologists to convey the likelihood of disease to referrers even in the absence of a visualized appendix. Ultrasound plays an important role in children with suspected appendicitis, reducing the need for CT and associated ionizing radiation concerns, without increasing the risk of morbidity through missed appendicitis. The value of ultrasound findings to clinicians can be improved through sonographer engagement in the recognition of secondary sonographic signs, leading to a higher visualization rate and reduced equivocal findings.
The authors would like to thank the patients and their families who consented to participate in this study, as well as the medical imaging assistants, sonographers, and radiologists of the Department of Medical Imaging and Nuclear Medicine at Lady Cilento Children’s Hospital.
1. Ferris M, Quan S, Kaplan BS, Molodecky N, Ball CG, Chernoff GW, et al. The global incidence of appendicitis: a systematic review of population-based studies. Ann Surg 2017;266:237–241.
2. Kirby A, Hobson RP, Burke D, Cleveland V, Ford G, West RM. Appendicectomy for suspected uncomplicated appendicitis is associated with fewer complications than conservative antibiotic management: a meta-analysis of post-intervention complications. J Infect 2015;70:105–110.
3. Imler D, Keller C, Sivasankar S, Wang NE, Vasanawala S, Bruzoni M, et al. Magnetic resonance imaging versus ultrasound as the initial imaging modality for pediatric and young adult patients with suspected appendicitis. Acad Emerg Med 2017;24:569–577.
4. Mostbeck G, Adam EJ, Nielsen MB, Claudon M, Clevert D, Nicolau C, et al. How to diagnose acute appendicitis: ultrasound first. Insights Imaging 2016;7:255–263.
5. Puylaert JB. Acute appendicitis: US evaluation using graded compression. Radiology 1986;158:355–360.
6. Cundy TP, Gent R, Frauenfelder C, Lukic L, Linke RJ, Goh DW. Benchmarking the value of ultrasound for acute appendicitis in children. J Pediatr Surg 2016;51:1939–1943.
7. Lee JH, Jeong YK, Park KB, Park JK, Jeong AK, Hwang JC. Operatordependent techniques for graded compression sonography to detect the appendix and diagnose acute appendicitis. AJR Am J Roentgenol 2005;184:91–97.
8. Alter SM, Walsh B, Lenehan PJ, Shih RD. Ultrasound for diagnosis of appendicitis in a community hospital emergency department has a high rate of nondiagnostic studies. J Emerg Med 2017;52:833–838.
9. Lofvenberg F, Salo M. Ultrasound for appendicitis: performance and integration with clinical parameters. Biomed Res Int 2016;2016:5697692.
10. Kessler N, Cyteval C, Gallix B, Lesnik A, Blayac PM, Pujol J, et al. Appendicitis: evaluation of sensitivity, specificity, and predictive values of US, Doppler US, and laboratory findings. Radiology 2004;230:472–478.
11. Quigley AJ, Stafrace S. Ultrasound assessment of acute appendicitis in paediatric patients: methodology and pictorial overview of findings seen. Insights Imaging 2013;4:741–751.
12. Trout AT, Towbin AJ, Fierke SR, Zhang B, Larson DB. Appendiceal diameter as a predictor of appendicitis in children: improved diagnosis with three diagnostic categories derived from a logistic predictive model. Eur Radiol 2015;25:2231–2238.
13. Lee MW, Kim YJ, Jeon HJ, Park SW, Jung SI, Yi JG. Sonography of acute right lower quadrant pain: importance of increased intraabdominal fat echo. AJR Am J Roentgenol 2009;192:174–179.
14. Reddan T, Corness J, Mengersen K, Harden F. Ultrasound of paediatric appendicitis and its secondary sonographic signs: providing a more meaningful finding. J Med Radiat Sci 2016;63:59–66.
15. Jones SR, Carley S, Harrison M. An introduction to power and sample size estimation. Emerg Med J 2003;20:453–458.
16. Reddan T, Corness J, Mengersen K, Harden F. Sonographic diagnosis of acute appendicitis in children: a 3-year retrospective. Sonography 2016;3:87–94.
17. Bain BJ. Blood cells: a practical guide. 5th ed. Chichester: John Wiley & Sons, Ltd, 2014.
18. Garcia-perez MA, Nunez-anton V. Cellwise residual analysis in twoway contingency tables. Educ Psychol Meas 2003;63:825–839.
19. Cohen B, Bowling J, Midulla P, Shlasko E, Lester N, Rosenberg H, et al. The non-diagnostic ultrasound in appendicitis: is a nonvisualized appendix the same as a negative study? J Pediatr Surg 2015;50:923–927.
20. Partain KN, Patel A, Travers C, McCracken CE, Loewen J, Braithwaite K, et al. Secondary signs may improve the diagnostic accuracy of equivocal ultrasounds for suspected appendicitis in children. J Pediatr Surg 2016;51:1655–1660.
21. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–845.
22. Alter SM, Walsh B, Lenehan PJ, Shih RD. Radiology report "disclaimers" increase the use of abdominal CT in the work-up of pediatric abdominal pain. Am J Emerg Med 2018;36:556–559.
23. Partain KN, Patel AU, Travers C, Short HL, Braithwaite K, Loewen J, et al. Improving ultrasound for appendicitis through standardized reporting of secondary signs. J Pediatr Surg 2017;52:1273–1279.
24. Xu Y, Jeffrey RB, DiMaio MA, Olcott EW. Lymphoid hyperplasia of the appendix: a potential pitfall in the sonographic diagnosis of appendicitis. AJR Am J Roentgenol 2016;206:189–194.
25. Mittal MK, Dayan PS, Macias CG, Bachur RG, Bennett J, Dudley NC, et al. Performance of ultrasound in the diagnosis of appendicitis in children in a multicenter cohort. Acad Emerg Med 2013;20:697–702.
SN, sensitivity; SP, specificity; PPV, positive predictive value; NPV, negative predictive value; LR+, positive likelihood ratio; LR-, negative likelihood ratio.