Synchronous tele-ultrasonography is helpful for a naive operator to perform high-quality thyroid ultrasound examinations
Article information
Abstract
Purpose
This study investigated the value of synchronous tele-ultrasonography (TUS) for naive operators in thyroid ultrasonography (US) examinations.
Methods
Ninety-seven patients were included in this prospective, parallel-controlled trial. Thyroid scanning and diagnosis were completed by resident A independently, resident B with guidance from a US expert through synchronous TUS, and an on-site US expert. The on-site expert’s findings constituted the reference standard. Two other off-site US experts analyzed all data in a blind manner. Inter-operator consistency between the two residents and the on-site US expert for thyroid size measurements, nodule measurements, nodule features, American College of Radiology (ACR) Thyroid Imaging Reporting and Data System (TI-RADS) categories, and image quality was compared. Two questionnaires were completed to evaluate the clinical benefit.
Results
Resident B detected more nodules consistent with the on-site expert than resident A did (89.4% vs. 56.5%, P<0.001). Resident B achieved excellent consistency with the on-site expert in terms of ACR TI-RADS categories, nodule composition, shape, echogenic foci, and vascularity (all intra-class correlation coefficients [ICCs] >0.75), while resident A achieved lower consistency in ACR TI-RADS categories, composition, echogenicity, margin, echogenic foci, and vascularity (all ICCs 0.40-0.75). Residents A and B had excellent consistency in target nodule measurements (all ICCs >0.75). Resident B achieved better performance than resident A for gray values, time gain compensation, depth, color Doppler adjustment, and the visibility of key information (all P<0.05). Furthermore, 61.9% (60/97) of patients accepted synchronous TUS, and 59.8% (58/97) patients were willing to pay for it.
Conclusion
Synchronous TUS can help inexperienced residents achieve comparable thyroid diagnostic capability to a US expert.
Introduction
Thyroid nodules are a common clinical problem, the incidence of which is increasing worldwide [1]. With the development of high?frequency ultrasonography (US), US has become the first-line imaging modality for the diagnosis and management of thyroid-related diseases [2,3]. Thyroid nodules have been detected by US in 20%-76% of the population, with about 7%-15% of thyroid nodules diagnosed as malignancies [4-6]. The World Health Organization considers US to be one of the most important medical technologies, and many medical institutions are equipped with US scanners [7]. However, the current major barrier in US examinations is the fact that US is highly operator-dependent, which may lead to different diagnoses for the same patient between experienced and inexperienced US performers. To overcome this barrier, the American College of Radiology (ACR) Thyroid Imaging Reporting and Data System (TI-RADS) was proposed for evaluating thyroid nodules [8]. When diagnosing a nodule using the ACR TI-RADS, it is important to acquire all key information of the thyroid nodule, such as its composition, echogenicity, and shape, to make a correct diagnosis. However, because of the global shortage of medical resources, experienced US performers are not always present, especially in resource-limited countries and areas [9]. As a result, many patients are misdiagnosed or never diagnosed. This situation offers new opportunities for reforms to address the changing times.
Telemedicine, as one such reform, has been defined as the provision of medical services using communication technology. It exchanges medical information among two or more sites remotely without the need for direct contact between doctors and patients [10-12]. In 1987, a paper in The Lancet first described the use of the telephone to make a medical diagnosis [13]. Tele-ultrasonography (TUS), as a new sub-type of telemedicine, could enable the provision of remote US examinations that yield accurate diagnoses. TUS might be a good solution for the scenarios with insufficient US medical resources [14,15]. In 2005, the United States National Aeronautics and Space Administration was the first organization to use TUS to evaluate shoulder integrity in symptomatic crew members in the International Space Station [16].
In routine clinical practice, synchronous TUS can provide patients with high-quality real-time medical care. In particular, it enables experienced US performers to monitor the scanning process in real-time and give feedback simultaneously, resulting in high imaging quality and a timely, accurate diagnosis. Hence, it was assumed that remote guidance via synchronous TUS would be helpful for a naive operator (i.e., an inexperienced resident) for thyroid US examinations. To confirm this hypothesis, this prospective and parallel controlled trial was designed to explore the effectiveness of applying synchronous TUS in aiding inexperienced residents for thyroid examinations, and further to investigate the views of inexperienced residents and patients on synchronous TUS for thyroid examinations.
Materials and Methods
Compliance with Ethical Standards
The trial protocol (No. SHSY-IEC-4.1/20-65/01) was approved by the ethical committee of the hospital. Each patient was informed about the details of the trial and informed consent was obtained at the beginning of US examination.
Design
This prospective clinical trial was conducted at a thyroid research institute based in a university hospital where more than 40,000 thyroid-related patients are seen every year.
Participants
Patients included in the trial were recruited at the authors’ institute. The investigator informed the patients about all examinations and procedures, and all patients who followed the inclusion criteria were enrolled: (1) age ≥18 and ≤80 years, (2) an initial thyroid US examination, and (3) the provision of consent to participate in this trial. The exclusion criteria were (1) patients with a treatment history of thyroid disease and (2) patients with incomplete US imaging data.
Three US experts and two residents participated in the trial. Two residents (referred to hereinafter as resident A and resident B) and one expert (L.H.G., with 8 years of experience in thyroid US examinations) were designated to participate in the US scanning procedure. Both residents were first employed after graduation from medical school. They had completed a full-time US course that covered basic US principles, the basic operation method of US, and the assessment categories of the ACR TI-RADS. However, they had limited clinical experience in thyroid US examinations (fewer than 10 patients). Two other experts (X.L.L. and Q.W., both with approximately 5 years of experience in thyroid US), as off-site experts, were designated to analyze all the data in a blind manner.
Synchronous TUS
In this trial, the synchronous TUS system was a combination of US equipment (Aplio500 US scanner, Toshiba Medical System, Tochigi, Japan), a data acquisition subsystem (Shanghai Keyi Intelligent Technologies Co. Ltd., Shanghai, China), two headsets, one web camera, one laptop, and networking infrastructure. A data acquisition subsystem is the key to two-wire synchronous data transmission, which can transmit US videos in Digital Imaging and Communications in Medicine format from the US equipment to the screen of the remote expert’s laptop via the network. Real-time voice and videos can be transmitted using headsets and a web camera. A high-speed network (300 Mbps downstream and 50 Mbps upstream) was available at the institute, which allowed real-time monitoring and high-quality communication of US images in this trial.
US Examination Protocol
All thyroid US examinations were performed with the same Aplio500 machine with a linear array transducer (14L5; frequency range, 5 to 14 MHz). Each patient was scanned three times: first, independently scanned by resident A; second, scanned by resident B with real-time guidance by a US expert (L.H.G.), who was located in another room and oversaw the scan using the synchronous TUS system (Fig. 1); and third, as a reference standard, the scan was performed on-site by the same expert who guided resident B. Resident A made the diagnoses independently, while resident B made them through discussions with the remote expert. Throughout the entire scanning procedure, resident A and resident B were blind to each other, and the on-site expert was blind to resident A.
All thyroid US examinations included two steps: first, the thyroid was scanned and measured, including the transverse and anterior-posterior (AP) diameters of the thyroid lobes and the thickness of the isthmus; second, if present, the thyroid nodule was detected and measured. If no nodule was found, the examination was finished at step 1. For patients with multiple nodules, the nodule most highly suspicious of malignancy on US was measured. Otherwise, if the nodules were equally suspicious of malignancy, the one with the largest diameter. The thyroid measurement and thyroid nodule measurement protocol is presented in Fig. 2. The size, location, and US features of the nodules were recorded on a case report form. According to the white paper of the ACR TI-RADS atlas, the US features are depicted as follows: composition, echogenicity, shape, margin, and echogenic foci. The total time of each US examination by resident A and resident B was also recorded. The total time was calculated from the time the US transducer touched the skin to the completion of the scan.
Questionnaire
After each US examination, each patient and resident B were asked to complete a separate questionnaire to evaluate the clinical benefit.
Outcomes
The background image quality (i.e., gray value, depth, focus position and time gain compensation adjustment) and the target nodule image quality (i.e., color Doppler adjustment and visibility of all key information) were evaluated by two off-site US experts. If no nodule was detected, only the image quality of the thyroid parenchymal background was evaluated. Based on the reference standard from the on-site US expert, a result was reached through consensus.
A comprehensive assessment was also carried out by the two off-site US experts using a 5-point Likert scale: 1 point, the image quality is poor and cannot be used for diagnosis; 2 points, the image quality is not good enough and could affect the diagnosis; 3 points, the image quality is flawed, but the image can be used for diagnosis; 4 points, the image quality is good and the image can be used for diagnosis with satisfaction; and 5 points, the image quality is excellent and can be used for diagnosis with a high level of satisfaction [17].
Inter-observer and Intra-observer Consistency
Thirty other thyroid nodules examined by the on-site US expert were consecutively selected to analyze the inter-observer and intra-observer (i.e., L.X.L. and Q.W.) reliability for US features. Intra-observer consistency was evaluated by the same off-site US expert at a time interval of 2 days.
Statistical Analyses
SPSS version 20.0 for Windows (IBM Corp., Armonk, NY, USA) was used for the statistical analyses. The independent t-test was used to compare quantitative data (i.e., patient age, thyroid size, and examination time), which were expressed as mean±standard deviation. The chi-square test was used to compare categorical variables, which were expressed as numbers and percentages. The Wilcoxon signed rank test was used to compare the total score of the comprehensive image quality assessment. The sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) for residents A and B were computed. The intra-class correlation coefficient (ICC) was used to evaluate the inter-operator consistency of the target nodule features, ACR TI-RADS categories, and target nodule measurement. Agreement was considered as excellent (ICC, 0.75-1), good (ICC, 0.40-0.75), fair (ICC, 0.20-0.40), and poor (ICC, 0.00-0.20). A P<0.05 was considered as indicating a significant difference.
The patients were divided into five cohorts (cohort 1: patients 1-20, cohort 2: patients 21-40, cohort 3: patients 41-60, cohort 4: patients 61-80, cohort 5: patients 81-97) in the order of their examinations. The mean values of the examination time and comprehensive assessment of the five cohorts were used to reflect trends in their image quality and resident B’s experience with synchronous TUS.
Results
Patients
From April 2020 to June 2020, 99 patients participated in the trial. Two patients were excluded because of incomplete imaging data. Finally, 97 patients (mean age, 43.2±13.7 years; range, 19 to 72 years) comprised the study cohort with complete imaging data (Fig. 3). Among them, 68 (70.1%) were female (mean age, 41.8±14.2 years; range, 19 to 72 years) and 29 (29.9%) were male (mean age, 46.5±11.9 years; range, 22 to 63 years).
Image Assessment
Thyroid nodules were detected in 68.0% (66/97) of patients by the on-site US expert as the reference standard. Among them, 27 patients (40.9%) had one nodule, 15 patients (22.7%) had two nodules, and 24 patients (36.4%) had three or more nodules. In accordance with the protocol presented above, 66 nodules from 66 patients were measured. Thirty-three nodules (50.0%) were located in the right lobe of the thyroid, 30 (45.5%) nodules were located in the left lobe, and three nodules (4.5%) were located in the isthmus. The mean size of the nodules was 12.0±9.9 mm (range, 2.2 to 46.9 mm). The ACR TI-RADS categories of the thyroid nodules were as follows: 3.0% (2 of 66) for category 2, 51.5% (34 of 66) for category 3, 43.9% (29 of 66) for category 4, and 1.5% (1 of 66) for category 5.
Thyroid nodules were detected in 71.1% (69/97) and 68.0% (66/97) of patients by resident A and resident B, respectively. Resident B detected the same nodules more consistently with the on-site US expert than resident A did (resident A, 56.5%; resident B, 89.4%; P<0.001). The diagnostic results of sensitivity, specificity, accuracy, PPV, and NPV for resident A and resident B are summarized in Table 1. In terms of sensitivity, specificity, accuracy, PPV, and NPV, resident B achieved a higher overall diagnostic performance than resident A (all P<0.05).
The inter-operator agreement for the target nodule features and ACR TI-RADS categories obtained by resident A, resident B, and the on-site expert are summarized in Table 2. In terms of the inter-operator agreement between resident B and the on-site US expert, there was excellent consistency in the ACR TI-RADS categories, composition, shape, echogenic foci, and vascularity (all ICCs >0.75). The inter-operator agreement for echogenicity and margin illustrated good consistency (both ICCs 0.40-0.75). Meanwhile, the inter-operator agreement between resident A and on-site US expert showed good consistency in the ACR TI-RADS categories, composition, echogenicity, margin, echogenic foci, and vascularity (all ICCs 0.40-0.75). Regarding the inter-operator agreement for the measurements of the target nodules, residents A and B had excellent consistency (all ICCs >0.75). The size of the transverse and AP diameters of the thyroid lobes and the thickness of the isthmus are summarized in Table 3. Statistically significant differences were found in the measurements of the left thyroid lobe size between resident A and resident B (both P<0.05).
The comprehensive assessment score was higher for resident B than for resident A (P=0.043). As the number of scanning cases increased, the mean comprehensive assessment score for resident B increased (Fig. 4).
In terms of gray values, time gain compensation, depth, color Doppler adjustment, and the visibility of key information, the quality of the images obtained by resident B was higher than that of the images obtained by resident A (all P<0.05). The results of the image quality analysis are shown in Table 4.
Reproducibility
The inter-observer reproducibility was good for the US features of composition between the two off-site experts (kappa coefficient, 0.75), while the inter-observer reproducibility for echogenicity, shape, margin, echogenic foci, and vascularity was excellent (all kappa coefficients >0.81). All correlation coefficients of intra-observer reproducibility were >0.61.
Total Time
The total time of the US examinations for resident B was 274.40±117.43 seconds (range, 110.00 to 706.00 seconds), while that for resident A was 193.43±63.93 (range, 100.00 to 359.00 seconds). Resident B spent more time than resident A (P<0.001). As their scanning experience increased, the mean scanning time for both resident A and resident B in the rolling cohorts decreased; however, resident B showed a more pronounced decreasing trend (Fig. 5).
Questionnaire for Resident B and Patients
The answers to the questionnaires for resident B and patients are summarized in Table 5. Resident B believed that the synchronous TUS was helpful in 62.9% (61/97) of the thyroid US examinations and had a training effect in 64.9% (63/97). A total of 61.9% (60/97) of the patients accepted the mode of synchronous TUS for thyroid US examinations, and 58 patients (59.8%) were willing to pay reasonable fees for it in the future.
Discussion
In this prospective study, synchronous TUS was found to be effective and acceptable for thyroid US examinations. In clinical practice, TUS includes synchronous and asynchronous modes. The asynchronous mode means that the transmission of TUS images and related communication cannot be performed in real time. As previously reported by Alfageme et al. [18], the mean response time using asynchronous TUS for evaluating dermatologic diseases was 52.3±2.2 hours. As a result, this mode may lead to delay in diagnoses and further treatment. The mode cannot alleviate the shortage of medical workers and the unbalanced distribution of medical resources. Based on the findings of this study, synchronous TUS enables inexperienced US residents to obtain stable and high-quality US images to make accurate diagnoses, so that people in resource-limited countries and regions can access high-quality medical care in a timely manner. In addition, fortunately, the availability of telemedicine with fourth-generation (4G) wireless transmission technology and 4G long term evolution wireless communication technology has improved significantly, and fifth-generation (5G) transmission is available in certain countries [19]. The improvement of transmission technology makes synchronous TUS increasingly accessible.
In the current study, the quality of the images obtained by the inexperienced resident under the guidance of the remote expert using synchronous TUS was higher than that of the independent resident. An important point to note is that there was no statistically significant difference in image quality with regard to the focus position (P=0.070). In general, the correct focus position should be in the lower middle thyroid. The independent inexperienced resident could correctly adjust the focus position. This means that to some extent, the focus position is not a key factor influencing the image quality for inexperienced residents.
In terms of thyroid nodule detection, the diagnostic performance of the inexperienced resident with remote guidance via synchronous TUS was higher than that of the independent resident. The guidance provided additional diagnostic sensitivity, specificity, accuracy, PPV, and NPV. The independent resident misdiagnosed 30 cases (43.5%). Low image quality and a lack of confidence may be the reason for this. In contrast, the resident with guidance from the remote expert misdiagnosed only seven (10.6%) cases. All seven of these patients had a heterogeneous thyroid background, which interfered with the diagnosis of nodules. In terms of target nodule features, no US features presented excellent inter-operator agreement between the independent resident and the expert (all ICCs, 0.40-0.75). As a result, the ACR TI-RADS categories of the target nodules were not sufficiently accurate. Fortunately, with synchronous TUS guidance, the other resident achieved higher consistency with the expert. In addition, a significant difference in the left thyroid lobe size measurements was found between residents A and B. It was observed that eight nodules in the left thyroid lobe were adjacent to the isthmus. Among them, five nodules were markedly large, with a size exceeding 20 mm. Therefore, the left lobe size measurements in these five cases may have been influenced by the large nodules, especially for the inexperienced resident. Fortunately, a consistency analysis of the measurements of the target nodules showed that residents A and B had excellent consistency (all ICCs >0.75). Therefore, the size measurements of thyroid nodules inside the left lobe were not affected by the results of the left thyroid lobe size measurement. Synchronous TUS was very helpful for the inexperienced resident to measure the thyroid and target nodules. The results of this study are thus thought to provide solid evidence of the benefit of synchronous TUS in a comprehensive way.
The application of synchronous TUS can lead to the following three problems: (1) the total time for thyroid examination is prolonged; (2) the cost of medical resources in hospitals is increased, and (3) the economic burden for patients becomes more substantial. However, the questionnaire results showed that most patients accepted synchronous TUS and were willing to pay for this new method in the future. After internal negotiation and consultation with the patients, an additional 25% of the US examination price is potentially reasonable.
In addition, synchronous TUS had a training effect on inexperienced US residents. As the number of examined patients increased, the time of the examinations decreased, especially for the resident who received guidance from the expert. This finding is a good illustration of the potential of synchronous TUS. Previous studies by Kim et al. [20] and Grant et al. [21] found that remote guidance could bring a significant improvement in diagnostic confidence for inexperienced residents. In the future, training by synchronous TUS could help inexperienced residents obtain high-quality US images and make accurate diagnoses. In addition, synchronous TUS in the present study is only helpful for inexperienced residents. When the on-site doctor is experienced, from the standpoint of both the patient and the experienced US performer, the synchronous TUS seems unnecessary. However, in the authors’ opinion, the synchronous TUS system may be beneficial in specific challenging cases for obtaining a prompt diagnosis by experienced US practitioners.
There were several limitations of this study. Firstly, this was a single-center study with a small sample size, which may reduce the generalizability of the findings to some extent. Secondly, this study lasted for about 2 months. It is not clear whether the inexperienced resident who received guidance will retain the skills in the long term. Thirdly, length measurements of the thyroid lobes were not obtainable. As reported in a previous study [22], the sagittal cine clips were split into upper and lower halves so that large measurement errors might have existed. Fourthly, in this study, the nodules detected by the on-site expert were used as the reference standard, without pathologic confirmation. This standard was used to calculate the sensitivity, specificity, accuracy, PPV, and NPV. However, as previously stated, the goal of this study was to demonstrate the value of applying synchronous TUS in thyroid US examinations and not for the diagnosis of specific diseases. Finally, the neck lymph nodes were not evaluated, even though such an evaluation is mandatory for a proper examination because the thyroid nodule depicted on US had a risk for malignancy. In the future, a multicenter trial with pathological confirmation will be carried out to provide more solid evidence to confirm these results. Neck lymph node evaluation using synchronous TUS will also be carried out to confirm the value of synchronous TUS.
In conclusion, the present prospective, parallel controlled trial affirms the value of applying synchronous TUS in thyroid US examination. This new method of examination enables inexperienced residents to achieve a similar diagnostic capability to that of an expert in thyroid US examination, and TUS is acceptable for patients where US experts are not accessible.
Notes
Author Contributions
Conceptualization: Li XL, Xu HX, Guo LH. Data acquisition: Wang Q, Chen ZT. Data analysis or interpretation: Sun YK, Qian ZB. Drafting of the manuscript: Li XL, Sun YK, Chen ZT, Qian ZB. Critical revision of the manuscript: Wang Q, Xu HX, Guo LH. Approval of the final version of the manuscript: all authors.
No potential conflict of interest relevant to this article was reported.
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China (grants 81901753, 81725008 and 81772849), Fundamental Research Funds for the Central Universities (grant 22120190213), Shanghai Hospital Development Center (grants SHDC22015005), Shanghai Municipal Health Commission (grant 2019LJ21 and SHSLCZDZK 03502), the Science and Technology Commission of Shanghai Municipality (grants 19DZ2251100 and 19441903200).
References
Article information Continued
Notes
Key point
Synchronous tele-ultrasonography (TUS) can help the naive operator to achieve the similar level of ultrasonography (US) expert in thyroid scanning. The naive operator and patients accepted the mode of synchronous TUS for thyroid US examination.