This was a bibliometric analysis based on literature databases.

First, the number of citable and non-citable articles by year was counted on the journal homepage of Ultrasonography. Citable articles included reviews, original articles, and case reports. Second, the countries of authors who published in Ultrasonography from 2014 to 2019 were counted on the journal homepage of Ultrasonography. Third, the number of total cites based on the Crossref metadata, Scopus, and Web of Science Core Collection by year was counted in those three databases. Fourth, the 2-year impact factor (i.e., cites per document over the course of 2 years) based on the Crossref metadata, Scopus, and Web of Science Core Collection according to year was manually calculated as follows.

The definition of the impact factor is as follows: the impact factor of the journal J in the year X=A/B, where A is the number of total citations in the year X received by all items published in the journal J in the years (X-1) and (X-2) and B is the total number of all citable items published in the journal J in the years (X-1) and (X-2).

Fifth, the Hirsch index was calculated based on the Crossref metadata, Scopus, and Web of Science Core Collection. The Hirsch index is defined as "the number of papers with citation number ≥h, and it has index h if h of its N_p papers have at least h citations each, and the other (N_p-h) papers have ≤h citations each" [6].

Sixth, the citing authors' countries and the source title of the articles that cited Ultrasonography were counted in the Web of Science Core Collection [cited 2020 Mar 16]. Seventh, the types and titles of sources cited by articles in Ultrasonography were counted in the Web of Science Core Collection.

Descriptive statistics were applied. A comparison with other related journals was described for the impact factor.

As shown in Fig. 1, the number of citable articles, including reviews, original articles, and case reports, was consistently 40 from 2014 to 2019.

The number of countries of authors from 2014 to 2015 was 4; that from 2016 to 2017 was 12; and that from 2018 to 2019 was 22 (Fig. 2, Supplementary Data 1). The proportion of Korean authors decreased to 45.7% in 2018-2019, although it was 90.1% in 2014-2015.

The number of total cites of Ultrasonography based on the Crossref metadata, Scopus, and Web of Science Core Collection are presented in Fig. 3 (Supplementary Data 2). It reached 632 in the Web of Science Core Collection, 595 in Scopus, and 552 in the Crossref metadata in 2019.

The 2-year impact factor and equivalent citation metrics are presented in Fig. 4. In the Web of Science Core Collection, the 2-year impact factor soared from 2.15 in 2016 to 3.20 in 2019. It also changed from 2.136 in 2016 to 3.075 in 2019 based on Scopus, while it decreased from 3.05 in 2016 to 2.75 in 2019 based on the Crossref metadata.

The Hirsch index was 20 based on Scopus (Table 1). The publication types of the 21 most frequently cited articles on Scopus were reviews (n=18), original articles (n=2), and perspective (n=1). The Hirsch index was also 20 based on the Web of Science Core Collection. The number of review articles out of the 20 most frequently cited articles in the Web of Science Core Collection was 15, and that of original articles was 5 (Table 2).

The number of countries of authors who cited Ultrasonography based on the Web of Science Core Collection was 76 (Supplementary Data 3). The top-ranking countries are presented in Fig. 5. Korea, the United States, and China were the top 3 countries. The number of source titles of citing articles was 668 (Supplementary Data 4). The top 9 most frequently citing journals are presented in Fig. 6.

The types of sources cited by articles in Ultrasonography from 2014 to 2019 were classified as follows: journals (n=7,087), books (n=166), conference proceedings (n=17), web sources (n=14), papers (n=5), and theses (n=2). The number of journals cited by articles in Ultrasonography was 1,246 (Supplementary Data 5). Of these citations, the most frequently listed journal titles are presented in Fig. 7.

The number of citable articles was consistently 40 per year from 2014 to 2015. The number of countries of the authors was 1 in 2014, but soared in 16 in 2019. The total cites reached more than 550 in 3 databases. The 2-year impact factor reached 3.20 in 2019. The Hirsch index was 20. The journal was cited by authors from 76 countries. The major reference types were journals (n=7,087) and books (n=166). The most frequently cited journals were Radiology, American Journal of Radiology, and Journal of Ultrasound Medicine.

The above results provide a vivid example of the degree to which a local journal could be promoted to a top-tier journal in a short period after changing its language to English. After changing its language policy to English-only, Ultrasonography became indexed in the three most essential literature databases (PubMed, Scopus, and SCIE). Furthermore, its citation frequency soared rapidly. The estimated 2019 Journal Citation Reports (JCR) impact factor of Ultrasonography is 3.2, which corresponds to a JCR ranking of 72.9%, near the first quartile. In the Radiology, Nuclear Medicine, Imaging category of the Scimago Journal & Country ranking (https://www.scimagojr.com/), its 2018 Scientific Journal Rankings (SJR) ranking was 76th out of 328 journals (76.8%), meaning that it was in the first quartile. What factors may explain this enormous achievement?

First, the society members' research activities and writing skills are excellent. The level of research conducted by the members of the Korean Society of Ultrasound in Medicine is internationally recognized. Korea is ranked third in the world in terms of the number of presentations at international meetings on ultrasonography. In the early stages of Ultrasonography as an English-language journal, most authors were from Korea; therefore, the citation frequency at the launching stage was dependent on the quality of articles published by Korean authors belonging to the Korean Society of Ultrasound in Medicine. If their competency had not been excellent, it would not have been possible for Ultrasonography to reach its present level.

Second, the two main reasons why this society journal was initially underappreciated were the local language of the main texts and the fact that the journal was not listed in literature databases. After overcoming those shortcomings, Ultrasonography could be widely read by international radiologists, physicians, and researchers. This phenomenon has consistently been observed for other medical journals in Korea. After adopting an English-only policy for publications and being indexed in PubMed, citation frequency began to increase rapidly [7].

Third, Ultrasonography accepted the following digital standards of journal publishing from the time of its launch: journal article tag suite (JATS) XML to be listed in PubMed Central [8]; PubReader and EPUB 3.0 format; digital object identifier (DOI) to provide a persistent direction to the landing page of the journal [9]; the cited-by function; Crossmark and Fundref [10]; the requirement for contributors to provide Open Researchers and Contributors ID (ORCID) profiles [11]; and QR codes [12].

Fourth, the editors and editorial board members have played a superb role in these developments. They introduced an international-level editorial policy, including careful review, and firmly adhere to international ethical guidelines, including the Principles of Transparency and Best Practice in Scholarly Publishing, 3rd version [13,14].

Fifth, the society has supported journal publishing and editorial work without any limitations. With the full financial support of the society, the members of the editorial team have been able to demonstrate their high level of competence.

The difference in total cites across the three databases originated from differences in the journals included in each database (Fig. 3). If all Scopus and Web of Science Core Collection journals deposited their reference metadata to Crossref, the frequency count based on the Crossref database would be expected to be the highest. However, many journals have not done so. The same issue arose in the estimation of the 2-year impact factor (Fig. 4). The impact factor equivalent based on the Crossref metadata was lower than the impact factors calculated based on Scopus and the Web of Science Core Collection because journals inconsistently deposited reference metadata in Crossref. Korea, the United States, and China have been found to be most common countries of citing authors in most medical journals in Korea (Fig. 5) [15]. This trend simply originates from the fact that the United States and China are the most productive countries in the literature databases at present.

The authors of Ultrasonography most frequently cited Radiology, American Journal of Roentgenology, Journal of Ultrasound Medicine, Ultrasound Medicine and Biology, and European Radiology (Fig. 7). Ultrasonography most frequently received citations from Ultrasound in Medicine and Biology, Journal of Ultrasound in Medicine, Korean Journal of Radiology, and European Radiology (Fig. 6). Of note, Radiology and American Journal of Roentgenology were cited by Ultrasonography frequently. However, the authors who published in those two journals did not cite Ultrasonography to a similar extent. These findings can be interpreted as indicating that the content of Ultrasonography was of interest for authors publishing in the above-listed most frequently citing journals.

The type of references should be added when producing JATS XML and Crossref XML. It was possible to trace reference types from those XML files. The authors of Ultrasonography cited journals most frequently (97.2%). In general, in Korean medical journals, citations of scholarly journals are most frequent than citations of other types of sources [16]. The findings for Ultrasonography are compatible with other journals regarding this phenomenon.

The present bibliometric results are comparable to those reported in earlier studies of other local journals. As noted in a previous study, "Out of 181 articles of Infection & Chemotherapy published between 2015 and 2018. There were 196 source journal titles that cited Infection & Chemotherapy in the 2015-2018 issues. The Hirsch index was 15" [17]. Furthermore, "The number of citable articles of Blood Research from 2013 to 2017 was 196. The number of citing journals in Web of Science Core Collection was 422." In 2016, the Hirsch index of Blood Research based on the Web of Science Core Collection was 10 [18]. In a third journal, it was noted that "From 2012 to 2017, there were 282 citable articles of Clinical Molecular Hepatology. The citing authors were from 85 countries, and their papers had been cited in 663 journals. The Hirsch index was 19" [19]. Thus, the bibliometric results of Ultrasonography are comparable to those of the above three journals.

Five journals with a scope including ultrasonography or ultrasound and a JCR ranking in the first and second quartiles were present in the 2018 JCR category of Radiology, Nuclear Medicine, and Medical Imaging (Table 3). Only two of those journals showed a higher impact factor than the estimated 2019 impact factor of Ultrasonography. Those five journals are published by commercial publishing companies located in Western Europe and North America, making Ultrasonography the only society-owned journal among the high-ranking journals in this category. Furthermore, there is still no author-side submission fee or article processing charge, although it is a gold open access journal. Therefore, Ultrasonography will be able to continue devoting itself to the wide propagation of high-quality information throughout the world without imposing any hindrances on authors.

There was no limitation in the bibliometric analysis of the journal because there was no loss of any relevant data.

What next steps or strategies should be taken for the journal to develop further into a top-tier journal in the ultrasonography field?

First, I suggest adopting a data-sharing policy for all types of articles other than clinical trials. Openly sharing data with other researchers has the following benefits: guaranteeing reproducibility, ensuring reliable scientific soundness, facilitating subsequent meta-analyses or mega-analyses, and resource-saving by avoiding the need to reproduce the same data [20]. In Korea, there are still only a few journals that have adopted a data-sharing policy. The results of a 2019 survey of 100 journals showed that 13 had adopted such a policy, with repository sites including Harvard Dataverse (2) and Mendeley Data (1) [21]. There is a Korean journal that mandates data sharing and peer-reviews data [22]. Adoption of a data-sharing policy will be beneficial for strengthening the transparency of the articles published in Ultrasonography.

Second, Ultrasonography should be added to MEDLINE. Although Ultrasonography has been searchable in PubMed since 2014 as a PMC journal, being indexed in MEDLINE would provide the following additional benefits: first, Medical Subject Heading terms would be added to the abstracts in PubMed, thereby enabling more specific search results; and second, it would enhance Ultrasonography's branding to have been acknowledged by the reviewers of the United States National Library of Medicine Literature Selection Technical Review Committee (LSTRC) [23]. During review by LSTRC, 3 points are usually checked: quality of the journal, importance to a variety of readers, and ethics policies/adherence.

Quality is checked in terms of the journal's "scientific merits (validity, the currency of information and references, originality, contribution to the field), editorial works (credibility of contents), and production quality (layout, printing, readability, usability, graphics; number and location of advertisements)" [23]. Scientific quality is a somewhat challenging criterion, although it can be determined by a specialist in the same field or a related field. Contribution to the field may be reflected by the citation frequency by other researchers. However, the editorial works and production quality of Ultrasonography are excellent in comparison to other MEDLINE-indexed journals. The main readers of the journal are radiologists in the field of ultrasonography in the clinical field. It also publishes some articles relevant for researchers on medical physics. Ultrasonography has already strictly adhered to research and publication ethics standards: to prevent plagiarism or duplicate publications, the editor screens all manuscripts using Crosscheck; patient anonymity is strictly kept; informed consent is obtained from all subjects; and conflicts of interest are declared in all articles [24,25]. There is a document indicating that the journal complies with the Principles of Transparency and Best Practice in Scholarly Publishing, 3rd version [26].

Nonetheless, being indexed in MEDLINE is no easy task. The recent passing rate of the MEDLINE review process is around 15%, and only a few medical journals in Korea have successfully been indexed in MEDLINE. However, it is time for the editors of Ultrasonography to do their best to be approved by the LSTRC.