Systematic Reviews
Copyright ©The Author(s) 2017. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Orthop. Jun 18, 2017; 8(6): 514-523
Published online Jun 18, 2017. doi: 10.5312/wjo.v8.i6.514
Worldwide orthopaedic research activity 2010-2014: Publication rates in the top 15 orthopaedic journals related to population size and gross domestic product
Erik Hohmann, Vaida Glatt, Kevin Tetsworth
Erik Hohmann, Medical School, University of Queensland, Herston 4006, Australia
Erik Hohmann, Medical School, Faculty of Health, University of Pretoria, Pretoria 0002, South Africa
Erik Hohmann, Department of Orthopedic Surgery and Sports Medicine, Valiant Clinic/Houston Methodist Group, Dubai 414296, United Arab Emirates
Vaida Glatt, University of Texas Health Science Center, San Antonio, TX 78229, United States
Kevin Tetsworth, Department of Orthopaedic Surgery, Royal Brisbane Hospital, Herston 4006, Australia
Kevin Tetsworth, Department of Surgery, School of Medicine, University of Queensland, Herston 4006, Australia
Author contributions: Hohmann E designed and performed the research; Hohmann E analyzed the data; Hohmann E, Glatt V and Tetsworth K wrote the paper; all authors read and approved the final manuscript.
Conflict-of-interest statement: All the authors declare that they have no competing interests.
Data sharing statement: The technical appendix, statistical code, and dataset are available from the corresponding author at ehohmann@hotmail.com.
Open-Access: This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Correspondence to: Erik Hohmann, FRCS, FRCS (Tr&Orth), MD, PhD, Department of Orthopedic Surgery and Sports Medicine, Valiant Clinic/Houston Methodist Group, Dubai 414296, United Arab Emirates. ehohmann@hotmail.com
Telephone: +971-4-3788818 Fax: +971-4-3788718
Received: October 25, 2016
Peer-review started: October 28, 2016
First decision: December 1, 2016
Revised: December 12, 2016
Accepted: March 23, 2017
Article in press: April 18, 2017
Published online: June 18, 2017

Abstract
AIM

To perform a bibliometric analysis of publications rates in orthopedics in the top 15 orthopaedic journals.

METHODS

Based on their 2015 impact factor, the fifteen highest ranked orthopaedic journals between January 2010 and December 2014 were used to establish the total number of publications; cumulative impact factor points (IF) per country were determined, and normalized to population size, GDP, and GDP/capita, comparison to the median country output and the global leader.

RESULTS

Twenty-three thousand and twenty-one orthopaedic articles were published, with 66 countries publishing. The United States had 8149 publications, followed by the United Kingdom (1644) and Japan (1467). The highest IF was achieved by the United States (24744), United Kingdom (4776), and Japan (4053). Normalized by population size Switzerland lead. Normalized by GDP, Croatia was the top achiever. Adjusting GDP/capita, for publications and IF, China, India, and the United States were the leaders. Adjusting for population size and GDP, 28 countries achieved numbers of publications to be considered at least equivalent with the median academic output. Adjusting GDP/capita only China and India reached the number of publications to be considered equivalent to the current global leader, the United States.

CONCLUSION

Five countries were responsible for 60% of the orthopaedic research output over this 5-year period. After correcting for GDP/capita, only 28 of 66 countries achieved a publication rate equivalent to the median country. The United States, United Kingdom, South Korea, Japan, and Germany were the top five countries for both publication totals and cumulative impact factor points.

Key Words: Bibliometrics, Orthopedic surgery, Impact factor, Publication productivity

Core tip: The total number of publications by a country is one of the best indicators of research output and productivity, and is an important aspect of clinical excellence. Our results demonstrate that the United States collectively published more articles and accumulated the highest number of impact factors during the study period, and confirms its overwhelming dominance of publications in the fifteen highest ranked journals in orthopaedics. However, after adjusting for population size, Switzerland was the most academically productive nation. Similarly, after adjusting the number of publications with respect to GDP, Croatia was the most productive, and “cost effective” country.



INTRODUCTION

The total number of publications by a country is one of the best indicators of research output and productivity[1], and is an important aspect of clinical excellence[2,3]. Prior bibliographic analyses of orthopaedic academic output have concentrated on the total number of publications per country over various periods ranging from five to ten years[4-6]. The United States, United Kingdom, Germany, Japan, and South Korea have all consistently ranked among the five most productive countries.

The availability of funding has been shown to result in higher publication output, favoring those countries with a larger population size and more powerful economies[6,7]. However, no prior bibliographic analysis of orthopaedic research and publications has accounted for population size or economic discrepancies. To adjust for these inconsistencies, the use of the gross domestic product (GDP) and gross domestic product per capita (GDP/capita) may provide a more meaningful result, and allow for a better comparison between countries[8]. Although the number of publications per capita is one simple way to minimize this inherent bias, it is not the only approach that can be used to determine how academically productive various nations have been. The reciprocal, population size per publication for example, is an equally valid metric that perhaps better expresses this relationship. This reciprocal approach has been employed instead in various iterations throughout this study, to more directly investigate how academically active each nation has been in the field of orthopaedics over the past five years.

Using the fifteen highest rated orthopaedic journals over a five year period, based on the 2015 impact factor, the purpose of this study was threefold: First, to investigate the number of publications and total impact factor from each country, and to then relate these variables to population size, GDP, and GDP per capita. Second, to determine the minimum number of publications required to be comparable to the country producing the median number of publications, when normalized for GDP per capita. Finally, to establish the number of publications that would be required from each country to be equivalent to the country having the highest research output, when normalized for GDP per capita.

MATERIALS AND METHODS

The 2015 Journal Citation report was accessed on the Web of Science (Thomson Reuters, New York, United States)[9], and the fifteen highest ranked journals based on their 2015 impact factor were selected from the category “orthopedics”. Journals were excluded from this list if they were not directly related to the field of orthopedics, or if their main purpose was to provide narrative review articles (Table 1). The abstracts of all articles published in these 15 journals between January 2010 and December 2014 were screened via the journals’ websites. Letters to the editor, editorials, editorial comments, historical articles, errata, proceeding papers, meeting abstracts, and notes were excluded. Only research articles (levels 1-4), systematic reviews, meta-analyses, non-solicited review articles, and case reports were included. The level of evidence was recorded for each published article; if the journal did not assign the level of evidence, the levels of evidence chart published by the Journal of Bone and Joint Surgery was used[10]. Each publication was assigned a country of origin defined by the location of the the authors’ principal institution, or defined by the country of origin of the corresponding author if the manuscript did not provide details about study location. Any discrepancies were resolved by agreement between the two senior authors. The total number of publications and the total number of impact factor points per country were collated.

Table 1 Impact factors (2015 Journal Citation Reports - Thomson Reuters) and number of included publications from 2010-2014.
JournalImpact pointsPublications 2010-2014
1Journal of Bone and Joint - American Volume5.2801833
2American Journal of Sports Medicine4.3621561
3The Bone and Joint Journal3.3091379
4Arthroscopy – The Journal of Arthroscopic and Related Surgery3.2061072
5Knee Surgery Sports Traumatology Arthroscopy3.0531747
6Journal of Orthopaedic Research2.9861301
7Acta Orthopaedica2.771565
8Clinical Orthopaedics and Related Research2.7652027
9Journal of Arthroplasty2.6661873
10Spine Journal2.4261029
11Spine2.2972848
12Journal of Shoulder and Elbow Surgery2.2891324
13Injury – International Journal of the Care of the Injured2.1371133
14International Orthopaedics2.1101477
15European Spine Journal2.0661852
Total number of publications23021

GDP and GDP per capita were sourced from the World Bank website[11], and population size was extracted from the CIA World Factbook[12]. To describe the relationship between population size and the number of publications from a given nation, the population size of that country was divided by their total number of publications. The resulting value describes the population size per publication (PSPP) for that nation; in other words, the calculated value defines the population size per published article, allowing for a better and more direct comparison accounting for population size. Likewise, to define the population size per impact factor point (PSIP) from a given nation the population of that country was divided by their total impact factor points.

Extending this analysis, the gross domestic product was also divided by the total number of publications and impact factor points. These values provide an overview of the gross cost associated with producing a manuscript (GDPP), as well as the gross cost associated with producing one impact factor point (GDPI) for each country. Finally, to simultaneously adjust for population size and economic strength, the GDP per capita was divided by either the total number of publications or by cumulative impact factor points. These values then provide information regarding the gross cost per capita associated with producing a manuscript (GDPCP), or the gross cost per capita associated with producing one impact factor point (GDPCI) for each country.

The list for GDPCP was next ranked lowest to highest to identify the median country. This median country then served as the benchmark, and a correction coefficient was calculated that was normalized to this median country. In this way the number of publications of the median country could then be used to calculate the number of publications every country would need to produce to be considered equivalent to that median country. Dividing the GDPCP of each country by this normalizing coefficient, (NCmed) determined the number of publications that would be necessary for each country to produce to be considered equivalent to the median country. This provides an excellent measure, corrected for economic power (GDP/capita) and population size, of the expected academic output of different countries, normalized to the output of the median nation.

Finally, a very similar process was followed where a correction coefficient was determined that was instead normalized to the publication output of the current global leader in orthopaedic research. The most active country then served as the benchmark, and a coefficient was calculated that was normalized to the academic activity of that country (NCtop). This value was then used to calculate the number of publications every country would need to produce to be considered equivalent to the global leader. Dividing the GDPCP of each country by this NCtop thus determines the number of publications that would be necessary for each country to produce to be considered equivalent to the global leader. This provides an excellent measure, corrected for economic power (GDP/capita) and population size, of the expected academic output of different countries, normalized to the output of the leading nation.

RESULTS

A total of 23021 orthopaedic articles were published in the 15 highest ranked orthopaedic surgery journals during the study period, between January 2010 and December 2014 (Table 1). Table 2 demonstrates the top ten countries for each of the fifteen journals, in terms of number of publications. The United States was consistently the leading country in ten of the fifteen journals, and was also the most productive country with a total of 8149 publications; they were followed by the United Kingdom and Japan, having 1644 and 1467 publications, respectively. A total of 66 countries had published at least one article (Table 3) during the study period. Similar to the number of publications, the United States also accumulated the largest number of impact factor points (24744) followed by the United Kingdom (4776) and Japan (4053) (Table 3). Overall, the top five countries were the United States, United Kingdom, Japan, South Korea, and Germany, and these countries were together responsible for 60.4% of all publications, and 61.4% of all impact factor points.

Table 2 Top 10 Number of publications per country for each of the 15 selected journals.
Journal12345678910
JBJS-AmUSA–1124CAN-107KOR-84UK-75JAP-52HOL-46GER-45FRA-39SWIS-37AUS-27
Am J Sports MedUSA-819KOR-117JAP-84GER-82UK-49AUS-40ITA-25CAN-34SWE-32SWIS-31
BJJUK-545USA-115KOR-76JAP-75HOL-50CAN-46AUS-43GER-41CHINA-35SWIS-31
ArthroscopyUSA-513KOR-105JAP-63GER-55CHINA-40CAN-34ITA-27UK-22FRA-18SPAIN-18
KSSTAUSA-242GER-195KOR-157ITA-149JAP-144UK-85HOL-76TURK-70SWE-64CHINA-62
J Orthopaedic ResearchUSA-535JAP-107GER-96CAN-69CHINA-67UK-48TAIW-45AUS-37KOR-31HOL-31
Acta OrthopaedicaSWE-125DEN-76NOR-69HOL-59FIN-40GER-34UK-34USA-21JAP-17AUS-13
CORRUSA-1155CAN-110KOR-98JAP-71UK-60SWIS-60GER-59FRA-49ITA-45HOL-33
J ArthroplastyUSA-934JAP-136CAN-124UK-117KOR-114AUS-72CHINA-64GER-37SPAIN-29HOL-26
Spine JournalUSA-491KOR-78CHINA-62JAP-56CAN-48HOL-29UK-24SWIS-23INDIA-21ITA-21
SpineUSA-1168JAP-307CHINA-255CAN-166KOR-163UK-73GER-65AUS-59HOL-57TAIW-49
J Shoulder Elbow SurgUSA-659JAP-79UK-72KOR-65CAN-60SWIS-49FRA-42GER-36ITA-35BELG-34
InjuryUK-215USA-126GER-114ITA-89CHINA-78HOL-57GREEC-48SPAIN-37SWIS-34AUS-34
International OrthopaedicsGER-232CHINA-198UK-101USA-97FRA-97JAP-81ITA-76A-76CRO-54SWIS-49
European Spine JournalCHINA-251JAP-182GER-161USA-150ITA-133UK-124FRA-104KOR-90SWIS-84HOL-81
Table 3 Highest number of publications and impact points for each country.
RankCountryPublicationsRankCountryImpact points
1United States81491United States24744
2United Kingdom16442United Kingdom4776
3Japan14673Japan4053
4South Korea13544South Korea3765
5Germany12725Germany3491
6China12226China3034
7Canada9307Canada2774
8Italy7378Holland2155
9Holland6639Italy1982
10France54810Switzerland1507
11Switzerland52711Australia1412
12Australia48512France1382
13Sweden40313Sweden1187
14Spain31114Spain833
15Austria29515Austria801
16Taiwan26416Norway755
17Denmark25417Taiwan729
18India24618Denmark710
19Norway24019India646
20Turkey23520Turkey630
21Belgium21921Belgium614
22Greece18222Greece508
23Finland16723Brazil408
24Brazil14724Finland402
25Hong Kong13025Hong Kong371
26Israel11926Israel315
27Ireland9827Singapore295
28Singapore8428Ireland262
29New Zealand7829New Zealand227
30Croatia7430Iran174
31Egypt6831Egypt168
32Iran6532Croatia159
33Poland6133Poland141
34Thailand5234Thailand128
35Czech Republic39Slovenia128
36Slovenia3235Czech Republic84
37Hungary2936Hungary71
38Portugal2537Portugal71
39Chile2438Chile66
40Malaysia2339Malaysia63
41South Africa2140South Africa59
42Argentina2041Argentina55
43Serbia1942Serbia43
44Luxemburg1443Luxemburg43
45Saudi Arabia1244Saudi Arabia29
46Mexico1045Mexico26
47Lebanon946Lebanon23
Lithuania9Lithuania23
Russia947Russia21
48Estonia748Estonia17
48Nigeria749Nigeria15
49Pakistan650Romania13
Romania6Philippines13
50Columbia551Pakistan12
Kuwait552Columbia11
Philippines5Tunisia11
Tunisia553Kuwait9
51Bulgaria354Iceland7
Iceland355Bulgaria6
Iraq3Iraq6
52Malawi256Malawi5
Morocco2Nepal5
Nepal2Uganda5
53Ethiopia157Morocco4
Sudan158Ethiopia3
Uganda1Sudan3

However, when adjusted for population size (PSPP), Switzerland was the leading country with one publication per 15300 people, followed by Norway with one publication per 21100, and Denmark with one publication per 22300. Switzerland was also the leader in the category of impact factor (PSPI), accumulating one impact factor point per 5400 people, followed by Norway with one impact factor point per 6700, and Holland with one impact factor point per 7800 (Table 4).

Table 4 Number of publications (PSPP) and impact (PSPI) normalized for population size (publication/impact point per in thousand populations).
RankCountryPSPPRankCountryPSIP
1Switzerland15.31Switzerland5.4
2Norway21.12Norway6.7
3Denmark22.33Holland7.8
4Sweden24.14Denmark7.9
5Holland25.45Sweden8.2
6Austria28.76Austria10.6
7Finland32.37Canada12.1
8Canada35.98Luxemburg12.6
9Luxemburg38.99United States12.9
10South Korea38.910United Kingdom13.4
11United Kingdom38.911Finland13.4
12United States39.312South Korea13.6
13Australia44.313Australia15.2
14Belgium51.114Belgium18.2
15Hong Kong55.315Singapore18.3
16New Zealand57.316Hong Kong19.4
17Croatia57.817New Zealand19.7
18Greece60.418Greece21.6
19Germany63.119Germany23
20Singapore64.320Slovenia24
21Slovenia64.321Ireland24.3
22Ireland65.122Israel25.6
23Israel67.723Croatia27
24Italy82.424Italy30.7
25Japan86.825Japan31.4
26Taiwan88.426Taiwan32
27Iceland107.727Iceland46.1
28France121.528France48.1
29Spain151.929Spain56.7
30Estonia185.730Estonia76.5
31Czech Republic269.231Turkey121.7
32Turkey326.232Czech Republic125
33Lithuania333.333Lithuania130.4
34Hungary341.434Hungary139.4
35Serbia379.535Portugal147.3
36Portugal418.436Serbia167.7
37Lebanon551.837Lebanon215.9
38Poland631.638Chile247.6
39Kuwait673.839Poland272
40Chile680.840Kuwait374.3
41China1110.541Iran443.5
42Egypt1176.542China447.3
43Iran1187.343Malaysia471.7
44Thailand1283.144Egypt476.2
45Malaysia1292.145Brazil491.2
46Brazil1363.246Thailand521.2
47Argentina2072.547Argentina753.6
48Tunisia2178.048South Africa915.2
49Saudi Arabia2402.549Tunisia990
50Bulgaria2421.750Saudi Arabia12108.3
51South Africa2571.451Bulgaria15353.8
52Romania3326.752Romania15353.9
53India5089.453India19380.8
54Malawi8180.054Malawi32720
55Ethiopia9410.055Columbia43649
56Columbia9602.856Mexico45536.5
57Iraq1114057Nepal55600
58Mexico11839.558Iraq55700
59Nepal1390059Russia68333.3
60Russia15944.460Uganda75160.0
61Morocco1650561Philippines75684.6
62Philippines1967862Morocco82525.0
63Nigeria2480063Nigeria115733.3
64Pakistan32695.764Sudan126533.3
65Uganda3758065Pakistan163478.3
66Sudan3797666Ethiopia313666.7

The number of publications, when normalized with respect to economic activity (GDPP), was highest for Croatia, with one publication per $772000, followed by Korea with $1042000, and Greece with $1294000. For impact factor (GDPI) Croatia was again the leader, and produced one impact factor point per $359000, followed by South Korea with $375000, and Holland with $408000 (Table 5). When adjusting for both GDP and population simultaneously (GDPCP) China was the leader, producing one publication per $6200, followed by India with $6400, and the USA with $6700. The United States was the leader in the impact factor category (GDPCI), producing one impact factor point per $2200, followed by India with $2400 and China $2500 (Table 6). However, these results need to be interpreted carefully, and it is probable that the extremely large population size of both China and India resulted in data distortion.

Table 5 Number of publications (GDPP) and impact points (GDPI) related to GDP ( in thousand dollars).
RankCountryGDPPRankCountryGDPI
1Croatia7721Croatia359
2South Korea10422South Korea375
3Greece12943Holland408
4Holland13264Greece464
5Switzerland13305Switzerland465
6Denmark13486Sweden481
7Sweden14177Denmark482
8Slovenia14178Slovenia576
9Austria15479Austria579
10Finland163010United Kingdom626
11United Kingdom181811Canada644
12Taiwan185212Norway662
13Canada192013Taiwan671
14Norway208314Finland677
15Malawi212915United States704
16United States213816Hong Kong784
17Hong Kong223717New Zealand829
18Serbia230918Malawi852
19New Zealand241219Belgium866
20Belgium242720Israel970
21Ireland255921Ireland975
22Israel256922Serbia1020
23Italy290523Australia1032
24Australia300324Singapore1044
25Germany304125Italy1080
26Japan313726Germany1108
27Turkey339827Japan1135
28Singapore366528Turkey1267
29Estonia378429Luxemburg1509
30Egypt421330Estonia1558
31Spain444231Spain1658
32Hungary447132Egypt1706
33Luxemburg463433Hungary1949
34Lebanon508134Lebanon1988
35France516335France2047
36Czech Republic526336Lithuania2102
37Lithuania537237Iceland2434
38Iceland567938Czech Republic2444
39Iran6543Iran2444
40Thailand778539Thailand3163
41India832740India3171
42China847441Portugal3241
43Poland893342China3413
44Portugal920443Poland3865
45Tunisia972244Chile3910
46Nepal988445Nepal3954
47Chile1075246Tunisia4419
48Malaysia1470047Malaysia5367
49Brazil1596048Uganda5400
50South Africa1667149Brazil5750
51Bulgaria1890650South Africa5934
52Argentina2683351Bulgaria9452
53Uganda2699852Argentina9757
54Kuwait3272253Romania15311
55Romania3317454Kuwait18179
56Pakistan4060555Ethiopia18540
57Morocco5500456Pakistan20303
58Ethiopia5562157Philippines21906
59Philippines5695558Sudan24734
60Saudi Arabia6218759Saudi Arabia25733
61Sudan7420260Morocco27502
62Iraq7450361Columbia34340
63Columbia7544862Iraq37251
64Nigeria8121563Nigeria37901
65Mexico12946964Mexico49796
66Russia20673365Russia88600
Table 6 Number of publications (GDPCP) and impact points (GDPCI) related to GDP per capita (in thousand dollars).
RankCountryGDPPRankCountryGDPI
1China6.21United States2.2
2India6.42India2.4
3United States6.73China2.5
4South Korea20.74South Korea7.4
5Japan24.75Japan8.9
6United Kingdom28.26United Kingdom9.7
7Germany37.67Germany13.7
8Turkey44.78Turkey16.7
9Egypt479Italy17.6
10Italy47.410Canada18.1
11Canada5411Egypt19
12Brazil77.412Holland24.2
13France7813Brazil27.9
14Holland78.714France30.9
15Iran83.715Iran31.3
16Spain95.416Spain35.6
17Thailand114.917Greece42.3
18Greece118.118Taiwan43.8
19Taiwan120.819Australia43.9
20Malawi127.520Thailand46.7
21Australia127.721Sweden49.6
22Sweden146.222Malawi51
23Switzerland162.423Switzerland56.8
24Austria173.524Austria63.9
25Croatia182.125Belgium77.1
26Belgium216.226Croatia84.7
27Pakistan219.527Denmark85.5
28Poland235.128Poland101.7
29Denmark23929Hong Kong108.3
30Finland298.330Pakistan109.7
31South Africa308.731South Africa109.9
32Hong Kong30932Israel118.1
33Israel312.733Finland123.9
34Serbia323.834Norway128.9
35Nepal35135Nepal140.4
36Norway405.436Uganda
37Nigeria457.637Serbia143.1
38Hungary483.738New Zealand166.9
39New Zealand485.839Malaysia179.5
40Malaysia491.640Singapore190.8
41Czech Republic500.841Ethiopia191.3
42Ireland554.842Hungary197.6
43Ethiopia57443Ireland207.5
44Philippines574.444Nigeria213.5
45Chile605.345Chile220.1
46Argentina625.446Philippines220.9
47Singapore67047Argentina227.4
48Uganda71548Czech Republic232.5
49Slovenia75049Slovenia279.1
50Tunisia884.250Portugal311.7
51Portugal885.351Sudan371.7
52Mexico1032.652Mexico397.1
53Sudan111553Tunisia401.9
54Lebanon1117.654Lebanon437.3
55Russia1415.155Russia606.5
56Columbia1580.856Lithuania717.8
57Morocco159557Columbia718.5
58Romania1666.258Romania769
59Lithuania1834.159Morocco797.5
60Saudi Arabia2013.460Saudi Arabia833.1
61Iraq214061Iraq1070
62Bulgaria261762Estonia1186
63Estonia2880.363Bulgaria1308.5
64Luxemburg8333.164Luxemburg2713.3
65Kuwait8718.865Kuwait4843.8
66Iceland17334.566Iceland7429.1

When ranked with respect to GDPCP Poland was the median country, publishing 61 articles, and served as the median academic output benchmark. The results showed that 28 countries were able to achieve this academic output (Table 7). As an example, for the United States to achieve this benchmark a minimum of 235 publications were required; however, a total of 8149 publications were recorded, which was 3,468% greater than the requisite number. For Norway, to achieve this benchmark a minimum of 414 publications were required, but only 240 publications were recorded; this was only 58% of the number of publications necessary to have achieved an academic output equivalent to the median activity (Table 7).

Table 7 Number of publications required to equivalent with the median (Poland n = 61) using the benchmark measure.
RankCountryPublished publica-tions 2010-2014Papers to be published% of published papers
1China1222323783
2India24673656
3United States81492353505
4South Korea13541191137
5Japan1467235952
6United Kingdom1644197833
7Germany1272203625
8Turkey23545525
9Egypt6814499
10Italy737148496
11Canada930214435
12Brazil14748303
13France548182301
14Holland663222298
15Iran6523280
16Spain311126246
17Thailand5225204
18Greece18291198
19Taiwan264136194
20Malawi21184
21Australia485263183
22Sweden403251160
23Switzerland527364145
24Austria295218135
25Croatia7457129
26Belgium219201109
27Pakistan66100
28Poland6161100
29Denmark25425898
30Finland16721279
31South Africa212876
32Hong Kong13017176
33Israel11915875
34Serbia192672
35Nepal2367
36Norway24041458
37Nigeria71450
38Hungary296049
39New Zealand7816148
40Malaysia234847
41Czech Republic398347
42Ireland9823142
43Ethiopia1250
44Philippines51241
45Chile246239
46Argentina205338
47Singapore8423935
48Uganda1333
49Slovenia3210231
50Tunisia51926
51Portugal259426
52Mexico104423
53Sudan1520
54Lebanon94321
55Russia95417
56Columbia53415
57Morocco21415
58Romania64214
59Lithuania97013
60Saudi Arabia1210312
61Iraq32711
62Bulgaria3339
63Estonia7868.1
64Luxemburg144962.8
65Kuwait51852.7
66Iceland32211.4

The United States was the leader when ranked with respect to GDPCP, publishing 8,149 articles, and served as the leading academic output nation. Using the NCtop to calculate the required number of publications to be equivalent with the global research leader (United States), only two other countries, China and India, were considered equivalent or superior (Table 8). For example, for Korea 4174 publications would have been needed to have an academic output equivalent to that of the United States, but only 1354 articles (32%) were published. Again, these results need to be interpreted carefully, and it is highly probable that the large population size of both China and India resulted in data distorsion.

Table 8 Number of publications required to equivalent with the leader (United States) the benchmark measure.
RankCountryPublished publica-tions 2010-2014Papers to be published% of published papers
1China12221132108
2India246236104
3United States81498149100
4South Korea1354417432
5Japan1467540227
6United Kingdom1644691524
7Germany1272713818
8Turkey235156915
9Egypt6847714
10Italy737521014
11Canada930749812
12Brazil14716998.6
13France54863788.6
14Holland66377878.5
15Iran658128
16Spain31144297
17Thailand528925.8
18Greece18232085.6
19Taiwan2648925.5
20Malawi2385.2
21Australia48592435.1
22Sweden40387974.6
23Switzerland527127754.1
24Austria29576403.9
25Croatia7420113.7
26Belgium21970683.1
27Pakistan61973
28Poland6121412.8
29Denmark25490912.7
30Finland16774362.2
31South Africa219682.1
Hong Kong13059952.1
Israel11955532.1
Serbia199182.1
32Nepal21051.9
33Norway240145231.6
34Nigeria74871.5
35Hungary2920941.4
New Zealand7856561.4
Malaysia2316881.4
36Czech Republic3929151.3
Ireland9881151.2
Ethiopia1861.2
Philippines54291.2
37Chile2421681.1
Argentina2018671.1
38Singapore8484011
39Uganda11070.94
40Slovenia3235820.89
41Tunisia56600.76
42Portugal2533030.75
43Mexico1015410.65
44Sudan11660.6
Lebanon915020.6
45Russia919010.47
46Columbia511800.42
47Morocco24760.42
48Romania614920.4
49Lithuania924640.36
50Saudi Arabia1236060.33
51Iraq39580.31
52Bulgaria311720.26
53Estonia730090.23
54Luxemburg14174120.08
Kuwait565070.08
55Iceland377620.04
DISCUSSION

These results demonstrate that the United States collectively published more articles and accumulated the highest number of impact factor points during the study period from 2010 through 2014, and confirms its overwhelming dominance of publications in the fifteen highest ranked journals in the field of orthopaedics. However, after adjusting for population size, Switzerland was the most academically productive nation. Similarly, after adjusting the number of publications with respect to GDP, Croatia was the most productive, and “cost effective” country.

Over the last 30 years, English has become the international language of medical science[13]. Of the current top 50 highest impact journals in orthopaedics, 45 are based in English speaking countries; all 50 of these journals publish their manuscripts in English only[9]. The majority of those countries where English is the primary language also enjoy a high standard of living, and would appear to have advantages in terms of research funding and academic opportunity. Although this suggests an inherent bias towards authors from those countries where English is the principal language, over this 5-year period articles were published by a total of 66 different countries; in many of those countries English is not the main language. Strategies were employed here to attempt to eliminate or minimize any of these potential socio-economic advantages, and therefore obtain a better measure of the relative academic activity and orthopedic research output from various nations around the world. This study has revealed superior academic activity outcomes has been achieved by several of these countries, when adjusted for population size and GDP.

Both GDP and GDP per capita are indicators of economic strength, representing the value of all goods and services produced over a specified time period[7]. The cost of producing a research paper per GDP/capita is theoretically a better indicator of a country’s research productivity, one that takes into consideration some of the socio-economic conditions that might favor more populous or prosperous nations. After adjusting for GPD per capita both India and China were the leading countries, but due to their inordinately large population size the calculated figures are most likely biased. After eliminating these two countries, the United States, South Korea, Japan, Germany, and the United Kingdom ranked among the top five countries with the highest number of both publications and impact factor points. One possible explanation could be that the research output of these countries is directly related to economic vitality, although none of these five leading countries had the highest GDP per capita. For example, the United States, ranked 8th, Germany 15th, the United Kingdom 17th, Japan 23rd and South Korea 27th. Earlier research by Meo et al[7] and Halpenny et al[8] also failed to demonstrate a correlation between GDP per capita, total number of publications, and h-index in different science fields and social science disciplines. However, they were able to confirm a strong and positive correlation between the number of publications and the percentage of GDP spent on research.

This study introduced a new metric to bibliographic analysis, normalizing the collective publications and impact factor points of individual nations to that of the output of the median nation, after first correcting for both population size and economic activity. Although this measure has not been validated yet and may lack the robustness of standard citation and content analysis, it is nevertheless similar to other accepted bibliometric measures. In our opinion it facilitates a better comparison between countries, by defining the number of publications that would be necessary for a particular country to produce to have an output equivalent to that of the median nation.

After normalizing research output, 28 countries exceeded this benchmark, whereas 38 were below the level of the median nation. These findings unequivocally demonstrated the dominance of the United States compared to all other countries. To have an output equivalent to the median nation, Poland, it was necessary for the United States to publish 235 articles: However, they collectively published 8149 and were the global leader by an overwhelming margin. China and India were ranked even higher by this metric, but this might demonstrate an inherent limitation of this methodology related to population size. Those countries with a very low GDP per capita, a large population size, and a relatively large number of publications will most likely result in a ceiling effect, and normalizing research output to that of the median nation would thus be unreliable. Therefore, further research is required to better define the extent of this problem and to validate this approach.

Research output is an important determinant of economic growth, and an increase in service delivery, education, and innovation is often an indicator of a society’s shift from a producing economy to a knowledge-based economy[14]. In fact, publications of scientific literature can indicate a nation’s growth and progress in science and technology[5]. Moir et al[15] observed a 21% increase in orthopaedic publications from 1980 to 1994 in six selected journals. More recently, Bosker and Verheyen[4] also reported an increased number of orthopaedic publications in the 15 major clinical orthopaedic journals from 2000-2004, with a total of 13311 articles. The present bibliometric analysis counted over 23000 articles, representing a 73% increase over a 10 years interval. Several authors have previously performed subspecialty analyses[1,16]. Luo et al[1] showed that high income countries published 90% of all articles in foot and ankle research, with the United States publishing the highest number; however, Switzerland took the lead when it was normalized to population size and GDP. Liang et al[16] reported that the United States published the largest number of publications in the subspecialty of arthroscopy, but when adjusted for population size Switzerland was again the country with the highest number of publications. Similar findings were reflected in our results, although in their study Korea ranked first when academic output was adjusted for GDP.

Bibliometric analysis has also been performed by other disciplines. In emergency medicine, the United States was the most productive country followed by the United Kingdom and Australia. When normalized to population size, Australia had the highest number of articles per million persons, but Germany had the highest mean impact factor and citations[17,18]. In the specialty of critical care medicine, the United States has published the most articles, followed by the United Kingdom, Germany, France, and Australia. The United States also had the highest number of randomized controlled trial publications, the highest total impact factor points, and the highest total citations[17,18]. Halpenny et al[8] performed a bibliographic analysis in radiology. In their study, the United States published 42% of the 10,925 papers, followed by Germany and Japan. When corrected for GDP, Switzerland (0.925), Austria (0.694), and Belgium (0.648) produced the most publications per billion of GDP. Robert et al[19] evaluated the pain medicine literature over a period of 30 years and reported that the United States, the United Kingdom and Germany were the highest ranking countries. The pattern of publication rates are comparable to orthopaedics and these findings can possibly be generalized to other disciplines of medicine.

This study has recognized limitations. While the total number of articles and cumulative impact factor points was determined for each nation, the value of individual articles was not assessed; it is possible that there was a significant discrepancy in the manuscript quality between countries, potentially introducing selection bias. Even the selection of impact factor as an outcome measure to evaluate publication quality has been criticized, as it is determined by technicalities that are not related to the scientific value of the research studies themselves[20,21]. Citation analysis was also not performed, and it is acknowledged that the number of citations are a proxy measure of influence reflecting the recognition and quality of the published research by its peers[22]. However, using the impact factor reflects citation counts indirectly, as article citation rates ultimately determine the journal’s impact factor[20]. Nevertheless, overcitation, biased citing, audience size, biased data, and ignorance of the literature are additional common criticisms of bibliometric studies[23]. Another potential limitation of this method is that the research output of the median nation was based on data collected over a specific five-year period from the fifteen currently highest ranked orthopaedic journals. These results will almost certainly change if more journals are included, or the time interval is either extended or shortened.

In conclusion, the results of this study demonstrate that five countries were responsible for 60% of the research output in orthopaedic surgery over a 5-year period, when restricted to the 15 highest ranked journals specific to the field. Only 28 of 66 countries were able to achieve a publication rate equivalent to that of the median nation, after first correcting for GDP per capita. The United States was unequivocally the global leader when judged by this measure, and exceeded the median production by more than 34 times. Although China and India ranked the highest after correcting for both GDP and population size, this probably reflects the inordinately large populations of both countries. The United States, United Kingdom, South Korea, Japan, and Germany placed in the top five countries with respect to both publication totals and cumulative impact factor points.

COMMENTS
Background

Bibliographic analysis of academic output has been performed for many indications and can be an indicator for academic excellence. However most studies have focussed on the total number of publications without accounting for gross domestic product or economic discrepancies between countries. The primary aim of this study was therefore to investigate the number of publications and total impact factor from each country, and to then relate these variables to population size, gross domestic product (GDP), and GDP per capita. Secondly they determined the minimum number of publications required to be comparable to the country producing the median number of publications, when normalized for GDP per capita. The final aim was to establish the number of publications that would be required from each country to be equivalent to the country having the highest research output, when normalized for GDP per capita.

Research frontiers

Over the last 30 years English has become the international language of medical science. In Orthopedics 45 of the 50 highest impact orthopaedic journals are based in English countries. Based on these facts the majority of publications in these journals should come from primary English speaking countries.

Innovations and breakthroughs

Based on the total number of publications and impact points the United States was the undebated leader for both the total number of publications and impact points. However when adjusting for publication size and GDP per capita, it was Switzerland respectively Croatia which were the most productive nations. When using a newly introduced benchmark to adjust for both population size and GDP, 28 countries exceeded and 38 nations were below the median nation.

Applications

This review suggests that the total number of publications and impact points are not representative of true research output and other factors should be included into bibliometric analysis.

Terminology

Bibliometric analysis is based on quantitative variables such as number of publications, impact points and citation rates. Analysis can be performed at the macro-level comparing countries performances, at the middle level analyzing Universities or other institutional output or at the microlevel investigating research output of departments or individuals.

Peer-review

The authors present a very interesting paper on the worldwide orthopaedic research activity. They relate the scientific production with the GDP, and per capita GDP. This sort of information, although known for general science, was unknown in the orthopaedic field. The relevance of this paper is not only related to science but also to politics.

Footnotes

Manuscript source: Unsolicited manuscript

Specialty type: Orthopedics

Country of origin: Australia

Peer-review report classification

Grade A (Excellent): A, A

Grade B (Very good): B

Grade C (Good): 0

Grade D (Fair): 0

Grade E (Poor): 0

P- Reviewer: Drobetz H, Guerado E, Vaishya R S- Editor: Gong ZM L- Editor: A E- Editor: Lu YJ

References
1.  Luo X, Liang Z, Gong F, Bao H, Huang L, Jia Z. Worldwide productivity in the field of foot and ankle research from 2009-2013: a bibliometric analysis of highly cited journals. J Foot Ankle Res. 2015;8:12.  [PubMed]  [DOI]
2.  Carey RM, Wheby MS, Reynolds RE. Evaluating faculty clinical excellence in the academic health sciences center. Acad Med. 1993;68:813-817.  [PubMed]  [DOI]
3.  Christmas C, Kravet SJ, Durso SC, Wright SM. Clinical excellence in academia: perspectives from masterful academic clinicians. Mayo Clin Proc. 2008;83:989-994.  [PubMed]  [DOI]
4.  Bosker BH, Verheyen CC. The international rank order of publications in major clinical orthopaedic journals from 2000 to 2004. J Bone Joint Surg Br. 2006;88:156-158.  [PubMed]  [DOI]
5.  Hui Z, Yi Z, Peng J. Bibliometric analysis of the orthopedic literature. Orthopedics. 2013;36:e1225-e1232.  [PubMed]  [DOI]
6.  Lee KM, Ryu MS, Chung CY, Choi IH, Kwon DG, Kim TW, Sung KH, Seo SG, Park MS. Characteristics and trends of orthopedic publications between 2000 and 2009. Clin Orthop Surg. 2011;3:225-229.  [PubMed]  [DOI]
7.  Meo SA, Al Masri AA, Usmani AM, Memon AN, Zaidi SZ. Impact of GDP, spending on R& amp; D, number of universities and scientific journals on research publications among Asian countries. PLoS One. 2013;8:e66449.  [PubMed]  [DOI]
8.  Halpenny D, Burke J, McNeill G, Snow A, Torreggiani WC. Geographic origin of publications in radiological journals as a function of GDP and percentage of GDP spent on research. Acad Radiol. 2010;17:768-771.  [PubMed]  [DOI]
9.  Journal Citation Reports for Scientific Information, 2015.  Available from: http://www.weofknowledge.com.  [PubMed]  [DOI]
10.  Marx RG, Wilson SM, Swiontkowski MF. Updating the assignment of levels of evidence. J Bone Joint Surg Am. 2015;97:1-2.  [PubMed]  [DOI]
11.  World Bank List of Economies. World Bank Data Development Group.  Available from: http://data.worldbank.org.  [PubMed]  [DOI]
12.  The World Factbook. Central Intelligence Agency.  Available from: https://www.cia.gov/library/publications/resources/the-world-factbook.  [PubMed]  [DOI]
13.  Maher J. The development of English as an international language of medicine. App Linguistics. 1986;7:206-219.  [PubMed]  [DOI]
14.  Wong CY, Goh KL. The pathway of development: science and technology of NIEs and selected Asian emerging economies. Scientometrics. 2012;92:523-548.  [PubMed]  [DOI]
15.  Moit JS, Sutherland AG, Maffulli N. International orthopaedic journals: a 15-year review. J Bone Joint Surg Br. 1998;80:6-8.  [PubMed]  [DOI]
16.  Liang Z, Luo X, Gong F, Bao H, Qian H, Jia Z, Li G. Worldwide Research Productivity in the Field of Arthroscopy: A Bibliometric Analysis. Arthroscopy. 2015;31:1452-1457.  [PubMed]  [DOI]
17.  Li Q, Jiang Y, Zhang M. National representation in the emergency medicine literature: a bibliometric analysis of highly cited journals. Am J Emerg Med. 2012;30:1530-1534.  [PubMed]  [DOI]
18.  Li Z, Qiu LX, Wu FX, Yang LQ, Sun YM, Lu ZJ, Yu WF. Assessing the national productivity in subspecialty critical care medicine journals: a bibliometric analysis. J Crit Care. 2012;27:747.e1-747.e5.  [PubMed]  [DOI]
19.  Robert C, Wilson CS, Donnadieu S, Gaudy JF, Arreto CD. Evolution of the scientific literature on pain from 1976 to 2007. Pain Med. 2010;11:670-684.  [PubMed]  [DOI]
20.  Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997;314:498-502.  [PubMed]  [DOI]
21.  Whitehouse GH. Impact factors: facts and myths. Eur Radiol. 2002;12:715-717.  [PubMed]  [DOI]
22.  Lefaivre KA, Shadgan B, O’Brien PJ. 100 most cited articles in orthopaedic surgery. Clin Orthop Relat Res. 2011;469:1487-1497.  [PubMed]  [DOI]
23.  Mac Roberts MH, Mac Roberts BR. Problems of citation analysis. Scientometric. 1996;36:435-444.  [PubMed]  [DOI]