Prospective Study
Copyright ©The Author(s) 2018. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Orthop. Jan 18, 2018; 9(1): 1-6
Published online Jan 18, 2018. doi: 10.5312/wjo.v9.i1.1
Dutch version of the Victorian Institute of Sports Assessment-Achilles questionnaire for Achilles tendinopathy: Reliability, validity and applicability to non-athletes
Inger Sierevelt, Maayke van Sterkenburg, Hans Tol, Bella van Dalen, Niek van Dijk, Daniel Haverkamp
Inger Sierevelt, Daniel Haverkamp, Department of Orthopaedic Surgery, Slotervaart Hospital, Amsterdam 1066 EC, The Netherlands
Maayke van Sterkenburg, Niek van Dijk, Department of Orthopedic Surgery, Academic Medical Center, Amsterdam 1105 AZ, The Netherlands
Hans Tol, Academic Center for Evidence Based Sports Medicine, Academic Medical Center Amsterdam, Amsterdam 1105 AZ, The Netherlands
Bella van Dalen, Bergmanclinics te Naarden, Naarden 1411 GE, The Netherlands
Author contributions: All the authors contributed to this manuscript.
Institutional review board statement: The local accredited ethics committee (Dutch acronym: METC) reviewed this study in an expedited manner and determined, based on the Dutch Medical Research Involving Human Subjects Act (Dutch acronym: WMO), that the research activities described meet the requirements for exemption from METC review under the WMO.
Informed consent statement: All patients received a patient information brochure.
Conflict-of-interest statement: The authors do not report a conflict of interest.
Data sharing statement: None.
Open-Access: This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Correspondence to: Daniel Haverkamp, MD, PhD, Department of Orthopaedic Surgery, Slotervaart Hospital, Louwesweg 6, Amsterdam 1066 EC, The Netherlands. daniel@drhaverkamp.com
Telephone: +31-6-45628276
Received: January 24, 2017
Peer-review started: February 2, 2017
First decision: June 26, 2017
Revised: November 30, 2017
Accepted: December 1, 2017
Article in press: December 1, 2017
Published online: January 18, 2018

Abstract
AIM

To translate the Victorian Institute of Sports Assessment-Achilles (VISA-A) questionnaire into the Dutch language (VISA-A-NL), and to assess its reliability, validity, and applicability to non-athletes.

METHODS

After translation according to a forward-backward protocol, 101 patients with complaints of Achilles tendinopathy were asked to fill out the VISA-A-NL at two time points together with visual analogue scale, the Foot and Ankle Outcome Score, and the Short Form-36 questionnaires. Reliability, internal consistency, construct validity, and content validity were tested.

RESULTS

The VISA-A-NL showed high reliability (0.97, 95%CI: 0.95-0.98). Cronbach’s alpha (internal consistency) was 0.80. It increased to 0.88 without activity domain. Correlation with other questionnaires was moderate or poorer.

CONCLUSION

The VISA-A-NL proved to be an excellent evaluation instrument for the Dutch physician. If applied to non-athletes, using a modified score (questions 1-6) should be considered.

Key Words: Achilles tendon, Victorian Institute of Sports Assessment-Achilles, Validity, Patient reported outcome measurement

Core tip: This manuscript shows the validity and reliability of the Dutch version of the Victorian Institute of Sports Assessment-Achilles (VISA-A) in patients with Achilles tendinopathy. The most important finding is that the athletes and non-athletes cannot be compared. The effect of treatment, when using the VISA-A score to measure outcome, is underestimated in non-athletes. If applied to non-athletes, using a modified score (questions 1-6) should be considered.



INTRODUCTION

Achilles tendinopathy is a major cause of chronic pain and disability, which may lead to suboptimal overall health as physical inactivity is a risk factor for cardiovascular disease[1]. Many studies have been published on a multitude of treatments for Achilles tendinopathy, but prospective series on the outcome are lacking. One of the factors limiting the quality of research may be the absence of standardised measures to evaluate the outcome of treatment[2]. A patient’s subjective assessment of treatment outcome such as pain, functional ability, and satisfaction fulfils the criteria of being valid, reliable, and sensitive to change if gathered by a correctly designed and tested patient-centred questionnaire[3]. The Victorian Institute of Sports Assessment-Achilles (VISA-A) questionnaire was created in 2001 to assess clinical severity for patients with Achilles tendinopathy. It is a self- administered questionnaire evaluating symptoms and their effect on physical activity, and displayed reliability and construct validity. Subjective scoring systems can be used in countries other than the ones in which they were developed if translated and validated for a specific language and population[4-6]. The VISA-A has been translated into Swedish, Italian, and German and proved to be able to determine the clinical severity and provide information about the effect of the management of patients with Achilles tendinopathy who speak these languages[1,7,8]. The aim of this study was to translate the VISA-A questionnaire into the Dutch language (VISA-A-NL) and assess its reliability and validity to provide a valid questionnaire for the Dutch population. Moreover, the questionnaire seems to be designed only for athletes as 40% of the points account for activity. As approximately 30% of patients with complaints of Achilles tendinopathy have a sedentary lifestyle[9], applicability to non-athletes was also evaluated in this study.

MATERIALS AND METHODS
Translation procedure

A Dutch translation was made using a forward-backward translation protocol according to the guidelines of Guillemin et al[4,5]. Three people independently translated the English version of the VISA-A questionnaire to Dutch. All three were in the medical field, and spoke English as a second language. One independent native speaker who was not active in the medical field translated this Dutch version back into English. Discrepancies were discussed and adjusted for the final Dutch questionnaire (VISA-A-NL). It was assumed that no major cultural differences in lifestyle exist between the Dutch and Canadian populations, and therefore cultural adaptation of the questionnaire was not required[4,10].

Patients

The local accredited ethics committee (Dutch acronym: METC) reviewed this study in an expedited manner and determined, based on the Dutch Medical Research Involving Human Subjects Act (Dutch acronym: WMO), that the research activities described meet the requirements for exemption from METC review under the WMO. According to the Consensus-Based Standards for the Selection of Health Measurement Instruments criteria, we decided to choose a sample size of at least 100 patients. Actually, 104 consecutive patients from the outpatient clinic of three participating Dutch hospitals were included, of whom 47% were female; 79 (76%) were athletes and 25 (24%) were non-athletes. Their mean age was 48.5 years (SD, 11.6 years).

All patients had complaints of Achilles tendinopathy (including mid-portion and insertional tendinopathy, paratendinopathy, and retrocalcaneal bursitis)[11].

They were clinically assessed and an American Orthopaedic Foot and Ankle Society (AOFAS) questionnaire was taken by the consulting physician. All patients were asked to fill out two sets of questionnaires; the first [A = VISA-A-NL, the Visual Analogue Scale (VAS) pain, VAS function, the Foot and Ankle Outcome Score (FAOS), and the Short Form-36 (SF-36)] was completed on the day of consultation, and the second (B = VISA-A-NL) was completed 5 d later[12]. Additionally, they were asked whether their complaints had changed since the first assessment.

Questionnaires

The original English version of the VISA-A questionnaire as designed by Robinson and co-workers[2] contains eight questions that cover three domains; pain (questions 1-3), function (questions 4-6), and activity (questions 7-8). Scores are summed to yield a total of 100 points in an asymptomatic subject: questions 1-7 score a maximum of 10 points each; question 8, on sporting activity, carries a maximum of 30. Pain on undertaking sports will automatically lead to a loss of 10-20 points.

Since questions 7 and 8 refer to sport activities (accounting for 40% of points) and the study population contained both athletes and non-athletes, we modified the VISA-A score by deleting both questions and investigated the psychometric properties of both the VISA and the modified VISA.

FAOS is a 42-item questionnaire divided into five subscales: pain (9 items), other symptoms (7 items), activities of daily living (17 items), sport and recreation function (5 items), and foot and ankle related quality of life (4 items). Each question can be scored on a 5-point Likert scale (0-4) and each of the five subscale scores is calculated as the sum of the items included. Raw scores are then transformed to 0-100, worst to best score[13].

SF-36 is a self-administered, generic HRQL (health related quality of life) instrument[14-17]. It comprises 36 items across eight dimensions (physical functioning, role limitation due to physical problems, bodily pain, perception of general health, energy and vitality, social functioning, role limitation due to emotional problems, and mental health). The eight dimensions of the SF-36 score are calculated on a 0-100 worst to best scale[18].

The VAS is a 100 mm visual analogue scale and is used to determine the seriousness of pain and functional problems[19].

In 1994, the AOFAS developed a questionnaire to provide a standard method for reporting clinical status of the ankle and foot. The AOFAS-ankle and hindfoot clinical rating system combines both subjective and objective factors into numerical scales to describe function, alignment, and pain. Since objective aspects are incorporated, this questionnaire has to be completed by the investigator[20].

Testing

When a questionnaire is developed or translated, the most important consideration is that it must be able to accurately measure that for which it is designed. To evaluate the psychometric properties of the VISA-A-NL, both reliability and validity were assessed.

Reliability

Reliability is defined as the extent to which patients can be distinguished from each other, despite measurement errors[21].

Test-retest reliability: Test-retest reliability refers to the repeatability of the test and measures the extent to which the same results are obtained on repeated administration when no change in physical functioning has occurred[22]. To determine the test-retest reliability, a second VISA-A-NL (B) questionnaire was given to all patients; 67 patients responded. In 15 patients, complaints had changed at re-test measured by an anchor question (7 item Likert). Test-retest reliability was therefore assessed in 52 patients, using the intra-class coefficient (ICCagreement, two-way random effects model). An ICC > 0.75 was considered good[23]. A t-test was performed to determine the presence of a systematic difference between the first and second assessment. Additionally, standard error of measurement (SEM) was calculated as the square root of the within-subject variance. The smallest detectable change (SDC) was calculated as 1.96 × √2 × SEM. The SDC is the smallest measurement change that can be interpreted as real change[24].

Internal consistency: Internal consistency of the scale is the extent to which the items are inter-correlated and cover the same construct (homogeneity of the scale). To evaluate the internal consistency of the VISA-A, Cronbach’s alpha was calculated. A Cronbach’s alpha of 0.7 was considered to represent an acceptable degree of internal consistency, 0.8 was considered as good, and 0.9 as excellent internal consistency[25].

Validity

Validity relates to the ability of a questionnaire to measure outcome parameter of interest.

Construct validity: Construct validity was tested by determining the association between the VISA-A-NL questionnaire and the FAOS, SF-36, VAS scores for pain and function, and the AOFAS, using Pearson correlation coefficients. We evaluated construct validity by hypothesizing that correlation coefficients between the VISA-A-NL (with and without the activity questions) and VAS pain, FAOS pain, symptoms, and sport and recreation, and SF-36 bodily pain and physical functioning would be higher than correlations with the other domains.

Content validity: Content validity examines the extent to which all concepts of interest are adequately represented by the items in the questionnaire[12]. It was evaluated by assessing distribution and floor and ceiling effects of the VISA-A-NL. These are considered to be present if more than 15% of responders achieve the lowest or highest possible score[12].

Statistical analysis

Statistical analyses were performed using PASW statistics 18.0 software (SPSS Inc., Chicago, IL, United States). A P-value < 0.05 was considered statistically significant.

RESULTS

Of 104 participants, 11 returned questionnaires that were filled out incompletely or erroneously and were therefore excluded from analysis. Thus, 93 patients were finally included in the analysis.

Reliability

Of 93 patients, 52 (56%) returned the second set of questionnaires. The ICCagreement of the questionnaire was 0.97 (95%CI: 0.95-0.98) and Cronbach’s alpha was 0.78 for the entire study population (Table 1). A statistically significant difference between the two assessments was not observed for both versions of the VISA-A-NL in athletes, nor in non-athletes (0.29 < P < 0.67).

Table 1 The ICCagreement and Cronbach’s alpha of the questionnaire.
Athletes (n = 39)Non-athletes (n = 13)Total (n = 52)
ICCVISA total (95%CI)0.95 (0.91-0.97)-0.97 (0.95-0.98)
ICCVISA modified (95%CI)0.96 (0.92-0.98)0.98 (0.93-0.99)0.97 (0.95-0.98)
SEMVISA total4.41 (4.4%)-4.07 (4.1%)
SEMVISA modified2.64 (4.4%)2.42 (4.0%)2.68 (4.5%)
SDCVISA total12.21 (12.2%)-11.28 (11.3%)
SDCVISA modified7.32 (12.2%)6.70 (11.2%)7.44 (12.4%)
Cronbach’s alphaVISA total0.720.820.78
Cronbach’s alphaVISA modified0.830.860.86
Validity

Pearson correlation coefficients of the VISA-A-NL with the other questionnaires are shown in Table 2. The subscale “SF-36 physical functioning” correlated well with the VISA-A-NL questionnaire, both with and without activity domain. Most other physical domains correlated moderately, but the subscales of FAOS quality of life showed a poor correlation with VISA-A. A poor correlation was also observed for the psychological domains (SF-36 social functioning, mental health, role emotional, vitality, and general health perception).

Table 2 Pearson correlation coefficients of the Victorian Institute of Sports Assessment-Achilles-NL with the other questionnaires (with and without activity domain).
VISA-ATotal
VISA-AModified
VISA-AModified
VISA-ATotal
VISA-AModified
Athletes (n = 71)Athletes (n = 71)Non-athletes (n = 22)Entire population (n = 93)Entire population (n = 93)
VAS pain-0.54a-0.58a-0.39-0.54a-0.57a
VAS function0.52a0.51a0.44a0.50a0.52a
AOFAS0.48a0.46a0.310.56a0.50a
FAOS symptoms0.45a0.52a0.53a0.58a0.60a
FAOS pain0.53a0.56a0.52a0.58a0.60a
FAOS ADL0.56a0.55a0.47a0.59a0.58a
FAOS sport0.56a0.61a0.43a0.55a0.59a
FAOS QOL0.29a0.33a0.140.38a0.37a
SF-36 physical functioning0.55a0.63a0.66a0.70a0.71a
SF-36 role physical0.130.120.58a0.31a0.32a
SF-36 bodily pain0.310.40a0.46a0.49a0.51a
SF-36 social functioning0.01-0.040.290.27a0.21
SF-36 mental health-0.11-0.070.390.210.20
SF-36 role emotional-0.060.020.65a0.37a0.39a
SF-36 vitality-0.26-0.250.26-0.05-0.09
SF-36 general health perception-0.01-0.020.280.25a0.21
SF-36 physical component scale0.43a0.47a0.360.52a0.51a
SF-36 mental component scale-0.34a-0.32a0.49a0.040.03

The mean scores of the VISA-A-NL questionnaire were 52.4 (SD 19.7) and 22.0 (SD 15.7) for athletes and non-athletes, respectively. The mean scores of the modified VISA-A-NL questionnaire were 36.2 (SD 13.9) and again 22.0 (SD 15.7) for athletes and non-athletes, respectively. Floor and ceiling effects were not observed in both versions of the questionnaire, as only one (1%) subject scored 0 points, nobody scored 100 points, and one patient scored the maximum of 60 points in the modified VISA-A score.

DISCUSSION

The aim of this study was to translate the original VISA-A questionnaire on the subjective complaints of patients with Achilles tendinopathy into the Dutch language, to validate it, and to assess its applicability to non-athletes.

The translation procedure did not create any problems, since the items are universal and there is no large cultural difference between Dutch and Canadian patients.

Reliability of the translation was excellent, with a statistically non-significant difference between assessments. This outcome may be explained by the fact that, as the procedure for the Dutch Oxford 12-item knee questionnaire taught us, we introduced a question if complaints had changed between assessments[26]. Fifteen (22%) of sixty-seven patients answered this question with “yes”, and therefore they were excluded from reliability testing.

The SDC in this study indicated that under a stable condition, the VISA-A score can vary up to 12 points. For clinical studies, this implies that clinical changes can only be detected if they exceed the 12 points.

The study questionnaire showed good internal consistency (Cronbach’s alpha = 0.80), but there indeed was a negative effect of the activity domain. Sub-analysis without these questions showed an increase of Cronbach’s alpha to 0.88. However, when measuring the effect of a treatment in a mixed group of athletes and non-athletes, the effect of treatment could be underestimated. For example, an athlete can score 0 points with questions 7 and 8 before treatment as complaints withhold him/her from being sports active. After treatment, the athlete is complaint-free and scores 100 points. The non-athlete also scores 0 points for questions 7 and 8 before treatment, and is also complaint-free after treatment. However, he/she will never score higher than 60 points as questions 7 and 8 will not be answered differently between assessments. The effect of treatment, when using the VISA-A score to measure outcome, is therefore underestimated in non-athletes.

When choosing the VISA-A questionnaire for a mixed population of athletes and non-athletes, deleting questions 7 and 8 can be considered, since the psychometric properties of the modified VISA-A were comparable with those of the original version.

Generally, Pearson correlation coefficients were higher for physical than psychological components. Convergent and divergent validity is confirmed as correlation coefficients were higher for the physical domains of the questionnaires. However, for non-athletes, correlation with socio-emotional components was also higher. This could imply that physical restrictions in this subgroup with chronic Achilles tendinopathy have greater emotional and social consequences than in athletes. Noticeable is the moderate correlation of VISA-A with VAS, which is a validated subjective outcome measure frequently used for scientific means.

Low correlations can be explained by the fact that none of the questionnaires except for the VISA-A were validated for Achilles tendinopathy. Initially it was intended to do so, but this study aim was departed to not further enlarge patient burden as many of these questionnaires are extensive. Given the laborious inclusion of 104 patients in 3.5 years and a 56% response rate to both assessments, this was well decided. This was also why responsiveness was not tested.

In conclusion, the VISA-A-NL questionnaire seems suitable for use in athletes. However, in a combined population of athletes and non-athletes, results will become incomparable as the highest possible score for non-athletes is 40 points lower than that for non-athletes. Psychometric properties of the VISA-A-NL, without questions 7 and 8, are satisfactory for both athletes and non-athletes. It is therefore proposed that the modified VISA-A-NL questionnaire is considered a mixed population of patients with Achilles tendinopathy, meaning a version with only questions 1-6 and the complete questionnaire (questions 1-8) is reserved for athletes only.

ARTICLE HIGHLIGHTS
Background

Achilles tendinopathy is a major cause of chronic pain and disability. Many studies have been published on a multitude of treatments for Achilles tendinopathy, but prospective series on the outcome are lacking. The Victorian Institute of Sports Assessment-Achilles (VISA-A) questionnaire was created in 2001 to assess clinical severity for patients with Achilles tendinopathy. It is a self-administered questionnaire evaluating symptoms and their effect on physical activity, and displayed reliability and construct validity.

Research frontiers

The VISA-A has been translated into Swedish, Italian, and German and proved to be able to determine the clinical severity and provide information about the effect of the management of patients with Achilles tendinopathy who speak these languages.

Innovations and breakthroughs

This work translated the VISA-A questionnaire into the Dutch language (VISA-A-NL), assessed its validity and reliability, and provided a valid questionnaire for the Dutch population. The questionnaire seems to be designed only for athletes as 40% of the points account for activity. As approximately 30% of patients with complaints of Achilles tendinopathy have a sedentary lifestyle, applicability to non-athletes was also evaluated.

Applications

Psychometric properties of the VISA-A-NL, without questions 7 and 8, are satisfactory for both athletes and non-athletes. It is therefore proposed that the modified VISA-A-NL questionnaire is considered for a mixed population of patients with Achilles tendinopathy.

Footnotes

Manuscript source: Invited manuscript

Specialty type: Orthopedics

Country of origin: Netherlands

Peer-review report classification

Grade A (Excellent): 0

Grade B (Very good): B

Grade C (Good): 0

Grade D (Fair): 0

Grade E (Poor): 0

P- Reviewer: Hernandez-Sanchez S S- Editor: Kong JX L- Editor: Wang TQ E- Editor: Li RF

References
1.  Maffulli N, Longo UG, Testa V, Oliva F, Capasso G, Denaro V. Italian translation of the VISA-A score for tendinopathy of the main body of the Achilles tendon. Disabil Rehabil. 2008;30:1635-1639.  [PubMed]  [DOI]
2.  Robinson JM, Cook JL, Purdam C, Visentini PJ, Ross J, Maffulli N, Taunton JE, Khan KM; Victorian Institute Of Sport Tendon Study Group. The VISA-A questionnaire: a valid and reliable index of the clinical severity of Achilles tendinopathy. Br J Sports Med. 2001;35:335-341.  [PubMed]  [DOI]
3.  Dawson J, Carr A. Outcomes evaluation in orthopaedics. J Bone Joint Surg Br. 2001;83:313-315.  [PubMed]  [DOI]
4.  Guillemin F. Cross-cultural adaptation and validation of health status measures. Scand J Rheumatol. 1995;24:61-63.  [PubMed]  [DOI]
5.  Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46:1417-1432.  [PubMed]  [DOI]
6.  Mathias SD, Fifer SK, Patrick DL. Rapid translation of quality of life measures for international clinical trials: avoiding errors in the minimalist approach. Qual Life Res. 1994;3:403-412.  [PubMed]  [DOI]
7.  Lohrer H, Nauck T. Cross-cultural adaptation and validation of the VISA-A questionnaire for German-speaking achilles tendinopathy patients. BMC Musculoskelet Disord. 2009;10:134.  [PubMed]  [DOI]
8.  Silbernagel KG, Thomeé R, Karlsson J. Cross-cultural adaptation of the VISA-A questionnaire, an index of clinical severity for patients with Achilles tendinopathy, with reliability, validity and structure evaluations. BMC Musculoskelet Disord. 2005;6:12.  [PubMed]  [DOI]
9.  Ames PR, Longo UG, Denaro V, Maffulli N. Achilles tendon problems: not just an orthopaedic issue. Disabil Rehabil. 2008;30:1646-1650.  [PubMed]  [DOI]
10.  Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25:3186-3191.  [PubMed]  [DOI]
11.  van Dijk CN, van Sterkenburg MN, Wiegerinck JI, Karlsson J, Maffulli N. Terminology for Achilles tendon related disorders. Knee Surg Sports Traumatol Arthrosc. 2011;19:835-841.  [PubMed]  [DOI]
12.  Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, Bouter LM, de Vet HC. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34-42.  [PubMed]  [DOI]
13.  Roos EM, Brandsson S, Karlsson J. Validation of the foot and ankle outcome score for ankle ligament reconstruction. Foot Ankle Int. 2001;22:788-794.  [PubMed]  [DOI]
14.  Aaronson NK, Muller M, Cohen PD, Essink-Bot ML, Fekkes M, Sanderman R, Sprangers MA, te Velde A, Verrips E. Translation, validation, and norming of the Dutch language version of the SF-36 Health Survey in community and chronic disease populations. J Clin Epidemiol. 1998;51:1055-1068.  [PubMed]  [DOI]
15.  Brazier JE, Harper R, Jones NM, O’Cathain A, Thomas KJ, Usherwood T, Westlake L. Validating the SF-36 health survey questionnaire: new outcome measure for primary care. BMJ. 1992;305:160-164.  [PubMed]  [DOI]
16.  Sullivan LM, Dukes KA, Harris L, Dittus RS, Greenfield S, Kaplan SH. A comparison of various methods of collecting self-reported health outcomes data among low-income and minority patients. Med Care. 1995;33:AS183-AS194.  [PubMed]  [DOI]
17.  Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30:473-483.  [PubMed]  [DOI]
18.  Insall JN, Dorr LD, Scott RD, Scott WN. Rationale of the Knee Society clinical rating system. Clin Orthop Relat Res. 1989;13-14.  [PubMed]  [DOI]
19.  Price DD, McGrath PA, Rafii A, Buckingham B. The validation of visual analogue scales as ratio scale measures for chronic and experimental pain. Pain. 1983;17:45-56.  [PubMed]  [DOI]
20.  Kitaoka HB, Alexander IJ, Adelaar RS, Nunley JA, Myerson MS, Sanders M. Clinical rating systems for the ankle-hindfoot, midfoot, hallux, and lesser toes. Foot Ankle Int. 1994;15:349-353.  [PubMed]  [DOI]
21.  Streiner DL. Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess. 2003;80:99-103.  [PubMed]  [DOI]
22.  Eechaute C, Vaes P, Duquet W, Van Gheluwe B. Test-retest reliability of sudden ankle inversion measurements in subjects with healthy ankle joints. J Athl Train. 2007;42:60-65.  [PubMed]  [DOI]
23.  Fleiss JL  The design and analysis of clinical experiments. 1986. New York: Wiley.  [PubMed]  [DOI]
24.  Beckerman H, Roebroeck ME, Lankhorst GJ, Becher JG, Bezemer PD, Verbeek AL. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res. 2001;10:571-578.  [PubMed]  [DOI]
25.  Dunbar MJ, Robertsson O, Ryd L, Lidgren L. Translation and validation of the Oxford-12 item knee score for use in Sweden. Acta Orthop Scand. 2000;71:268-274.  [PubMed]  [DOI]
26.  Haverkamp D, Breugem SJ, Sierevelt IN, Blankevoort L, van Dijk CN. Translation and validation of the Dutch version of the Oxford 12-item knee questionnaire for knee arthroplasty. Acta Orthop. 2005;76:347-352.  [PubMed]  [DOI]