Minimum Detectable Change of Visual Acuity Measurements Using ETDRS Charts (Early Treatment Diabetic Retinopathy Study)

International Journal of Environmental Research and Public Health Article Minimum Detectable Change of Visual Acuity Measurements Using ETDRS Charts (Early Treatment Diabetic Retinopathy Study) María Carmen Sánchez-González 1,* , Raquel García-Oliver 1, José-María Sánchez-González 1, María-JoséBautista-Llamas 1, José-Jesús Jiménez-Rejano 2and Concepción De-Hita-Cantalejo 1   Citation: Sánchez-González, M.C.; García-Oliver, R.; Sánchez-González, J.-M.; Bautista-Llamas, M.-J.; Jiménez-Rejano, J.-J.; De-Hita-Cantalejo, C. Minimum Detectable Change of Visual Acuity Measurements Using ETDRS Charts (Early Treatment Diabetic Retinopathy Study). Int. J. Environ. Res. Public Health 2021,18, 7876. https://doi.org/10.3390/ ijerph18157876 Academic Editors: Francesc Medina-Mirapeix, Rodrigo Martín-San Agustín and Mª Piedad Sánchez-Martínez Received: 25 June 2021 Accepted: 23 July 2021 Published: 25 July 2021 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). 1Department of Physics of Condensed Matter, Optics Area, University of Seville, 41012 Seville, Spain; [email protected] (R.G.-O.); [email protected] (J.-M.S.-G.); [email protected] (M.-J.B.-L.); [email protected] (C.D.-H.-C.) 2Physiotherapy Department, Faculty of Nursing, Physiotherapy and Podiatry, University of Seville, 41009 Seville, Spain; [email protected] *Correspondence: [email protected] Abstract: In our work, we determined the value of visual acuity (VA) with ETDRS charts (Early Treatment Diabetic Retinopathy Study). The purpose of the study was to determine the measurement reliabilities, calculating the correlation coefficient interclass (ICC), the value of the error associated with the measure (SEM), and the minimal detectable change (MDC). Forty healthy subjects took part. The mean age was 23.5 ± 3.1 (19 to 26) years. Visual acuities were measured with ETDRS charts (96% ETDRS chart n º 2140) and (10% SLOAN Contrast Eye Test chart n º 2153). The measurements were made (at 4 m) under four conditions: Firstly, photopic conditions with high contrast (HC) and low contrast (LC) and after 15 min of visual rest, mesopic conditions with high and low contrast. Under photopic conditions and high contrast, the ICC = 0.866 and decreased to 0.580 when the luminosity and contrast decreased. The % MDC in the four conditions was always less than 10%. It was minor under photopic conditions and HC (5.83) and maximum in mesopic conditions and LC (9.70). Our results conclude a high reliability of the ETDRS test, which is higher in photopic and high contrast conditions and lower when the luminosity and contrast decreases. Keywords: ETDRS; minimal detectable change; visual acuity 1. Introduction Visual acuity (VA) allows for the evaluation of the integrity of the central and peripheral visual pathway, detect refractive errors and control the progression of eye diseases [ 1 ]. There are many tests for VA measurement. Currently, for research purposes, test symbols based on the Logarithm of the Minimum Angle of Resolution (LogMAR) design are used, along with the Bailey–Lovie visual acuity chart [ 2 ], including the version produced by Ferris et al. [3] and the Early Treatment Diabetic Retinopathy Study (ETDRS) test [4]. The ETDRS visual acuity charts have five letters in each row and a uniform space between each letter and each row of letters. This is currently considered by many authors as the gold standard of visual acuity testing in research [ 4 ]. However, Solomon et al. [ 5 ] along with a narrative history of the evolution, modification, and legacy of the ETDRS classification system concluded that new updated must be implemented on the test. One of the main objectives of a VA measurement is the detection of clinical changes in the subject. The reliability of a visual acuity measurement is important in the assessment of a subject, because it allows for the differentiation of whether a change in visual acuity represents a clinical change or corresponds to a measurement error. Any measurement method has associated systematic errors. Hence, small variations could not be attributed to changes in a patient’s health status. Int. J. Environ. Res. Public Health 2021,18, 7876. https://doi.org/10.3390/ijerph18157876 https://www.mdpi.com/journal/ijerph Int. J. Environ. Res. Public Health 2021,18, 7876 2 of 8 The inherent error of the measurement, even when using standardized optotypes, can cause a failure where a true change in the VA, a false negative or a change in diagnosis when in fact there is none, or a false positive are not detected [ 6 ]. Computerized repetition and an average visual acuity measurement reduces test–retest variability [ 6 , 7 ]. A computer is used to produce random stimuli, along with automatic processing and statistical analysis of the subject’s response, which is not possible using printed graphics. In clinic, this is usually referred to as repeatability, usually calculated as the test–retest variability (TRV). For the evidence diagnostics to be valid and have good discriminatory capacity, they must have high repeatability. The ability to detect true visual change decreases as the TRV increases within the acuity data [7]. Some studies have reviewed and evaluated the repeatability of best corrected visual acuity in the ETDRS tests of healthy adults [ 8 ] with retinal pathology [ 9 , 10 ], as well as in children [ 1 , 11 ], Other authors have compared the ETDRS test with other tests [ 4 , 12 – 15 ] and evaluated the test–retest reliability of the electronic visual acuity [ 16 ] algorithm of the Diabetic Retinopathy Study of Early Treatment (E-ETDRS) [ 17 – 19 ]. They have concluded that ETDRS provides high repeatability coefficients. The precision or sensitivity of an instrument or scales is defined as the measurement’s ability to distinguish small differences. The more accurate an instrument is, the more likely it is to also capture small changes, while an insensitive instrument will require big changes to show any changes in its scores. The accuracy of a scale is usually measured by the standard error of the measurement (SEM), which determines a threshold, above which the change is considered real (or the minimal detectable change). The minimal detectable change (MDC) represents the minimal change in a score that reflects an actual clinical change and not a change by mis measurement [20]. In our work, we determined the value of VA with ETDRS in high and low contrast conditions, since visual function can remain normal in high contrast and be affected in low contrast [ 21 ]. The purpose of the study was to determine the measurement reliabilities, calculating the correlation coefficient interclass (ICC), the value of the error associated with the measure (SEM) and the minimal detectable change (MDC). 2. Materials and Methods 2.1. Design This test–retest reliability study of ETDRS chart was conducted in accordance with the 2013 Helsinki Declaration (Fortaleza, Brazil). The study protocol was approved by the Virgen Macarena—Virgen del Rocio Hospital Institutional Review Board (code number 1260-N-18). Informed consent was obtained from all participants. The study procedure was conducted in the facilities of the University. 2.2. Subjects Forty healthy subjects (40 eyes, 24 females and 16 males) took part in the study. The mean age was 23.5 ±3.1 (19 to 26) years. Participants were University of Seville students, teachers and administration staff. They were invited to participate in the study through a mailing system sent to the pharmacy faculty university community. The inclusion criteria were: (I) visual acuity (VA) between +0.10 and − 0.30 LogMAR (20/25 to 20/10 Snellen Scale) measured with a Snellen Optotype; (II) a stable refraction for at least one year, meaning a change ≤ 0.50 diopters in the spherical and cylindrical refraction; (III) had not worn soft or rigid gas permeable contact lenses within the previous 24 h, and (IV) had not used near electronic devices within the last six hours. Furthermore, the exclusion criteria included: (V) eye diseases, such as glaucoma and cataracts; (VI) out of the norm values of accommodation amplitude, negative relative accommodation, positive relative accommodation and monocular and binocular accommodative facility testing; (VII) pregnant or lactating participants, and (VIII) participants with disorders of the eye muscles (IX) participants with strabismus or nystagmus, or any other disorder affecting ocular fixation. The exclusion criteria were checked with a comprehensive optometry exam that Int. J. Environ. Res. Public Health 2021,18, 7876 3 of 8 included: slit lamp examination, intraocular pressure measurement, binocular vision, and accommodation assessment. 2.3. Study Procedure From a total sample of 55 participants, 15 were excluded from the study due to the aforementioned criteria. Investigations were done monocularly. The eye to be measured was randomly selected. The random arrangement was (1:1) generated by an independent researcher using Epidat ® 4.2 software (Department of Health, Government of Galicia, Spain). The participants carried their glass correction, if needed, and the study eye was examined when the contralateral eye was occluded. Visual acuity examinations were repeated two hours after the first examination. The measurements were made by a single expert and licensed optometrist. All of the measurements obtained were performed in the same way for all of the participants. Visual acuity was measured with ETDRS charts (96% ETDRS chart n º 2140) and (10% SLOAN Contrast Eye Test chart n º 2153) displayed in the standard Lighthouse Low Vision Products light box (Lighthouse Low Vision Products, Long Island City, NY, USA). The measurements were made (at 4 m) under four conditions: Firstly, photopic conditions with high contrast (HC) and low contrast (LC) and after 15 min of visual rest, mesopic conditions with high contrast (HC) and low contrast (LC). The room was lit using a halogen lamp, and the lighting conditions were measured with a Lux LCD Illuminance Meter (Precision Vision ® , Woodstock, IL, USA) to obtain a photopic luminance level of 85 cd/m2or a mesopic luminance level of 0.75 cd/m2[1,8]. The charts were front-illuminated by two 40-watt fluorescent tubes. For the photopic VA measurements, the room was lit. For the mesopic VA measurements, the lights in the room were off during the entire test. The patient was asked to read the letter starting at the top row and continuing to the next row until they incorrectly identified a complete row, at which point the test was terminated [8]. The letter scores were converted to the LogMAR equivalent using the formula: Log- MAR = 1.7 −(0.02) ·(letter score) [17]. 2.4. Statistical Analysis Statistical analysis was carried out with SPSS statistics 25.0 (IBM Corporation, Armonk, NY, USA). Intraclass correlation coefficients (ICCs) were calculated on the basis of a two-way mixed model with an absolute agreement for a single rater/measurement [ 22 ]. Additionally, ICC 3,1 were based on a two-way mixed model with a consistency agreement for single rater/measurement and ICC 1,1 were based on one-way random effects model with an absolute agreement for a single rater/measurement being calculated to verify the first ICC. In all of the cases, the average measurements were obtained. The criteria for the values of the ICCs were as follows: >0.80, excellent reliability; 0.61–0.80, good reliability; 0.41–0.60, moderate reliability; and ≤0.40, poor reliability [23]. The standard error measurement (SEM) is an index that can be used to define the difference needed between the separate measures on a subject in order for the difference in the measures to be considered real. SEMs were calculated as arranged by Weir as [ 24 ]; SEM =SD√1−ICC , where SD was the pooled standard deviation from all of the measurements and ICC was a two-way mixed model with an absolute agreement for a single rater/measurement intraclass correlation coefficient. Minimal detectable change (MDC) can be defined as the minimal difference needed for a change to be considered real. MDC was calculated as follows: MDC =SEM × 1.96 ×√2 . In addition, the MDC percentage was also calculated according to the following formula: MDC%=MDC mean × 100. An MDC % score of ≤ 30% was acceptable, whereas, an MDC% score of ≤ 10% was excellent. Bland–Altman plots were constructed with 95% limits of agreement (LOA) to represent the agreement between the two ETDRS visual acuity measurements, where the differences in Int. J. Environ. Res. Public Health 2021,18, 7876 4 of 8 the two measurements were compared to the visual acuity measurement mean. The LOA were assessed as the mean change ±1.96 ×SD of the difference [25]. 3. Results Eighty measurements from 40 eyes (40 participants) were included in the following results. The ETDRS visual acuity mean, standard deviation and range were expressed in LogMAR unit and letter scores. Furthermore, ICC, ICC 3,1 , ICC 1,1 , SEM, MDC, and MDC % were also reported for all conditions. 3.1. Photopic and High Contrast The first measurements obtained were 0.0065 ± 0.01607 ( − 0.22 to +0.20) LogMAR/ 84.67 ± 5.08 (75 to 96) letters and − 0.0190 ± 0.01504 ( − 0.20 to +0.20) LogMAR/ 85.95 ±4.75 (75 to 95) letters in the second measurements. ICC, ICC 3,1 , and ICC 1,1 were 0.866, 0.893, and 0.864, respectively. All of the ICC values obtained excellent reliability results. SEM was 0.0361 LogMAR (1.80 letters score). MDC reported a change of 0.100 LogMAR (4.98 letters score) and the MDC percentage was 5.83%, obtaining an excellent score in MDC % . The Bland–Altman plot difference between the measurements was 0.0178, upper LOA was 0.08836 and lower LOA was −0.05276. 3.2. Photopic and Low Contrast The first measurements obtained were 0.0130 ± 0.01563 ( − 0.20 to +0.22) LogMAR/ 84.35 ± 4.94 (74 to 95) letters and 0.0018 ± 0.01421 ( − 0.20 to +0.24) LogMAR/84.91 ± 4.49 (73 to 95) letters in the second measurements. ICC, ICC 3,1 , and ICC 1,1 were 0.777, 0.779, and 0.777, respectively. All of the ICC values obtained good reliability results. SEM was 0.0444 LogMAR (2.21 letters score). MDC reported a change of 0.123 LogMAR (6.12 letters score) and MDC percentage was 7.23%, obtaining an excellent score in MDC % . The Bland–Altman plot difference between the measurements was 0.0179, upper LOA was 0.10512 and lower LOA was −0.06932. 3.3. Mesopic and High Contrast The first measurements obtained were − 0.0060 ± 0.01492 ( − 0.24 to +0.20) Log- MAR/85.30 ± 4.71 (75 to 97) letters and − 0.0060 ± 0.01427 ( − 0.20 to +0.24) LogMAR/ 85.30 ±4.51 (73 to 95) letters in the second measurements. ICC, ICC 3,1 , and ICC 1,1 were 0.733, 0.728, and 0.734, respectively. All of the ICC values obtained good reliability results. SEM was 0.0474 LogMAR (2.36 letters score). MDC reported a change of 0.131 LogMAR (6.54 letters score) and the MDC percentage was 7.66%, obtaining an excellent score in MDC % . The Bland–Altman plot difference between the measurements was 0.0075, upper LOA was 0.10158 and lower LOA was −0.08658. 3.4. Mesopic and Low Contrast The first measurements obtained were 0.0565 ± 0.01492 ( − 0.18 to +0.28) LogMAR/ 82.17 ± 4.71 (71 to 94) letters and − 0.0225 ± 0.01315 ( − 0.16 to +0.22) LogMAR/83.87 ± 4.15 (74 to 93) letters in the second measurements. ICC, ICC 3,1 , and ICC 1,1 were 0.580, 0.617, and 0.567, respectively. All of the ICC values obtained moderate, good and moderate reliability results, respectively. SEM was 0.0583 LogMAR (2.91 letters score). MDC reported a change of 0.161 LogMAR (8.06 letters score) and the MDC percentage was 9.70% obtaining an excellent score in MDC % . The Bland–Altman plot difference between the measurements was 0.0292, upper LOA was 0.12151 and lower LOA was −0.06311. (Figure 1). Int. J. Environ. Res. Public Health 2021,18, 7876 5 of 8 Int. J. Environ. Res. Public Health 2021, 18, x 5 of 8 Figure 1. Bland–Altman plots between the measurements of VA. (A) Photopic and High Contrast; (B) Photopic and Low Contrast; (C) Mesopic and High Contrast; (D) Mesopic and Low Contrast. The central line indicates the mean between measurements. The external lines indicate limit of the 95 % agreement superior and inferior. 4. Discussion The purpose of this study was to calculate the minimal detectable change (MDC) and examine the test–retest reliability of the Early Treatment Diabetic Retinopathy Study (ETDRS) in four different conditions of luminosity and contrast, along with the photopic and mesopic high (HC) and low contrast (LC). This study was conducted in healthy subjects with excellent VA due reproducibility in eyes with ocular disease or poor optical quality of the image would be affected. In the review carried out, we found studies evaluating the test–retest reliability of (ETDRS) [1,6,8–11,13]. However, any measurement process has associated systematic errors in the measured variables, and therefore, changes smaller than the magnitude of the error cannot be attributed to changes in a patient’s health status. In our work, we calculated the values of the error associated with the measurements (SEM) and the minimal detectable change (MDC), as they allowed us to determine whether an observed change between two measurements corresponded to a real change in the subject [20]. We also calculated the interclass correlation coefficients (ICC) to compare reliability between the measurements [24]. To our knowledge, ETDRS has only been measured for repeatability and test–retest reliability and none of the existing papers have addressed the study of the MDC and SEM. Figure 1. Bland–Altman plots between the measurements of VA. ( A ) Photopic and High Contrast; ( B ) Photopic and Low Contrast; ( C ) Mesopic and High Contrast; ( D ) Mesopic and Low Contrast. The central line indicates the mean between measurements. The external lines indicate limit of the 95 % agreement superior and inferior. 4. Discussion The purpose of this study was to calculate the minimal detectable change (MDC) and examine the test–retest reliability of the Early Treatment Diabetic Retinopathy Study (ETDRS) in four different conditions of luminosity and contrast, along with the photopic and mesopic high (HC) and low contrast (LC). This study was conducted in healthy subjects with excellent VA due reproducibility in eyes with ocular disease or poor optical quality of the image would be affected. In the review carried out, we found studies evaluating the test–retest reliability of (ETDRS) [ 1 , 6 , 8 – 11 , 13 ]. However, any measurement process has associated systematic errors in the measured variables, and therefore, changes smaller than the magnitude of the error cannot be attributed to changes in a patient’s health status. In our work, we calculated the values of the error associated with the measurements (SEM) and the minimal detectable change (MDC), as they allowed us to determine whether an observed change between two measurements corresponded to a real change in the subject [ 20 ]. We also calculated the interclass correlation coefficients (ICC) to compare reliability between the measurements [ 24 ]. To our knowledge, ETDRS has only been measured for repeatability and test–retest reliability and none of the existing papers have addressed the study of the MDC and SEM. According to ICC values, ETDRS showed excellent test–retest reliability in all of the four measurement conditions. The ICCs can theoretically vary between 0 and 1.0, where an ICC of 0 indicates no reliability and an ICC of 1.0 indicates perfect reliability [ 24 ]. Under photopic conditions and high contrast, the ICC = 0.866 and decreased to 0.580 when the luminosity and contrast decreased. Int. J. Environ. Res. Public Health 2021,18, 7876 6 of 8 Our test–retest reliability results, measured with the interclass correlation coefficient, were in line with other studies. Beck et al. [ 17 ] measured test–retest reliability at 4 or 3 m in luminosity conditions (85 to 105 cd/m 2 ) and HC (98%) in three groups of subjects with different visual acuity (VA) values. The ICC values obtained were ICC = 0.86 (VA > 20/40), ICC = 0.87 (VA 20/40 to 20/100), and ICC = 0.95 (VA < 20/100). In all of the cases, the value of the ICC was high, although the variability was slightly higher in participants with low visual acuity values. Camparini et al. [ 26 ] demonstrated values of 0.94 when measured in standard lighting conditions at 4 m. Barrio et al. [ 8 ] determined the high repeatability of the test in far and near mesopic conditions (0.75 cd/m 2 ), in both high (96%) and low contrast (10%). Similar repeatability was observed in HC and LC in both far ( ± 0.11 logMAR) and near ( ± 0.16 logMAR/ ± 0.15 logMAR) conditions. Chaikitmongkol et al. [ 9 ] compared the repeatability of the ETDRS number charts, ETDRS Landolt C and ETDRS alphabet charts in four groups of subjects (healthy and with eye pathologies). They concluded that the ETDRS number chart was the one with the most repeatability in the measurement of VA for clinical practice. Patel et al. [10] described the repeatability between sessions of visual acuity measurements obtained with TDRS in patients with age-related macular degeneration (AMD). The measurements of patients with small druse were the most repeatable (coefficient of repeatability [CR] = 9 letters), and the measurements of patients with late AMD were the least repeatable (CR = 17 letters). This variability between the measurements may be due to the eye pathology itself. The ICC refers to how repeated measurements vary in subjects, and a smaller variation indicates a higher reliability [ 27 ]. As in our work, in all the studies reviewed, high ICC values suggested a high reliability of the ETDRS test. In addition, we calculated the value of the standard measurement error (SEM) and the minimal detectable change (MDC) as reliability indexes. However, current literature lacks data on other errors in the measurement of VA with ETDRS with which to compare our results. Still, the % MDC in the four conditions was always less than 10%. It was minor under photopic conditions and HC (5.83) and maximum in mesopic conditions and LC (9.70). In addition, the number of letters that the subject failed increased as the luminosity and contrast decreased, as determined by SEM error values that progressively increased from 1.80 letters under photopic conditions and high contrast to 2.91 under mesopic conditions and low contrast. The SEM and % MDC values were low, indicating that absolute reliability was high overall and the random measurement error was small. These values strengthen the high reliability of the ETDRS test, as referred to by other authors who measured only its repeatability [ 8 – 10 , 17 , 26 ]. Within the future research of lines, a new approach with an automated-ETDRS in near and intermediate distance [ 28 ] or a comparative assessment between Snellen scale and ETDRS [29] could improve scientific literature about ETDRS. 5. Conclusions Our results concluded that a high reliability of the ETDRS test, which is higher in photopic and high contrast conditions and lower when the luminosity and contrast decreases. With respect to clinical implications, a change in a VA score greater than the value of the MDC should be considered as a real change with 95% certainty. Therefore, we feel that the MDC value should be used in clinical research when evaluating VA to know exactly the score of each patient. Author Contributions: Conceptualization, M.C.S.-G. and C.D.-H.-C.; methodology, R.G.-O. and M.C.S.-G.; software, J.-M.S.-G. and J.-J.J.-R.; validation, J.-M.S.-G., M.-J.B.-L., and R.G.-O.; formal analysis, J.-M.S.-G. and J.-J.J.-R.; investigation, M.C.S.-G.; resources, M.C.S.-G. and C.D.-H.-C.; data curation, M.-J.B.-L.; writing—original draft preparation, M.C.S.-G.; writing—review and editing, M.C.S.-G. and R.G.-O. visualization, J.-M.S.-G. and C.D.-H.-C.; supervision, J.-M.S.-G.; project administration, M.-J.B.-L. All authors have read and agreed to the published version of the manuscript. Int. J. Environ. Res. Public Health 2021,18, 7876 7 of 8 Funding: This research received no external funding. Institutional Review Board Statement: Data acquisition and analysis was performed in compliance with protocols approved by the Ethical Committee of Virgen Macarena—Virgen del Rocio Hospital Institutional Review Board (code number 1260-N-18). Date of approval: 7 November 2018. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study. Data Availability Statement: The data presented in this study are available on request from the corresponding author. Acknowledgments: We would like to thank all the participants involved in the project. Also, thanks to the staff of the Optics and Optometry Titling of the University of Seville, where this study was carried out, for their support. Conflicts of Interest: The authors declare no conflict of interest. References 1. Manny, R.E.; Hussein, M.; Gwiazda, J.; Marsh-Tootle, W. Repeatability of ETDRS Visual Acuity in Children. Investig. Opthalmol. Vis. Sci. 2003,44, 3294–3300. [CrossRef] [PubMed] 2. Bailey, I.L.; Lovie, J.E. New design principles for visual acuity letter charts. Am. J. Optom. Physiol. Opt. 1976 ,53, 740–745. [CrossRef] [PubMed] 3. Ferris, F.L., 3rd; Kassoff, A.; Bresnick, G.H.; Bailey, I. New visual acuity charts for clinical research. Am. J. Ophthalmol. 1982 , 94, 91–96. [CrossRef] 4. Koenig, S.; Tonagel, F.; Schiefer, U.; Bach, M.; Heinrich, S.P. Assessing visual acuity across five disease types: ETDRS charts are faster with clinical outcome comparable to Landolt Cs. Graefes Arch. Clin. Exp. Ophthalmol. 2014 ,252, 1093–1099. [CrossRef] [PubMed] 5. Solomon, S.D.; Goldberg, M.F. ETDRS Grading of Diabetic Retinopathy: Still the Gold Standard? Ophthalmic Res. 2019 , 62, 190–195 . [CrossRef] [PubMed] 6. Shah, N.; Laidlaw, D.A.H.; Shah, S.P.; Sivasubramaniam, S.; Bunce, C.; Cousens, S. Computerized Repeating and Averaging Improve the Test-Retest Variability of ETDRS Visual Acuity Measurements: Implications for Sensitivity and Specificity. Investig. Opthalmol. Vis. Sci. 2011,52, 9397–9402. [CrossRef] [PubMed] 7. Rosser, D.A.; Murdoch, I.E.; Fitzke, F.W.; Laidlaw, D.A.H. Improving on ETDRS acuities: Design and results for a computerised thresholding device. Eye 2003,17, 701–706. [CrossRef] 8. Barrio, A.; Antona, B.; Puell, M.C. Repeatability of mesopic visual acuity measurements using high- and low-contrast ETDRS letter charts. Graefe’s Arch. Clin. Exp. Ophthalmol. 2015,253, 791–795. [CrossRef] 9. Chaikitmongkol, V.; Nanegrungsunk, O.; Patikulsila, D.; Ruamviboonsuk, P.; Bressler, N.M. Repeatability and Agreement of Visual Acuity Using the ETDRS Number Chart, Landolt C Chart, or ETDRS Alphabet Chart in Eyes With or Without Sight-Threatening Diseases. JAMA Ophthalmol. 2018,136, 286–290. [CrossRef] 10. Patel, P.J.; Chen, F.K.; Rubin, G.S.; Tufail, A. Intersession repeatability of visual acuity scores in age-related macular degeneration. Investig. Ophthalmol. Vis. Sci. 2008,49, 4347–4352. [CrossRef] 11. Lin, Z.; Wu, C.; Chen, X.; He, M. [Repeatability of ETDRS visual acuity measurement in children]. Yan Ke Xue Bao = Eye Sci. 2008 , 24, 48–52. 12. Ruamviboonsuk, P.; Tiensuwan, M.; Kunawut, C.; Masayaanon, P. Repeatability of an automated Landolt C test, compared with the early treatment of diabetic retinopathy study (ETDRS) chart testing. Am. J. Ophthalmol. 2003,136, 662–669. [CrossRef] 13. Kuo, H.-K.; Kuo, M.-T.; Tiong, I.-S.; Wu, P.-C.; Chen, Y.-J.; Chen, C.-H. Visual acuity as measured with Landolt C chart and Early Treatment of Diabetic Retinopathy Study (ETDRS) chart. Graefes Arch. Clin. Exp. Ophthalmol. 2011 ,249, 601–605. [CrossRef] [PubMed] 14. Falkenstein, I.A.; Cochran, D.E.; Azen, S.P.; Dustin, L.; Tammewar, A.M.; Kozak, I.; Freeman, W.R. Comparison of Visual Acuity in Macular Degeneration Patients Measured with Snellen and Early Treatment Diabetic Retinopathy Study Charts. Ophthalmology 2008,115, 319–323. [CrossRef] 15. Kaiser, P.K. Prospective evaluation of visual acuity assessment: A comparison of snellen versus ETDRS charts in clinical practice (An AOS Thesis). Trans. Am. Ophthalmol. Soc. 2009,107, 311–324. 16. Jolly, J.K.; Juenemann, K.; Boagey, H.; Nadsady, M.; Bridge, H.; Maclaren, R.E. Validation of electronic visual acuity (EVA) measurement against standardised ETDRS charts in patients with visual field loss from inherited retinal degenerations. Br. J. Ophthalmol. 2020,104, 924–931. [CrossRef] [PubMed] 17. Beck, R.W.; Moke, P.S.; Turpin, A.H.; Ferris, F.L.; SanGiovanni, J.P.; Johnson, C.A.; Birch, E.E.; Chandler, D.L.; Cox, T.A.; Blair, R.C.; et al. A computerized method of visual acuity testing: Adaptation of the early treatment of diabetic retinopathy study testing protocol. Am. J. Ophthalmol. 2003,135, 194–205. [CrossRef] Int. J. Environ. Res. Public Health 2021,18, 7876 8 of 8 18. Cotter, S.A.; Chu, R.H.; Chandler, D.L.; Beck, R.W.; Holmes, J.M.; Rice, M.L.; Hertle, R.W.; Birch, E.E.; Moke, P.S. Reliability of the electronic early treatment diabetic retinopathy study testing protocol in children 7 to. Am. J. Ophthalmol. 2003 ,136, 655–661. [CrossRef] 19. Laidlaw, D.A.H.; Tailor, V.; Shah, N.; Atamian, S.; Harcourt, C. Validation of a computerised logMAR visual acuity measurement system (COMPlog): Comparison with ETDRS and the electronic ETDRS testing algorithm in adults and amblyopic children. Br. J. Ophthalmol. 2008,92, 241–244. [CrossRef] 20. Turner, D.; Schünemann, H.J.; Griffith, L.E.; Beaton, D.E.; Griffiths, A.M.; Critch, J.N.; Guyatt, G.H. The minimal detectable change cannot reliably replace the minimal important difference. J. Clin. Epidemiol. 2010,63, 28–36. [CrossRef] [PubMed] 21. Lam, A.K.; Tong, C.; Tse, J.; Yu, M. Repeatability of near visual acuity measurement at high and low contrast. Clin. Exp. Optom. 2008,91, 447–452. [CrossRef] 22. Koo, T.K.; Li, M.Y. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J. Chiropr. Med. 2016,15, 155–163. [CrossRef] 23. Landis, J.R.; Koch, G.G. The measurement of observer agreement for categorical data. Biometrics 1977,33, 159–174. [CrossRef] 24. Weir, J.P. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J. Strength Cond. Res. 2005 , 19, 231–240. [CrossRef] [PubMed] 25. Bland, J.M.; Altman, D.G. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986 , 1, 307–310. [CrossRef] 26. Camparini, M.; Cassinari, P.; Ferrigno, L.; Macaluso, C. ETDRS-fast: Implementing psychophysical adaptive methods to standardized visual acuity measurement with ETDRS charts. Investig. Ophthalmol. Vis. Sci. 2001,42, 1226–1231. 27. Lee, H.S.; Park, S.W.; Chung, H.K. The Korean version of relative and absolute reliability of gait and balance assessment tools for patients with dementia in day care center and nursing home. J. Phys. Ther. Sci. 2017,29, 1934–1939. [CrossRef] 28. Pang, Y.; Sparschu, L.; Nylin, E. Validation of an automated-ETDRS near and intermediate visual acuity measurement. Clin. Exp. Optom. 2020,103, 663–667. [CrossRef] 29. Yu, H.J.; Kaiser, P.K.; Zamora, D.; Bocanegra, M.; Cone, C.; Brown, D.M.; Sadda, S.R.; Wykoff, C.C. Visual Acuity Variability: Comparing Discrepancies between Snellen and ETDRS Measurements among Subjects Entering Prospective Trials. Ophthalmol. Retin. 2021,5, 224–233. [CrossRef] [PubMed]