Poor Reproducibility in the Evaluation of Paranasal Sinus X-Rays in Chronic Rhinosinusitis

Open Journal of Radiology, 2017, 7, 23-34 http://www.scirp.org/journal/ojrad ISSN Online: 2164-3032 ISSN Print: 2164-3024 DOI: 10.4236/ojrad.2017.71003 March 7, 2017 Poor Reproducibility in the Evaluation of Paranasal Sinus X-Rays in Chronic Rhinosinusitis A. Luukkainen1,2*, E. Terna1*, J. Numminen3, A. Markkola4, P. Dastidar5, J. Jarnstedt5, H. Huhtala6, M. Karjalainen3, K. Blomgren7, P. Kauppi8, M. Rautiainen3,9, S. Toppila-Salmi1,8 1Transplantation Laboratory, Haartman Institute, University of Helsinki, Helsinki, Finland 2Department of Otolaryngology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore 3Department of Ear and Oral Diseases, Tampere University Hospital, Tampere, Finland 4University of Helsinki and HUS Imaging, Helsinki, Finland 5Medical Imaging Centre, Department of Radiology, Tampere University Hospital, Tampere, Finland 6School of Health Sciences, University of Tampere, Tampere, Finland 7Department of Otorhinolaryngology, University of Helsinki and Helsinki University Hospital, Helsinki, Finland 8Department of Allergy, University of Helsinki and Helsinki University Hospital, Helsinki, Finland 9Department of Otorhinolaryngology, University of Tampere, Tampere, Finland Abstract Objective: The aim of this study was to evaluate intraand inter-observer r eproducibility of sinus xrays in comparison to sinus computed tomography (CT) in chronic rhinosinusitis (CRS) patients. Methods: This was a prospe ctive controlled study for which 14 adult CRS patients were recruited. Patients underwent a sinus multi-detector CT scan as well as additional sinus xrays at the same time. Symptom interview and skin prick tests were performed. Lund-Mackay (LM) scores and 43 other findings in p aranasal sinuses were analyzed by three blinded observers from CT-scans and xrays. We compared agreement between sinus CT and x-rays (intraobserver reproducibility) and between three observers (interobserver reproducibility) by Cohen’s kappa. Results : In at least 90% of the cases, the status of 47/49 structures was detec table in CT scans, whereas the status of only 8/49 structures was detectable in x-rays. The majority of the 25 visualized structures had poor intraobserver and inter-observer reproducibility. Conclusion : Only a few structures can be visualized in paranasal sinus x-rays and compared to paranasal sinus CT - scans, their reproducibility is poor. Our results strongly support the current consensus of radiation dose reduction by limiting the number of x-rays. *The authors contributed equally to this work. How to cite this paper: Luukkainen, A., Terna, E ., Numminen, J., Markkola, A., Dastidar, P ., Jarnstedt, J., Huhtala, H., Karjalainen, M ., Blomgren, K., Kauppi, P., Rautiainen , M. and Toppila-Salmi, S. (2017 ) Poor Reproducibility in the Evalu a tion of Paranasal Sinus X -Rays in Chronic Rhinosinusitis . Open Journal of Radiology , 7 , 2334 . https://doi.org/10.4236/ojrad.2017.71003 Received: January 24, 2017 Accepted: March 4, 2017 Published: March 7, 2017 Copyright © 201 7 by authors and Scientific Research Publishing Inc. This work is licensed under the Creative Commons Attribution International License (CC BY 4.0). http://creativecommons.org/licenses/by/4.0/ Open Access A. Luukkainen et al. 24 Keywords Sinusitis, Paranasal Sinus, Computed Tomography, Magnetic Resonance Imaging, X-Ray 1. Introduction Chronic rhinosinusitis (CRS) is a multifactorial and variable disease with a prevalence of 10.9% [1]. Computed tomography (CT) scans and/or nasal endoscopy are the recommended imaging modality for CRS [1]. Correlation between sinonasal symptoms and endoscopic or radiologic signs is poor [2]. The main findings of CRS are mucosal changes within the ostiomeatal complex and/or sinuses [1]. Paranasal sinus anatomical variants are very common and several critical anatomical structures (such as big vessels, orbit and central nervous system) are closely located to the sinonasal surgical area [3] [4]. CRS surgery has improved many patients’ quality of life with hard to treat CRS [1]. Imaging of the nose and paranasal sinuses has progressed rapidly during the past decade. CT can demonstrate sinus anatomy specifically as well as anatomical variants and other important structures, when preparing for surgery or evaluating the cause of long lasting sinus symptoms. X-rays are not recommended for CRS imaging [2]. A simple radiogram of the nose and paranasal sinuses carries a radiation dose of 0.03 mSv (four days of natural background radiation). Although x-rays are inferior to CT in detecting bony structures and mucosal changes, little comparative data exists on the agreement between sinus CT and x-rays [2]. Despite the novel low dose sinus CT scan modalities, the number of performed sinus x-rays is high in Finland (13.1 per 1000 inhabitants in 2011, according to the statistics of the Radiation and Nuclear Safety Authority) [5]. The aim of this prospective controlled study was to evaluate intraand inter-observer reproducibility of sinus x-rays in comparison to sinus CT, in order to contribute to the need for reduction of unnecessary radiation. 2. Materials and Methods 2.1. Ethical Consideration The study was approved by the ethics committee of the Pirkanmaa Hospital District (no 96032) and was conducted in accordance with the Helsinki Declaration of 1975, as revised in 1983. Written informed consent was obtained from each participant. Volunteer patients were exposed to an extra radiation dose of 0.02 years (6 days of natural background radiation in Finland). 2.2. Patients This study was carried out in the Department of Otorhinolaryngology, at Tampere University Hospital, Finland from 2006 to 2015. A random sample of 14 A. Luukkainen et al. 25 adult CRS patients, who also had a history of AR and requiring sinus CT scans during 2007-2011, were enrolled. 3 (21%) out of 14 patients also had concomitant asthma. Having another severe disease was an exclusion criterion. 2 (85.7%) out of 14 patients reported as having other diseases (one had arrhythmia, one had resolved melanoma). None of the patients reported using regularly other medication than those due to inflammatory airway diseases. There was no patient record report of any other diseases, nor of psychiatric/psychologic disorders. Patient data was collected from medical records and by a questionnaire at the time of sinus CT scans, as previously described [6]. Follow-up data was collected from patient records of the Tampere University Hospital or Tampere City Hospital in 2015. The median (min-max) follow up time was 6.0 (0 - 8) years after sinus CT scans were performed. None of the subjects had undergone aspirin desensitization, allergen immunotherapy or anti IgE therapy prior to or during sinus CT scans or during follow-up. 2.3. CT Scans Patients underwent routine sinus multiple detector CT scans for clinical purposes. Two different CT scanners were used: GE LightSpeed 16 (GE Healthcare, Milwaukee, Wisconsin) and Philips Brilliance 64 (Philips, Best, Netherlands). Patients were scanned in a supine position with a kilovoltage of 120 kV and a milliampere second of 100 mAs. With the GE scanner, slice thickness was 0.625 mm with coronal reconstructions at 1.5 mm and a radiation dose of 0.8 mSv. With the Philips scanner, slice thickness was 0.9 mm with coronal reconstructions at 0.9 mm and a radiation dose of 0.9 mSv. Both were three dimensional (3D) in nature without any gaps. In all cases, imaging was performed using a bone filter technique. Scans covered the entire sinonasal area in both axial and coronal directions, starting from the nasal tip and ending at the posterior wall of the sphenoid sinuses. 2.4. X-Rays X-rays were performed at the same time as sinus CT scans. Two X-ray projections, Waters and PA-projection (Caldwell), were taken with Philips Pendo Diagnost (Philips, Best, Netherlands) skull unit in a sitting position. Imaging parameters were 85 kV, 12 mAs with Waters, and in PA-projection 85 kV and 8 mAs. The images were captured using photostimulable phosphor plates, pixel size 0.1 mm and read with AGFA CR 25 CR (Agfa-Gevaert N. V. Mortsel, Belgium) system. The radiation dose of x-rays with two projections was 0.06 mSv. 2.5. Evaluation of CT Scans and X-Rays CT scans and x-rays were observed by three independent observers blinded to each other and to patient history data: an experienced head and neck radiologist (AM), an experienced Ear Nose Throat (ENT)-and rhinosurgeon (JN), and a fifth year ENT resident (ST-S). Examination of the same patient’s images took place at least a week apart. The three observers filled a 49-item form of sinonasal A. Luukkainen et al. 26 structures from both CT-scan and x-ray bilaterally for each patient (Table 3). Each structure listed in the form could be scored by 2 - 5 different listed choices. Before evaluation of the CT scans, all choices were discussed between observers. Before this study, the observers had participated a pilot study with 15 CT scans [6]. The radiologist did not respond to the question: “Need for septoplasty”. 2.6. Data Analysis Statistical analysis was carried out by SPSS Base 15.0 Statistical Software Package (SPSS Inc., Chicago, IL, USA). Cohen’s kappa was used to compare the degree of agreement between CT scans and x-rays (e.g. intra-observer agreement); and the inter-observer agreement of x-rays. The calculation is based on the difference between how much agreement is actually present compared to how much agreement would be expected to be present by chance alone. The established interpretation of Kappa-value is classified into 6 subgroups: Poor < 0.2, Fair 0.21 - 0.4, Moderate 0.41 - 0.6, Good 0.61 - 0.8 and Very Good 0.81 - 1.0. A value under zero means that the agreement is worse than that by chance [7]. Two-tailed P-values of <0.05 were considered statistically significant. 3. Results 3.1. Patient Characteristics Patient characteristics are shown in Table 1. The median (min-max) age was 36.9 (20.0 - 54.3) years. 57.1% of patients underwent sinonasal operation within a year after the CT scans (Table 1). 2 (14.3%) of the patients reported suffering from diseases besides CRS ± AR ± asthma with regular need for medication. One of the patient reported having melanoma and the other patient cardiac arrhythmia. 3.2. Visualized Structures in X-Rays The number of visualized structures in x-rays was 25, e.g. the structures that were visualized in at least one patient´s paranasal sinus x-rays, whereas the number of visualized structures in at least one patient’s CT scan was 49 (Table 2). The visualized 25/49 structures in x-rays were Lund-Mackay (LM)-scores (except ostiomeatal unit), mucosa and size of paranasal sinuses, mucosa of inferior and middle turbinates, nasal mucosal oedema and septal abnormalities (Table 2). Yet, reliability to even detect these structures in most cases of x-rays was poor (Table 2). In at least 90% of the cases, the status of 47/49 structures was detectable in CT scans, whereas for x-rays it was only 8/49 (Table 2). Several diagnostically and surgically important structures were not visible in x-rays, such as ostiomeatal complex, insertion of the uncinate process, lamina papyrachea and anterior ethmoid artery (Table 2). 3.3. Intra-Observer Agreement We compared the degree of agreement between x-rays and CT scans of the 25 structures that were visualized in x-rays. In general, the intra-observer agree- A. Luukkainen et al. 27 Table 1. Characteristics of the patients. Chronic rhinosinusitis patients n = 14 % Gender Male 2 14.3 Female 12 85.7 Age <45 years 11 78.6 ≥45 years 3 21.4 Smoking No 9 64.3 Ex 3 21.4 Current 1 7.1 Unknown 1 7.1 Allergic rhinitis No 2 14.3 Yes 11 78.6 Unknown 1 7.1 SPT positivity No 3 21.4 Only pollen(s) 5 35.7 Only animal dander(s) 1 7.1 Multiple allergen types 5 35.7 Asthma No 10 71.4 Yes 3 21.4 Unknown 1 7.1 Nasal polyps No 12 85.7 Yes 1 7.1 Unknown 1 7.1 AERD No 13 92.9 Yes 0 0.0 Unknown 1 7.1 A. Luukkainen et al. 28 Continued Other diseases2 No 12 85.7 Yes 2 14.3 Current use of intranasal corticosteroids No 1 7.1 Yes 12 85.7 Unknown 1 7.1 ≥1 peroral corticosteroid course(s) during the past 1 year No 13 92.9 Yes 0 0.0 Unknown 1 7.1 Previous sinonasal operation(s) No 11 78.6 Yes 2 14.3 Unknown 1 7.1 Radiological signs in CT scans of previous sinus operation No 11 78.6 Yes 3 21.4 Total Lund-Mackay score of CT scans 0 - 3 7 50.0 4 - 12 7 50.0 13 - 24 0 0.0 Sinonasal operation performed within a year after the CT scans No 5 35.7 Yes 8 57.1 Unknown 1 7.1 ≥1 sinonasal operation(s) during the 6-year follow-up No 12 85.7 Yes 1 7.1 Unknown 1 7.1 Number of antibiotic courses during the past 2 years, median (min-max) 6.0 (2 - 15) Duration of symptoms in years, median (min-max) 2.5 (0.3 - 25.0) Current symptoms by VAS, mean (min–max) Sense of smell 3.7 (0.0 - 7.3) Post-nasal drip 5.5 (1.0 - 8.1) Obstruction 6.1 (0.7 - 9.8) Facial pain 5.5 (2.4 - 9.9) Abbreviations: SPT = skin prick test; AERD = patient-reported aspirin exacerbated respiratory disease; CT = computed tomography; VAS= visual analogue scale (0 - 10). 1At least one peroral corticosteroid treatment during the last 12 months. 2Self-reported and patient-record information. A. Luukkainen et al. 29 Table 2. The list of visualized or non-visualized sinonasal structures of paranasal sinus x-rays. Visualized in x-rays Non-visualized in x-rays Non-detectable cases% Non-detectable cases% CT-scans x-rays CT-scans x-rays Atrophy-normal hypertrophy of inferior turbinate 0.0 28.6 Anterior ethmoidal artery 25.0 100.0 Atrophy-normal hypertrophy of middle turbinate 3.6 50.0 Atrophy-normal hypertrophy of superior turbinate 21.4 100.0 Hypoplasia/normal/hyperplasia of anterior ethmoidal sinus 0.0 14.3 Contact to middle turbinate of orbital lamina of ethmoidal bone 0.0 100.0 Hypoplasia/normal/hyperplasia of frontal sinus 0.0 14.3 Frontal recess 0.0 100.0 Hypoplasia/normal/hyperplasia of maxillary sinus 0.0 14.3 Infraorbital cell 0.0 100.0 Hypoplasia/normal/hyperplasia of posterior ethmoidal sinus 0.0 14.3 Keros classification 0.0 100.0 Hypoplasia/normal/hyperplasia of sphenoid sinus 0.0 14.3 Lund-Mackay ostiomeatal unit 0.0 100.0 Lund-Mackay anterior ethmoidal sinus 0.0 0.0 Mucosa of pneumatized middle turbinate 0.0 100.0 Lund-Mackay frontal sinus 0.0 7.1 Mucosa of pneumatized superior turbinate 0.0 100.0 Lund-Mackay maxillary sinus 0.0 0.0 OMC region, accessory maxillary sinus ostium 7.1 100.0 Lund-Mackay posterior ethmoidal sinus 0.0 7.1 OMC region, hiatus 0.0 100.0 Lund-Mackay sphenoid sinus 0.0 3.6 OMC region, infundibulum 0.0 100.0 Mucosa of nasal cavity (extent of edema) 0.0 14.3 OMC region, maxillary antrum 0.0 100.0 Mucosa of nasal cavity (normal-polypous) 0.0 14.3 OMC region, pneumatized superior attachment of uncinate process 0.0 100.0 Need for septoplasty1 0.0 35.7 OMC region, prominent ethmoid bulla 0.0 100.0 Septal deviation obstructing middle meatus 0.0 32.1 OMC region, superior attachment of uncinate process 7.1 100.0 Septum, crest 0.0 17.9 Optic nerve 0.0 100.0 Septum deviation 0.0 21.4 Sphenoethmoidal recess 3.6 100.0 Septum turbinate 0.0 32.1 Paradoxical middle turbinate 0.0 100.0 Septum, spur 0.0 21.4 Paradoxical superior turbinate 7.1 100.0 Sinus mucosal abnormalities of anterior ethmoidal sinus 0.0 7.1 Pneumatized middle turbinate 0.0 100.0 Sinus mucosal abnormalities of frontal sinus 0.0 17.9 Pneumatized superior turbinate 7.1 100.0 Sinus mucosal abnormalities of maxillary sinus 0.0 3.6 Previous sinus surgery performed 0.0 100.0 Sinus mucosal abnormalities of posterior ethmoidal sinus 0.0 7.1 Thickness of orbital lamina of ethmoidal bone 0.0 100.0 Sinus mucosal abnormalities of sphenoid sinus 0.0 10.7 The CT scans and x-rays were taken from 14 patients with chronic rhinosinusitis symptoms. Each patient underwent CT scans and x-rays at the same time. The columns show in alphabetical order the evaluated 49 structures from paranasal sinus CT-scans and x-rays. Visualized in x-rays = the 25 structures that were visualized in at least one patient’s paranasal sinus x-rays (left column); Non-visualized in x-rays = the 24 structures that were non-visualized in paranasal sinus x-rays of all cases (right column); OMC = Ostiomeatal complex; Not detectable = the percentage of the observer´s responses “The status of the structure is not detectable” of both sides. 1Evaluated by the ENT surgeon. Other structures evaluated by the radiologist. A. Luukkainen et al. 30 ment was poor (kappa < 0.2) in the majority of structures, such as LM scores. Moderate and good agreement was only achieved for gross anatomical structures on the right hand-side only, concerning respectively, nasal septum deviation and size of the frontal sinus (Table 3). Fair or poor agreement was observed for the rest of the structures. 3.4. Inter-Observer Agreement The 25 structures that were visualized in x-rays were evaluated by a radiologist, an ENT surgeon and an ENT resident. The inter-observer agreement between radiologist and ENT resident for x-rays was poor (kappa ≤ 0.02) in 88% of the structures and fair (kappa 0.21 - 0.4) for the rest (12%) of the structures. The agreement between radiologist and ENT surgeon for x-rays was poor or fair in 80% of the structures and the agreement between ENT surgeon and ENT resident was poor or fair in 92% of the structures. 4. Discussion This study was carried out to evaluate intraand inter-observer reproducibility of sinus x-rays in comparison to sinus CT scans. When this study was started, multidetector CT scans had a high radiation dose (in average 0.9 mSv) and x-rays were still relative widely used due to a clearly smaller radiation dose (in average 0.03 mSV per image). After this, low-dose CT scans (such as cone beam CT scans) have emerged (radiation dose between 0.08 - 0.27 mSv) and hence have largely replaced both sinus x-rays and high-dose sinus CT scans [8]. The number of performed sinus x-rays is still high in Finland [5]. Consequently, this study could contribute to the reduction of unnecessary radiation. Our main finding was that a small proportion of structures can be visualized in x-rays; and x-ray evaluations have poor reproducibility. CRS specific changes that should be observed in CT-scans including degree of opacification of the paranasal sinuses and/or obstruction of the ostiomeatal complex cannot be visualized or only poorly in simple x-rays, making CRS diagnosis unreliable [1]. Similarly, others found in a small study that engorged turbinates and opaque nasal fossa could be observed from paranasal sinus x-rays. In the x-rays of 19.7% of patients with nasal and/or paranasal symptoms during at least 8 weeks, no changes were observable [9]. Intra-observer agreement was fair to poor regarding most of the 25 structures that could be visualized in plain x-rays. Similarly to us, others report in a study with 47 patients with acute rhinosinusitis that plain sinus radiograms have a low sensitivity for detecting sinus inflammatory changes in other paranasal sinuses besides the maxillary sinus, in comparison to CT-scans [10]. Inter-observer agreement was poor for the majority of structures that could be visualized by the radiologist, ENT surgeon and ENT resident. Very good interobserver agreement was only achieved in regard to structures that could not be visualized, between ENT resident and surgeon. A. Luukkainen et al. 31 Table 3. Comparison of the degree of intra-observer agreement from paranasal sinus x-rays and computed tomography (CT)-scans. Computed tomography scans vs. x-rays Right kappa P Left kappa P Lund-Mackay frontal sinus −0.057 <0.001 −0.050 <0.001 Lund-Mackay anterior ethmoidal sinus 0.109 1.00 0.243 0.357 Lund-Mackay posterior ethmoidal sinus 0.155 <0.001 0.200 <0.001 Lund-Mackay sphenoid sinus −0.148 <0.001 0.323 0.286 Lund-Mackay maxillary sinus 0.125 0.560 0.087 1.00 Sinus mucosal abnormalities of frontal sinus −0.120 1.00 0.192 1.00 Sinus mucosal abnormalities of anterior ethmoidal sinus 0.155 <0.001 0.114 1.00 Sinus mucosal abnormalities of posterior ethmoidal sinus 0.142 <0.001 0.067 1.00 Sinus mucosal abnormalities of sphenoid sinus 0.058 <0.001 0.233 1.00 Sinus mucosal abnormalities of maxillary sinus 0.355 <0.001 0.339 <0.001 Hypoplasia/normal/hyperplasia of frontal sinus 0.650 <0.001 0.344 <0.001 Hypoplasia/normal/hyperplasia of anterior ethmoidal sinus 0.000 <0.001 0.000 <0.001 Hypoplasia/normal/hyperplasia of posterior ethmoidal sinus 0.000 <0.001 0.000 <0.001 Hypoplasia/normal/hyperplasia of sphenoid sinus 0.192 <0.001 0.208 <0.001 Hypoplasia/normal/hyperplasia of maxillary sinus 0.000 <0.001 0.000 <0.001 Need for septoplasty1 0.114 1.00 0.114 1.00 Septal deviation obstructing middle meatus 0.079 1.00 0.114 1.00 Septum turbinate 0.000 <0.001 0.000 <0.001 Septum deviation 0.462 <0.001 0.385 <0.001 Septum, crest 0.133 <0.001 −0.061 1.00 Septum, spur 0.023 <0.001 −0.094 <0.001 Atrophy-normal-hypertrophy of inferior turbinate 0.250 <0.001 0.381 .019 Atrophy-normal-hypertrophy of middle turbinate −0.083 0.143 −0.051 <0.001 Mucosa of nasal cavity (extent of edema) 0.030 0.250 −0.054 <0.001 Mucosa of nasal cavity (normal-polypous) 0.175 <0.001 −0.054 1.00 Agreement kappa Poor ≤0.2 Fair 0.21 - 0.4 Moderate 0.41 - 0.6 Good 0.61 - 0.8 Very good 0.81 - 1.0 The CT scans and x-rays were taken from 14 patients with chronic rhinosinusitis symptoms. Each patient underwent CT scans and x-rays at the same time. Agreement is presented only of the 25 structures that were detected in x-rays (Table 2). The 24 structures that were non-detectable in x-rays have been withdrawn from evaluation. The order of the structures is the same as they were in the evaluation form. Structures with substantial to almost perfect agreement level by Kappa-coefficient. 1Evaluated by the ENT surgeon. Other structures evaluated by the radiologist.