A novel cloud-deployed data pipeline for cervical spine fracture detection in 3D CT images

A novel cloud-deployed data pipeline for cervical spine fracture detection in 3D CT images Fatah Bouchebbah1, Rayane Aggoune2, and Chahinez Amrane2 1LIMED Laboratory, Faculty of Exact Sciences, University of Bejaia, 06000 Bejaia, Algeria, [email protected] 2Department of Computer Science, Faculty of Exact Sciences, University of Bejaia, 06000 Bejaia, Algeria Abstract A cervical spine fracture is a serious medical emergency that can lead to permanent paralysis or even death. In addition, rapid and accurate detection of such fractures is essential for optimal patient care. However, manually interpreting computed tomography (CT) images to detect possible fractures in the cervical spine, as traditionally done, is time-consuming and requires the experience of experienced radiologists. Fortunately, the integration of artificial intelligence and cloud computing technologies in healthcare has the potential to revolutionize cervical spine fracture detection by providing fast, accurate, and automated solutions. In this context, we present a couple of contributions in this paper. In the first contribution, we develop a new multifaceted computational pipeline based on the combination of Faster R-CNN and Next-ViT models to detect fractures within the cervical spine. The new computational pipeline has been trained and evaluated on the large RSNA public dataset containing cervical spine CT scans. Hence, the new system has achieved encouraging results. Furthermore, the new proposed data pipeline’s ability to detect subtle and complex fractures has motivated us to integrate it in a cloud-based architecture that we present as a second contribution in the setting of this paper. The proposed cloud-based architecture has the potential to be used as a distant clinical decision-support tool to help radiologists identify fractures quickly and reliably, and to be continuously improved through a feedback mechanism. Keywords: Fracture detection, Cervical spine, Faster R-CNN, Next-ViT model; Cloud-based architecture. 1 Introduction Cervical spine fractures, often caused by accidents or falls, pose a challenging medical dilemma. These kinds of injuries, which occur in a delicate part of the human skeletal structure, require swift and precise identification to prevent serious neurological damage. Moreover, according to Savage et al. [5], more than 1.5 million people in the United States alone suffer spine fractures every year, a significant proportion of which affect the delicate architecture of the cervical spine. For the elderly and those with pre-existing conditions like osteoporosis, such fractures can be fatal. The situation is further complicated by the fact that cervical spine fractures often require immediate attention, yet rapid and accurate diagnosis remains elusive. Fortunately, in the current age of rapid technological advancement, Artificial Intelligence (AI) and cloud computing are making profound inroads into various domains. In fact, as reported by Voter et al. [9], the combination of AI’s cutting-edge technologies and the prowess characteristics of cloud-based systems offer innovative solutions featured with computational capabilities and abilities to decipher intricate medical data patterns. Especially when it comes to challenging medical situations like cervical spine fractures which are marked by complex diagnostics and the potential for severe neurological consequences if mishandled. Our main objective throughout this paper is to improve patient outcomes and to assist healthcare professionals by presenting an accurate, rapid, automatic, secured, and continuously improving advanced system for cervical spine fracture detection. The system relies on a combination of deep learning algorithms like Faster R-CNN and Next-ViT, that have gained significant attention in the computer vision community due to their recent remarkable state-of-the-art performances, as well as cloud computing and human expertise. The rest of this paper is organized as follows: Section 2exhibits a state of the art of the main works established in the literature on the cervical spine fracture detection problem. Section 3presents in details our first contribution in this paper, which is a new multifaceted computational pipeline based on a 20 combination of Faster R-CNN and Next-ViT. Section 4describes and discusses our second contribution, that is a proposed cloud-based architecture deployed in Google Cloud Platform that offers an end-toend cervical spine fracture detection service to medical professionals as well as to patients. Section 5is dedicated to testing and evaluating the whole presented system by comparing it to an existing system in the literature by considering the RSNA 2022 Cervical Spine Fracture Detection dataset. The paper ends with a conclusion, given in Section 6, summarizing the main of the contributions while highlighting some limitations and interesting perspectives worth to consider to improve the work. 2 Related work As a research problem, detection of injuries, and especially fractures, in cervical spine has been a topic of interest lately. Therefore, a variety of methods that are based on different techniques of machine learning have been presented in the literature as attempts to find solutions to the problem. Among the exhibited methods, the approached based on deep learning have demonstrated encouraging features. However, they still require several improvements to be effective enough when integrated in clinical routines. For instance, Small et al. [7] have investigated the application of a Convolutional Neural Network (CNN) architecture that was developed by Aidoc, known as FDA-approved CNN, for the detection of cervical spine fractures. The findings of the study have emphasized the potential of the tested model to improve cervical fracture detection. However, they have also acknowledged certain limitations of CNNs when they are applied to detect fractures. Notably, CNNs may struggle to detect areas of gross bony translation and fractures characterized by distraction rather than linear bony features. Moreover, Merali et al. [3] have conducted a study with the objective of developing a deep-learning model capable of detecting cervical spinal cord compression in patients diagnosed with Degenerative Cervical Myelopathy (DCM) in T2-weighted MRI scans. For this aim, the authors have employed ResNet-50 architecture and have tested multiple network configurations to determine a suitable setup for the used dataset. The used architecture, with a proper settings, has achieved an encouraging accuracy, however the results in terms of specificity have been relatively low. Furthermore, Shaolong et al. [6] have presented a comprehensive investigation into the utilization of deep learning techniques applied to MRI scans for the detection and classification of lesions associated with cervical spinal cord diseases. For this reason, the researchers have employed Faster R-CNN (Region Convolutional Neural Network) approach, which combines a backbone convolutional feature extractor utilizing both ResNet-50 and VGG-16 networks. This integration of latter networks yielded promising results in terms of prediction accuracy and speed for lesion detection and recognition within cervical spinal cord MRIs. In addition, Tuan et al. [8] have conducted an extensive investigation to develop an efficient and accurate method for the early detection and localization of spine fractures. Through their experimentation, they have explored multiple machine learning models and hence have identified a two-stage approach utilizing Deep CNN (DCNN) with RNN and attention layers. The presented approach have had commendable performance in terms of running time, resource utilization, and accuracy. In addition, Salehinejad et al. [4] have introduced a DCNN with a Bidirectional Long Short-Term Memory (BLSTM) layer as the baseline architecture, that has been specifically tailored for an automated detection of cervical spine fractures in CT axial images. The performed study has shaded light on the potential of deep learning techniques in fracture detection and has provided a foundation for future investigations aimed at refining and advancing automated fracture detection algorithms in clinical settings. Unfortunately, the approaches presented by Small et al. [7], Merali et al. [3], Shaolong et al. [6], Tuan et al. [8], and Salehinejad et al. [4] remain beneficial to the health professionals who own the programs only. Therefore, they are restricted to a local use solely. Recently, Showmick Guha et al. [1] have studied the performance of a variety of CNN models adapted to cervical spine fracture detection using transfer-learning. The adapted methods include MobileNetV2, InceptionV3, and Resnet50V2. Performed tests have revealed a superiority of MobileNetV2, which was trained with data augmentation technique, over the other approaches. Consequently, the model has been deployed for clinical use in the form of an Android application for smartphones. However, this kind of deployment is not quietly proper to a clinical use, unless the attention is restricted to personal or emergency use. On the other hand, many matters like constrained resources and model evolution need to be resolved for a better usage. 21 3D CT image Volumetric image slicing Data augmentation Object detection and region cropping Classification via Next-ViT Decision Figure 1: A representation of the proposed data pipeline. 3 Proposed multifaceted computational pipeline for the detection cervical spine fractures In light of the importance and challenge of cervical spine fractures detection, we synergies cutting-edge algorithms to present a new data pipeline specifically designed for CT images analysis. The proposed data pipeline leverages the proven capabilities of Faster R-CNN for object localization and Next-ViT for image classification, and adds insight from attention maps to focus on the region(s) of interest within an analyzed CT image (i.e. the eventual fracture(s)). Explicitly, the presented data pipeline is essentially composed of four stages, namely: volumetric image slicing,data augmentation to train Faster R-CNN, object localization using Faster R-CNN and image cropping, and finally classification via Next-ViT. A schematic representation of the proposed framework is presented in Fig. 1and necessary details and descriptions about the proposed data pipeline are given in the subsections below. 3.1 Volumetric image slicing Cervical CT scans are 3D images rich in anatomical information. However, these latter are quite complex to process by computerized approaches due to their excessive amount of data. The question raised here is therefore how to exploit the richness of these data without getting trapped in computational bottlenecks? Herein lies the critical importance of the image-slicing process. In fact, it transforms an intricate 3D spatial problem into a more manageable 2D problem space. Thus, slicing serves, on one hand, as a strategic maneuver to reduce computational cost; on the other hand, it prepares the ground for expeditious and focused downstream data processing. Specifically, the produced 2D slices can be orientated to emphasize anatomical planes that are most relevant for the diagnosis of cervical spine fractures. This ensures to retrain the most pertinent and diagnostically relevant information in the slices. Furthermore, in this initial phase of the pipeline, slices are extracted from the original DICOM files of CT scans using a specific function in 512×512 pixels format. Which are later resized into 224×224 pixels format to match the input size expected by the Next-ViT model. In addition, windowing techniques are applied to the extracted 2D slices to enhance their contrast. The window width and level are set to 1800 and 400, respectively. 3.2 Data augmentation to train Faster R-CNN Data augmentation is a widely used technique to increase the size and diversity of the training datasets. This is especially important as Faster R-CNN object detection model requires a large amount of labelled data to be trained effectively. In the context of this study, the data augmentation is mainly performed 22 by using random horizontal flipping, which can help the model learn to detect objects from different perspectives. 3.3 Object detection and region cropping In this stage, Faster R-CNN is used to detect and isolate regions of interest. Specifically, we take advantage of Faster R-CNN’s ability to operate as a computerized lens, to scrupulously navigate through the 2D slice images to discern and delineate regions that house potential fracture sites within the cervical spine and to underscore their associated vertebra with bounding boxes. This act of object localization constructs a vital foundational tier, guiding the ensuing procedures in the pipeline, which are designated to further refine, dissect, and classify these pronounced areas suspected of fractures. The localized regions of interest are subsequently cropped from the rest of their associated slices to form small imagettes. Specifically, the aim of this phase is dual: firstly, to drastically curtail computational excess, and secondly, to concentrate the ensuing analysis on clinically pertinent regions. Explicitly, the sectors of the cervical spine believed to harbour fractures, as pinpointed by Faster R-CNN. 3.4 Classification via Next-ViT In this final stage, Next-ViT model is used to binary classify the imagettes previously produced to distinguish between those really containing fractures and those that are not. This model was selected for its unique set of attributes that align impeccably with our research goals. One of the standout qualities of Next-ViT is its data efficiency. The model demonstrates impressive performance even when subjected to small, annotated datasets. In addition, Next-ViT diverges from CNNs by incorporating self-attention mechanisms. These latter mechanisms excel at identifying complex spatial and contextual relationships within images, a feature invaluable for interpreting the complex imagery commonly found in cervical spine studies. Moreover, given the underwhelming results of our initial attempt to train a vision transformer from scratch, we have chosen to adopt a pre-trained Next-ViT architecture, which led to a marked improvement in our system’s efficacy. However, to better fit Next-ViT to our problem, we have performed a refinement training of the model, specifically using data augmentation techniques by applying simple transformations to the training dataset (i.e. rotation, scaling, and flipping). We hypothesize that these simple transformations assist the model in understanding underlying data patterns, thereby improving its learning capability. 4 Proposed cloud-based architecture for cervical spine fracture detection As a second contribution in this paper, we describe a robust and scalable cloud-based system that is dedicated to the detection of cervical spine fractures. The cloud infrastructure serves as the backbone supporting the entire multifaceted computational pipeline presented in Section 3and offers unique advantages both in terms of computational resources and data management. 4.1 Motivations and goals The presented architecture is motivated by several compelling incentives for coupling cloud computing and deep learning models in the arena of cervical spine fracture detection. The impetus for adopting a cloud-based approach originates from a critical need to address challenges in scalability, data integrity, and real-time analytics. Below are the main key motivations: 1. Superior diagnostic accuracy: Traditional diagnostic approaches, although useful, sometimes fail to identify complex or subtle fractures. The marriage of cloud-based computational power and well established deep learning models has the potential to usher in a new era of nuanced and precise diagnoses. 2. Operational efficiency: Utilizing the distributed computing power of the cloud alongside deep learning models that can efficiently parse large sets of image data enhances the operational efficiency of the diagnostic process. This could significantly reduce the time radiologists need to reach a diagnosis. 23 3. Scalability and adaptability: The inherent scalability of cloud infrastructure is well-suited for handling the voluminous medical imaging data generated daily. This removes the need for healthcare organizations to make significant investments in local computing resources. 4. Broadened access to advanced tools: Cloud-based systems democratize access to cutting-edge diagnostic technologies. This model allows healthcare providers, regardless of their size or location, to benefit from state-of-the-art tools without prohibitive upfront costs. 5. Augmentation of clinical decision-making: The synergy between cloud technology and deep learning models can act as a potent decision-support mechanism. It can provide preliminary evaluations that assist healthcare professionals in making timely and well-informed decisions. 6. Future-ready integration: The modular architecture of cloud-based systems makes them ripe for seamless integration with existing electronic health records. This offers the possibility for more integrated, collaborative approaches to healthcare delivery in the future. Thus, the integration of cloud computing and deep learning models in the detection of cervical spine fractures has the potential to surmount existing limitations, refine diagnostic protocols, democratize access to state-of-the-art technologies, and fundamentally transform clinical practices in this vital area of healthcare. 4.2 Description of the proposed cloud-based architecture The proposed architecture is designed to be deployed in Google Cloud Platform (GCP), integrating its services to offer an efficient end-to-end cervical spine fracture detection workflow. A general view of the proposed cloud-based architecture for cervical spine fracture detection is illustrated in Fig. 2. Initially, the overarching vision of crafting an integrated end-to-end diagnostic workflow for enhanced cervical spine fracture detection stemmed from comprehensive brainstorming sessions. Significantly, it was our deep dive into the vast capabilities of the Google Cloud Platform (GCP) that galvanized our alignment with this mission. Building upon this foundation, our hands played a pivotal role in the ensuing architectural design and execution phase. Furthermore, recognizing the paramount importance of data integrity, we have channelled significant efforts into devising an efficient automatic ingestion mechanism for CT scans. Simultaneously, with an acute awareness of the sensitive nature of medical data, we have championed the incorporation of a robust encryption protocol, ensuring that data remain secured. Transitioning from data acquisition, our focus then have gravitated towards the multi-layered data pipeline. Specifically, we have integrated in the proposed architecture the data pipeline elaborated in Section 3, that meticulously optimize the mechanisms of pre-processing, feature extraction, and fracture detection. On the other hand, we believe in the interdependent nexus between machine learning methods and human expertise and its capacity to offer better solutions, especially when they are combined appropriately. This conviction has led to the establishment of a systematic feedback loop, where the invaluable insights of medical professionals continuously enrich our cloud-based system. Through this mechanism, their diagnostic evaluations directly inform and steer the iterative enhancements of the integrated models in the proposed system. Moreover, with an ever-evolving medical landscape, we need to ensure that the used models in the architecture underwent consistent training sessions. By leveraging insights from the analytical database, our diagnostic algorithms remain at the cutting edge, always adaptive to the latest nuances in medical diagnostics. Beyond the technical realm, we endeavour to foster a culture of interdisciplinary collaboration. By orchestrating synergy between cloud experts, data scientists, and medical professionals, we strive to ensure that our collective expertise coalesced seamlessly. This unity of purpose and knowledge-sharing became instrumental in shaping our presented solution. In summary, witnessing the transformative potential of our architecture in the realm of medical diagnostics has been both a privilege and a testament to the collaborative prowess of our team. Our journey exemplifies the boundless possibilities that emerge when cloud computing and machine learning converge, especially in the ever-critical domain of healthcare. The amalgamation of GCP’s advanced services presents a promising horizon for medical diagnostics. While this overview provides a highlevel design, the actual implementation should be tailored according to specific requirements, ensuring 24 Epoch Precision Recall mAP0.5 80 0.9287 0.8857 0.9424 100 0.9687 0.9057 0.9724 Table 1: Performance metrics of the Faster R-CNN on the RSNA dataset. a balance between functionality, budget, and privacy concerns. Collaboration with cloud and domain experts is essential for the successful realization of such a system. Figure 2: A representation of the proposed cloud-based system. 5 Evaluation and discussion of the cervical spine fracture detection system The proposed multifaceted data pipeline has been trained and evaluated using the large RSNA public dataset containing cervical spine CT scans [2]. In the setting of this work, the images of the dataset was split into training (80%) and validation (20%) sets. The slices and their corresponding label files (.txt files) are then organized appropriately into separate directories for training and validation. Subsequently, we have downloaded Faster R-CNN’s code from TensorFlow, adjusted it and trained it to meet our purpose. Thus, the performance of Faster R-CNN on the used dataset is assessed using standard evaluation metrics, namely: precision,recall, and mean average precision at IoU (mAP50). The obtained results after 80 and 100 epochs are presented in Table 1. The yielded results showcase the model’s potential in both recognizing and pinpointing objects within images after 80 and 100 epochs. A summary of Faster R-CNN train loss metrics from one of the epochs during the model’s training phase are present in Table 2. The table summarizes important performance indicators and parameters that provide insights into the model’s training dynamics, namely : Loss, Loss Classifier, Loss Box Reg, Loss Objectness, and Loss RPN Box Reg. For visual illustration of the cropping operation results, we give in Fig. 3cropped images obtained from different slices. Concerning the classification stage of the multifaceted data pipeline, the implementation of Next-ViT requires setting appropriate values for model’s parameters. This is specifically done to insure a satisfying 25 Parameter Best value Averaged value Loss 0.1576 0.3236 Loss Classifier 0.0490 0.1163 Loss Box Reg 0.0900 0.1130 Loss Objectness 0.0088 0.0790 Loss RPN Box Reg 0.0040 0.0153 Table 2: Training Loss results. Figure 3: Illustrations of cropped vertebra. balance between computational efficiency and detail resolution to make the model highly applicable in clinical settings for which timely and accurate diagnosis is paramount. In the context of this work, we have considered the parameter tuning exhibited in Table 3. Parameter Value Patch size 16 ×16 Latent space dimension 192 Number of encoder blocks 12 Number of MLP heads 3 Total parameters ≈5.5M Table 3: Next-ViT model parameter tuning. Also, it is worth to note that we have applied other adaptations to Next-ViT to meet our specific needs. For instance, the output shape is printed and should be [16, 2] of shape. This is to say that for each input image, we get 2 values as output, corresponding to fracture and no fracture results respectively. In addition, to optimize the neural network, we have employed RAdam optimizer and used a learning rate of 0.001. Specifically, the value of 0.001 is considered a moderate choice, which is neither too high to cause instability nor too low to slow down the learning process. This value is often recommended for Adam and its variants like RAdam due to its effectiveness in a wide range of scenarios. To validate the robustness and effectiveness of Next-ViT model, we have used two metrics: accuracy and loss. Hence, the obtained validation results of the model on the RSNA 2022 Cervical Spine Fracture Detection dataset are shown in the graphs presented in Fig. 4. From the latter figure, it is easy to notice that the performance of the Next-ViT model improved through the epochs for both accuracy and loss validation metrics, until achieving a validation accuracy of 95.5% and a validation loss of 2%. This is particularly promising because it suggests that the Next-ViT could be used to develop a fast and accurate 26 AI-based system for cervical spine fracture detection. Such a system could be used to help radiologists identify fractures more quickly and reliably, and it could also be used to screen patients for suspected fractures in emergency settings. Figure 4: Validation results of Next-ViT. Moreover, a comparison of the work presented herein with the concurrent work of Showmick Guha et al. [1] that is recently exhibited in the literature is reported in Table 4. From the latter, it is easy to notice that the model of Showmick Guha et al. [1] presents the best accuracy currently. However, the proposed model is more subtitle to offer a superior diagnostic accuracy in the future. In fact, the exhibited architecture foresees continuous model refinement training by taking into consideration the capacity of accepting new unseen data as well as correcting feedback from experts who use the system. So, the two features guarantee a continuously improving diagnostic accuracy. Furthermore, despite the fact that the two works ensure a real time response, nevertheless, the proposed system is clearly more adapted to clinical routines considering the fact that it is deployed on the cloud. Hence, it offers a better scalability and adaptability in terms of resources, brocaded access to advanced tools, and future-ready integration compared to its concurrent work which is designed for miniaturized systems essentially made for a personal use. Feature Proposed work Showmick Guha et al. [1] Best accuracy 95,5 % 99.75 % Deployment Cloud Android application Real-time response Considered Considered Model refinement possibility Considered Not considered Data integrity Considered Not considered Storage and processing capacity High Very low Table 4: Comparison between the proposed work and a concurrent work according to few features. 6 Conclusion The confluence of AI, medical imaging, and cloud computing represents a promising avenue for revolutionizing the healthcare domain. In this setting, we have made a couple of contributions which are exhibited in this document. Mainly, we have introduced a new comprehensive computational data pipeline tailored for the detection of cervical spine fractures. Specifically, the proposed data pipeline is composed of four 27 stages, each of which fulfils a unique role to achieve high diagnostic precision and reliability. Furthermore, motivated by several key goals such as improving diagnostic accuracy, increasing scalability, and enhancing data security, we have exhibited, as our second main contribution, a new cloud-based system to extend the capabilities of our computational data pipeline. The new cloud-based architecture represents a paradigm shift in how cervical spine fractures can be detected and managed. The proposed cloud-based system not only streamlines the workflow but also allows for continuous improvement through real-time feedback mechanisms. Furthermore, the experimental study comprising the implementation, training, and validation of the presented comprehensive computational data pipeline over the RSNA 2022 Cervical Spine Fracture Detection dataset has shown an encouraging performance with regard to a concurrent work in the literature. While the findings of this paper are compelling, they raise several salient questions that could form the basis of future scholarly inquiry. These include prototyping the proposed cloud-based diagnostic system and its convenience to resource-constrained devices, as well as further refinements and improvements of the proposed multifaceted data pipeline by the integration visualization mechanisms or with the adoption of emerging artificial intelligence paradigms such as deep reinforcement learning and federated learning. Acknowledgment The authors are thankful to the anonymous reviewer for his valuable comments that helped to improve the paper. References [1] P. Showmick Guha, S. Arpa, and A. Md. A real-time deep learning approach for classifying cervical spine fractures. Healthcare Analytics, 4:100265, 2023. [2] H.M Lin, E. Colak, T. Richards, F.C. Kitamura, L.M. Prevedello, J. Talbott, R.L. Balland E. Gumeler, K.W. Yeom, M. Hamghalam, et al. The RSNA cervical spine fracture CT dataset. Radiology: Artificial Intelligence, 5:e230034, 2023. [3] Z. Merali, J. Wang, J.H. Badhiwala, C.D. Witiw, J.R. Wilson, and M.G. Fehlings. A deep learning model for detection of cervical spinal cord compression in mri scans. Scientific Reports, 11(1):14620, 2021. [4] H. Salehinejad, E. Ho, H.M. Lin, P. Crivellaro, and O. Samorodova. Deep sequential learning for cervical spine fracture detection in computed tomography imaging. IEEE Transactions on Medical Imaging, 40(6):1642–1652, 2021. [5] J.W. Savage, G.D. Schroeder, and P.A. Anderson. Vertebroplasty and kyphoplasty for the treatment of osteoporotic vertebral compression fractures. J Am Acad Orthop Surg, 22(10):653–664, Oct 2014. [6] M. Shaolong, H. Yang, C. Xiangjiu, and G. Rui. Faster rcnn-based detection of cervical spinal cord injury and disc degeneration. Medical Physics, 48(2):801–813, 2020. [7] J.E. Small, P. Osler, A.B. Paul, and M. Kunst. Ct cervical spine fracture detection using a convolutional neural network. Journal of Computer Assisted Tomography, 45(4):578–585, 2021. [8] D.T. Tuan, Q.H. Le, and T.H. Nguyen. Cervical Spine Fracture Detection via Computed Tomography scan. PhD thesis, FPT University, 2022. [9] A.F. Voter, M.E. Larson, J.W. Garrett, and J.P.J. Yu. Diagnostic accuracy and failure mode analysis of a deep learning algorithm for the detection of cervical spine fractures. AJNR Am J Neuroradiol, 42(8):1550–1556, Aug 2021. 28