A Study on the Performance of Distributed Storage Systems in Edge Computing Environments

A Study on the Performance of Distributed Storage Systems in Edge Computing Environments Antonios Makris1,2, Ioannis Kontopoulos1,2, Stylianos Nektarios Xyalis1, Evangelos Psomakelis1,2, Theodoros Theodoropoulos1,2, Andreas Varvarigos1, Konstantinos Tserpes1,2 1Department of Informatics and Telematics, Harokopio University of Athens, Greece 2School of Electrical and Computer Engineering, National Technical University of Athens, Greece Abstract—Edge computing presents a promising paradigm for the management and processing of the vast volumes of data generated by Internet of Things (IoT) devices. By merging cloud services with decentralized processing at the edge of the network, edge computing optimizes resource utilization while mitigating communication overhead and data transfer delays. Despite advancements, there are issues regarding cloud/edge-based application requirements. A distributed edge storage solution is crucial, ensuring data proximity, minimizing network congestion, and adapting to changing demands. Nevertheless, implementing or selecting an efficient edge-enabled storage system presents numerous challenges due to the distributed and heterogeneous nature of the edge, as well as its limited resource capabilities. Hence, it is essential for the research community to actively contribute towards clarifying the objectives and delineating the strengths and weaknesses of different storage solutions. This work presents an overview and performance analysis of three storage solutions in the edge computing context, namely MinIO, IPFS, and BigchainDB. The evaluation considers a set of Quality of Service (QoS) and resource utilization metrics. The systems are deployed on a cluster of four Raspberry Pis, which function as a network of edge devices. The results demonstrate the superiority of IPFS and provide insights into the performance of the evaluated storage systems for edge deployments. Index Terms—edge, storage solutions, performance evaluation, minio, ipfs, bigchaindb, distributed databases I. INTRODUCTION The widespread expansion of the Internet of Things (IoT) has introduced a profound and transformative shift in the field of IT service utilization. It has also introduced a novel computing paradigm that necessitates the processing of data at the network edge. According to the International Data Corporation, it is estimated that by 2025, around 70% of data generated by IoT devices will be processed at the edge of the network [1], [2]. To enhance the computational and storage capabilities of these devices, many applications rely on Cloud computing solutions. However, the requirement to establish connections with data centers operated by major providers such as Amazon, Google, and Microsoft results in substantial network latency, thereby hindering the deployment of a multitude of services. Additionally, existing cloud infrastructures have been proved inadequate to support, manage, and process the massive amounts produced by IoT applications, as end devices are usually distant from the cloud servers. This geographic separation introduces additional processing and network overhead, leading to elevated latency, limited bandwidth, and overall performance degradation. This confluence marks the birth and realization of Edge computing, bringing together the power of cloud services with decentralized processing at the network edge. Edge computing is a promising paradigm that effectively addresses network bottlenecks, communication overhead, and data transfer delays [3]–[6]. This approach strategically places resourcerich computational resources closer to mobile and/or IoT devices, offering superior scalability and availability compared to traditional cloud platforms [7]–[9]. Furthermore, storing and processing data at the edge, facilitates the utilization of contextual information to enhance data localization and decision-making [10]. Edge nodes, in general, possess constrained computation, storage, network, and power resources. They often exhibit heterogeneous hardware and software architectures and are distributed across multiple locations. Efficient data sharing between edge nodes poses one of the main challenges in developing edge applications, and it can be achieved either within application frameworks or by leveraging an external storage service. Despite significant advancements in providing efficient edge storage solutions, there are still some issues to be addressed related to the functional and non-functional requirements of cloud/edge-based applications [11]. To meet the ever-increasing demands of modern applications, the general requirements that an edge-enabled storage system should encompass are as follows: low data retrieval latency, high availability through fault tolerance and migration techniques, data locality, high scalability, data integrity, handling potential storage resource scarcity at edge nodes, managing high heterogeneity within the system, edge cloud server pool isolation and data accessibility in partitioning conditions. These requirements can be successfully achieved by optimizing resource usage, allocation, and data management plans on edge devices as well as by leveraging the distributed nature of storage solutions. The distributed nature ensures that data is stored and processed in close proximity to its source, minimizing network congestion, and reducing reliance on centralized cloud infrastructure. Moreover, the scalability and flexibility of distributed edge storage enable seamless adaptation to evolving application demands, providing a robust foundation for the efficient delivery of cloud/edge-based services. This work offers a comprehensive overview and performance analysis of three different storage systems in the context of edge computing: MinIO, an object storage solution; IPFS, a Peer-to-Peer (P2P) distributed file system that incorporates the concepts of BitTorrent; and BigchainDB, a blockchain database. The reason for including these storage solutions is their relevance and applicability in the edge computing domain. MinIO is widely recognized for its efficient object storage capabilities, IPFS leverages P2P technology for distributed file sharing, and BigchainDB utilizes blockchain principles for secure and decentralized data management. By assessing their performance in an edge context, valuable insights can be extracted regarding their suitability and effectiveness for edge storage deployments. The evaluation of each storage solution entails the use of several metrics for QoS and resource utilization. Each system is installed and deployed on a cluster comprising four Raspberry Pis (RPis), which function as a network of edge devices with limited processing and storage capabilities. The experimental findings demonstrate that IPFS outperforms the other systems across all scenarios, with MinIO emerging as the second-best performer in most cases. The rest of this paper is organized as follows: Section II explores related work concerning storage solutions for edge computing infrastructures. Section III provides an overview of the three evaluated storage systems, highlighting the different distributed architectures employed by each. Section IV assesses their performance based on resource utilization and performance metrics. Finally, Section V summarizes the merits of our work and highlights some perspectives that require further attention in the future. II. RELATED WORK A new generation of world wide web, called WEB 4.0 is quickly emerging in more and more applications concerning both our daily life and the industry. The rising popularity of IoT devices is enhancing this emergence, leading to a faster evolution and a series of new challenges that have posed significant concerns for researchers during the last decade [12], [13]. The minimization of data latency and network overhead, especially in fog and edge networks, is one of the most relevant problems in this category [14]. One commonly employed approach involves the adoption of edge storage methodologies, aiming to relocate a portion or even the entirety of required data and its processing closer to the edge devices that will utilize them. The solution involves leveraging Edge storage technology, which actively decentralizes the load, prioritizing resource efficiency to accommodate the limited resources commonly found in IoT devices within edge networks. In fact, decentralization and resource efficiency serve as the primary driving forces motivating researchers in this domain. To meet these demands, numerous conventional storage technologies have been adapted and even integrated. Examples of such technologies include blockchain and block storage. Bitcoin stands out as one of the most well-known cryptocurrencies. To operate, it relies on Blockchain, a secure and traceable transaction-based storage technology. Researchers across various fields have extensively explored this decentralized system, giving it an independent direction [15]. In essence, Blockchain establishes a central repository that facilitates a series of linked exchanges, forming transactions. Before these exchanges are recorded as valid transactions in the repository, multiple nodes collaboratively validate them in a decentralized manner. However, when applying Blockchain to edge storage, two significant challenges emerge. Firstly, transaction validation demands substantial computational resources. Second, a centralized database is required to store the transaction chain [16]. This means that the decentralization and lightweight requirements of edge storage services are not really compatible with Blockchain at first glance. Nonetheless, researchers are diligently exploring various approaches, combining Blockchain with technologies such as P2P networks, to address and overcome these incompatibilities. A decentralized technology, such as a P2P network, serves as a file storage and sharing solution. Communication between its nodes, commonly referred to as peers, is facilitated through a set of protocols that enforce data security [17]. The lightweight nature of these protocols, in terms of both added overhead and processing power requirements, makes P2P networks particularly well-suited for edge applications [18]. Modern networks leverage Distributed Hash Tables (DHTs), which enhance functionality and security. Furthermore, encryption algorithms can be seamlessly integrated to further bolster data security [19]. However, the primary drawback of these networks lies in maintaining data integrity, immutability, and network reliability, which inadequately address essential security aspects [20]. Consequently, it becomes necessary to combine these networks with other more secure technologies for real-life use cases. For example, Blockchain is a popular choice that enhances P2P networks by adding the missing security controls. Researchers are actively striving to achieve equilibrium among the existing frameworks through comparative assessments of their throughput, resource efficiency, and limitations, whether in isolation or when integrated with one another. In this pursuit, Blockchain and P2P networks have gained significant attention, particularly as P2P networks emerge as strong contenders for edge storage solutions, provided that the aforementioned drawbacks can be addressed. In relevant experiments, the interaction between these frameworks appears to offer an efficient solution to the edge storage challenge. Blockchain proves effective in mitigating most of the weaknesses associated with P2P networks without substantially increasing overhead in read/write operations, throughput, and network traffic [21]–[24]. However, a limitation arises from the redundancy requirements of Blockchain mechanisms, necessitating greater available disk space in the edge clusters hosting these solutions. As a consequence, this imposes constraints on the network architecture options for IoT and fog networks. Research in the field of edge storage can be broadly classified into two main areas: security and resource efficiency, depending on the researchers’ priorities and use cases. However, in many cases, these two priorities are inherently contradictory. Enhancing resource efficiency often involves relaxing security measures, while bolstering security requires a greater commitment of resources. For example, in systems that rely on Blockchain and cryptographic security controllers, a substantial amount of middleware and network orchestrators are needed. These components enable the framework to execute encryption, decryption, and security validations for every data transaction [25], [26]. Similarly, in certain secure edge storage architectures, emphasis is given to a pre-specified set of data security objectives, such as availability and integrity. These approaches require significant redundancy, leading to the creation of resource-intensive platforms [27], [28]. The two most prevalent methodologies for ensuring availability and integrity, erasure coding and data replication, demand the incorporation of additional nodes. These additional nodes are responsible for holding the replicated data and coordinating data reading and recovery efforts. On the other hand, systems that prioritize high resource efficiency often tend to disregard data security entirely. Their focus remains solely on data transfer and storage between nodes, neglecting the resources required to secure data packets transmitted over the internet or the communication links connecting the nodes within the edge network [29]–[31]. These networks are frequently designed and assessed with the assumption that data and network security are addressed at a different level of data transfer and storage, which falls beyond their aims. Great examples of systems that completely disregard security include CREIM [32], DIMA [33] and FastCache [34]. CREIM is a system that implements decentralized indexing of data, focusing on elasticity, decentralization and resource efficiency. It establishes a decentralized index, enabling nodes within the cluster to serve as both indexing and data servers. This design facilitates swift data retrieval in edge clusters, even as the number of nodes continuously fluctuates. DIMA, a framework tailored for micro-service caching, operates on comparable principles to CREIM. However, it incorporates AI capabilities to intelligently position cached data among the nodes comprising the edge cluster, enhancing its overall efficiency. FastCache’s primary focus lies in handling a large volume of concurrent data micro-transactions, which presents a notable challenge, particularly in IoT edge clusters where numerous small messages are exchanged between nodes. Consequently, this results in substantial overhead as the protocols’ additional costs per transaction may equal or even exceed the size of the actual message in certain cases. On the other hand, there are examples of solutions that prioritize data security, such as DSRP [35], along with various other edge storage and pre-caching distributed systems discussed in relevant literature [36], [37]. Most of these solutions concentrate on optimizing encryption protocols by distributing the workload and implementing data partitioning across the edge cluster. This approach transforms resource-demanding tasks like encryption, hashing, and integrity validation into distributed, decentralized tasks, thereby reducing their overall resource requirements. In contrast, DSRP focuses solely on integrity and doesn’t address the entire CIA trinity (confidentiality, integrity, and availability), making it a more lightweight solution. Specifically, it generates a set of keys that provide evidence of data transfers, storage, and replications, allowing nodes within the edge cluster to establish trust communities among themselves, thereby ensuring data integrity. III. STORAGE SOLUTIONS In this section, the three different storage solutions that are evaluated are presented. These systems belong to distinct storage categories. More specifically, MinIO is an object storage system, IPFS functions as a file storage system, and BigchainDB serves as a blockchain database. Each storage format stores, organizes, and presents data in different ways, possessing unique capabilities and limitations. Object storage links data with the associated metadata, file storage utilizes a hierarchy of files in folders, while blockchain constitutes a shared database that stores data as signed blocks interconnected in an immutable chain. In the following subsections, a comprehensive overview of each storage solution is provided. Table I presents a comparison of the evaluated storage solutions. A. MinIO MinIO1is an open-source framework created by IBM. MinIO stands as a cloud-native solution, designed specifically to be decentralized and highly scalable in a P2P fashion. It has the capability to operate within lightweight containers, managed efficiently by external orchestration services like Kubernetes. By employing a hierarchical structure, MinIO enables the creation of federations of clusters. In particular, MinIO combines the storage of data and metadata as unified objects, eliminating the necessity for a dedicated metadata database. Furthermore, MinIO performs crucial operations such as erasure coding, bitrot check, and encryption in an inline and strictly consistent manner. The result is that MinIO is exceptionally resilient. It uses object storage over block storage so it is in fact a combination of the two systems, preserving the lightweight distributed nature of block storage while providing a plethora of metadata and easy usage of the object storage. In contrast to other object storage solutions primarily intended for archival purposes, MinIO is specifically engineered to provide the high-performance object storage demanded by modern big data applications. Figure 1 illustrates a four-node MinIO cluster. Node 1 Node 2 Node 3 Node 4 Internal RESTful API Internal RESTful API Internal RESTful API Fig. 1: Cluster of MinIO with four nodes 1https://min.io/ TABLE I: Comparison of the evaluated storage solutions: MinIO, IPFS, and BigchainDB Feature MinIO IPFS BigchainDB Description High-performance object storage system Decentralized P2P file system Blockchain database Storage Mode Objects Files Signed blocks Consensus Not applicable Distributed consensus Byzantine fault tolerance Data Redundancy Supports erasure coding and replication Content-addressed storage with replication Replication and distribution Scalability Scalable horizontally Scalable horizontally and vertically Scalable horizontally and vertically Data Integrity Strong Strong Strong Data Access Method Content Queries Filepaths Transactions Main Strengths Unstructured and Scalable Simple and Secure Security, Immutability, and Transparency B. InterPlanetary File System The InterPlanetary File System (IPFS)2, is a P2P distributed file system, designed to store versioned file data in a decentralized manner [16]. IPFS has been developed on top of the BitTorrent protocol [38] and the Kademlia DHT [39]. The BitTorrent protocol, a popular P2P file sharing system, enables effective relocation of objects between peers in the IPFS infrastructure. Meanwhile, the Kademlia DHT serves as a widely used DHT for managing metadata within the IPFS framework. On top of these, IPFS builds a Merkle DAG, which is a directed acyclic graph where links between objects are cryptographic hashes of the targets embedded in the sources. In general, IPFS is considered as a single BitTorrent swarm, exchanging objects within one Git repository. In other words, IPFS provides a high-throughput content-addressed block storage model, with content-addressed hyperlinks. IPFS operates as a P2P system, devoid of privileged nodes and lacking a single point of failure. In this decentralized architecture, IPFS nodes store objects locally and uphold a DHT that facilitates the discovery of network addresses for other peers. Figure 2 illustrates an IPFS cluster of four nodes. IPFS cluster functions as a distributed application that operates alongside IPFS peers. It serves as a sidecar, responsible for managing a global cluster pinset and intelligently allocating its items to the respective IPFS peers. Peer A Peer CPeer B Peer D IPFS swarm Cluster swarm Fig. 2: IPFS swarm with four peers 2https://docs.ipfs.io/ C. BigchainDB BigchainDB3is a blockchain database that combines the characteristics of both blockchain (decentralization, immutability, autonomy, and owner-controlled assets) and traditional databases (high transaction rate, low latency, indexing, and structured data querying) properties [40]. BigchainDB supports two transaction operations: CREATE and TRANSFER. A BigchainDB transaction is a JSON string that conforms to the BigchainDB Transactions specification. Transactions of the BigchainDB contain Asset, Input, Output, Metadata and Transaction ID. Any physical or digital objects can be represented by the asset. The CREATE operation of a transaction is used to create new assets, whereas the TRANSFER operation of a transaction is used to update the state or ownership of the asset. After obtaining a transaction, it is possible to transmit it to a BigchainDB network using the BigchainDB HTTP API4. A BigchainDB network, consisting of four nodes, is depicted in Figure 3. Each node is a virtual concept consisting of three parts: •MongoDB database: each node has its own local MongoDB, which is used for the data storage locally. MongoDB offers high throughput, massive data storage capabilities, NoSQL query language support, efficient query and rights management. The node operator has administrator privileges to access the local MongoDB. That means that each node operator has access to the full power of MongoDB for indexing and querying the stored data (transactions, assets, metadata and blocks). •BigchainDB server: a server node that is responsible for sending and processing requests, permission controls, signatures, encryption, decryption, transactions, transaction verification and so on. •Tendermint communication node: Byzantine Fault Tolerant (BFT) middleware for networking and consensus. Each node has its own local MongoDB database as mentioned above, and all communication between nodes is utilized using Tendermint protocols. One consequence is that the resulting system is BFT, because Tendermint is Byzantine as well. 3https://www.bigchaindb.com/ 4https://docs.bigchaindb.com/projects/server/en/latest/ http-client-server-api.html Node 1 Node 3 Node 2 Node 4 Fig. 3: Communications in a four-node BigchainDB network IV. EXPERIMENTAL EVALUATION The experimental evaluation presented is performed on a cluster consisting of 4 RPis, each with a Quad-core (CortexA72 (ARM v8) 64-bit SoC @ 1.5GHz), 8GB of RAM (LPDDR4-3200 SDRAM), and equipped with a 16GB SD card, all running Raspberry Pi OS with Python 3.6. The behavior of each storage system is evaluated using a collection of small to medium binary files, evenly distributed in size and ranging from 15KB to 10MB, which form the evaluation dataset that is stored in the examined systems (200 files in total). The performance evaluation was conducted using Locust5, a load-testing framework that is open-source and allows for the specification of user behavior. It has the capability to run load tests across multiple machines and simulate a large number of simultaneous user requests. To facilitate the experiments, 20 different users were configured to execute distributed query requests. The evaluation metrics employed are divided into two categories: resource consumption and performance. More specifically, four aspects were taken into consideration: i) transaction rate, ii) number of user requests, iii) response time, and iv) resource utilization. To enhance the validity of our findings, each experiment was conducted over five iterations, thereby enhancing the reliability of our results and mitigating potential biases. During the experiments, operations occurred within the cluster, which were interconnected within the same local area network configuration. For the first aspect, we measured the average transaction rate of each storage system. The transaction rate is donated as TR =Ptransactions/total time, where the numerator represents the total number of transactions, and the denominator corresponds to the time required to perform those operations. In this case, the total time for measurements was set at five minutes. Figure 4 illustrates the transaction rate achieved by each storage solution. The results indicate that IPFS outperformed both MinIO and BigchainDB, achieving a transaction rate of 6.7. MinIO followed with a rate of 3.6, while BigchainDB yielded the worst result, with a rate of 0.1. 5https://locust.io/ MinIO IPFS BigchainDB System 0 1 2 3 4 5 6 7 Transaction rate Fig. 4: Transaction rate achieve by each storage system read_requests write_requests Requests 0 200 400 600 800 1000 Number of requests MinIO IPFS BigchainDB Fig. 5: Number of read and write requests for each storage system The second aspect relates to the total number of read and write requests carried out by each storage system. As in the case of the first aspect, the total time for measurements was set at five minutes. Figure 5 illustrates these results, with the left side depicting read requests and the right side representing write requests. The results showcase IPFS as the top performer, successfully handling approximately 1000 read and write requests, respectively. MinIO follows with approximately 550 read and write requests, respectively, while BigchainDB ranks last with approximately 52 requests in total. IPFS is capable of handling more requests than MinIO and BigchainDB due to its decentralized P2P architecture. MinIO, on the other hand, is primarily optimized for efficient object storage and retrieval, which enables it to handle a substantial number of requests. BigchainDB, operates on a consensusdriven model where each transaction is recorded and validated by multiple nodes in the network. This distributed consensus mechanism introduces additional overhead, which can result in lower throughput and a lower number of requests handled per unit of time compared to IPFS and MinIO. The third aspect refers to the evaluation of the response time of each storage system, i.e. how long it takes to respond to user queries. To assess this, we measured the average response time of a single request, as well as the average response time of all requests in both read and write operations. Figures 6a and 6b illustrate the average response time (measured in milliseconds) MinIO IPFS BigchainDB System 101 102 103 104 Response Time (a) Average read operation of each request MinIO IPFS BigchainDB System 100 101 102 103 104 Response Time (b) Average write operation of each request MinIO IPFS BigchainDB System 102 103 104 105 Response Time (c) Average read operation of all requests MinIO IPFS BigchainDB System 103 104 105 Response Time (d) Average write operation of all requests Fig. 6: Performance of read/write operations of each storage system of a single request in read and write operations, respectively. On the other hand, Figures 6c and 6d illustrate the average response time for all users’ requests. Overall, as evidenced by the figures, IPFS exhibits superior performance in both read and write operations. MinIO follows IPFS while BigchainDB falls short when compared to both MinIO and IPFS. BigchainDB prioritizes blockchain features, which can lead to reduced performance in terms of rapid data read and write operations. For instance, the average read operation of each request takes nearly four times longer compared to IPFS. On the other hand, IPFS leverages its content-addressed methodology and peerto-peer architecture, allowing for exceptionally fast response time. The final and fourth aspect of the experimental evaluation refers to the resource utilization when each storage system is engaged in read or write operations. The resources evaluated are the percentage of RAM utilized and the DiskIO time during the operations. Figures 7 and 8 illustrate the resource usage in terms of RAM and DiskIO for the read (Figure 7) and the write (Figure 8) operations, respectively. Although CPU usage was also recorded, it was negligible and thus not included in the plots. These findings indicate that the storage systems are lightweight enough to be deployed on various edge devices, including the Raspberry Pi cluster used in the evaluation. It is worth noting that a standard deviation line has been omitted due to the insignificance of observed differences. Figure 7a showcases that during read operations, BigchainDB utilized the least amount of RAM (1.4%) compared to the other storage systems, while IPFS consumed the highest amount of RAM (3.5%). MinIO closely trails behind BigchainDB with a RAM usage of 1.5%. On the other hand, Figure 7b illustrates the average DiskIO time in seconds for read operations, where MinIO exhibits the best performance, followed by both BigchainDB and IPFS with the same DiskIO time. Figure 8a provides an overview of the RAM usage for each storage system during write operations. These results are similar to the results of the read operations in terms of the bestand worst-performing systems. Similar to Figure 7a, BigchainDB demonstrates the least amount of RAM utilization (1.4%), while IPFS exhibits the highest RAM usage (2.8%), and MinIO falls behind BigchainDB. However, the results diverge when it comes to the DiskIO time of the write operations, as depicted in Figure 8. Notably, all systems showcase approximately the same DiskIO time during write operations. BigchainDB’s underlying structure and blueprint are centered on ensuring that data remain immutable, which drives the implementation of highly efficient memory management strategies. These tactics enable BigchainDB to use a relatively small amount of RAM when handling read and write operations, showcasing its proficiency in resource utilization. In contrast, IPFS, leveraging content-addressing and peer-to-peer features, demands a comparatively larger RAM allocation to effectively handle its extensive network of interconnected nodes and data chunks. MinIO, tailored for high-performance object storage, demonstrated RAM utilization results almost on par with BigchainDB, further highlighting its optimization for efficient and rapid data storage. RAM utilization 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 Usage (%) MinIO IPFS BigchainDB (a) RAM Disk IO time 0 5 10 15 20 Time MinIO IPFS BigchainDB (b) Disk IO time Fig. 7: Statistics for the read operation of each storage system (all requests) RAM utilization 0.0 0.5 1.0 1.5 2.0 2.5 Usage (%) MinIO IPFS BigchainDB (a) RAM Write (all requests) Disk IO time 0.0 0.5 1.0 1.5 2.0 Time MinIO IPFS BigchainDB (b) Disk IO time Fig. 8: Statistics for the write operation of each storage system (all requests) V. CONCLUSION The challenges of implementing an efficient edge-enabled storage system are amplified by the heterogeneous and diverse nature of the edge, as well as its limited resources. With numerous storage systems and technologies available, researchers and practitioners are unsure about the optimal choice for managing and processing the vast amounts of data generated by IoT devices. Therefore, in this study we provide a comprehensive review and a performance analysis of three major storage systems used in edge computing, namely MinIO, IPFS, and BigchainDB. The evaluation of each storage system involves the use of several metrics for Quality of Service and resource utilization. These storage systems are deployed and installed on a cluster consisting of four Raspberry Pis, acting as a cluster of edge devices with limited processing and storage capabilities. The experimental results demonstrate that IPFS outperforms the other systems in all scenarios, with MinIO being the second-best performer in most cases. It is important to note that the response times of each storage system are comparable, and additional workload and stress testing are necessary to further validate our research findings. Additionally, the absence of diverse traffic scenarios in our evaluation limits the breadth of our findings, as real-world IoT applications often exhibit varying patterns of data flow. As part of our future work, we plan to assess the scalability of the storage systems by expanding the cluster with a higher number of Raspberry Pis and increasing the number of users. This evaluation aims to identify potential bottlenecks and examine the systems’ ability to handle increased workloads when deployed on a larger scale of edge devices. Furthermore, it is essential to consider the potential influence of network utilization and the impact of load balancing on the obtained results. Addressing these factors in future evaluations could provide a more comprehensive understanding of the performance dynamics. Finally, we intend to conduct a thorough analysis of additional storage solutions like Gluster and Ceph. This will allow for a more refined comparison of the diverse range of storage options available for edge computing environments. ACKNOWLEDGMENT This project has received funding from the Horizon Europe Framework Programme under Grant agreement No 101135775 (PANDORA). Additionally, it received funding from the European Union’s Horizon Europe Research and Innovation Programme under grant agreement No. 101120990 (SOPRANO). This paper reflects only the authors’ view and the Commission is not responsible for any use that may be made of the information it contains. REFERENCES [1] W. Shi, G. Pallis, and Z. Xu, “Edge computing,” Proceedings of the IEEE, vol. 107, no. 8, pp. 1474–1481, 2019. [2] M. Zwolenski and L. Weatherill, “The digital universe: Rich data and the increasing value of the internet of things,” Journal of Telecommunications and the Digital Economy, vol. 2, no. 3, pp. 47–1, 2014. [3] M. Chiang and T. Zhang, “Fog and iot: An overview of research opportunities,” IEEE Internet of things journal, vol. 3, no. 6, pp. 854– 864, 2016. [4] Z. Hao, E. Novak, S. Yi, and Q. Li, “Challenges and software architecture for fog computing,” IEEE Internet Computing, vol. 21, no. 2, pp. 44–53, 2017. [5] T. Theodoropoulos, A. Makris, J. Violos, and K. Tserpes, “An automated pipeline for advanced fault tolerance in edge computing infrastructures,” in Proceedings of the 2nd Workshop on Flexible Resource and Application Management on the Edge, 2022, pp. 19–24. [6] T. Theodoropoulos, A. Makris, I. Kontopoulos, A.-C. Maroudis, and K. Tserpes, “Multi-service demand forecasting using graph neural networks,” in 2023 IEEE International Conference on Service-Oriented System Engineering (SOSE). IEEE, 2023, pp. 218–226. [7] A. Makris, A. Boudi, M. Coppola, L. Cordeiro, M. Corsini, P. Dazzi, F. D. Andilla, Y. G. Rozas, M. Kamarianakis, M. Pateraki et al., “Cloud for holography and augmented reality,” in 2021 IEEE 10th International Conference on Cloud Networking (CloudNet). IEEE, 2021, pp. 118– 126. [8] T. Theodoropoulos, A. Makris, A. Boudi, T. Taleb, U. Herzog, L. Rosa, L. Cordeiro, K. Tserpes, E. Spatafora, A. Romussi et al., “Cloud-based xr services: A survey on relevant challenges and enabling technologies,” Journal of Networking and Network Applications, vol. 2, no. 1, pp. 1–22, 2022. [9] T. Theodoropoulos, A. Makris, E. Psomakelis, E. Carlini, M. Mordacchini, P. Dazzi, and K. Tserpes, “Gnosis: Proactive image placement using graph neural networks & deep reinforcement learning,” in 2023 IEEE 16th International Conference on Cloud Computing (CLOUD). IEEE, 2023, pp. 120–128. [10] S. A. Hossain, M. A. Rahman, and M. A. Hossain, “Edge computing framework for enabling situation awareness in iot based smart city,” Journal of Parallel and Distributed Computing, vol. 122, pp. 226–237, 2018. [11] A. Makris, I. Kontopoulos, E. Psomakelis, S. N. Xyalis, T. Theodoropoulos, and K. Tserpes, “Performance analysis of storage systems in edge computing infrastructures,” Applied Sciences, vol. 12, no. 17, p. 8923, 2022. [12] E. Psomakelis, F. Aisopos, A. Litke, K. Tserpes, M. Kardara, and P. M. Campo, “Big IoT and Social Networking Data for Smart Cities,” in Proceedings of the 6th International Conference on Cloud Computing and Services Science-Volume 1 and 2, 2016, pp. 396–405. [13] T. Rausch, A. Rashed, and S. Dustdar, “Optimized container scheduling for data-intensive serverless edge computing,” Future Generation Computer Systems, vol. 114, pp. 259–271, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167739X2030399X [14] A. Makris, E. Psomakelis, T. Theodoropoulos, and K. Tserpes, “Towards a distributed storage framework for edge computing infrastructures,” in Proceedings of the 2nd Workshop on Flexible Resource and Application Management on the Edge, 2022, pp. 9–14. [15] M. Nofer, P. Gomber, O. Hinz, and D. Schiereck, “Blockchain,” Business & Information Systems Engineering, vol. 59, no. 3, pp. 183–187, 2017. [16] S. Kumar, A. K. Bharti, and R. Amin, “Decentralized secure storage of medical records using blockchain and ipfs: A comparative analysis with future directions,” Security and Privacy, vol. 4, no. 5, p. e162, 2021. [17] W. Hou, Y. Jiang, W. Lei, A. Xu, H. Wen, and S. Chen, “A p2p network based edge computing smart grid model for efficient resources coordination,” Peer-to-Peer Networking and Applications, vol. 13, no. 3, pp. 1026–1037, 2020. [18] D. Tracey and C. Sreenan, “How to see through the fog? using peer to peer (p2p) for the internet of things,” in 2019 IEEE 5th World Forum on Internet of Things (WF-IoT). IEEE, 2019, pp. 47–52. [19] E. Daniel and F. Tschorsch, “Ipfs and friends: A qualitative comparison of next generation peer-to-peer data networks,” IEEE Communications Surveys & Tutorials, vol. 24, no. 1, pp. 31–52, 2022. [20] G. Subathra, A. Antonidoss, and B. K. Singh, “Decentralized consensus blockchain and IPFS-based data aggregation for efficient data storage scheme,” Security and Communication Networks, vol. 2022, pp. 1–13, jul 2022. [Online]. Available: https://doi.org/10.1155\%2F2022\ %2F3167958 [21] L. Zhang, W. Zeng, Z. Jin, Y. Su, and H. Chen, “A research on traceability technology of agricultural products supply chain based on blockchain and IPFS,” Security and Communication Networks, vol. 2021, pp. 1–12, nov 2021. [Online]. Available: https://doi.org/10.1155\%2F2021\%2F3298514 [22] P. Weerapanpisit, S. Trilles, J. Huerta, and M. Painho, “A decentralized location-based reputation management system in the iot using blockchain,” IEEE Internet of Things Journal, vol. 9, no. 16, pp. 15 100– 15 115, 2022. [23] B. Confais, A. Lebre, and B. Parrein, “An object store service for a fog/edge computing infrastructure based on IPFS and a scale-out NAS,” in 2017 IEEE 1st International Conference on Fog and Edge Computing (ICFEC). IEEE, may 2017. [Online]. Available: https://doi.org/10.1109\%2Ficfec.2017.13 [24] B. W. Nyamtiga, J. C. S. Sicato, S. Rathore, Y. Sung, and J. H. Park, “Blockchain-based secure storage management with edge computing for iot,” Electronics, vol. 8, no. 8, p. 828, 2019. [25] Y. Ren, , Y. Leng, Y. Cheng, J. Wang, , and and, “Secure data storage based on blockchain and coding in edge computing,” Mathematical Biosciences and Engineering, vol. 16, no. 4, pp. 1874–1892, 2019. [Online]. Available: https://doi.org/10.3934\%2Fmbe.2019091 [26] L. Yuan, Q. He, F. Chen, J. Zhang, L. Qi, X. Xu, Y. Xiang, and Y. Yang, “CSEdge: Enabling collaborative edge storage for multi-access edge computing based on blockchain,” IEEE Transactions on Parallel and Distributed Systems, vol. 33, no. 8, pp. 1873–1887, aug 2022. [Online]. Available: https://doi.org/10.1109\%2Ftpds.2021.3131680 [27] A. Javed, K. Heljanko, A. Buda, and K. Fr¨ amling, “Cefiot: A faulttolerant iot architecture for edge and cloud,” in 2018 IEEE 4th world forum on internet of things (WF-IoT). IEEE, 2018, pp. 813–818. [28] M. Nijim and H. Albataineh, “Secure-stor: A novel hybrid storage system architecture to enhance security and performance in edge computing,” IEEE Access, vol. 9, pp. 92 446–92 459, 2021. [Online]. Available: https://doi.org/10.1109\%2Faccess.2021.3092732 [29] J. Liu, M. L. Curry, C. Maltzahn, and P. Kufeldt, “Scale-out edge storage systems with embedded storage nodes to get better availability and {Cost-Efficiency}at the same time,” in 3rd USENIX Workshop on Hot Topics in Edge Computing (HotEdge 20), 2020. [30] H. Jin, R. Luo, Q. He, S. Wu, Z. Zeng, and X. Xia, “Cost-effective data placement in edge storage systems with erasure code,” IEEE Transactions on Services Computing, pp. 1–1, 2022. [Online]. Available: https://doi.org/10.1109%2Ftsc.2022.3152849 [31] A. E. Elgazar, M. Aazam, and K. A. Harras, “{SMC}: Smart media compression for edge storage offloading,” in 2nd USENIX Workshop on Hot Topics in Edge Computing (HotEdge 19), 2019. [32] S. Tang, X. Du, Z. Lu, K. Gai, J. Wu, P. C. Hung, and K.-K. R. Choo, “Coordinate-based efficient indexing mechanism for intelligent iot systems in heterogeneous edge computing,” Journal of Parallel and Distributed Computing, vol. 166, pp. 45–56, 2022. [33] H. Tian, X. Xu, T. Lin, Y. Cheng, C. Qian, L. Ren, and M. Bilal, “Dima: Distributed cooperative microservice caching for internet of things in edge computing by deep reinforcement learning,” World Wide Web, vol. 25, no. 5, pp. 1769–1792, 2022. [34] L. Qian, Z. Qu, M. Cai, B. Ye, X. Wang, J. Wu, W. Duan, M. Zhao, and Q. Lin, “Fastcache: A write-optimized edge storage system via concurrent merging cache for iot applications,” Journal of Systems Architecture, vol. 131, p. 102718, 2022. [35] C. Wu, Y. Chen, Z. Qi, and H. Guan, “Dspr: Secure decentralized storage with proof-of-replication for edge devices,” Journal of Systems Architecture, vol. 125, p. 102441, 2022. [36] D. Liu, Z. Li, and D. Jia, “Secure distributed data integrity auditing with high efficiency in 5g-enabled software-defined edge computing,” Cyber Security and Applications, vol. 1, p. 100004, 2023. [37] J. Chen, Y. Wang, M. Ye, and Q. Jiang, “A secure cloud-edge collaborative fault-tolerant storage scheme and its data writing optimization,” IEEE Access, 2023. [38] B. Cohen, “Incentives build robustness in bittorrent,” in Workshop on Economics of Peer-to-Peer systems, vol. 6. Berkeley, CA, USA, 2003, pp. 68–72. [39] P. Maymounkov and D. Mazieres, “Kademlia: A peer-to-peer information system based on the xor metric,” in International Workshop on Peer-to-Peer Systems. Springer, 2002, pp. 53–65. [40] T. McConaghy, R. Marques, A. M¨ uller, D. De Jonghe, T. McConaghy, G. McMullen, R. Henderson, S. Bellemare, and A. Granzotto, “Bigchaindb: a scalable blockchain database. white paper, bigchaindb (2016),” 2016.