Autonomous agents for Hyper-distributed applications: a role-based approach for the Swarm

Autonomous agents for Hyper-distributed applications: a role-based approach for the Swarm Francesc Lordana, Xavier Casas-Morenoa, Philip Cumminsa, Javier Conejeroa, Rosa M. Badiaa, Raül Sirventa aBarcelona Supercomputing Center - Centro Nacional de Supercomputación, Barcelona, Spain Abstract Hyper-distributed applications leverage the Compute Continuum to offer highly-available, trustworthy and secure services with low latency. However, given the number of managed devices and their heterogeneity, finding an optimal distribution of these application’s functionalities is a challenging problem tackled either manually or with a resourcehungry, centralized solution. This article proposes a ground-breaking programming environment where applications are composed of multiple cooperative roles to be executed by the devices. A novel software stack transforms each device into an Autonomous ageNT (ANT) able to decide which roles it executes, according to its capabilities and context, and enabling the construction of a cooperative, self-organized platform. In this article, we present the COLMENA framework and the programming model that defines a service as a set of roles, behaviors, and abstractions, leveraging declarative programming. We present the architecture design and current implementation of the building blocks of the COLMENA software stack. Using a custom-built simulator, we compare different policies for role selection, including rule-based and reinforcement learning approaches. Finally, we demonstrate the validity of the programming model and the middleware by deploying a pilot use-case over an infrastructure of heterogeneous hardware devices and show reactivity towards the events of adding and removing devices in the platform. Keywords: Swarm, Edge, Cloud, Compute Continuum, Distributed System, Programming Model, Decentralized Platform, Autonomous Agent 1. Introduction Recent advancements in Cloud Computing have enabled the widespread deployment and accessibility of distributed applications. Although IoT devices are extensively used for data collection, nearly 80% of data processing still takes place in centralized computing facilities and data centers [1]. The Cloud Computing paradigm relies heavily on the network infrastructure, which introduces higher communication latency and significant energy consumption, while leaving computational resources available at the edge largely underexploited. Such dependence on centralized infrastructures poses a significant barrier to achieving climateneutrality and sustainable societal goals. Hyper-distributed applications, which integrate data and resources from multi-tenant, geographically dispersed infrastructures interconnected by heterogeneous Email addresses: [email protected] (Francesc Lordan), [email protected] (Xavier Casas-Moreno) and often unreliable networks, demand low latencies to ensure high availability, trust, and security [2]. The Compute Continuum addresses these challenges by unifying Edge and Cloud resources into a cohesive computing platform [3]. This platform harnesses the capabilities of IoT, Edge, and Far-Edge devices, thereby minimizing reliance on Cloud infrastructures and network stability. By doing so, it enhances the responsiveness, accuracy, and energy efficiency of IT systems. However, the challenge of defining and deploying applications across the Continuum while minimizing human intervention remains an open research topic. Swarm Computing is an emerging paradigm applicable to the Compute Continuum characterized by the collaboration of multiple autonomous agents operating in a fully decentralized manner to achieve a shared global objective. Swarm Computing offers numerous advantages. By exploiting data locality, the Continuum strengthens the security and privacy of the services and enhances their energy and resource usage effi- ciency, thus, reducing operational costs. Its lower communication latencies enable higher-performance solutions. Furthermore, the decentralization of the control plane provided by Swarm technologies significantly improves the scalability and fault tolerance of applications. The adoption of the Continuum and Swarm technologies is anticipated to positively influence stakeholders involved in all the stages of the life-cycle of a hyperdistributed application: construction (Developers), deployment (Service Providers and Infrastructure Owners), operation (Service Providers, Infrastructure Owners and end-users) and maintenance (Developers). The novelty of these concepts, coupled with the lack of supporting tools for stakeholders, poses a major barrier to the widespread adoption of these technologies. Current hyper-distributed applications are predominantly custom-built and highly specialized. The vast heterogeneity and extreme scale of the infrastructure, encompassing both computing devices and networks, significantly complicates the management of the Continuum and the orchestration of applications. This complexity is further exacerbated by the dynamic nature of the Continuum. For example, devices joining or leaving the infrastructure, unreliable networks, and workloads driven by fluctuating user demands, introduce additional layers of unpredictability and challenges to efficient operation. Automatically optimizing the management of this complex ecosystem often requires resourceintensive solutions. For simplicity, these solutions are typically centralized in the Cloud, which undermines the decentralized potential of the Continuum. Moreover, this management complexity often permeates service development. Hyper-distributed applications typically consist of numerous sub-services, often encapsulated within virtual environments such as containers, which combine data processing components with communication solutions to integrate them. The choice of technologies and solutions for these components is usually driven by a predefined deployment strategy taking the infrastructure’s characteristics into account. However, these strategies are predominantly static, giving priority to management simplicity over flexibility. As a result, hyper-distributed applications struggle to adapt in a dynamic and evolving environment, which negatively impacts their performance, efficiency, and cost. The work presented in this article embraces this paradigm shift toward the Swarm by introducing COLMENA 1: an open framework designed to develop, 1https://proyecto-colmena.com/ deploy and operate hyper-distributed applications on large, heterogeneous, and dynamic groups of devices across the Continuum. To simplify the development and maintenance of hyper-distributed applications targeting swarms, the framework includes a programming model that allows developers to describe application logic as a composite of interacting roles. Each role is defined by its functionality or behavior (program logic) and a set of hardware requirements. This abstraction enables developers to concentrate on the functionality of the application, leaving the complexities of low-level infrastructure management to the underlying runtime system. For distributing and running these roles, each infrastructure component - whether a hardware node, virtual environment, or entire virtualization platform - hosts a lightweight, energy-efficient software known as the COLMENA Agent. This software endows devices with the intelligence to become self-aware of their capabilities and context and transforms the device into an Autonomous ageNT (ANT). By hosting the COLMENA Agent, ANTs can self-organize to form a flexible, agile, secure and fully decentralized platform with a mesh architecture: the COLMENA platform. The platform effectively integrates diverse device capabilities — including computing, storage, connectivity, and cyber-physical functions — into a unified ecosystem that promotes a seamless collaboration among devices through inter-device communication, data sharing, and computational workload exchange. This integration operates transparently across various providers, network zones, and connectivity types. The COLMENA Agent software running on each ANT drives the distribution of roles by dynamically selecting which roles are executed locally, based on the ANT’s capabilities, current context, and the roles’ requirements. This process is further guided by application-specific indicators such as expected performance, energy consumption, and quality of service (QoS) and experience (QoE), compared to those currently delivered. By enabling distributed, autonomous decision-making, COLMENA eliminates the need for resource-intensive, centralized management mechanisms, paving the way for extreme-scale elastic and heterogeneous systems. Besides, it enhances the robustness and efficiency of applications with the flexibility to quickly react to infrastructure or workload changes by dynamically reassigning the roles played by each ANT with zero setup and reconfiguration effort for the operator. This article continues by providing an overview of the state of the art of the related fields of knowledge 2 and contributes to it by i) describing a pilot use case, offering a practical context for COLMENA, ii) detailing a software stack, which forms the backbone of the framework, iii) specifying a syntax for the programming model,iv) proposing some decision-making policies for ANTs, empowering them to autonomously select roles for local execution and achieve the desired collective behavior., v) providing a single-node multithread simulator capable of emulating the behavior of the platform for testing and validation purposes, vi) evaluating the proposed policies using the simulator to assess their effectiveness and impact, vii) building and integrating those components of the software stack to deliver a minimal-viability prototype of the COLMENA framework, viii) and demonstrating the viability of the proposal running the pilot use case using the developed prototype. This article, along with the COLMENA framework, contributes to the definition of next-generation computing and data technologies by easing the programming, deployment and maintenance of hyper-distributed applications on the Continuum. To enhance the trust of end-users in these systems, COLMENA aims at keeping a transparent and comprehensible operation, open to the adoption of new generations of hardware (processors, network or smart systems) and, thus, improve the interoperability of the system and avoid vendor lock-in. 2. Related work The orchestration of services in the Cloud is often achieved through the use of Cloud Operating Systems (OS) that manage the creation of multiple Cloud environments, as well as the deployment and operation of services as compositions of virtual entities. Vendorproprietary solutions are the most well-known examples (Amazon Web Services, Google Cloud Platform, and Microsoft Azure). Still, there are also open-source, interoperable solutions such as OpenStack 2and OpenNebula 3. Due to its popularity, Kubernetes 4(K8s) is a widespread solution supported by most Cloud OSs to orchestrate containers in clusters. To the best of our knowledge, all these solutions heavily rely on a centralized architecture to manage both the infrastructure and the virtual environments. The Edge presents a heterogeneous scenario with constrained resources and unstable networking. Some 2https://www.openstack.org/ 3https://opennebula.io/ 4https://kubernetes.io/ lightweight implementations of Kubernetes (e.g., K3s) focus on the resource constraints issue. Frameworks like Cilium 5, Skupper 6and Clusterlink [4] provide networking and security functionalities to Kubernetes edge clusters. However, the Edge demands a decentralized management solution for not remaining a surrogate of the Cloud. Some projects are already working in that direction. PiCasso [5] combines virtual environments with decentralized community mesh networks. OpenWhisk 7, Colony [6] or Serverledge [7] provide the execution of serverless functions (FaaS) in the Edge. Developing services deployed across the Continuum is based on tailored solutions where data is collected (and often preprocessed) on the Edge and then sent to the Cloud for archiving or analysis using publishsubscribe standards (e.g., MQTT), Apache Kafka or Remote Procedure Call (RPC) for synchronous communication. Data management systems proposed for IoT [8]8 support the fast ingestion and analysis of time series in the Cloud, requiring continuous data transfers to and from edge devices, thus disabling fast responses and the flexibility required by swarms. Other solutions [9]9 focus on resource heterogeneity combined with restrictions, avoiding data transfers to and from the Cloud, but assuming fixed infrastructures. The fully decentralized nature of swarms requires new approaches for managing and processing data. For instance, dataClay [10] is an object-based data storage solution optimized for dynamic environments. Another way of defining distributed applications in the Continuum is by using programming models. Traditional programming models for distributed systems— such as data analytics frameworks [11,12] and generalpurpose workflow managers [13,14]—have centralized global orchestrators. Another common paradigm for developing concurrent and distributed systems is the actor model, where an application can be defined as a composition of standalone actors communicating among them. Erlang [15] and Akka [16] implement the actor model, but their runtime systems are not prepared to face the challenges set by the continuum. Programming models targeting swarms (e.g., Buzz [17]) have focused on designing simple reactive controllers, neural networks, state machines, or behavior trees for individual devices that give rise to desired swarm behaviors [18,19,20]. Mandrake [21] 5https://cilium.io/ 6https://skupper.io/ 7https://openwhisk.apache.org/ 8BWorld Robot Control Software. https://www.influxdata.com 9ExtremeDB https://www.mcobject.com/extremedbfamily 3 proposes a programming model to define decentralized applications that deal with autonomous agents built on Protocols Over Things [22]. However, unlike COLMENA, Mandrake and Buzz have not been applied to the Continuum nor considered the dynamic allocation of roles on autonomous devices. In the literature, role assignment in multi-agent systems (allocation of items between different entities) is typically solved in a centralized manner by running classical optimization algorithms [23,24,25]. There are attempts for a distributed allocation, which are market-based and built on contract net protocol. Discovering controllers for individuals that result in desired emergent properties at a collective level is notoriously difficult. COLMENA is a novel approach to develop and orchestrate services across the Continuum. On the one hand, the decomposition of the service into several roles extends the actor model adapting it to the Compute Continuum by enabling the scaling of the number of instances of each role. On the other hand, COLMENA proposes a novel mechanism for orchestration where devices autonomously decide the role assignments based on QoS metrics. There already exist a myriad of frameworks to measure different metrics – e.g., Scaphandre for energy consumption –, extract and standardize them – e.g., OpenTelemetry – and aggregate them for archival or real-time analysis – Prometheus. COLMENA can incorporate some of these frameworks for capturing the metrics that are relevant for both the infrastructure and the operation of services. 3. Pilot use case: Video surveillance To provide a practical context for evaluating the capabilities of COLMENA, we introduce a pilot use case centred on a video surveillance system, which is relevant in a variety of domains, including road and traffic management, detection of missing people, and security [26]. While COLMENA is designed to support a broad range of applications, this use case offers a tangible, relatable scenario to assess key functionalities such as autonomous role management, QoS enforcement and scoped communications. Additionally, this use case serves as a consistent narrative throughout the paper, guiding readers through the framework’s concepts and its implementation and evaluation. The envisioned use case focuses on a company operating across several buildings, some of which have multiple floors subdivided into rooms equipped with cameras. The company seeks to deploy a hyper-distributed application to identify potential intruders entering the premises by processing images captured by cameras installed in all the lobbies of these buildings. Figure 1 illustrates the infrastructure expected to be used in this example highlighting the cameras whose images should be processed (i.e., in blue, the cameras located at the lobby of each floor). lobby hhrr office office Company Headquarters lobby DCP Figure 1: Infrastructure of the video surveillance use case highlighting the cameras whose images should be processed The installed cameras, constrained by limited computational capacity, cannot execute the computer vision algorithms necessary for intrusion detection. Consequently, the image processing workload must be offloaded to computing nodes with sufficient resources. In this use case, the computing nodes are servers located in a centralized data processing center (DPC) at the company’s headquarters. However, the infrastructure could also leverage edge servers in smaller DPCs in each company building or even virtual instances hosted in the cloud. Quick processing of captured images is crucial in the described use case due to the real-time nature of intrusion detection. Especially, during periods of increased activity, like peak entry/exit hours, prolonged processing times could result in a backlog of images, leading to degraded system performance and an inability to respond swiftly to threats. Establishing a threshold for the Quality of Service allows the system to adapt the amount of allocated resources to fluctuating workloads while maintaining operational efficiency and responsiveness. 4. The COLMENA framework As outlined in the Introduction section, COLMENA is designed to support all stakeholders involved in hyper-distributed applications by addressing issues in 4 every phase of their life-cycle: construction, deployment, and operation. During the construction phase, it aims at simplifying the coding process for Developers. At the deployment stage, it assists Providers in instantiating services on infrastructure owned by one or more Infrastructure Owners to whom it provides tools to control the usage of their devices. In the operation phase, COLMENA offers Providers mechanisms to monitor and manage the application behavior—including the possibility of deploying the continuous software updates facilitated by the Maintainers—and delivers to End-Users a highly secure, reliable, and available solution tailored to their needs. COLMENA is built around the concept of a unified platform that integrates a vast, heterogeneous set of IoT devices, smart systems, edge servers, and Cloud platforms, collectively forming a Compute Continuum. The platform design draws inspiration from successful examples of natural systems, such as organic colonies, where each member autonomously determines its contributions to the community based on its capabilities and the needs of the community by playing specific roles. Similarly, in COLMENA, devices within an IT infrastructure act as Autonomous AgeNTs (ANTs). These ANTs leverage their individual capabilities and contextual awareness to decide how best to contribute to the overall system’s objectives. The ANTs self-organize to establish a decentralized mesh architecture that supports multiple collaboration mechanisms: peer-to-peer communication, data and software sharing, and exchanging serverless computation. In this analogy, developing a service for such a platform resembles designing the "society" formed by these individuals. Specifically, it involves defining the roles composing the society (functionalities in the system), specifying their associated behaviors, and detailing the skills and capabilities (both hardware and software) required to carry out each role effectively. Optimally distributing functionalities across the Compute Continuum remains a significant challenge in deploying hyper-distributed services. The challenge arises due to the difficulty of managing a vast scale infrastructure. The complexity of the problem grows when considering the heterogeneity of the system. Furthermore, the dynamic nature of networks and the variability in workloads complicates establishing and applying general rules to simplify the problem. This requires continuous evaluation and re-adaptation. Currently, services are typically deployed using static, and often suboptimal, strategies or rely on resource-intensive centralized solutions. In contrast, we propose leveraging the autonomy of devices and embracing a fully decentralized approach that gives these devices the freedom to fail. Within this paradigm, each ANT autonomously determines which roles it is best suited to execute, based on the capabilities of the ANT and the current needs of the services. To enable ANTs to evaluate their performance and contribution when running a role and assess the overall quality of service (QoS) and experience (QoE) being delivered, a set of key performance indicators (KPIs) and their expected values must be defined. These KPIs allow devices to identify the specific needs and surpluses of each functionality within the system. Using these indicators, COLMENA agents autonomously select roles that align with their capabilities and context, aiming to contribute to an optimal distribution of functionalities. This self-organizing approach reduces dependency on centralized coordination, improves adaptability to dynamic conditions, and facilitates efficient resource utilization across the Continuum. 4.1. Software stack To construct and operate services on such a platform, we propose (and implement as described in section 6) the four-layer software architecture shown in Figure 2. At its foundation (layer 0), there are the IT resources that compose the underlying infrastructure. These can be either individual hardware nodes such as single board computers, edge servers, robots, virtual instances; virtual machines and containers; or whole virtualization platforms, Clouds and Container-as-a-Service clusters (e.g., Kubernetes or K3s). This base layer reflects the heterogeneity inherent in the system, encompassing variations in available network interfaces, computing resources, smart systems (sensors and actuators), and storage capabilities. Such diversity in infrastructure components highlights the need for a flexible and adaptable software stack to unify and effectively use these resources across the Continuum. The first software layer, ANT Handler, transforms each IT device into an interoperable ANT, ensuring an efficient operation by tailoring its functionality to the specific compute, network, storage, and sensing/actuating capabilities of the device and their current availability. The ANT Monitor component gathers and monitors information about the ANT’s capabilities and their usage, providing a real-time understanding of the device’s operational state. The Role Optimiser focuses on maximising resource utilisation, performance, and energy efficiency adapting the execution of roles to align with the specific hardware capabilities of the device. The Communication Controller monitors the current 5 Layer 1: ANT Handler Layer 4: Service Service A Layer 3: Service Tools Role Operation Abstraction Swarm Intelligence Library Distributed Service Manager QoE & QoS Assessment AI-driven Role Selector Sharing Manager Software Repository Data Compute Layer 2: Swarm Platform Context Manager Self Awareness Environment Awareness Communication Controller Smart System Controller ANT Monitor Role Optimiser Service Construction Service Deployment Service Operation Layer 0: Hardware Node Virtual Instance Virtual Platform E N D - U S E R P R O V I D E R Service Deployment Tool Service Operation Dashboard D E V E L O P E R </> Swarm Programming Model O W N E R ANT Configuration Manager Service B Network Interfaces Sensors & Actuators Compute Devices Data Storage 5G Trust Video Surveillance Figure 2: COLMENA framework software stack network conditions to dynamically establish communication channels that meet specific requirements such as latency, bandwidth, and reliability. The Smart System Controller provides a generic interface to interact securely and remotely with the embedded sensors and actuators, facilitating seamless integration with the physical environment. Designed for Infrastructure Owners, the ANT Configuration Manager provides a mechanism to restrict the software to run on the device, the access to its smart systems, the amount of shared computational and storage resources and its electric power consumption. All these components are deployed on each device and operate locally without requiring external interaction with other ANTs. The second layer establishes a collaboration plane among ANTs, transforming the infrastructure into a cohesive Swarm Platform rather than merely a collection of heterogeneous devices. While all components of this layer are deployed in each ANT, its components work cooperatively with their counterparts in other devices to achieve collective goals and ensure seamless operation. The Context Manager equips ANTs with comprehensive context awareness. It integrates self-context information, such as capacities (via the ANT Monitor), identity, location, and ownership, with environmentcontext information, including the capabilities, proximity, volatility, and trustworthiness of neighboring ANTs. The Sharing Manager facilitates seamless cooperation among ANTs by establishing mechanisms to share software images, data and even computational workload seamlessly across the continuum. Additionally, this component becomes an interface to enable external interaction offering a unified and cohesive view of the Swarm platform, thus simplifying operations over the platform as a whole. The Distributed Service Manager is responsible for selecting which roles to be hosted locally aiming to contribute to the overall service performance. To achieve this purpose it includes a QoE & QoS Assessment module that evaluates the status of the QoS/QoE metrics defined by the Developer as KPIs. 6 ANT 2 COLMENA Agent Role 3 Service A ANT 1 Role 5 - Service B Role 2 - Service A Role 2 - Service A Role 1 - Service A Behaviour Role Operation Abstractions ANT Monitor Role Optimizer ANT Configuration Manager COLMENA Agent Communication Controller Smart System Controller Sharing Manager Software Repository Compute Data Distributed Service Manager QoE & QoS Assessment AI-driven Role Selector Context Manager Selfawareness Environment Awareness ANT 3 COLMENA Agent Role 4 Service B Network Interfaces 5G Sensors & Actuators Compute Devices Data Storage Compute Abstractions Data Abstractions Communication Abstraction Smart Systems Abstraction Figure 3: Deployment model of the components of the software stack This information is fed into the Role Selection module, which uses decision-making policies to determine the number of instances of each role to be played by the device, considering its current conditions and owner preferences. Continuously reconsidering this decision allows a dynamic adaptation of the deployment delivering an agile, robust and efficient platform. Section Decision-making policies details the initial policies implemented in the prototype version of the Role Selection module. The Service Tools layer provides a suite of tools to support service stakeholders relying on the Swarm. These tools do not necessarily run on an ANT of the swarm; the can run on any device with access to the platform. The layer offers three tools designed to assist Developers in implementing and maintaining the service. The Swarm Programming Model allows the description of a service as a composite of different roles indicating their requirements and establishing their KPIs. Role Operation Abstractions streamline the development of role behaviors by providing generic interfaces to communicate, share data, interact with smart systems, and distribute the computational load of the role. Finally, the layer offers the Swarm Intelligence library that facilitates the development of intelligent services by incorporating continuous learning and Swarm Intelligence algorithms. At the end of the construction phase, Developers generate a software bundle that includes a description of the service, all the source code and necessary configuration files. This package is then delivered to the Service Providers, who use the Service Deployment Tool to instantiate the service. This tool processes the bundle to generate all necessary software images for running the service, such as role-specific images. Leveraging the Sharing Manager within the Swarm Platform layer, the tool publishes these generated images to the shared software repository and propagates the service description across the platform to enable its deployment. Following the publication of the service description, ANTs autonomously determine which roles to execute. During the operational phase, the Service Operation Dashboard enables Service Providers to monitor and manage deployed services. This dashboard supports continuous service improvements by facilitating software updates, thereby integrating a Continuous Integration/Continuous Deployment (CI/CD) methodology into the service life-cycle. The topmost layer of the stack comprises those Services relying on the platform to deliver solutions characterized by exceptionally high availability, reliability, and trustworthiness to End-Users. These services are the ultimate beneficiaries of the platform’s robust and adaptive architecture, ensuring seamless operation and responsiveness to dynamic demands while maintaining the highest standards of performance and security. To ensure secure service execution and facilitate software management, COLMENA employs virtual environments and containerization technologies like Docker or Balena. Figure 3illustrates the deployment of the components of the software stack. Each ANT operates a dedicated container, referred to as the COLMENA Agent, which encompasses the two lowest layers of the architecture for handling the underlying resources, providing efficient performance and supporting all the collaboration mechanisms. Service roles are executed 7 within individual containers, each containing the specific logic for their behavior alongside the implementation of the Role Operation Abstractions which interact with the components of the COLMENA Agent, promoting modularity, adaptability, and efficient integration. 4.2. Programming model The COLMENA Programming Model is designed to simplify the development of hyper-distributed applications by providing a clear framework for defining services and their roles, along with the performance indicators (KPIs) that will drive their deployment across the network of devices (ANTs). The model enables developers to break down services into modular, interacting roles, each with specific behaviors and responsibilities. Throughout this section, the use case described in Section Pilot use case: Video surveillance will be used to illustrate the application of the model, as demonstrated by the code snippet in Listing 1. In COLMENA, a service represents the overall structure of a hyper-distributed application and is defined by extending the Service class. Roles are the building blocks of these services and represent individual functionalities. These are implemented as inner classes of the Service extending the Role class. Using decorators on the constructor method of the service, developers specify which cooperation components are required for the service’s functionality. These decorators can define shared data spaces and communication channels for roles within the service, facilitating smooth cooperation between different instances of the roles. The @Data decorator designates a shared data space, and the @Channel decorator sets up communication channels to facilitate message passing between role instances. For roles to access a shared data space or publish/receive messages from a channel, developers have to decorate the constructor method of the role to indicate that it uses them. Within each role, the behavior method implements the core logic of the role and defines the actions it will perform. The behavior of a role can either be continuously running a specific task (Persistent roles), like reading a sensor or streaming data, or being triggered asynchronously reacting to events (Asynchronous roles) — such as messages received through one of the communication channels. To indicate the type of role, developers annotate the behavior function with the @Persistent decorator for the continuous roles, while the @Async decorator marks roles that should execute in response to incoming messages, specifying which message channel triggers the role’s behavior. In the example, the video surveillance service (defined in line 1) declares two roles, Sensing (line 9) and Processing (line 23). These roles rely on a communication channel named pending_images (declared in line 4). The Sensing role uses the pending_images channel (line 12) to continuously (line 17) publish images captured by the camera. The Processing role also uses the pending_images channel (line 25) but executes its behavior asynchronously whenever an image is retrieved from the channel (line 31), applying a computer vision algorithm to the image. 1class V i d e o S u r v e i l l a n c e ( S e r v i c e ) : 2 3@Context (" p r e m i s e s " ,c l a s s =Premises) 4@Channel ("pending_images" , scope=" " ) 5@Metric("processing_time") 6def __init__(s e l f ,*a r gs , ** kwargs ) : 7pass 8 9class Sensi n g ( Role ) : 10 @Requirements("/devices /camera" ) 11 @Context (" p r e m i s e s " , scope="*/lobb y " ) 12 @Channel ("pending_images") 13 @KPI(" nu m_play ers =1 " ) 14 def __init__ ( s e l f ,*arg , ** kwarg ) : 15 # I n i t i a l i z e t h e r o l e 16 17 @Persistent ( ) 18 def b e h a v i o r ( s e l f ) : 19 image =# Get image from camera 20 time =time . time ( ) 21 s e l f . p end ing _i mag es . p u b l i s h ( [ time , image ] ) 22 23 class P r o c e s s i n g ( Role ) : 24 @Requirements("/processor /GPU" ) 25 @Channel ("pending_images") 26 @Metric("processing_time") 27 @KPI("processing_time [2s] <3 " ) 28 def __init__ ( s e l f ,*arg , ** kwarg ) : 29 # I n i t i a l i z e t h e r o l e 30 31 @Async( message="pending_images") 32 def b e h a v i o r ( s e l f , message ) : 33 t_capture =message [ 0 ] 34 image =message [ 1 ] 35 pr o c ess_ i m a ge ( image ) 36 t_processed =time . time ( ) 37 delay =t_processed −t_capture 38 s e l f . p r o c e s s i n g _ t i m e . p u b l i s h ( d el ay ) 39 40 def process_image ( s e l f , image ) : 41 # C a ll Computer V i s i o n L i b r a r y 42 # R e t u r n s p r o c e s s i n g t i m e 43 44 class Pr e m i ses ( Con t e x t ) : 45 def __init__ ( s e l f ,*args , ** kwargs ) : 46 super ( ) . _ _ i n i t _ _ ( *args , ** kwargs) 47 48 def locate (s e l f , d e v i c e ) : 49 # O bt ai n t h e c o r r e s p o n d i n g l o c a t i o n u s i n g 50 # d e v i c e s c o n f i g u r a t i o n and s ens or data . 51 # location =" Headquarters . f l o o r 2 . lobby " 52 return l o c a t i o n Listing 1: Implementation of the video surveillance use case In addition to defining the role’s behavior, developers can also specify the hardware and software requirements needed for a role to function properly. These requirements are automatically taken into account by ANTs when deciding which roles to run. The @Requirements decorator on a role constructor sets 8 constraints on the hardware and software a device must possess to host the role execution. For example, the Sensing role specifies that the host device must be equipped with a camera (line 10); and the Processing role requires a GPU (line 24). A critical factor for effectively deploying a hyperdistributed application in the Compute Continuum is the contextual circumstances of each device. Attributes such as the geographical region (e.g., country or city), precise GPS coordinates, or the ownership of the device can significantly influence the deployment strategy. To address this, COLMENA encapsulates a specific situational attribute of the devices within the Context concept. Rather than relying solely on predefined contexts, COLMENA enables the definition of custom contextual circumstances by extending the Context class. This involves implementing the locate method to determine the device’s current classification based on its configuration and sensor data. This approach provides COLMENA with the flexibility to tailor contextual definitions to the unique needs of each application. Using the @Context decorator on the constructor method of a role, developers can restrict the deployment of the role to those ANTs that currently meet that circumstantial aspect. For example, the video surveillance use case defines a new Context to describe the location of the devices within the company premises (line 44). For controlling people entering the company buildings, only those devices in any of the lobbies within the premises are needed to host the Sensing role (line 11). While the example demonstrates the use of a single context to constrain the deployment of a role to specific circumstances, COLMENA supports more complex conditions that combine multiple contexts. For example, deployment criteria could simultaneously require devices to be located in the lobby, manufactured by ACME Inc., and owned by the company. In addition to restricting the deployment of roles, contexts can also limit the scope of role cooperation and facilitate the creation of multiple shared data spaces or communication channels tailored to specific cooperation environments. Developers can specify the sharing scope by setting the scope parameter in the @Data and @Channel decorators on the service constructor. The video surveillance example only establishes one single communication channel for the whole system (line 4); however, other services using the same context may need to establish a communication channel for each floor or keep a private instance for a piece of data in each ANT. Quality of Experience (QoE), Quality of Service (QoS), and performance indicators play a crucial role in determining the instantiation of roles within COLMENA. By annotating the constructor method of a role with @KPI, specific conditions can be set to evaluate whether the execution of the role is necessary or not. These conditions rely on metrics that are collected dynamically during runtime. COLMENA collects some predefined metrics which are broadly applicable to most of the services; for instance, the number of devices currently playing a specific role. In addition, applicationspecific metrics can be introduced by annotating the service constructor with the @Metric decorator specifying the name of the metric. The responsibility for managing these custom metrics relies entirely on the application logic; their roles, also annotated with the @Metric decorator with a matching name, must explicitly expose their values to ensure proper integration. Often, the conditions defined by a KPI are not concerned with the global, instantaneous value of a metric but instead rely on an aggregated value over a specific timeframe. For example, the aggregated metric over the last five seconds of operation may be more relevant for decision-making. Similarly, KPIs may also impose constraints based on a given contextual scope. For instance, they can limit the number of devices running the role belonging to the same owner or monitor the amount of workload pending to be processed in the building where the ANT is located to adjust the number of nodes playing the role processing it locally. The @KPI decorator accepts expressions following the pattern MetricName("ContextName"="scope") [timeframe] enabling developers to specify both the desired timeframe and the contextual scope of the metric. The pilot use case introduces a custom metric (line 5), processing_time, to measure the time required to process each image. This metric enables dynamic role deployment by providing a feedback loop based on performance. The Sensing role contributes to this metric by adding a timestamp to each captured image before publishing it to the pending_images channel. The Processing role calculates and exposes the processing_time metric(lines 38); since the behavior of the role explicitly needs the metric, it is required to annotate also the role construction with the @Metric decorator (line 26). The deployment of the Processing role is governed by a @KPI condition relying on this custom metric. Specifically, the KPI requires the average value of processing_time during the last two seconds to remain below three seconds. If this threshold is exceeded, additional instances of the Processing role should be instantiated on suitable ANTs. Figure 4illustrates how 9 (a) Number of active devices (b) Number of active roles (c) Processing time KPI Figure 8: Evaluation results of the distributed implementation of COLMENA. composition of interacting roles (or functionalities) and leverages the lower layers of the stack to transform each device into an autonomous agent aware of its capabilities and current context. To further facilitate the programming, the framework also provides communication and shared data abstractions, enabling seamless cooperation among roles and devices. Unlike existing solutions that rely on centralized orchestration engines or consensus mechanisms, COLMENA adopts a groundbreaking approach based on the "freedom to fail," allowing devices to autonomously select roles based on the current and desired levels of key indicators, such as QoS, performance, or application-specific requirements. We demonstrated the framework with a pilot use case: a video surveillance system comprising two roles, Sensing and Processing. The use case is implemented using the programming model and evaluated within both a custom-built simulator and a first version of the agent’s software stack. The custom-built simulator mirrors the framework’s core architecture. This simulator includes agent modules such as the service manager, sharing manager, and context manager, and provides an API for developing and evaluating services in a manner consistent with the programming model. Decision-making policies are crucial in enabling autonomy in agents. In this article, we introduce three RL-based algorithms (SARSA, Q-Learning, and interconnected Learning Automata) and a rule-based algorithm, which are implemented and evaluated in the simulator. Results indicate that agents successfully learn behaviours that converge toward desired QoS levels in typical environments. However, our experiments revealed challenges for some algorithms in dynamic environments. Future work will focus on enhancing adaptability by incorporating cooperation mechanisms common in multi-agent RL and exploring multi-objective algorithms that optimise multiple KPIs. Additional policies, such as auction-based mechanisms, will also be investigated. We plan to implement these policies directly within the agent’s software stack (within the Role Selector component), to assess their performance in heterogeneous and distributed environments. For example, resource-constrained devices may use lightweight, rulebased algorithms, while more capable devices can leverage complex, AI-driven policies. Additionally, we have developed an initial version of the COLMENA framework, which is publicly available on GitHub. This release includes foundational components, such as the ANT components, programming model and role abstractions. The video surveillance use case is deployed on a testbed of heterogeneous devices, demonstrating the system’s reactivity to dynamic events, such as adding or removing devices, under a rule-based policy. Future work will iterate on the middleware to integrate peer-to-peer solutions for message passing, data management, and compute sharing. We will also expand the architecture to address diverse use cases where COLMENA can create meaningful impacts, including healthcare, renewable energy integration, transportation, logistics, and beyond. Acknowledgments This work is co-financed by the COLMENA project of the UNICO I+D Cloud program that has the Ministry for Digital Transformation and of Civil Service and the EU-Next Generation EU as financing entities, within the framework of the PRTR and the MRR. It has also been partially supported by projects CEX2021-001148-S and PID2023-147979NB-C21 from the MCIN/AEI and MICIU/AEI /10.13039/501100011033 and by FEDER, 16 UE, and by the Departament de Recerca i Universitats de la Generalitat de Catalunya, research group MPiEDist (2021 SGR 00412). References [1] E. Commission, D.-G. for Communication, The European data strategy – Shaping Europe’s digital future, Publications Office, 2020. [2] Y. Ai, et al., Edge computing technologies for internet of things: a primer, Digital Communications and Networks 4 (2) (2018) 77–86. [3] A. J. Ferrer, et al., Towards a cognitive compute continuum: An architecture for ad-hoc selfmanaged swarms, in: 2021 IEEE/ACM 21st Int. Symp. on Cluster, Cloud and Internet Computing (CCGrid), IEEE, 2021, pp. 634–641. [4] K. Toledo, et al., Clusterlink: A multi-cluster application interconnect, in: Proceedings of the 16th ACM International Conference on Systems and Storage, SYSTOR ’23, Association for Computing Machinery, New York, NY, USA, 2023, p. 138. doi:10.1145/3579370.3594747. [5] M. Selimi, et al., Picasso: Enabling informationcentric multi-tenancy at the edge of community mesh networks, Computer Networks 164 (2019) 106897. [6] F. Lordan, et al., Colony: Parallel functions as a service on the cloud-edge continuum, in: L. Sousa, N. Roma, P. Tomás (Eds.), Euro-Par 2021: Parallel Processing, Springer International Publishing, Cham, 2021, pp. 269–284. [7] G. R. Russo, et al., Serverledge: Decentralized function-as-a-service for the edge-cloud continuum, in: 2023 IEEE International Conference on Pervasive Computing and Communications (PerCom), IEEE, 2023, pp. 131–140. [8] C. Adams, et al., Monarch: Google’s planet-scale in-memory time series database, Proc. VLDB Endowment 13 (12) (2020) 3181–3194. [9] C. Wang, et al., Apache iotdb: time-series database for internet of things, Proc. the VLDB Endowment 13 (12) (2020) 2901–2904. [10] J. Martí, et al., Dataclay: A distributed data store for effective inter-player data sharing, J. Systems and Software 131 (2017) 129–145. [11] M. Zaharia, et al., Spark : Cluster Computing with Working Sets, HotCloud’10 Proc. the 2nd USENIX Conf. on Hot topics in cloud computing (2010). [12] J. Dean, S. Ghemawat, Mapreduce: simplified data processing on large clusters, Communications of the ACM 51 (1) (2008) 107–113. [13] E. Deelman, et al., Pegasus, a workflow management system for science automation, Future Generation Computer Systems 46 (2015) 17–35. [14] F. Lordan, et al., ServiceSs: An Interoperable Programming Framework for the Cloud, J. Grid Computing 12 (1) (2014) 67–91. [15] R. Virding, C. Wikström, M. Williams, Concurrent Programming in ERLANG (2Nd Ed.), Prentice Hall International (UK) Ltd., Hertfordshire, UK, UK, 1996. [16] M. K. Gupta, Akka Essentials, Packt Publishing, 2012. [17] C. Pinciroli, G. Beltrame, Swarm-oriented programming of distributed robot networks, Computer 49 (12) (2016) 32–41. [18] M. Brambilla, et al., Swarm robotics: a review from the swarm engineering perspective, Swarm Intelligence 7 (2013) 1–41. [19] S. Jones, et al., Evolving behaviour trees for swarm robotics, in: Distributed Autonomous Robotic Systems: The 13th Int. Symp., Springer, 2018, pp. 487–501. [20] J. Kuckling, et al., Behavior trees as a control architecture in the automatic modular design of robot swarms, in: Swarm Intelligence: 11th Int. Conf., ANTS 2018, Rome, Italy, October 29–31, 2018, Proc. 11, Springer, 2018, pp. 30–43. [21] S. H. Christie, et al., Mandrake: multiagent systems as a basis for programming fault-tolerant decentralized applications, Autonomous Agents and Multi-Agent Systems 36 (1) (2022) 16. [22] D. Smirnova, et al., Protocols over things: A decentralized programming model for the internet of things, Computer 53 (12) (2020) 60–68. [23] B. P. Gerkey, M. J. Matari´ c, A formal analysis and taxonomy of task allocation in multi-robot systems, The Int. J. of robotics research 23 (9) (2004) 939–954. 17 [24] G. A. Korsah, et al., A comprehensive taxonomy for multi-robot task allocation, The Int. J. of Robotics Research 32 (12) (2013) 1495–1512. [25] A. Campbell, A. S. Wu, Multi-agent role allocation: issues, approaches, and multiple perspectives, Autonomous agents and multi-agent systems 22 (2011) 317–355. [26] V. Karagiannis, et al., Edge computing with peer to peer interactions: Use cases and impact, in: Proc. Workshop on Fog Computing and the IoT, 2019, pp. 46–50. [27] S. V. Albrecht, F. Christianos, L. Schäfer, Multiagent reinforcement learning: Foundations and modern approaches, MIT Press, 2024. [28] C. J. Watkins, P. Dayan, Q-learning, Machine learning 8 (1992) 279–292. [29] G. A. Rummery, M. Niranjan, On-line Q-learning using connectionist systems, Vol. 37, University of Cambridge, Department of Engineering Cambridge, UK, 1994. [30] M. A. Wiering, M. Van Otterlo, Reinforcement learning, Adaptation, learning, and optimization 12 (3) (2012) 729. [31] R. Wheeler, K. Narendra, Decentralized learning in finite markov chains, IEEE Transactions on Automatic Control 31 (6) (1986) 519–526. [32] P. Vrancx, K. Verbeeck, A. Nowé, Decentralized learning in markov games, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 38 (4) (2008) 976–981. [33] M. Tokic, Adaptive ε-greedy exploration in reinforcement learning based on value differences, in: Annual conference on artificial intelligence, Springer, 2010, pp. 203–210. 18