Decentralized Multi-Agent Reinforcement Learning for the Green Serverless Cloud-Edge Continuum

Decentralized Multi-Agent Reinforcement Learning for the Green Serverless Cloud-Edge Continuum Yashwant Singh Patel§, Anurag Choubey†, Anil Singh‡, Paul Townend§ §Department of Computing Science, Ume˚ a University, Ume˚ a, Sweden †School of Computer Science Engineering & Technology, Bennett University, Greater Noida, India ‡Department of Computer Science and Engineering, Thapar Institute of Engineering and Technology, Patiala, Punjab, India E-mail:{yashwant.patel, paul.townend}@umu.se, [email protected], [email protected] Abstract—The Cloud-Edge Continuum systems are inherently complex and massive, often featuring federated multi-provider stakeholders (e.g. cloud/edge service providers, energy providers), heterogeneous platforms, and dynamic infrastructures; this significantly increases the complexity of developing, deploying, and managing applications. The Serverless computing offers a powerful tool to simplify and speed up the Continuum application development. However, existing scheduling mechanisms for Serverless platforms focus primarily on performance metrics such as latency, model accuracy, and throughput, often neglecting critical factors such as energy efficiency and sustainability. This gap is further exacerbated in Continuum environments, where computational nodes may rely on unpredictable and intermittent green energy sources, leading to availability bottlenecks and energy constraints. This work investigates the design of a decentralized green energy-aware approach for scheduling Serverless functions across the Cloud-Edge Continuum. To achieve this, we introduce a formal model of the green energy-aware workload scheduling problem. We then develop a consensus-based upper confidence bound (UCB) approach for cooperative multi-agent reinforcement learning (MARL) that leverages distributed agents to consider energy awareness and quality-of-service (QoS) requirements of different functions into their scheduling decisions. To demonstrate the practicality of our approach, we implement a realworld prototype using a cluster of Raspberry Pis, Cloud servers, Kubernetes, and OpenFaaS. Experimental results show that our approach maximizes the green energy utilization by (44%) and reduces total latency by (25%) compared to the centralized technique, highlighting its energy efficiency, scalability, and overall sustainability in Continuum settings. Index Terms—Serverless Computing, Function-as-a-Service, Cloud-Edge Continuum, Multi-Agent Reinforcement Learning, Green Computing. I. INTRODUCTION Cloud infrastructures have evolved from centralised largescale systems to distributed federations covering cloud data centers, edge & fog nodes, 5G/6G networks - collectively known as the Cloud-Edge Continuum, facilitating ultra-low latency, greater flexibility, improved scalability, and enhanced efficiency [1]. The Cloud-Edge Continuum system shown in Fig. 1, is a spectrum of federated regions that comprises several IoT devices, edge & fog data centers (small/medium scale) and Cloud data centers (large scale), powered by a mix of green and brown energy sources, and managed by multiple service providers through the backbone network. To meet the demands of modern applications, a Continuum system must effectively manage the massive complexity of federated infrastructure and energy providers, heterogeneous resources, and dynamic environments, while ensuring a fully automated and secure orchestration over public networks [2]. These requirements pose significant integration challenges for service providers, and add layers of complexity for resource management across multi-tiered and multi-provider infrastructures. Fig. 1. Serverless Cloud-Edge Continuum system. Serverless computing substantially reduces the complexity of application development by abstracting infrastructure concerns such as availability, scalability, fault tolerance, over/under-provisioning of resources, and other infrastructure issues, allowing developers to focus on functionality, implementing it as invokable services commonly known as Function-as-a-Service (FaaS); while service providers handling critical tasks such as deployment, provisioning, and scaling, with a focus on optimizing efficiency, load distribution, and cost-effectiveness [3]. Although serverless computing models reduce the operational overhead for developers, they shift challenges such as resource management and energy efficiency to service providers. On a large scale, Continuum’s data centers and networks are estimated to consume up to 18% of global electricity consumption by 2030 [4], placing a serious strain on national power grids [5], and raising significant environmental risks, particularly given the fact that nearly 80% of global energy demand still comes from non-renewable (brown energy) sources [6]. As global energy demand continues to grow, power grids face mounting pressure to balance the demands of CloudEdge Continuum systems in addition to other major energy consumers. This escalating strain underscores the urgent need for strategies that mitigate, optimize, and, where possible, reduce energy consumption. One of the most promising solutions is energy-aware resource scheduling in Serverless CloudEdge systems; by continuously monitoring large federated infrastructures, this technique enables the dynamic placement of functions to the most energy-efficient resources - while still accounting for factors related to pricing, QoS, and performance. However, most of the existing scheduling approaches for Serverless Cloud-Edge platforms largely overlook energy efficiency during function execution - and none fully incorporate critical energy-related factors such as the availability of green versus brown energy sources, multi-provider settings, energy and pricing policies, or the need to balance energy consumption, etc. [3]. Energy-awareness is especially critical in the context of Cloud-Edge Continuum, where autonomous resource management systems must operate across large, federated, multi-provider infrastructures. To address these key omissions, we propose a decentralized scheduling mechanism based on multi-agent reinforcement learning, where distributed agents incorporate energy awareness into their scheduling decisions, prioritizing regional nodes powered by renewable energy while ensuring Serverless functions completion within required QoS constraints. The multi-agent reinforcement learning (MARL) approach presented here focuses on creating a shared environment in which multiple agents interact through a message-passing mechanism. Each agent is assigned to a specific region within the Continuum ecosystem and collaborates with others to form a dynamic, cooperative system. This system adapts to the decisions of its peers in order to maximize cumulative rewards across all matches. The agents’ decision-making process, aimed at optimizing these matches, is guided by a hierarchical leader election strategy. The key contributions of this paper are as follows: •Problem: We derive the problem of decentralized energy management for Serverless and multi-provider Green Cloud Continuum. •Research approach: We design a consensus-based upper confidence bound approach for cooperative multi-agent reinforcement learning, in which multiple agents interact through a message-passing mechanism to dynamically manage resource allocations to Serverless functions. •Prototype: We implement a real-world prototype using a cluster of Raspberry Pis, Cloud servers, Kubernetes, and OpenFaaS to validate the system’s performance and scalability by comparing centralized and decentralized approaches under varying serverless workloads. II. RELATED WORK Energy-aware scheduling within the Serverless Cloud Continuum remains underexplored, with few solutions developed specifically for Serverless Edge or Cloud systems. For instance, Rastegar et al. [7] introduce ‘EneX’, an energyconscious execution scheduler for Serverless providers. Their work explores online and offline scheduling strategies; while considering performance, scalability, and computational overhead. Whereas ActionWhisk [8] prioritizes cost optimization over energy efficiency, Rausch et al. [9] aim to reduce latency by exploring functions and tasks relocation strategies within a cluster. Adeppady et al. [10] develop a threshold-based technique to minimise energy usage in Serverless edge environments by efficiently managing container states while maintaining target service latency. Similarly, Verma et al. [11] propose ‘LEASE’, a resource allocation framework for serverless functions supporting microservices with strict deadline requirements. To enhance scheduling efficiency, LEASE employs a prioritybased mechanism that dynamically offloads functions from over-provisioned to under-provisioned nodes, balancing energy consumption with service completion time. Russo et al. [12], [13] highlight that most existing FaaS frameworks are primarily designed for cloud environments, often overlooking edge specific requirements. To address this gap, they introduce ‘Serverledge’, an open-source FaaS framework that bridges the gap between Serverless computing at the Edge and in the Cloud. Their framework supports advanced features such as asynchronous function invocation, live function migration, and compatibility with Docker and Podman as function runtimes. Aslanpour et al. [14] address the challenges of energy variability and operational imbalance in Serverless edge computing, particularly for IoT applications powered by green energy. They introduce the ‘faashouse’ system design, which optimizes resource utilization and function-to-node allocation to guarantee balanced energy distribution and enhance overall system sustainability. The framework integrates computation offloading and introduces a novel scoring-based assignment strategy inspired by Kubernetes. Experimental results show substantial improvements, with a 46% increase in operational availability and a 44% boost in throughput. In a related study, Aslanpour et al. [15] explore energy-aware resource scheduling challenges for edge nodes powered by green energy and batteries, operating within a Serverless environment. They propose an algorithm that incorporates features of warm scheduling,sticky offloading, and priority-based scheduling across multiple zones. Patel et al. [3] propose a green energy-aware scheduling approach for Serverless functions based on a stable matching heuristic. Their method uses distributed controllers (e.g., local controllers and a global controller) to maximize green energy utilization while satisfying QoS requirements of functions. However, dependence on a global controller creates a single point of failure, potentially affecting the resilience of the Continuum system. Agiollo et al. [16] present an energy-efficient federated learning framework for resource-constrained edge environments. By dynamically managing energy consumption, computational capacity, and QoS needs across distributed and heterogeneous edge nodes; their approach achieves energy savings of 30% to 60% compared to existing methods. The framework also integrates serverless computing and container orchestration to facilitate efficient worker selection and realtime energy monitoring across edge devices. Gu et al. [17] investigate a deep reinforcement learning approach for energyaware scheduling and service management in edge environments. Their method utilizes real-time system state observations to dynamically optimize decisions, aiming to minimize long-term operational costs. Beyond the works discussed above, few studies have explored the concept of ‘MARL’ in decentralized matching environments. For example, Saha et al. [18] propose a generic bandit learning model that creates a trade-off between maximizing cardinality and ensuring stability in MARL settings. Shao et al. [19] introduce a self-organizing multi-agent system featuring aconductor election mechanism to automate decision-making. Taywade et al. [20], [21] design a random agent selection strategy for matching decisions, incorporating a fairness factor to harmonize fairness with stability. Collectively, these works contribute to the development of MARL environments for optimal decision-making within their respective frameworks. TABLE I COMPARISON OF EXISTING APPROACHES Reference Comparative Factors Green Energy Decentralization MARL Message-Passing Scalability [3] ! # # # ! [7] # # # # ! [9] # # # # ! [10] # # # # # [12] # ! # # ! [13] # ! # # ! [14] ! # # # # [15] ! # # # # [16] # # # # # [17] ! # # # # Proposed ! ! ! ! ! Our analysis of existing research (as summarized in Table I) highlights the lack of approaches considering green-energy awareness, decentralization, scalability, message-passing, and multi-agent reinforcement learning-driven approaches. To address this, we propose a consensus-based upper confidence bound approach for cooperative multi-agent reinforcement learning, where multiple agents communicate via message passing to dynamically manage resource allocation for Serverless functions. This design enables a self-organizing multiagent system that orchestrates function execution across multiprovider Cloud-Edge Continuum to maximize cumulative rewards across all Serverless functions and Continuum resource matches. III. SYSTEM MODELING AND PROBLEM FORMULATION A. System Architecture The overall system architecture is illustrated in Fig. 1, and follows the design proposed in [1], [3]. The Continuum comprises federated regions, each encompassing multiple cloud and edge/fog nodes equipped with heterogeneous computational resources. These regions are interconnected through high-speed backbone networks and powered by a hybrid energy system that includes mix of green energy, brown energy, and energy storage devices, each with varying energy generation rates. Energy demand typically peaks during certain hours and drops during off-peak periods. To optimize energy usage, green energy can be stored during low-demand periods and utilized during peak times, reducing dependence on brown energy. The Continuum adopts a decentralized scheduling model, where each node is capable of independently scheduling incoming requests; which is a critical feature for managing edge-generated workloads. Each request consists of multiple microservices, and serverless functions execute tasks associated with them. To support this dynamic environment, each region includes an agent called a ‘regional agent’, responsible for autonomic and adaptive decision-making based on localized data and Continuum behavior. B. System Modeling We model the system using discrete, equal-length time slots, defined as T={t|t∈[0, T],(t+ 1) −t= ∆t}, where ∆tis the time step duration in seconds. The Continuum consists of a set of geographical regions denoted as R={rv|v∈[1, n]}, each hosting a set of heterogeneous mnodes of different models and computational capabilities denoted as H={hv u|u∈[1, m], v ∈[1, n]}. A regional agent, which is not part of H, is responsible for managing the resources. The renewable energy input to node uat time tis modeled as G={gv ut,|u∈[1, m], v ∈[1, n], t ∈ T , gv ut ≥0}. The corresponding energy consumption is captured by E= {pv ut|u∈[1, m], v ∈[1, n], t ∈ T , pv ut ≥0}. In each geographical region, function invocations are generated at varying rates. The State of Charge (SoC), expressed by the set SOC = {sv ut,|u∈[1, m], v ∈[1, n], t ∈ T , θ ≥sv ut ≥0}represents the remaining battery charge of computing nodes, where θis the maximum battery capacity. The processing capability of each regional node is expressed in MIPS (Million Instructions Per Second), expressed by the set C={cpv ut,|u∈[1, m], v ∈ [1, n], t ∈ T , cpv ut ∈[0,Γ]}, where Γindicates the maximum computational capacity. Regional nodes are eligible to host serverless functions only if their SoC satisfies the minimum energy threshold sv ut ≥Ψ, where Ψis a predefined minimum energy level. Node availability is defined by the binary variable Z={zu,v t|u∈[1, m], v ∈[1, n], t ∈T, zu t∈ {0,1}}, where zu,v t= 0 indicates the regional node is unavailable at time t. The overall state of the nodes across all geographical regions in the Continuum can thus be mathematically modeled using the above variables. hv u∈ H :zu,v t=(1,if sv ut ≥Ψ 0,if sv ut <Ψ(1) The SoC sv ut is determined by three key factors: (i) the green energy input gv ut, (ii) the SoC from the previous time slot sv u(t−1), and (iii) the energy consumption pv ut at t. For each node hv u∈H, the SoC sv ut can be expressed as [14], [15]: ∀hv u∈H:su,v t= min(θ, max(0, gv ut +sv u(t−1) −pv ut)) (2) Next, we define the incoming function invocations as F= {fu,v,w t|u∈[1, m], v ∈[1, n], w ∈[1, l], t ∈ T } for βw microservices managed by regional node hv uat time t. Each serverless function instance is characterized by five tuples: fu,v,w =< fu,v,w ID , fu,v,w pri , fu,v,w st , fu,v,w ft , fu,v,w q>, where fu,v,w ID is the identifier of the invokable function, fu,v,w pri is the execution priority (e.g., low or high), fu,v,w st is the function’s arrival time, fu,v,w ft is the execution deadline, and fu,v,w q is the resource requirement for function deployment. The resource demand for function execution is further defined as Q={qw|w∈ {1, l}, qw∈[0,Γ]}, where qwmust not exceed the hosting node’s maximum capacity Γ. C. Problem Formulation The objective of the Serverless function scheduling in multiprovider Cloud-Edge Continuum is to reduce the usage of brown energy by maximising the green energy utilization at ttime slot while satisfying QoS constraints. However, our approach is flexible and supports multiple scheduling priorities and objective functions beyond the one discussed here. Formally, the scheduling objective can be expressed as: min X t m X u=1 n X v=1 (max(0, pv ut −(gv ut +sv u(t−1)))) (3) s.t. l X w=1 (ξu,v,w t×fu,v,w q)!≤(zu,v t×Γ), t ∈ T ,∀hv u∈ H (4) m X u=1 n X v=1 ξu,v,w t!= m X u=1 n X v=1 κu,v,w t!,∀t∈ T (5) fu,v,w wt +fu,v,w pt ≤fu,v,w ft (6) SOV(fu,v,w t, hv u)=SOV(fu,v,w t, hp) =⇒fu,v,w t×zu,v t= 0 (7) Constraint 4 guarantees that the total capacity demand resulting from the placement of serverless functions should not exceed the maximum processing capacity Γof any regional node, regardless of whether the functions are executed locally or offloaded to remote nodes. Furthermore, the variable zu,v t prevents allocations to nodes that are currently unavailable. The function assignments are represented by the set A= {ξu,v,w t|u∈[1, m], v ∈[1, n], w ∈[1, l], t ∈ T , ξ ≥0}, where each ξu,v,w texpresses the total number of assigned function replicas, combining local assignments λu,v,w tand remote assignments ρu,v,w t. For any time period t, constraint 5 guarantees that all required function replicas κu,v,w tare assigned. Constraint 6 enforces that the maximum completion time fu,v,w ft for a function does not exceed its specified deadline. In this context, fu,v,w wt denotes the waiting time before the function is scheduled, and fu,v,w pt represents its processing time. Constraint 7 enforces that the serverless functions can only be placed to nodes (e.g., candidate node hp); where the functions maintain complete control and exclusive authority over the allocated resources. Here, data sovereignty (SOV) can be represented as a set of constraints-such as blacklists of resource providers or nodes, along with other policy rules-as suggested in [1]. However, the Serverless function scheduling problem is NPhard, making it computationally intractable to solve optimally in polynomial time. Therefore, we introduce a novel multiagent reinforcement learning-based solution. This approach enables decentralized decision-making and allows regional agents to learn adaptive scheduling policies through interaction with the Continuum environment. IV. DECENTRALIZED MULTI-AGENT REINFORCEMENT LEARNING APPROACH In this section, we first introduce preliminary concepts to establish the foundation for our proposed cooperative MARL approach. We then describe the consensus-based upper confidence bound approach in detail, covering the initialization, submarket formation, and exploitation phases. A. Preliminary Concepts •Elementary components: We propose a cooperative multi-agent reinforcement learning (MARL) framework in which each agent independently learns the matching tendencies between regional nodes and serverless functions to identify optimal matches for itself. A reinforcement learning agent is deployed in each region, where it not only learns from its own interactions but also communicates with other agents through a messagepassing mechanism to update its parameters. Each agent perceives the current state of the environment, takes an action in coordination with others, and causes the environment to transition to a new state at every time step. The agent then receives a reward that reflects the quality of the transition, aiming to maximize its expected cumulative reward over time. Although agents learn independently, their actions influence the shared environment and consequently affect the learning dynamics of other agents. Since each agent receives an individual, intrinsic reward, we formulate our problem as a Markov decision game; an extension of the Markov decision process suitable for decentralized multiagent control, where each regional agent optimizes its own reward function. During each round of the matching process, a leader election mechanism is employed, in which a subset of predefined agents is stochastically nominated as potential leaders. From these candidates, a consensus requiring over two-thirds majority is reached to resolve conflicting matches. This process is repeated for all such conflicts across the various regions of the Cloud-Edge Continuum. The decentralized Markov decision process (DMP) among Nagents is modeled as DMP =< SP, AP, RP,¯ P, ag, ¯ Y , IM, η >, where Si represents each state in the environment, at each time slot t.Airepresents the action taken by each agent corresponding to its learning from the environment and by interacting with the other agents as A≡AN P.¯ P(Si P|Si P+ 1, Ap) : S X A X S →[0,1]. ag represents the agents in action, with the reward vector Rdesigned similarly for each agent and ηdenotes the discount factor. The function ¯ Yrepresents the observation function for each agent under the initial matching IMi, with stochastic policy ¯ P, which denotes the concatenation of individual agents’ action with their decision making vector. We refer to this as the Action-Decision function. •Message-passing among agents: At each time slot, each agent Ai Psends a message mjto its nearest neighbor, which is gradually broadcast across all regional agents and similarly received from each of them. At the event of consensus decision-making, only the message-type changes; with all other properties remain the same as in the original message, along with an additional flag of either 0 or 1 indicating the consensus status. Additionally, these messages are preserved to ensure that each agent has equal participation in the election process. •Leader election-based acute decision-making among cooperating MARL agents: The proposed leader election mechanism plays a critical role in resolving matching conflicts within each region, aiming to maximize the number of successful matches while allowing for some compromise on preferences to ensure eventual maximization of cumulative rewards. At each time slot, a manyto-one matching process takes place within each region between serverless functions i.e. triggered by the devices and regional nodes of the Cloud-Edge Continuum system as illustrated in Fig. 2. These matchings are guided by predefined preferences, which are modeled based on the system constraints described earlier. To manage soft constraints, preference relaxation is applied, allowing some flexibility in matching priorities. Conflicts arising from this relaxation are resolved through consensus among all participating agents using a message-passing mechanism with binary decisionmaking. In each round, a subset of pre-selected agents participates in the decision-making process, and a consensus is reached if at least two-thirds of the agents agree. This process continues iteratively until no further conflicts remain in any region during that round, and the maximum possible matching cardinality is achieved. Fig. 2. Proposed multi-agent reinforcement learning. B. Proposed Algorithm We propose a Consensus-based Upper Confidence Bound algorithm for the cooperative multi-agent reinforcement learning problem, modeled as a Multi-Arm Bandit (MAB) Learning for many-to-one stable matching. The steps of the algorithm (described in Algorithm 1) are discussed as follows: •Initialization Phase: The process begins by setting up a MAB environment corresponding to the matching iteration. Within each region, a native agent; which is aware of all the preferences - is responsible for creating an initial reinforcement learning tuple vector. Each agent is assigned an index, and each node (with both serverless functions and Continuum nodes) within a region (monitored by its respective agent), learns its rank corresponding to the preference vector. Each node then attempts to pull the best possible arm combination to maximize its reward. Initially, the action space is restricted to either preferred or non-preferred pairs, under a local ranking policy. •Submarket Phase: In each round r, some regional agents settle on certain arms satisfactorily and enter the exploitation phase. Agents and arms that remain unsettled stay active in the market, continuing to learn in search of a stable match. To maximize the cardinality of matches, agents enter a conflict phase, where unmatched pairs are considered using a relaxed preference list. Once agents enter in the conflict phase, conflicts are resolved through a consensus mechanism based on a leader election-based cooperation approach, by updating the parameter of a global policy. A set of agents is selected to participate in the consensus process, which converges once a two-third majority is reached. It is ensured that the policy of each agent is not dominant in nature and flexible for adaptation to a new preference list bounded by a maximum threshold value for instability. •Exploitation Phase: Every agent will try to choose its empirical optimal arm based on the steps of the upper-confidence bound (UCB) approach; which is well known for solving multi-arm bandit problems. The UCB algorithm, applied per region and per node, resolves uncertainty by optimistically picking the highest upper bound and lower bound based on the action-value estimate driven from the mean and standard deviation of each arm for every agent and round. It should be noted that the choice of arms is also limited by their number of uses and their feasibility, which are native concepts in the UCB approach. C. Algorithmic Analysis Theorem 1. The message passing complexity of the proposed multi-agent consensus-UCB algorithm (shown in Algorithm 1) is O(τ. n. d(F+H+δ)). Proof. : The message complexity of the proposed consensusenabled multi-agent UCB algorithm depends on the number of rounds, regional agents participating in the consensus process for decision-making, and the participating nodes. Suppose there are nregions, each with a single agent, and kpreselected agents participating in the leader election. Assuming there are τrounds for satisfying user requests and δconflicts to be resolved. Finally, sharing the matching decision with Ffunctions and regional nodes H, the total number of messages exchanged during the process is governed within O(τ.n.k(F+H+δ)). However, since only a subset of agents din kneed to communicate with each other, where d≪n, the overall complexity can be rewritten as O(τ.n.d(F+H+δ)). Compared with the centralized approach of message-passing among the participating entities for τrounds, where nagents work within the centralized ecosystem, we get a message complexity of O(τ.n.(F+H)). Figure 3 shows the overall message exchange mechanism employed in the proposed MARL approach. Theorem 2. The time complexity of the proposed multiagent consensus-UCB algorithm (shown in Algorithm 1) is O(τ. F.H. n. log2n)). Proof. : The overall complexity of the proposed consensusUCB algorithm can be broken down into distinct phases. The initialization phase takes O(n)time for the action space per region with a single agent. The submarket phase requires mapping unmatched nodes with a preference list in a many-to-one matching, which runs in O(F.(n.H+ nlog2n). Finally, the exploitation phase has a complexity of O(F.H.n). The overall time complexity for τrounds becomes O(τ. F.H. n. log2n). V. PERFORMANCE EVALUATION A. Testbed Settings The testbed emulates a Cloud-Edge Continuum setting with a master-worker setup, comprising of one master node and four worker nodes including two cloud-based workers and two edge-based workers (shown in Table II). Serverless workloads are orchestrated through Kubernetes, with OpenFaaS enabling Algorithm 1: Multi-agent consensus-UCB algorithm 1Parameters: Initialize discount factor ←η, Action space Aand Observation space ¯ Y 2End: Repeat until convergence 3while ϵin episodes do 4while t≤Tdo 5Phase 1: Initialization: 6/* Regions */ 7R← {R|ru,v,w t∈R, ru t=ω} 8/* Regional nodes */ 9H← {H|hv u∈H, u =null} 10 /* Resource capacities */ 11 C← {C|cpv ut ∈C, cpv ut = Γ} 12 /* State of Charge */ 13 SOC← {S|sv ut ∈S, sv ut =current SoC} 14 /* Serverless functions */ 15 F← {F|fu,v,w t∈F, w =null} 16 /* Creating a local matching policy */ for each agent u→in IMu 17 Choose action Au P 18 Receive reward Ru P 19 Local cumulative reward = Mean reward µu P+ Standard deviation reward p(2 ∗log(t))/Atu(t) 20 where Atu(t)is the number of per-arm attempts 21 Phase 2: Submarket formation: 22 /* Creating a global matching policy */ for each function ∈do 23 1. Find the unmatched nodes from each region per agent ag and the deferred preference list with some relaxed edges; 24 2. Add hpv uinto the preference list of node fu,v,w tbased on consensus among the pre-selected agents agT 25 3. Ensure consensus reached iff Vote <2/3of agT 26 end 27 Phase 3: Exploitation: 28 while ∃fu,v,w t∈F is not matched do 29 hv u←Update the rank of each node in each region; 30 Initialize the round tand feasible arms v per agent agu P 31 Choose action based on : confidence bound (upper) = µu P+SDu Pand 32 confidence bound (lower) = µu P-SDu P 33 Quantize each agent using the updated bound as: Pvµu v+SDu v 34 Quantize each round using the update bound as: 35 q2 log(t) Pv(µu v+SDu v)+SDu 36 end 37 end 38 end Fig. 3. Message exchange mechanism in the proposed MARL approach. FaaS deployment. This setup evaluates regional node selection strategies using multiple scheduling preferences, including green energy utilization, latency, throughput, and cost. TABLE II SPECIFICATIONS OF CLOUD AND EDGE NODES Device CPU RAM Name Role Architecture Cores GHz GB Intel i5-7600T Master Node x86 4 2.8 32 Intel i5-7600T Cloud Worker Node 1 x86 4 2.8 32 Intel i5-7600T Cloud Worker Node 2 x86 4 2.8 32 Raspberry Pi 4 Edge Worker Node 1 ARM Cortex-A72 4 1.5 8 Raspberry Pi 4 Edge Worker Node 2 ARM Cortex-A72 4 1.5 8 B. Baseline Polices We evaluate the proposed algorithm against four distinct policies [22] using a Kubernetes cluster, with each policy is associated to specific application requirements. This comparison allows us to analyze the algorithm’s efficiency across diverse scenarios and serverless workloads. •Green energy-aware policy: In this policy, weights are allocated to Cloud-Edge Continuum nodes based on their green energy availability, making it beneficial for applications focused on maximizing the use of green energy. •Latency-aware policy: This policy prioritizes the CloudEdge Continuum nodes based on their response time, making it ideal for mission-critical applications [23] that demand minimal latency. For example, health monitoring systems, where immediate responses are critical for saving lives [22]. •Throughput-aware policy: This policy allocates weights to Cloud-Edge Continuum nodes based on their observed throughput, making it ideal for high-throughput applications. For example, it is well suited for smart parking systems [22], where real-time updates are essential to ensure efficient parking space management and a seamless user experience. •Cost-aware policy: This policy defines weights based on the cost of Cloud-Edge Continuum nodes, making it a natural fit for scenarios involving resource costs. An example use case is smart traffic lights in smart cities [22], where utilizing cost-efficient edge nodes helps minimize overall operational expenses. C. Results Discussion The experiments are conducted on a testbed, with its specifications described in Table II and the underlying assumptions detailed in Table III. TABLE III RESOURCE CONFIGURATIONS Resource Energy Levels Latency Throughput Cost 1 Cost 2 Cost 3 Edge 1 High Green Very Low High Equal Low High Edge 2 Low Green Medium Medium Equal Low High Cloud 1 High Green High High Equal High Low Cloud 2 High Brown Very High Very High Equal High Low As shown in Fig. 5(a), the decentralized approach demonstrates significantly lower execution time at smaller scales (e.g., F= 5 to F= 50) and scales more gracefully with larger workloads. This is primarily due to parallel decisionmaking by regional agents, which eliminates the single-point bottleneck inherent in centralized systems (see Fig. 4 (a)). As a result, Serverless functions are dispatched and executed faster, especially under high system load. Green energy usage (refer Fig. 5(b)) is also markedly higher in the decentralized system, with utilization consistently above 65%, and peaking at 100% for smaller F. The centralized system lags behind, with utilization hovering around 45-56% (Fig. 4 (b)). The observed improvement arises from localized awareness of energy availability, allowing each regional agent to prioritize energy-efficient nodes without waiting for global state updates. As shown in Fig. 5(c), latency remains lower across all scales in the decentralized model. The decentralized approach benefits from localized scheduling and minimal inter-agent communication, reducing scheduling delays and network congestion. Even with the increasing workload, it maintains sublinear latency growth, unlike the centralized model, where latency rises steeply due to centralized decision delays (Fig. 4 (c)). While Cost 1 remains constant (used as a baseline), Cost 2 and Cost 3-representing dynamic or energy-adjusted pricingare slightly more optimized in the decentralized setting (refer Fig. 5(d)). This variation indicates that better green energy utilization and reduced latency in the decentralized system translate into marginally lower operational and environmental costs. Similarly, Fig. 5(e) shows that throughput in the decentralized system is consistently higher at smaller workloads (F ≤ 25) and remains stable at higher workloads. In contrast, throughput in the centralized system drops more sharply as F Fig. 4. Comparison of performance metrics in a centralized setting. Fig. 5. Comparison of performance metrics in a decentralized setting. Fig. 6. Analysis of green energy utilization. Fig. 7. Analysis of message passing mechanism. increases (shown in Fig. 4 (e)). The throughput is decreasing continuously because the computation requirements of functions keeps increasing for bigger value of F. These results highlight the scalability advantage of decentralization, where task distribution remains balanced across regional nodes due to multi-agent coordination. D. Scalability Studies Fig. 6 presents a comparison of green energy utilization between the centralized and decentralized approaches. The Fig. 8. Analysis of overall latency. decentralized strategy consistently shows higher green energy usage across all function counts, with values ranging from 72% to 100%, compared to 45% to 56.25% in the centralized system. This significant improvement is attributed to the architectural design of the decentralized system, where regional agents possess more precise, real-time information about the state and energy profile of their respective local nodes. As a result, they are better equipped to schedule serverless functions on nodes with higher green energy availability. Moreover, since each regional agent communicates with a limited set of nodes and only exchanges minimal state information with other agents, the global overhead is reduced, allowing faster and more frequent green energy-aware decisions. In contrast, the centralized approach faces scalability and latency issues in gathering and processing global state information, leading to suboptimal green energy decisions. Fig. 7 compares the total messages exchanged in centralized vs. decentralized agent systems for a fixed number of Serverless functions and regional nodes. We can observe a linear growth in the number of messages exchanged per round in the decentralized approach as compared to the centralized one. This is primarily due to the additional messages exchanged required for forming consensus among pre-selected nodes by factor d, and a scalar factor δfor conflict resolution. Importantly, this design further ensures that an increasing number of agents in the system does not flood the system with additional messages. Fig. 8 compares the latency trends for both approaches, plotted on a logarithmic scale due to the wide variation in values. The decentralized approach consistently demonstrates lower latency, particularly as the number of functions increases. For example, at F= 500, the decentralized latency is 35,500 ms, while the centralized system reaches 47,200 ms. At smaller scales (F= 5 to F= 50), the decentralized system maintains significantly faster response times; which is critical for time-sensitive applications. The lower latency in the decentralized system is again due to its distributed decision-making mechanism. By decentralizing the scheduling logic and limiting the communication radius to smaller subsets of regional nodes, the system reduces congestion and computational bottlenecks. Furthermore, decisions can be made closer to the edge, minimizing delays introduced by a central controller analyzing the entire network’s state. VI. CONCLUSION In this paper, we address the problem of green energy maximization and propose a decentralized multi-agent reinforcement learning approach, supporting multiple policies for multi-provider Serverless Cloud-Edge Continuum such as green energy-aware, latency-aware, throughput-aware, and cost-aware. In our cooperative multi-agent reinforcement learning strategy, each agent independently learns the matching tendencies between regional nodes and serverless functions to identify optimal matches for itself. The reinforcement learning agent not only learns from its own interactions, but also communicates with other agents across the regions through