A Flexible Billing Life Cycle for Cloud Services Using Augmented Customer Agreements

Received February 4, 2021, accepted February 16, 2021, date of publication March 17, 2021, date of current version March 25, 2021. Digital Object Identifier 10.1109/ACCESS.2021.3066443 A Flexible Billing Life Cycle for Cloud Services Using Augmented Customer Agreements JOSÉ MARÍA GARCÍA 1,2, OCTAVIO MARTÍN-DÍAZ 2, PABLO FERNANDEZ1,2, CARLOS MÜLLER 2, AND ANTONIO RUIZ-CORTÉS 1,2, (Member, IEEE) 1Smart Computer Systems Research and Engineering Laboratory (SCORE), Universidad de Sevilla, 41012 Seville, Spain 2Research Institute of Informatics Engineering (I3US), Universidad de Sevilla, 41012 Seville, Spain Corresponding author: José María García ([email protected]) This work was supported in part by the European Commission (FEDER) and the Spanish Government through Project HORATIO under Grant RTI2018-101204-B-C21, and in part by the Andalusian Administration through Project APOLO under Grant US-1264651 and the Project EKIPMENT-PLUS under Grant P18-FR-2895. ABSTRACT Cloud computing constant evolution requires dynamic adjustments to service pricing and billing terms, considering provider infrastructures, customer requirements, and discount policies, among others. In this context, Customer Agreements (CA) are used to regulate the service provision including, among other information, the agreed service level in SLAs, pricing, and billing terms. Although many existing proposals and industrial tools support the definition of pricing and billing of cloud services, there is a lack in terms of customisation and automated monitoring tools integrated with the billing process. In this article, we provide a flexible billing proposal supporting CA that considers not only SLA terms but also customisable pricing and billing terms, possibly including compensations (i.e. discounts or overcharges) that apply when specified conditions are met. In addition, we developed an automated monitoring and analysis tool that we validate in a real-world industrial scenario. Moreover, we analyse and compare more than 50 existing industrial tools with our proposal, highlighting the advantages of its rule-based approach. INDEX TERMS Billing, cloud services, customer agreements, monitoring, pricing, service level agreements. I. INTRODUCTION Within the highly dynamic and evolving cloud computing market, a proper service pricing and billing life cycle definition is crucial for providers and consumers alike. On the one hand, providers need to customise their service offerings, taking into account several aspects such as their own infrastructure, the target market, incurred costs, discount policies, purchasing options, and the actual billing process life cycle, which can vary from fixed billing during a period of time to a real-time, fluctuating price based on demand-and-supply chains [1]–[3]. On the other hand, customers face an ever growing number of service offerings to choose from, where they need to analyse which are the optimal options with respect to their requirements and budget, as well as carefully monitor that the billing conforms to the Customer Agreement (CA) and especially to its associated Service Level Agreement (SLA) [4], [5], i.e. services are correctly billed according to the consumed resources, the service level actu- The associate editor coordinating the review of this manuscript and approving it for publication was Peng-Yong Kong . ally offered, the contractual price, and applicable discounts, amongst other variables. In this context, CAs usually consist of a textual document that is predefined by providers, and which is common to all their consumers. Mostly, their associated SLAs specify a number of service level objectives (SLO) on known metrics such as latency or availability, together with one or more compensations to be applied if not fulfilled [6]. In absence of automated tools, consumers are solely responsible for checking any SLA violations that may occur in order to claim for penalties to be applied in the following billing cycles. Nevertheless, CAs in general and SLAs in particular can play a more relevant role in cloud infrastructure governance, enabling its automatic monitoring, dynamic adaptation to comply with the SLOs, and the application of compensations during the billing process. In order to do so, agreement documents should be augmented with additional information and features, such as higher flexibility and expressiveness on the metrics or SLOs, pricing models to be applied at service provision time, billing customisation, and compensations containing discount rules based on such pricing models, to be applied at billing time. 44374 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ VOLUME 9, 2021 J. M. García et al.: Flexible Billing Life Cycle for Cloud Services Using Augmented CA In this article, we propose a flexible service billing process governed by CAs and their associated SLAs, so that providers can customise their billing life cycle with respect to the pricing model, discounts, and the metrics that are relevant to the process. The main contributions of our work can be summarised as follows: 1) We provide a model to specify pricing and billing information augmenting CA documents. Thus, we leverage billing life cycle definition as a first-class citizen in CAs and discuss how SLAs influence the activities carried out during the billing process. 2) We propose an automatic billing generation based on rules derived from our pricing and billing model. Our solution enables the dynamic optimisation of revenues according to actual use of infrastructures and services, gathered from service monitoring, while benefiting consumers with more accurate and possibly discounted bills. 3) We present an analysis of automated billing systems and a prototype tool that allows providers to customise the life cycle and generate bills according to the CAs they offer. We apply the developed tool in an industrial use case, evaluating its performance under different workloads, and comparing it with the 53 industrial tools we analysed. The rest of the article is structured as follows. First, in Sec. II we present the real-world scenario that motivated this work and we identify the faced challenges. Then, Sec. III presents our proposal to dynamically generate bills with respect to monitored service level and discount rules. We validate our solution in Sec. IV, applying it to the motivating scenario, evaluating its performance, and also comparing some tools which can be used to support billing.1We discuss in Sec. Vthe related work regarding cloud pricing models and billing mechanisms. Finally, Sec. VI concludes the article, discussing our results and identifying future work. II. MOTIVATION AND CHALLENGES The contributions presented in this work are motivated in the context of a real scenario. In the following, we describe this scenario, from which we derive the actual challenges related to the cloud services billing life cycle. A. MOTIVATING SCENARIO Our scenario consists on a distributed provision of Software as a Service (SaaS) for educational platforms in the public administration of Andalusia (the biggest region of Spain in terms of size and population), which have to be dynamically deployed and scaled up to support online teaching needs. In this context, the Educational Administration (EA) contracts a SaaS provider (SP) to deliver the platforms for a network of over 2,000 schools distributed geographically across 1A landing page with supplementary material including our prototype and the comprehensive comparative analysis is available at https://isagroup.github.io/2020-10-billing-lifecycle/ the region; each of those schools represent a community of end-users (EU) for a given provided educational platform. Specifically, the SP has to set and maintain a flexible infrastructure that can scale to dynamically deploy the requested instances of educational platforms (based on the Moodle Framework) by the EA. Furthermore, privacy regulations oblige the chosen provider to satisfy all the privacy and data protection needs, initially resulting in a requirement to host those platforms in a private cloud infrastructure deployed in the data centres owned by EA. In order to address these requirements, a container-based solution was implemented; this approach provides a flexible deployment and operation model that allows the reconfiguration and evolution required in the scenario. Specifically, Fig. 1shows the architecture of the platform based on the contaniner orchestrator Kubernetes where all the components are deployed. In this environment, the platforms manager operates (from SP) the system with a provisioning tool that generates the educational platforms instances on demand, based on the requests from the EA, setting up the appropriate persistency layer by means of a clustered database with the appropriate configuration registry and data backups. From a consumer standpoint, each school represents a set of EU that access their dedicated instance by means of the Kubernetes proxy. All those components are orchestrated by the k-master core module that is responsible to integrate and monitor all the containers. In this context, the initial scenario was a fixed reserved physical server owned and operated by the SP but installed in the EA data centers. This resulted in a recurring bill depending on the number of platforms deployed without taking into account the actual usage of each platform. Unfortunately, this initial model proved to be unsatisfactory for both the EA and the SP. On the one hand, from the EA perspective, even though there was a dynamic billing depending on the number of platforms used on every day, there was also a minimum base cost derived from the maintenance and operation of the fixed reserved infrastructure. On the other hand, from the SP perspective, there was a continuous under-usage of the infrastructure that could not be used for other customers in order to generate new business possibilities, and increase its revenue. To overcome these limitations, both the EA and the SP acknowledged to explore an alternative pricing model to migrate the reserved physical server to a data center owned by the SP as a private cloud for its customers. By using this shared infrastructure model, it is still feasible to reserve parts of the infrastructure, but during periods when usage loads by the administration are low (e.g. at nights or on vacations) the provider can allocate more computing resources to other customers with the subsequent gain, while the administration gets a discount for the low usage load. In this scenario, a CA was settled defining a set of rewards for the administration in case the reserved resources (in terms of computing, memory or disk) were not used during a certain period. VOLUME 9, 2021 44375 J. M. García et al.: Flexible Billing Life Cycle for Cloud Services Using Augmented CA FIGURE 1. Motivating scenario. B. CHALLENGES TOWARDS CUSTOMISATION OF THE BILLING LIFE CYCLE The dynamic scenario previously described poses a series of requirements with respect to scalability, adaptability, cost optimisation, and privacy issues that impose significant challenges to achieve a truly flexible and customisable billing life cycle: 1) Specification of all billing-related information within the same CA together with the relevant service information. Since billing rules are based on service metrics, which in turn are usually defined in relevant CAs, they should be considered as a first-class citizen. Thus, CAs augmented with billing specifications should drive the billing life cycle of associated services, including contracting, monitoring, charges and billing generation. By addressing this challenge, solutions will enable easier adaptation and customisation of service offerings. 2) Highly-expressive billing rules, supporting the definition of not only pricing terms depending on different metrics, but also discounts and compensations. This expressiveness will facilitate cost optimisations for both consumers and providers. 3) Automated analysis tools, supporting metrics monitoring, billing generation, and simulation. Billing support systems should provide analytical platforms to automatically compute bills according to metrics gathered from the cloud infrastructure. Additionally, these tools should take scalability and privacy issues into account. III. CUSTOMISABLE BILLING GENERATION PROCESS In order to allow the level of customisation required for the dynamic billing scenario described before addressing the identified challenges, we propose a billing generation framework based on CAs that are augmented with SLAs, pricing and billing rules, in addition to the service terms. In the following sections, we exemplify our proposal using the results from an industrial research project named POETISA, which was conducted to provide a solution to the challenges identified in Sec. II-B. A. CHARACTERISING THE BILLING LIFE CYCLE Figure 2shows a representation of a prototypical billing life cycle using a BPMN diagram. The process begins with the service provision, which establishes the relationship between provider and customer, enacting a CA which usually contains the pricing information, billing rules that defines billing periods and conditions to generate the actual bills, as well as various terms that specify the provided service. This information is considered within the proper billing cycle, as delimited by the billing rules included in the CA, where the monitoring and the billing subprocesses are performed iteratively for the provided service, until the service is decommissioned. All these different tasks comprising the billing life cycle may vary from one provider to another, offering custom made alternatives depending on their business capabilities and the market characteristics. These tasks and some common alternatives are further discussed in the following. 1) SERVICE PROVISION Whenever a customer wants to make use of a service, they must choose between offerings that provide different configurations regarding the service capabilities as well as the billing life cycle instantiation. The latter results in varying schemata that define billing periods, charge dates, and discounts based on the options finally chosen. Among the most common options, not necessarily exclusive, are the following: •Pay-on-demand. The most simple and usual option, where customers pay for services which they have actually used during a billing period. Customers usually just need to register on the service, so that any later usage implies implicitly the CA acceptance, which is usually predefined, and the subsequent billing. However, resources can be unavailable because of outages or limitations, such as too many customers accessing the service at the same time. •Reservation. Customers request the exclusive use of a resource for a period, such as a year, in order to avoid problems of unavailability. Reservations make the customers eligible for discounts. Amazon was one of first providers to offer reserved instances to customers. •Auction. Providers offer a dynamic, usually cheaper price for unused resources, such as reservations not currently being active, to increase their revenue. Thus, customers bid for their usage, with the risk that such 44376 VOLUME 9, 2021 J. M. García et al.: Flexible Billing Life Cycle for Cloud Services Using Augmented CA FIGURE 2. Process diagram representing a prototypical billing life cycle. usage can be interrupted in case the actual customers who reserved them need to use those resources. As an example, Amazon offers the so-called spot instances as a kind of auction to its customers. •Subscriptions. Customers request a period of flat-rate usage of resources, such as a year, or a personalised plan. Subscriptions usually have the largest discounts. Rackspace was one of first providers to offer prepayment options to customers. 2) MONITORING AND BILLING CYCLE Once the CA has been enacted after provisioning the service, the actual billing cycle starts according to the conditions stated in the CA. During a billing cycle, providers monitor the service usage to gather relevant metrics so that they can afterwards compute the bills with relevant data. Both monitoring methods and data should be accessible to customers, in order to allow them to validate the billing process and claim any miscalculation due to SLA violations not considered by the provider. Unfortunately, it is not always that way [7]. After the billing cycle ends, the billing subprocess depicted in Fig. 2takes the monitoring data as well as the pricing and billing terms in order to generate bills and send due charges to the customer account. Depending on the billing life cycle chosen at service provision time, we can classify charge models as follows: •Prepayment. Customers pay a single, preliminary charge, including larger discounts, in case of subscriptions, such as Rackspace options. •Upfront. Customers pay a preliminary payment in case of reservations, like with Amazon reserved instances. Later, ordinary usage is charged as well, but with some discounts. •Ordinary. Customers pay periodically for ordinary usage, usually monthly, including discounts in case of reservation or auction. 3) SERVICE DECOMMISSION Finally, the billing life cycle comes to an end depending on the conditions and terms agreed upon in the CA. Most common conditions that trigger the service decommission subprocess are the following: •Expiration. CAs may include an expiration date in their service terms, as when subscriptions are in place. •Disuse. Customers simply stop using the resources. On pay-on-demand schemata, if resources are not used, then there are no charges, since the CA does not apply. •Cancellation. CAs may include explicit termination conditions or other limitations, such as a maximum number of API invocations during a period (as Amazon API Gateway), or under a termination fee (as Rackspace’s Master Services Agreement). 4) CONTEXT OF OUR SOLUTION As we discussed in Sec. II-B, in order to address the identified challenges we need to provide a high degree of flexibility to the whole billing life cycle presented above. Focusing on the monitoring and billing subprocesses, which are fundamental to our proposal, both providers and customers have different perspectives and expectations towards dynamic management and customisation of the billing life cycle: •Billing generation. On the one hand, providers clearly need to bill their customers according to the actual usage of the relevant services. In order to do so, both accessing the monitoring data and appropriately applying billing rules (pricing and discounts) are mandatory features. •Billing validation. On the other hand, customers should have the capacity for using the same billing rules to check that the billing carried out by the provider conforms to their agreement, thus avoiding discrepancies between them that could lead to legal claims. •Billing estimation. Apart from validation, customers may also require the simulation of subsequent billing periods [8]. This opens opportunities for comparing with other providers for strategical and/or tactical purposes. Nevertheless, there are situations where a business entity may act as service consumer and provider at the same time, which leads to intertwined needs related to billing generation, VOLUME 9, 2021 44377 J. M. García et al.: Flexible Billing Life Cycle for Cloud Services Using Augmented CA FIGURE 3. UML model excerpt of iAgree elements including our proposed Billing extension. validation, and estimation. In the following, we focus our contribution on the billing generation scenario, proposing a solution that addresses the previously identified challenges based on dynamic pricing models specified alongside SLAs, serving as the foundations to customise the billing generation process. B. PRICING MODEL SPECIFICATION Our proposed pricing model is included within iAgree,2an agreement model that is partially depicted in Fig. 3. iAgree faces the first research challenge we identified in Sec. II-B since it supports modelling CAs including both SLA terms (c.f. dark grey area) and billing terms by means of our proposed extension (c.f. light grey area). To further illustrate this extension, we also provide in Fig. 4an excerpt of the CA which corresponds to the motivating scenario exposed in Sec. II-A. Line references in the text below refer to such excerpt. The iAgree model is basically comprised of a set of Terms and some Context information, such as the parties involved in the agreement and a set of JSON-Schema Definitions to be used in metrics, objectives, and compensations. As an example, the BillingDiscount schema at lines 8-10 is used in the reward describing a billing discount at line 28. In this model, Pricing appears as a first-class term, just as Metrics or Guarantees. On the one hand, a Metric term includes its schema, and a computer URL that references a RESTful API that is required to monitor its current values. As an example, the URL at line 40 provides the average memory consumption (AMC) of the cloud infrastructure. On the other hand, a Guarantee term specifies the service level objective to be fulfilled during a time window, as the Uptime ≥99.00 condition at line 45. Optionally, as presented in [6] 2http://iagree.specs.governify.io/Specification/ a compensable guarantee may penalise or reward the under or over-fulfilling of the service level objective, respectively. Focusing in our proposed extension, a Pricing term specifies the current currency and a base cost. Starting from this base cost, additional quantities can be added or subtracted for a specific billing period by considering a set of compensation values over the cost schema in a similar way as proposed compensations for guarantees. Note that a pricing compensation refers to either an overcharge or a discount instead of penalties and rewards on service level. As an example, the Daily Platform Cost (DPC) is established at line 19 as the base cost per educational platform with a value of 0.78. Since our proposed model supports highly-expressive cost specifications by means of rules, the final cost at lines 21 and 24 not only considers the DPC, but also the Number of Moodle platforms (NM) and the Number of Days of the Month (NDM), which are additional metrics. These final cost formulas apply if conditions on NM in lines 22 and 25 hold, respectively. Note that the billing rule specified in line 24 denotes that the billing process in this example follows a reservation schema, because the final cost is not affected up to 150 NMs. If the provider wanted to offer a pay-on-demand option instead, it would just specify a higher base cost (e.g. 0.9), removing the conditional terms with respect to the NMs. Furthermore, the final cost in the scenario can be also modified by a monthly billing discount of 10% if the AMC metric is less than 30 Gb, as defined in the reward in line 28. C. AUTOMATIC BILLING GENERATION From the pricing and billing terms specified using the format described above, we derive a set of rules that are evaluated using a rule engine [9]. Thus, as shown in Fig. 5, our proposal first analyses the iAgree specification in order to extract the metrics information by using each computer URLs 44378 VOLUME 9, 2021 J. M. García et al.: Flexible Billing Life Cycle for Cloud Services Using Augmented CA FIGURE 4. Excerpt of POETISA Customer Agreement in iAgree. declared (as in line 40 of Fig. 4). We query those computers components specifying the relevant time window to generate the corresponding bill. The values obtained are injected into the rule engine’s working memory as facts. For instance, if the AMC obtained for a given time window were 20 GB, our solution would introduce the fact AMC =20 into the working memory of the rule engine. Together with the metrics information, our proposal transforms the iAgree conditions and values into a rule with the format condition →action. For instance, the discount compensation specified in lines 28-32 of Fig. 4corresponds to the following rule: AMC <30 →BillingDiscount += 10 Note that line 29 specifies the aggregation operator in case several rules are applied over the same billing parameter (i.e. BillingDiscount). These rules are injected into the rule engine’s production memory. Then, after transforming the pricing model specification into rules and obtaining the metrics values as facts, we make use of a rule engine (we use json-rules-engine3in the prototype described in Sec. IV-A), so that we can compute discounts and final price according to the iAgree specification. 3https://www.npmjs.com/package/json-rules-engine FIGURE 5. Transformation from iAgree to rules. For instance, considering the previously described rule and the fact containing the AMC metric value, the rule engine would fire that rule, since the condition holds, modifying its working memory to indicate a 10% increment for the billing discount parameter. After the rule engine finishes processing the rules, our solution queries the working memory to retrieve the corresponding billing parameters to generate the final bill. By using this approach, our solution addresses the second challenge identified in Sec. II-B, taking advantage of the expressiveness of rule engines to describe complex conditions and generate bills using arbitrary formulas for computing discounts and service costs, in terms of the action part of the rules obtained from the analysis and transformation of the iAgree document. IV. TOOLING SUPPORT AND VALIDATION In order to evaluate the suitability of our conceptual solution, we developed a software prototype that was deployed in the context of the real motivational scenario described in Sec. II, addressing the third challenge discussed in Sec. II-B. Based on this deployment, a validation process was carried out to gather metrics from the live platform and perform a billing analysis for the provider during the course of an industrial project called POETISA. In this section, we detail the structure of the implemented prototype, outline the POETISA use case validation, and analyse its performance under different conditions. Finally, a review of the current industrial tooling support is offered, not only providing an analysis of the features supported by most relevant commercial tools, but also pointing out the strengths and weaknesses of our proposed tooling. A. PROTOTYPE ARCHITECTURE From an architectural point of view, Fig. 6depicts the different elements that conform the validation scenario; specifically, this architecture includes an ecosystem of components (at the bottom part of the figure) that are integrated with the preexisting provisioning infrastructure (at the top part of the figure) for educational platforms. The overall system is VOLUME 9, 2021 44379 J. M. García et al.: Flexible Billing Life Cycle for Cloud Services Using Augmented CA FIGURE 6. Prototype architecture. deployed into a Kubernetes platform that provides a management infrastructure for architectures based on containers to support reliability and availability. Specifically, the prototype is deployed in a single-node Kubernetes cluster (k-master) that could be extended with additional nodes (k-minions) to extend the computing capabilities; in this context, the k-master node provides an extensive API to actuate and monitor all the artefacts deployed in the infrastructure. From a global standpoint, in Fig. 6we can see how the Customer Agreements are managed and updated in the system by the CA Manager using the Render component that triggers (on behalf of the manager) the different billing computations showing back its results; the billing computation itself is orchestrated by the Billing component that gather the appropriate Metrics Values from the Metric component and generates the set of Rules and Facts that should be resolved in the Evaluator component that calls a rule engine. It is important to highlight, that this general process described is based on background parallel monitoring activity developed by the Monitor component that is configured by the specific Metric configurations sent from the Billing component in order to harvest the actual monitoring data using the Heapster framework and collect them in the time series database InfluxDB, which the Metric component can efficiently query later on. Specifically, the prototype consists of the following elements: •Monitor component integrates Heapster (responsible for gathering events monitored from the cluster), InfluxDB (a persistence store with query capabilities specialised in time-series and events), and Grafana (providing a visual dashboard for service operation). Fig. 7shows a screenshot of the Grafana dashboard we created. •Metrics component manages the calculation of an specific element that is significant for the CA, based on the event data harvested from the cluster. •Billing component includes the parsing and analysis of the CA and the gathering of metrics in order to generate a set of rules and facts that can be computed by the evaluator component to calculate a bill. •Evaluator component encapsulates the rule engine to compute the actual cost and compensation for a given set of rules and facts. •Finally, the Render component provides a user interface (as shown in Fig 8) for the management of princing and SLA terms of the CA. Note that in the screenshot presented, a billing has been generated so that objectives and condition rewards are marked in green if met, red otherwise. All of them can be changed in order to simulate variations in the billing process. The prototype follows the principles of microservices architectures where each component have a well-defined responsibility and can be evolved and deployed in an independent way; in this context, each component implements a RESTful interface modeled based on the OpenAPI Specification that is enriched with interactive documentation and testing portals generated using the Swagger-UI Node.js module. The microservices paradigm deployed in a Kubernetes platform represents a scalable model that allows the seamless replication of each microservice in order to prevent bottlenecks. For example, in a scenario with a high number of metrics or complex rules the appropriate component could be scaled with multiple instances. From a metric gathering perspective, as the monitor is based on the InfluxDB persistency platform, the current prototype could be easily extended with the usage of Telegraf Plugins4that provide the mechanism to incorporate metrics from different public cloud providers (such as AWS,Azure or Google) and distributed tracing system implemented with Zipkin that can be used to monitor microservice architectures 4https://www.influxdata.com/products/integrations/ 44380 VOLUME 9, 2021 J. M. García et al.: Flexible Billing Life Cycle for Cloud Services Using Augmented CA FIGURE 7. Screenshot of the Grafana dashboard component. or Function as a Service (Faas) platforms as Apache OpenWhisk. B. INDUSTRIAL VALIDATION RESULTS Our prototype implementation was developed and deployed within the course of an industrial research project (POETISA), whose setting resembled the same motivating scenario described in Sec. II. The service provider that were running the educational platforms service infrastructure tested our proposal directly on the production environment, thanks to the capabilities of the Kubernetes platform to hot deploy the components presented before. As a result, this deployment provided support to the shift from the pre-existing billing model to a fine-grained, dynamic billing. On the one hand, the pre-existing model was defined as: MonthlyBill =BaseCost ∗PlatformCount ∗ActiveDays where ActiveDays is independent of the usage of each platform and it just represents the availability of the platform. On the other hand, the evolved billing were tailored with compensations based on the real-time usage of each platform; concretely, the discount model defined is based on three usage metrics in the infrastructure (Memory consumption, CPU load, and disk space) which could potentially result in much competitive billings with up to 75% of total reductions over the pre-existing cost. Figures 7and 8show actual data from the industrial validation and the CA applied, correspondingly.5 C. PERFORMANCE ANALYSIS Billing management is typically a crucial task for organizations and, depending on the size and business model, the 5The complete CA used in this validation scenario and the InfluxDB dataset generated during the project be found at https://isagroup.github.io/2020-10-billing-lifecycle/ execution of this process could be challenging. Consequently, in order to analyse the operational limits of our approach, in this section we detail the results of a set of performance experiments to assess the system under different conditions in order to characterise an estimation of its performance in a wide range of scenarios in terms of the following dimensions: •D1: Number of metrics used in the CA. Each metric may correspond with monitored data harvested in the platform, a single artefact measure (direct metric), or an aggregation calculated in terms of other measures (indirect metric). •D2: Number of pricing or SLA terms of the CA. Each term correspond with a guarantee that should be analysed along with a general penalty or reward definition that is derived from the over- or under-usage of the platform, correspondingly. •D3: Rules complexity. The complexity of expressions used in the definition of the billing rules can vary from a simple, fixed threshold (e.g. TotalAverageCPU <90) to multiple combinations of logical operators that result in complex conditions (e.g. (Node1CPU <TotalAverageCPU)∨(Node1MEM < TotalAverageMEM)). •D4: Simultaneous billing requests. In realistic settings multiple billings for different customers should be calculated in parallel. In order to calculate the throughput of the system, it is important to characterise the parallelism limit6where the system have a billing generation capacity that provides a stable performance level. We can overcome these operational limits by scaling up the number of microservices on the affected components 6This limit is highly dependent from the underlying infrastructure and deployment mechanism. VOLUME 9, 2021 44381 J. M. García et al.: Flexible Billing Life Cycle for Cloud Services Using Augmented CA (typically the billing and evaluator components) and/or deploying the system in a more advanced instance in an IaaS provider. The study of how the system could perform in different types of IaaS instances is out of the scope of this work. •D5: Data points volume per metric. Different scenarios would require different types of metrics with variable resolutions that represent a particular volume; e.g. a given metric could have higher resolution depending if the monitoring data is harvested every second or every minute. In the following, we first introduce the infrastructure used in the performance analysis tests, then we discuss the experimental process outlining the different kinds of experiments developed in order to assess the identified dimensions, and finally we present the results of the analysis highlighting the key performance aspects found. 1) TESTBED In order to define a controlled set of experiments, the developed prototype has been isolated from the production kubernetes platform and deployed separately in a single dedicated machine using a set of docker containers, as opposed to the industrial validation described in Sec. IV-B. Nevertheless, we exported the real dataset of metrics harvested in that scenario, so that we were able to use it for several of the experiments as we detail later in Sec. IV-C2. From an infrastructure perspective, the experimental testbed consists of a single dedicated machine with an Intel Core i7-7700HQ 2.80GHz running Windows 10 with a total 16 GB of RAM. In this host, the prototype were deployed over a Docker desktop platform with a configuration of 2 CPUs and 2 GB of RAM, while the experiment runner (built on a Node.js stack) were configured with a limit of 8 GB of RAM. Specifically, the experiments ecosystem7of the testbed is based on a core runner that generates a CA and metrics data based on a set of parameters, and then requests a billing generation from the prototype, measuring its performance. In this context, the different experiments are defined by means of scripts that parameterize the core runner with a specific configuration. In addition, the experiment runner includes a metric generator which has been designed to complement the real dataset to analyse the capabilities of the platform concerning D1 dimension. Specifically, the configuration input for the core runner can be grouped in three sets as follows: •Customer agreement generation parameters. These parameters include the number of terms for the CA to be generated (metrics, pricing, guarantees, and compensations) and the rule set to generate the guarantee objective expression. Currently, three different complexity generation rules are available (simple, medium, and complex) ranging from simple conditions that contains 7The complete list of scripts and datasets generated for the different experiments are available at https://github.com/isa-group/billing-performance a single comparison operator with literals, to complex aggregated expressions combining logical, algebraic, and comparison operators. •Metrics generation parameters. These parameters guide the generation of the data points that will be inserted in the metrics database that serves as a source of information for the billing generation. The interval of the metrics data points generated corresponds with the same interval of the real dataset monitored in the industrial validation (i.e. 13 months) in order to have a comparable context, though the resolution (data points per second) is parameterized to allow different data point densities. •Invocation parameters. These parameters include the iteration number and parallelism used to invoke the prototype allowing a repetition of calculations in a sequential and parallel model. 2) EXPERIMENTAL PROCESS In order to assess the performance of our approach amongst the different identified dimensions, we designed an experimental study. Specifically, for each dimension, we performed a corresponding experiment using the testbed and adjusting the core runner parameters accordingly with different configurations that explore those dimensions. Thus, each experiment run (using a specific configuration) is repeated a number of times (100 in our case) in order to calculate the average and standard deviation, and to assure a statistical significance of the results. Specifically, we developed 5 successive experiments with 43 different configurations (for a total duration of over 20 hours considering experimental setup and runtime): •Exp1. In order to explore D1 dimension, we used 10 different configurations to generate CAs with a number of metrics ranging from 1 to 1024 following a growth factor of 2. •Exp2. With respect to D2, we used 10 different configurations to generate CAs with a number of guarantee terms ranging from 1 to 1024 following a growth factor of 2. •Exp3. In order to consider D3 impact, we used 3 different configurations to generate CAs with different complexity degrees for guarantee objectives. •Exp4. For exploring D4, we used 10 different configurations to assess the behaviour with different scenarios of parallel billing generations, ranging from 1 to 10 concurrent users. •Exp5. In order to explore D5, we set up 10 different configurations of data point densities per metric, starting with 6 data points and following a growth factor of 3. 3) EXPERIMENTAL RESULTS The summary of the results of each experiments are depicted in Figure 9. As a global conclusion, we can identify a linear correlation between the number of CA terms and the performance obtained while there is no correlation with the data points density for each metric or the complexity of terms. 44382 VOLUME 9, 2021 J. M. García et al.: Flexible Billing Life Cycle for Cloud Services Using Augmented CA [13] A. Mazrekaj, I. Shabani, and B. Sejdiu, ‘‘Pricing schemes in cloud computing : An overview,’’ Int. J. Adv. Comput. Sci. Appl., vol. 7, no. 2, pp. 80–86, 2016. [14] K.-W. Park, J. Han, J. Chung, and K. H. Park, ‘‘THEMIS: A mutually verifiable billing system for the cloud computing environment,’’ IEEE Trans. Services Comput., vol. 6, no. 3, pp. 300–313, Jul. 2013. [15] L. Chen and K. Chen, ‘‘BitBill: Scalable, robust, verifiable peer-to-peer billing for cloud computing,’’ in Proc. 6th USENIX Conf. Hot Topics Cloud Comput. (HotCloud), 2014, p. 20. [16] Q. Zhang, A. Haller, and Q. Wang, ‘‘CoCoOn: Cloud computing ontology for IaaS price and performance comparison,’’ in The Semantic Web—ISWC 2019. Cham, Switzerland: Springer, 2019, pp. 325–341. [17] A. Roy, S. Misra, and S. Nag, ‘‘PRIME: An optimal pricing scheme for mobile sensors-as-a-service,’’ IEEE Trans. Mobile Comput., early access, Sep. 14, 2020, doi: 10.1109/TMC.2020.3023885. [18] M. Huang, W. Liu, T. Wang, Q. Deng, A. Liu, M. Xie, M. Ma, and G. Zhang, ‘‘A game-based economic model for price decision making in cyber-physical-social systems,’’ IEEE Access, vol. 7, pp. 111559–111576, 2019. [19] X. Zhang, Z. Huang, C. Wu, Z. Li, and F. C. M. Lau, ‘‘Dynamic VM scaling: Provisioning and pricing through an online auction,’’ IEEE Trans. Cloud Comput., vol. 9, no. 1, pp. 131–144, Jan. 2021. [20] M. Tortonesi and L. Foschini, ‘‘Business-driven service placement for highly dynamic and distributed cloud systems,’’ IEEE Trans. Cloud Comput., vol. 6, no. 4, pp. 977–990, Oct. 2018. [21] G. V. Prasad, A. S. Prasad, and S. Rao, ‘‘A combinatorial auction mechanism for multiple resource procurement in cloud computing,’’ IEEE Trans. Cloud Comput., vol. 6, no. 4, pp. 904–914, Oct. 2018. [22] S. Patanjali, B. Truninger, P. Harsh, and T. M. Bohnert, ‘‘CYCLOPS: A micro service based approach for dynamic rating, charging & billing for cloud,’’ in Proc. 13th Int. Conf. Telecommun. (ConTEL), Jul. 2015, pp. 1–8. [23] E. Elmroth, F. G. Márquez, D. Henriksson, and D. P. Ferrera, ‘‘Accounting and billing for federated cloud infrastructures,’’ in Proc. 8th Int. Conf. Grid Cooperat. Comput., Aug. 2009, pp. 268–275. [24] B. Rochwerger, D. Breitgand, E. Levy, A. Galis, K. Nagin, I. M. Llorente, R. Montero, Y. Wolfsthal, E. Elmroth, J. Caceres, M. Ben-Yehuda, W. Emmerich, and F. Galan, ‘‘The reservoir model and architecture for open federated cloud computing,’’ IBM J. Res. Develop., vol. 53, no. 4, pp. 1–4, 2009. [25] P. Harsh, Y. Jegou, R. G. Cascella, and C. Morin, ‘‘Contrail virtual execution platform challenges in being part of a cloud federation,’’ in Towards a Service-Based Internet-ServiceWave. Berlin, Germany: Springer, 2011, pp. 50–61. [26] R. Zarnekow, ‘‘The impact of cloud computing adoption on IT service accounting approaches—A customer perspective on IaaS pricing models,’’ Ph.D. dissertation, Dept. Inf. Commun. Manage., Teschnische Universität Berlin, Germany, 2012. [27] K. Cho and H. Bahn, ‘‘Real-time IaaS cost model and estimation results for cloud PC laboratory service,’’ in Proc. Int. Commun. Eng. Cloud Comput. Conf., Oct. 2019, pp. 14–18. [28] M. Lindner, F. Galán, C. Chapman, S. Clayman, D. Henriksson, and E. Elmroth, ‘‘The cloud supply chain: A framework for information, monitoring, accounting and billing,’’ in Proc. 2nd Int. ICST Conf. Cloud Comput. (CloudComp), Oct. 2010, pp. 1–22. JOSÉ MARÍA GARCÍA received the M.Sc. degree (Hons.) in computer science and software engineering and the Ph.D. degree in technology and software engineering from the University of Seville, Spain, in 2008 and 2012, respectively. From 2006 to 2013, he was a Research Assistant with the University of Seville. From 2013 to 2015, he was a Postdoctoral Researcher and an Assistant Professor with the Semantics Technology Institute, University of Innsbruck, Austria. He rejoined the University of Seville, as an Assistant Professor, in 2015. Since 2019, he has been an Associate Professor with the Smart Computer Systems Research and Engineering Laboratory, Applied Software Engineering Group, Research Institute of Computer Engineering, University of Seville. His research interests include blockchain, semantic web technologies, service-oriented architectures, and linked data. OCTAVIO MARTÍN-DÍAZ received the M.Sc. and Ph.D. degrees in computer science from the University of Seville, Spain, in 1994 and 2007, respectively. He is currently a Lecturer of software engineering with the University of Seville. He is also with the Applied Software Engineering Group and the Research Institute of Computer Engineering. His current research interests include purchasing, pricing, and billing in cloud computing and service-oriented computing, mainly focusing on aspects related to time management in service level agreements. PABLO FERNANDEZ received the Ph.D. degree in technology and software engineering from the University of Seville, Spain, in 2013. Since 2019, he has been an Associate Professor with the Smart Computer Systems Research and Engineering Laboratory, Applied Software Engineering Group, Research Institute of Computer Engineering, University of Seville. His current research interest includes automated governance of organizations and infrastructures based on service level agreements and commitments. CARLOS MÜLLER received the Ph.D. degree in technology and software engineering from the University of Seville, Spain, in 2013. He is currently a Lecturer of software engineering with the University of Seville. He is also with the Applied Software Engineering Group and the Research Institute of Computer Engineering. His current research interest includes the automated analysis of SLAs and the application of such analysis at SLA design and monitoring. ANTONIO RUIZ-CORTÉS (Member, IEEE) received the B.Sc. degree in computer science and the M.Sc. and Ph.D. degrees in informatics engineering from the University of Sevilla, Spain, in 1992, 1996, and 2002, respectively. From 1990 to 1998, he was in the Computer Industry. From 1996 to 1998, he was a part-time Lecturer with the University of Huelva. He joined the University of Sevilla, as a full-time Lecturer, in 1998. He founded the Applied Software Engineering Group, University of Sevilla, in 2004. Since 2016, he has been a Full Professor of software and service engineering. Since 2019, he heads the SCORE Laboratory, University of Sevilla. His current research interests include service-oriented computing, business process management, and testing and software product lines. Dr. Ruiz-Cortés is an Elected Member of the Academy of Europe. He was a recipient of the Most Influential Paper of SPLC, in 2017, and the VAMOS Award, in 2020. He received the Extraordinary Award for his B.Sc., M.Sc., and Ph.D. degrees. Since 2018, he has been the President of the Spanish Society on Software Engineering (SISTEDES). He is also an Associate Editor of Computing (Springer) and the International Journal of Cooperative Information Systems. VOLUME 9, 2021 44389