Automating SLA-Driven API Development with SLA4OAI

Automating SLA-Driven API development with SLA4OAI Antonio Gamez-Diaz, Pablo Fernandez, and Antonio Ruiz-Cortes Universidad de Sevilla?, Seville, Spain {antoniogamez,pablofm,aruiz}@us.es Abstract. The OpenAPI Specification (OAS) is the de facto standard to describe RESTful APIs from a functional perspective. OAS has been a success due to its simple model and the wide ecosystem of tools supporting the SLA-Driven API development lifecycle. Unfortunately, the current OAS scope ignores crucial information for an API such as its Service Level Agreement (SLA). Therefore, in terms of description and management of non-functional information, the disadvantages of not having a standard include the vendor lock-in and prevent the ecosystem to grow and handle extra functional aspects. In this paper, we present SLA4OAI, pioneering in extending OAS not only allowing the specification of SLAs, but also supporting some stages of the SLA-Driven API lifecycle with an open-source ecosystem. Finally, we validate our proposal having modeled 5488 limitations in 148 plans of 35 real-world APIs and show an initial interest from the industry with 600 and 1900 downloads and installs of the SLA Instrumentation Library and the SLA Engine. 1 Introduction In the last decade, RESTful APIs are becoming a clear trend as composable elements that can be used to build and integrate software [18,7]. One of the key benefits this paradigm offers is a systematic approach to information modeling leveraged by a growing set of standardized tooling stack from both the perspective of the API consumer and the API provider. Specifically, during the last years, the OpenAPI Specification1(OAS), formerly known as Swagger specification, has become the de facto standard to describe RESTful APIs from a functional perspective providing an ecosystem that helps the developer in several aspects of the API development lifecycle2. As an example, from the API provider perspective, there are tools that aim to ?This work is partially supported by the European Commission (FEDER), the Spanish Government under projects BELI (TIN2015- 70560-R) and HORATIO (RTI2018- 101204-B-C21), and the FPU scholarship program, granted by the Spanish Ministry of Education, Culture and Sports (FPU15/02980). 1https://github.com/OAI/OpenAPI-Specification 2https://openapi.tools 2 A. Gamez-Diaz, P. Fernandez and A. Ruiz-Cortes automate the server scaffolding, an interactive documentation portal creation or the generation of unit test cases; from the perspective of the consumer, there are tools to automate the creation of API clients, the security configuration or the endpoints discovery and usage [15,16,1]. However, as APIs are deployed and used in real settings, the need for nonfunctional aspects is becoming crucial. In particular, the adoption of Service Level Agreements (SLAs) [13] could be highly valuable to address significant challenges that the industry is facing, as they provide an explicit placeholder to state the guarantees and limitations that a provider offers to its consumers. For example, these limitations (such as quotas or rates) are present in most common industrial APIs [3] and both API providers and consumers need to handle how they monitor, enforce or respect them with the consequent impact in the API deployment/consumption. In this paper, we address the challenge of SLA modeling and management in APIs by providing the following contributions: –SLA4OAI, an open SLA specification that is integrated with the OpenAPI Specification joint with a Basic SLA Management Service (i.e., a minimum definition of endpoints required for the SLA enforcing in the APIs) that can be used to promote the vendor independence. –A set of tools to support the different activities of the API development lifecycle when it becomes aware of the existence of an SLA. –An initial validation over 5488 limitations in 35 of real-world APIs showing the expressiveness coverage and the potential evolution roadmap for the specification. The rest of the paper is structured as follows: in Section 2, we describe the related work and motivate the need for our proposal. In Section 3 we describe in brief words the OpenAPI Specification focusing on its extension’s capabilities. In Section 4 we describe our SLA4OAI model proposal. In Section 5 we show the ecosystem of tools that have been built around our proposal. In Section 6 we validate our proposal by modeling 5488 limitations in 35 of real-world APIs. Finally, in Section 7 we show some remarks and conclusions. 2 Motivation and Related Work The software industry has embraced integration as a key challenge that should be addressed in multiple scenarios. In such a context, the proliferation of APIs is a reality that has been formally analyzed: in [14], authors performed an analysis of more than 500 publicly-available APIs to identify the different trends in the current industrial landscape. Specifically, regarding the documentation, there is a clear trend with respect to the functional description of the service: during the last years, the OpenAPI Specification has consolidated as a de-facto standard to define the different functional properties an API provides. For instance, in [12], authors study on the presence of dependency constraints among input parameters in web APIs in industry. Automating SLA-Driven API development with SLA4OAI 3 With such a consolidated market of APIs, non-functional aspects are also becoming a key element in the current landscape. In [3], authors analyze a set of the 69 real APIs in the industry to characterize the variability in its offerings, obtaining a number of valuable conclusions about real-world APIs, such as: (i) Most APIs provide different capabilities depending on the tier or plan of the API consumer is willing to pay. (ii) Usage limitations are a common aspect all APIs describe in their offerings. (iii) Limitations over API requests are the most common including quotas over static periods of times (e.g., 1.000 request each natural day) and rates for dynamic periods of times (3 request per second). (iv) Offerings can include a wide number of metrics over other aspects of the API that can be domain-independent (such as the number of returned results or the size in bytes of the request) or domain-dependent (such as the CPU/RAM consumption during the request processing or the number of different resource types). Based on these conclusions, we identify the need for non-functional support in the API development life-cycle and the high level of expressiveness present in the API offerings. From the perspective of the API development life-cycle, the lack of a standard spec for non-functional aspects integrated with existing standards OpenAPI, prevents the tooling ecosystem to grow and provide support advanced issues: as an example, to support the API consumer, it could be possible to develop tools to automate the generation of SLA-aware API clients able to self-adapt the request rate to the API limitations; to support the API provider, it could be possible to create of SLA-aware API testers enriching the habitual tests with information about limitations in order to analyze the actual performance capabilities to decide the maximum number of API consumers to be allowed with a certain SLA that explicitly states the limitations in their usage. We have analyzed the most prominent academic and industrial proposals that aim to the definition of SLAs in both traditional web services and cloud scenarios in order to outline their scope and limitations. Specifically, in Table 1, we have considered 7 aspects to analyze in each SLA proposal, namely: F1 determines the format in which the document is written; F2 shows whether the target domain is web services; F3 indicates if it can model more than one offering (i.e., different operations of a web service); F4 determines if it allows modeling hierarchical models or overriding properties and metrics; F5 shows whether temporal concerns can be model (e.g., in metrics); F6 indicates if there exists a tool for assisting users to model using this proposal; F7 determines if there exists a tool/framework for enacting the SLA. Based on this comparison of the different SLA models, we highlight the following conclusions: (i) None of the specifications provides any support or alignment with the OpenAPI Specification; (ii) Most of the approaches provide a concrete syntax on XML, RDF (some of them they even lack concrete syntax) and there is no explicit support to YAML or JSON serializations. (iii) An important number of proposals are complete, but others leave some parts open to being implemented by practitioners. (iv) Besides the fact that a number of proposals are that aims to model web services, they are focused on traditional 4 A. Gamez-Diaz, P. Fernandez and A. Ruiz-Cortes Table 1. Analysis of SLA Models Name F1 F2 F3 F4 F5 F6 F7 SLAC [19] DSL 3 3 CSLA [9] XML 3 3 L-USDL Ag. [6] RDF 3 3 = 3 rSLA [17] Ruby 3 3 3 3 SLAng [10] XML 3 WSLA [11] XML 3 3 3 SLA* [8] XML 3 3 3 WS-Ag. [2] XML 3 3 3 = =Supported with minor enhancements or modifications. SOAP web services rather than RESTful APIs. In this context, they do not address the modeling standardization of the RESTful approach: i.e., the concept of a resource is well unified (a URL), and the amount of operations is limited (to the HTTP methods, such as GET, POST, PUT and DELETE). This lack of support of the RESTful modeling prevents the approaches to have a concise and compact binding between functional and non-functional aspects. (v) They do not have enough expressiveness to model limitations such as quotas and rates, for each resource and method and with complete management of temporally (static/sliding time windows and periodicity) present in the typical industrial API SLAs. (vi) Most proposals are designed to model a single offering and they mostly lack support to modeling hierarchical models or overriding properties and metrics (F4); in such a context, they cannot model a set of tiers or plans that yield a complex offering that maintains the coherence by model and instead they rely on a manual process that is typically error-prone. (vii) finally, the ecosystem of tools proposed in each approach (in the case of its existence) is extremely limited and that aims to be solely as a prototype; moreover, they apparently are not integrated into a developer community nor there is evidence of this usage by practitioners in the industry. In order to overcome the limitations of existing approaches, the main goals of this paper can be summarized as follows: (i) An interoperable model fullyintegrated with leading API description language (OAS) to express the API limitations. (ii) an initial ecosystem of tools to provide support to different parts of the SLA-Driven API development lifecycle. (iii) validation of this model in real-world scenarios to assess its expressiveness. 3 OAS in a Nutshell In this section, we briefly present the OpenAPI Specification (OAS), considering its goals, structure and extension capabilities. OAS, formerly known as Swagger, is a vendor-neutral, portable and open specification for the functional deception of APIs. It is promoted by the OpenAPI Initiative (OAI), an open source consortium hosted by The Linux Foundation and supported by a growing number of leading industry stakeholders, such as Google, IBM, Microsoft or Automating SLA-Driven API development with SLA4OAI 5 Oracle, amongst others. Both API clients and vendors are able to benefit from the formal definition using the OAS: from the clients’ point of view, they can use any tool from the extensive ecosystem created around the OAI; conversely, from the vendors’ point of view, they can generate interactive documentation portals, create auto-generated prototypes and perform automatic API monitoring and testing. Specifically, as a minimum content, an OAS document should describe a set of aspects including API general information (such as title, description and version), a list of Resources,Paths and Methods allowed, and set of Schemas (following the JSON-schema specification) to identify the structure of the data to be exchanged with the API (e.g., a resource structure). In order to have a more concise description, it is possible to reuse definitions of schemes by means of the $ref constructor as proposed in the JSON-schema standard. Complementary, API provider can include optional elements such as the different API endpoints, where the API can be accessed. This is especially useful in scenarios with different endpoints for development and production stages.   1openapi : 3.0.0 2info: 3title : Simple petstore API 4description : ... 5version: ... 6x -sla : ./ pets - plans . yaml 7servers: 8- url: .... 9paths : 10 /pets: 11 get: 12 description : ... 13 parameters : .. 14 responses : 15 200: 16 description : pet response 17 content: 18 application / json : 19 schema: 20 $ref : "#/ components / schemas / pet " 21 post: 22 ... 23 components : 24 schemas: 25 pet: 26 title : pet model 27 ...   Listing 1.1. RESTful API in OAS   1context: 2id: plans 3sla: ’1.0 ’ 4type : plans 5... 6infrastructure: ... 7metrics: 8requests : 9type : integer 10 format : int64 11 description : # requests 12 resolution: consumption 13 ... 14 plans : 15 free: 16 pricing: 17 cost : 0 18 currency : USD 19 billing: monthly 20 quotas: 21 /pets: 22 post: 23 requests : 24 - max: 100 25 period : daily 26 rates : 27 /pets: 28 get: 29 requests : 30 - max: 2 31 period : secondly 32 scope : tenant 33 pro: 34 ...   Listing 1.2. SLA written in SLA4OAI As an example, listing 1.1 shows an OAS fragment from a basic RESTful API that corresponds with a single endpoint (/pets) and two methods. Lines 9-22 describe the definition of the pet resource including the GET and POST methods for retrieving and creating resources; specifically, line 11 starts modeling 6 A. Gamez-Diaz, P. Fernandez and A. Ruiz-Cortes the GET method with a description and the parameters that the request might be able to handle and responses section (lines 14-20) describe the model of a successful HTTP response (i.e., status code 200) returning a pet resource conforming with the appropriate schema reference (line 20). Finally, in lines 24- 27, the data model (schema) of the pet object is being defined. A key feature of the OAS is the capability of being extended with the definition of custom properties starting with x-, paving the way for customizing or adding additional features according to specific business needs. As an example, line 6 shows the use of the xextension point to include a reference to the SLA description of the API following our proposal (c.f., Section 4). 4 Our Proposal 4.1 SLA4OAI Language SLA4OAI3is a language which provides a model for describing SLA in APIs in a vendor-neutral way by means of extending the main specification. This proposal is open for evolution based on the discussion with the community and other partners of the OpenAPI Initiative, hosted by the Linux Foundation. For the sake of completeness, always refer to the online version so as to have a complete reference of the language. The figure available online4depicts an abstract syntax of an SLA4OAI description. Starting with the top-level placeholder (denoted as SLA4OAI Document in the figure) we can describe basic information about the context, the infrastructure endpoints that implement the Basic SLA Management Service, the metrics and a default value for quotas,rates,guarantees and pricing. Context contains general information, such as the id, the version, the URL pointing to the api OAS document, the type and the validity of the document; in this context, the type field can be either plans or instance and it indicates whether the document corresponds with the general plan offering or it correspond with a specific SLA agreed with a given customer. The Metrics enables the definition of custom metrics which will be used to define the limitations, such as the number of requests, or the bandwidth used per request. For each metric, the type,format,unit,description, and resolution should be defined. The Plan configuration (configuration parameters for the service tailored for the plan), availability (availability of the service for this plan expressed via time slots using the ISO 8601 time intervals format), and the rest of the elements that will override the default with plan-specific values: quotas, rates and guarantees, pricing. In this context, it is important to highlight that the Plan section maps the structure in the OAS document to attach the specific limitations (quotas or rates) for each path and method. Specifically, after defining the configuration, the availability,pricing,guarantees, the limitations quotas and rates can be modeled; particularly, the limitations are described in the Limit with a max value that can be accepted, a period (i.e., secondly, minutely, hourly, daily, monthly 3https://sla4oai.specs.governify.io 4https://isa-group.github.io/2019-05-sla4oai/files/sla4oai_diagram.png Automating SLA-Driven API development with SLA4OAI 7 or yearly) and the scope where they should be enforced; as an extensible scope model, we propose two possible initial values (tenant or account as default) corresponding with a two-level structure: a limitation or guarantee with a tenant scope will be applicable to the whole organization while an account scope would be applicable to each specific user or account (typically with a different API key) in the organization. Considering the features of the existing SLA proposals previously analyzed and available in the online appendix, SLA4OAI is a proposal serialized using the YAML/JSON syntax (F1) specifically designed for web services (F2), concretely, RESTful APIs. It is able to model one or more offerings (F3) in a hierarchical model (F4) since plans can override the default values for the limitations. Furthermore, our proposal takes into account the temporality (F5), since each limitation is scoped to a precise period of time and each plan has its own availability information. Finally, as stated in following sections, SLA4OAI has a set of tools for assisting users to write the model (F6) and an initial ecosystem of tools to support parts of the development lifecycle (F7). Let us consider the aforementioned example (as modeled in Listing 1.1) to be extended with a basic SLA: as a provider, it would be useful to limit, on the one hand, the number of requests a consumer is allowed to make in a static window (quota) of 1 day depending on the plan purchased and, on the other hand, the requests allowed to be made in a sliding window (rate), differing from GET and POST methods to avoid the API saturation derived from abusive customers. Specifically, Listing 1.2 illustrates the model in SLA4OAI of the limitations of this example API: in lines 14-34 the free and pro plans are being modeled. Focusing on the first, line 15 define a specific plan by its limitations quotas (lines 20-25) and rates (lines 26-32). For instance, a quota of 100 POST requests over the resource /pets in a static window of 1 day is defined in lines 23-25. Conversely, a rate of 2 requests per second is defined for /pets GET requests (lines 29-32). Finally, note that line 4 indicates that this document is for describing plans. Whenever a client accepts a specific plan, type field would become an instance one. It is interesting to highlight the scope: tenant (line 32) in the rates for the GET request represents a limitation for the whole consumer organization affecting all the accounts of the organization, while the rest of the quotas and rates are enforced on a default per-account basis. 4.2 SLA-Driven API Development Lifecycle In spite of the fact that each organization could address the API development lifecycle with slightly different approaches, a minimal set of activities can be identified: a first activity corresponds with the actual Functional Development of the API implementing and testing the logic; next a Deployment activity where the developed artifact is configured to be executed in a given infrastructure; finally, once the API is up and running, an Operation activity starts where the requests from consumers can be accepted. This process is a simplification that can be evolved to add intermediate steps (such as testing) or to include an evolutive cycle where different versions are deployed progressively. In order to 8 A. Gamez-Diaz, P. Fernandez and A. Ruiz-Cortes incorporate SLAs in this process, we expand this basic lifecycle where both API Provider and API Consumer can interact (as depicted in Figure 1). Developer Product manager Product operator API provider Functional development SLA modeling Operation SLA enactment Deployment SLA registry Offer analysis Offer selection Consumption SLA instance SLA instrumentation Provider SLA-Driven API Development Lifecycle Consumer API consumer Consumer Fig. 1. SLA-Driven API development lifecycle Specifically, from the provider’s perspective, the Functional Development can be developed in parallel with a SLA modelling where the actual SLA offering (type plans) is written and stored in a given SLA Registry. Once both the functional development and the SLA modeling has concluded, the SLA instrumentation must be carried out, where the tools and/or developed artifacts are parameterized so they can adjust their behavior depending on a concrete SLA and provide the appropriate metrics to analyze the SLA status. Next, while the deployment of the API takes place, a parallel activity of SLA enactment is developed where the deployment infrastructure should be configured in order to be able to enforce the SLA before the API reaches the operation activity. Complementary, from consumer’s perspective, once the provider has published the SLA offering (i.e., Plans) in the SLA Registry, it starts the offer analysis to select the most appropriate option (offer selection activity) and to create and register its actual SLA (type instance); finally, the API Consumption is carried out as long as the API is the Operation activity and its regulated based on the terms (such as quotas or rates) defined in the SLA. In order to implement this lifecycle, it is important to highlight that the SLA instrumentation,SLA enactment and Operation activities should be supported by an SLA enforcement protocol that aims to define the interactions for checking if the consumption of the API for a given consumer is allowed (e.g., it meets the limitations specified in its SLA) and to gather the actual values of the metrics from the different deployed artifacts that implement the API. Automating SLA-Driven API development with SLA4OAI 9 4.3 Basic SLA Management Service The Basic SLA Management Service (BSMS) is a basic non-normative API description to provide basic support for the SLA enforcing protocol as motivated in the SLA-Driven API development lifecycle (c.f., Section 4.2) and addresses the following features: (i) Checking the current state of a given SLA (SLA Check). (ii) Reporting metrics to calculate the current state of a given SLA (SLA Metrics). To this end, this BSMS proposal represents a descriptive interface that could be implemented in different technologies and acts as a decoupling mechanism to the underlying infrastructure that actually provides support to the development lifecycle. Moreover, the definition of a BSMS paves the way to define multiple SLA enforcing architectures that could be selected depending on the performance or technological constraints of a given scenario. Specifically, Figure 2 and Figure 3 represent an overview of two different SLA enforcing architectures: on the one hand, the Standalone enforcing define an SLA instrumentation as part of the API with a direct communication with the SLA management infrastructure; on the other hand, a Gateway enforcing relays on the front load balancer to connect with the SLA management infrastructure so a potential set of API instances do only provide the functional logic. SLACheck SLAMetrics Request workload API 1 2 3 4 5 6 Fig. 2. Standalone SLA enforcing arch. SLACheck SLAMetrics Request workload API APIGateway API API 1 2 3 4 5 6 Fig. 3. Gateway SLA enforcing arch. In order to illustrate the interactions and behavior of each component implementing (or interacting with) the BSMS, we will focus on the Gateway enforcing architecture (See Figure 3) as it is a more complete scenario: 1. Requests will pass through the API Gateway until they are directed to the node that will serve it (step 1). 2. The API Gateway query the SLA Check component to determine if the request is authorized to develop the actual operation based on the appropriate SLA (step 2). (a) If it is authorized, the actual API is invoked and the response is returned (step 3). (b) If it is not authorized, a status code and a summary of the reason (as generated by the SLA check component) is returned (step 3). 3. After the consumption ends (step 4), the metrics are sent to the SLA Metrics component (step 5). This component is in charge of updating the status of the agreement with the new metrics introduced (step 6). This new information