The IDL Tool Suite: Inter-parameter Dependency Management in Web APIs

The IDL Tool Suite: Inter-parameter Dependency Management in Web APIs Saman Barakat1(B), Alberto Martin-Lopez2, Carlos Müller1, and Sergio Segura1 1SCORE Lab, I3US Institute, Universidad de Sevilla, Seville, Spain {salias,cmuller,sergiosegura}@us.es 2SEART @ Software Institute, Universitá della Svizzera Italiana, Lugano, Switzerland [email protected] Abstract. Web APIs contain inter-parameter dependencies that restrict the way in which input parameters can be combined to form valid calls to the service. Inter-parameter dependencies are extremely common and pervasive: they appear in 4 out of every 5 APIs across all application domains and types of operations. In this demonstration paper, we present the IDL tool suite, a comprehensive collection of tools designed to facilitate dependency management in web APIs. The IDL tool suite includes a specification language for inter-parameter dependencies (IDL), and OAS extension (IDL4OAS), a web editor for IDL specifications, an analysis engine (IDLReasoner), a web API for the analysis of IDL, and a website with detailed information about the tool suite and a playground. In addition to these tools, we present a catalog of applications where the IDL tool suite has already proven useful, including automated testing, code generation, and dependency-aware API gateways. We trust that the IDL tool suite will enable promising new research and applications in the area of web API management. The demo video of the IDL tool suite is available at https://www.youtube.com/watch?v=Hy5HYGK8Yn4. Keywords: Web API ·REST ·OpenAPI Specification ·IDL 1 Introduction Web Application Programming Interfaces (APIs) facilitate communication between software systems over the Internet, which makes them ideal for software integration. Modern web APIs typically adhere to the Representational State Transfer (REST) architectural style [3], and are often described using the OpenAPI Specification (OAS) [9], which provides a human- and machine-readable description of the API’s supported operations, parameters and responses. OAS documents are a valuable asset for automating tasks in the API lifecycle such as testing, code generation, and documentation. Despite this, OAS (and other specification languages) currently lack official support for describing inter-parameter chttps://doi.org/10.1007/978-981-97-0989-2_26 312 S. Barakat et al. dependencies, i.e., restrictions between parameters that must be satisfied to form a valid API request [5,6]. For instance, when searching for businesses in the Yelp API [11], the parameter location is “required if either latitude or longitude is not provided”. A recent study on 40 industrial APIs revealed that inter-parameter dependencies are extremely common and pervasive, as they appear in 4 out of every 5 APIs across all application domains and types of operations [6]. In previous work [5], we introduced the Inter-parameter Dependency Language (IDL) and IDLReasoner as a first step to specify and automatically analyze dependencies, respectively. We also created IDL4OAS, an OAS extension to integrate our approach in this language. Built upon this foundation, in this demonstration paper we present the IDL tool suite, a comprehensive collection of tools designed to facilitate dependency management in web APIs. The IDL tool suite extends IDL and IDL4OAS with the following novel contributions: 1) a web editor for IDL specifications, 2) an enhanced version of IDLReasoner, 3) a new analysis operation (explanation), 4) a web API for automated analysis of IDL with interactive documentation, and 5) a website with detailed information about the tool suite and a playground. In the rest of the paper, we provide an overview of the tools included in the IDL tool suite and the applications that these have already enabled, as well as future promising applications. 2 The IDL Tool Suite All our tools are publicly available on the IDL website [4]. The tool suite includes the IDL language, the IDL4OAS extension, an enhanced version of IDLReasoner, an IDLReasoner web API, and a web-based editor for IDL specifications. In the following sections, we provide an overview of each of these tools (Fig. 1). Fig. 1. IDL tool suite contributions. 2.1 Inter-parameter Dependency Language (IDL) IDL is a textual domain-specific language (DSL) that allows to formally specify all the seven patterns of dependencies identified in our previous study [6]. Listing 1shows some real-world examples of inter-parameter dependencies from the Google Maps Places API. For instance, line 3 shows a Requires dependency: if parameter rankby is set to ‘distance’, then either keyword,name or type must be present in the API request. Similarly, line 4 defines an Arithmetic The IDL Tool Suite: Inter-parameter Dependency Management in Web APIs 313 dependency, which states that parameter maxprice must be greater than or equal to minprice. IDL supports five other dependency patterns, namely, Or, ZeroOrOne,OnlyOne,AllOrNone and Complex dependencies. We refer the reader to our previous work for a detailed description of these dependencies, the IDL language and its grammar [5]. 1// Operation: Search for places within specified area: 2ZeroOrOne(radius, rankby=='distance'); 3IF rankby=='distance' THEN keyword OR name OR type; 4maxprice >= minprice; 5 6// Operation: Query information about places: 7AllOrNone(location, radius); 8Or(query, type); 9maxprice >= minprice; 10 11 // Operation: Get photo of place: 12 OnlyOne(maxheight , maxwidth); 13 14 // Operation: Automcomplete place name: 15 IF strictbounds THEN location AND radius; Listing 1. IDL specification of the Google Maps Places API. 2.2 IDL4OAS IDL4OAS is an OAS extension that allows to describe inter-parameter dependencies within an OAS document, as shown in Listing 2, which depicts the specification of the search operation of the Google Maps Places API. As illustrated, the property x-dependencies is added at the operation level, allowing to specify dependencies among its parameters with IDL. An IDL4OAS document can serve as the basis for automated dependency-aware testing, code generation and monitoring, as will be shown in Sect. 3. 1paths: 2/search: 3get: 4[...] 5x-dependencies: 6-ZeroOrOne(radius, rankby=='distance'); 7-IF rankby=='distance' THEN radius OR name OR type; 8-maxprice >= minprice; Listing 2. OAS document extended with IDL4OAS. 2.3 IDLReasoner IDLReasoner is a Java library for automatically analyzing IDL specifications [5]. It supports analysis operations for multiple purposes, for example, to check if an IDL specification is consistent, or to detect dead parameters (i.e., parameters that can never be used in API requests due to contradictory constraints). For instance, the IDL4OAS document shown in Listing 2actually contains a mistake, since the first dependency (line 6) states that radius and rankby==‘distance’ cannot be used together, while the second dependency 314 S. Barakat et al. states that if rankby==‘distance’, parameter radius may be included in the API request. IDLReasoner can detect these inconsistencies automatically. In this paper, we introduce a completely refactored version of IDLReasoner that leverages Choco [10]—a popular Java library for constraint programming— as the default constraint solver, making it more time- and memory-efficient. In addition, we implement a novel explanation analysis operation, which returns a human-readable explanation of the output of any of the existing analysis operations. For example, for the previous error, the explanation message would indicate the presence of two contradictory dependencies in lines 6 and 7. 2.4 Web API One of our novel contributions to the IDL tool suite is the IDLReasoner API.1 It consists in a RESTful API described in OAS, complemented with a Swagger documentation portal. The API supports all IDLReasoner analysis operations via simple HTTP requests. It is a key asset for integrating IDLReasoner into external projects such as test case generators or documentation portals. As a matter of fact, the IDLReasoner API is integrated into the IDL web editor. 2.5 IDL Web Editor Our last novel contribution for the IDL tool suite is a web-based editor for IDL.2 This editor acts as a playground for external users to learn and understand the capabilities of IDL, IDLReasoner and IDL4OAS, but also to analyze their own APIs on the fly, without the need of installing additional software. 3 Applications In this section, we outline some of the useful applications that the IDL tool suite has enabled in varied domains, including automated testing, code generation, and API gateways. 3.1 Automated Testing API specifications enhanced with IDL4OAS can be automatically analyzed with IDLReasoner. Testing tools for web APIs generally derive test cases automatically from the API specification. Thus, given an OAS document enriched with IDL4OAS, the IDL tool suite can support the automated generation of valid test cases (i.e., those satisfying all inter-parameter dependencies). This is the case of RESTest [7], a constraint-based testing tool which has automatically found over 200 bugs in industrial APIs such as Spotify and YouTube [8]. 1http://idl-reasonerapi.tech/swagger-ui/index.html. 2http://idl-playground.tech. The IDL Tool Suite: Inter-parameter Dependency Management in Web APIs 315 3.2 Code Generation An OAS document can drive the generation of source code for API client libraries or server stubs in a variety of programming languages [9]. Recently, we proposed IDLGen [1], an approach to automatically generate code to handle interparameter dependencies in servers written in Java and Python. Based on an OAS document extended with IDL4OAS, IDLGen generates assertions to validate the inter-parameter dependencies of incoming API requests, as well as code to handle invalid requests that violate one or more dependencies. Our results show that IDLGen can generate up to 10 times more code than current generators, and it can save up to 25 min of development time per API operation [1], potentially avoiding input validation failures in production [8]. 3.3 Smart API Gateway When deployed in an API gateway, IDLReasoner can be used to detect and block invalid requests that violate inter-parameter dependencies as well as to automatically generate explanations for such violations. In a recent study [2], we found that a dependency-aware API gateway could reduce response times of invalid requests by up to 80%, while incurring a negligible overhead in terms of latency and throughput for valid requests. 4 Conclusion and Future Work The IDL tool suite is a comprehensive solution for inter-parameter dependency management in web APIs. These assets have already proven useful for the development of dependency-aware approaches for automated testing of web APIs, code generation for client and server applications, and API gateways. We envision new promising applications of the IDL tool suite in the future, including enhanced documentation portals, monitoring solutions, and API design tools. Acknowledgements. This work has been partially supported by grants PID2021-126 227NB-C22 and TED2021-131023B-C21, funded by MCIN/AEI/10.13039/501100 011033 and by European Union “NextGenerationEU”/PRTR». References 1. Barakat, S., Sanchez, A.B., Segura, S.: IDLGen: automated code generation for inter-parameter dependencies in web APIs. In: Monti, F., Rinderle-Ma, S., Ruiz Cortés, A., Zheng, Z., Mecella, M. (eds.) ICSOC 2023. LNCS, vol. 14419, pp. 153– 168. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-48421-6_11 2. Barakat, S., Sanchez, A.B., Segura, S.: Toward dependency-aware API gateways. In: JCIS (2023) 3. Fielding, R.T.: REST: architectural styles and the design of network-based software architectures. Doctoral dissertation, University of California (2000) 4. The IDL Tool Suite. https://isa-group.github.io/IDL/. Accessed July 2023 316 S. Barakat et al. 5. Martin-Lopez, A., Segura, S., Müller, C., Ruiz-Cortés, A.: Specification and automated analysis of inter-parameter dependencies in web APIs. IEEE Trans. Serv. Comput. 15(4), 2342–2355 (2021) 6. Martin-Lopez, A., Segura, S., Ruiz-Cortés, A.: A catalogue of inter-parameter dependencies in RESTful web APIs. In: Yangui, S., Bouassida Rodriguez, I., Drira, K., Tari, Z. (eds.) ICSOC 2019. LNCS, vol. 11895, pp. 399–414. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33702-5_31 7. Martin-Lopez, A., Segura, S., Ruiz-Cortés, A.: RESTest: black-box constraintbased testing of RESTful web APIs. In: Kafeza, E., Benatallah, B., Martinelli, F., Hacid, H., Bouguettaya, A., Motahari, H. (eds.) ICSOC 2020. LNCS, vol. 12571, pp. 459–475. Springer, Cham (2020). https://doi.org/10.1007/978-3-030- 65310-1_33 8. Martin-Lopez, A., Segura, S., Ruiz-Cortés, A.: Online testing of RESTful APIs: promises and challenges. In: ESEC/FSE, pp. 408–420 (2022) 9. OpenAPI Specification. https://www.openapis.org/. Accessed July 2023 10. Prud’homme, C., Fages, J.G.: Choco-solver: a Java library for constraint programming.J.OpenSourceSoftw.7(78), 4708 (2022) 11. Yelp API. https://docs.developer.yelp.com/reference. Accessed July 2023