From Text to Performance Measurement: Automatically Computing Process Performance Using Textual Descriptions and Event Logs

From Text to Performance Measurement: Automatically Computing Process Performance Using Textual Descriptions and Event Logs Manuel Resinas1(B), Adela del-Río-Ortega1, and Han van der Aa2 1SCORE Lab, I3US, Universidad de Sevilla, Seville, Spain {resinas,adeladelrio}@us.es 2University of Mannheim, Mannheim, Germany [email protected] Abstract. Process performance measurement assesses how well a process is running, covering various dimensions such as time, cost, and quality. This task involves the definition of measurable Process Performance Indicators (PPIs), which in many cases are calculated based on data recorded in an event log. An inhibitor of effective performance analysis is that establishing PPI definitions measurable from event logs is highly complex, because it requires process analytical expertise, as well as in-depth knowledge about the structure and contents of the available event data. Given that managers typically do not have such knowledge, this means that those stakeholders that are generally most interested in measuring process performance cannot do so in a convenient manner. Recognizing this, we bridge this gap by proposing an approach for the measurement of process performance based on textual descriptions and event logs, which combines state-of-the-art natural language processing techniques with matching strategies that are tailored to the task at hand. Evaluation experiments using textual descriptions provided by both industry and academic users demonstrate the accuracy of our approach. Keywords: Process performance measurement ·process mining · natural language processing ·matching 1 Introduction Process Performance Measurement is the practice of evaluating various dimensions of business processes, such as time, cost, and quality, to determine if business processes are achieving strategic and operational goals, and to assist in their This work has been partially supported by projects PID2021-126227NB- C21/ AEI/10.13039/501100011033/ FEDER, UE; TED2021-131023B-C22/ AEI/10.13039/501100011033/ Unión Europea NextGenerationEU/PRTR, and US-1381595 (Junta de Andalucía/FEDER, UE). https://doi.org/10.1007/978-3-031-41620-0_16 From Text to Performance Measurement 267 optimization. It includes the definition, collection, visualization and analysis of Process Performance Indicators (PPIs), which are quantifiable metrics used to evaluate the efficiency and effectiveness of one or more business processes [4]. PPIs can be calculated from different sources, with recorded execution data, stored event logs, being among the main ones [15] and the focus of this paper. The definition of PPIs calculated from event logs consists of two primary parts: (1) establishing a formal definition of the manner in which process performance should be measured, according to a certain metamodel for PPIs [13,16,21], and (2) linking that definition to the data structure and contents of a specific event log. For example, to measure the “averagetimeuntil reimbursement” in a travel reimbursement process, part (1) involves the recognition that this corresponds to a PPI that should compute the average (an aggregate measure) time (a measure type) between receiving a request (start moment) and the reimbursement being paid (end moment). Part (2), in turn, requires one to recognize that this measure should be linked to the time between “receive request ”and“payment handled” activities in a particular event log. The problem here is that both parts of this task involve expertise from users regarding process performance measurement and process mining (how to properly define PPIs, how event logs work, etc.), as well as in-depth knowledge about the data in an event log (which activity or attribute value corresponds to an occurrence of interest, e.g., that the moment of reimbursement, non-straightforwardly, corresponds to a “payment handled” activity). Given that such expertise and domain knowledge are rarely (both) held by managers, process performance currently cannot be conveniently measured by those stakeholders most interested in it. Instead, managers need to involve process analysts and domain experts to obtain the insights they desire, which can lead to considerable hindrance in terms of additional effort and delays, as well as possibly incorrect measurements caused by miscommunication [1]. In this work, we overcome this barrier by proposing an approach for measuring process performance based on event logs and textual descriptions. In this manner, our work allows managers to conveniently and quickly obtain useful insights by describing desired performance measures in a textual manner, such as “the fraction of requests that are rejected”or“the maximum time between receiving and delivering orders that were approved”. With this goal, our work complements recent work on using natural language querying in process mining [2,11], with an approach tailored to the task of process performance measurement. Our approach builds on a language model fine-tuned to the task of extracting entities from PPI descriptions. To align these extracted entities to the contents of an event log, we propose various matching functions, as well as heuristics to infer missing information. Evaluation experiments using real-world event logs and PPI descriptions collected from industry and academic users highlight the potential of our approach, yet also reveal the challenging nature of the task. In the remainder, Sect. 2describes the challenges of automated transformation, whereas Sect. 3provides essential definitions. Section 4describes our 268 M. Resinas et al. automated approach to compute PPIs described in natural language. Section 5 presents a quantitative evaluation of our approach. Streams of related work are described in Sect. 6and we conclude our paper in Sect. 7. 2 Problem Illustration This section illustrates the main challenges associated with the transformation of textual PPI descriptions into measurable definitions. As a basis for this, we use the well-known domestic declarations event log from the 2020 BPI Challenge [5] and the PPI descriptions in Table 1. This process involves the submission, approval or rejection, and payment of travel declarations by employees. Table 1. Exemplary PPI descriptions ID Description ppi1 The average duration between submission and payment of a declaration ppi2 The average time it takes for a declaration to be paid after its submission ppi3 The amount of time until reimbursement ppi4 The percentage of rejected requests ppi5 The number of denied declarations as a fraction of the total submitted ones ppi6 The number of declarations above 100 euros ppi7 The total amount paid per year C1: High Flexibility of Natural Language. Textual PPI descriptions can describe the same measure in many different ways. For example, ppi1 to ppi3 all describe the time between submission and payment of declarations, using different structures (e.g., starting point first or last) and terminology (e.g., duration versus time). Similarly, whereas ppi4 immediately indicates that this is a fractional measure, this information comes later in ppi5, which starts in the exact same manner (“The number of [..]”) as is common for count measures, such as ppi6. These examples are only the tip of the iceberg, though, which means that a transformation approach must be able to deal with highly variable input. C2: Differences Between Description and Data. When describing a measure of interest, users do not necessarily account for the way that a process is recorded, which can result in large differences between the contents of a textual description and an event log. This often results in the use of synonyms (denying versus rejecting), though differences may also be process specific. For example, ppi3 refers to the time until reimbursement, yet there is no activity in the event log that contains this term. Rather, the moment of reimbursement corresponds to the Payment handled activity. Therefore, when matching information extracted from a PPI description to the contents of an event log, an approach must be able to find challenging correspondences. From Text to Performance Measurement 269 C3: Missing Information. Finally, PPI descriptions may leave certain information implicit that is required to define a measure. Common examples are: missing aggregation functions (does ppi3 refer to the average, total, or individual time until payment?), missing starting points of time measures (what is the starting point of ppi3?), and missing denominators of fractions (e.g., in ppi4). A transformation approach needs to be able to make the right choices in such situations, in order to still be able to compute a value for the desired measure. 3 Measurable PPI Definitions This section presents the formal PPI definitions that our approach uses. These definitions are inspired by the PPINOT metamodel [4], which we specifically adapted to the way in which PPIs are commonly described in natural language, allowing for easier and more accurate transformation from textual description to a formal PPI. The values for PPIs defined in this manner can be automatically measured using a PPI computation tool (cf., [15]). Scope. Our work covers a broad range of PPI definitions, allowing users to combine the following aspects. Each PPI definition should correspond to one of three types of base measures: count,time,anddata. These can be complemented with additional operators such as aggregation functions (e.g., min.,max.,average), group-by conditions (e.g., request per year or department), negation (e.g., requests not accepted), and filters (such as above 100 euros or in the form of a fraction, such as the fraction of rejected declarations). Definitions. We formalize measurable PPIs through the following definitions. Definition 1 (Universes). We define the following universes: –Uatt and Uval are the universes of attribute names and values in an event log, including an activity attribute to refer to activities and their names. For each a∈U att,weusedom(a)⊆U val to refer to values that acan take. –Uagg is the universe of aggregation functions. In this paper we consider Uagg = {avg, max, min, sum, perc}. –Uop is the universe of operations. In this paper Uop ={==,=,>,<,≥,≤}. –Ucase is the universe of possible conditions that refer to a case. In this paper, Ucase ={begin, end}, which refer to the beginning and end of a case. –Umval is the universe of possible values of base measures computed on an event log. This includes integers (for count measures), time intervals like 7daysfor time measures, and Uval for data measures. We next define the different kinds of conditions used to specify measures: Definition 2 (Conditions). We define two sets of conditions: –Instant conditions CI=CE∪C Ccomprise event and case conditions. Event conditions CE=Uatt ×Uop ×Uval are tuples that relate an attribute name to a value using a comparison operator. Case conditions CC=Ucase refer to either the beginning or end of a case. 270 M. Resinas et al. –Measure conditions CM=Uop ×U mval are tuples that define a boolean expression based on a comparison against a possible measure value (Umval). An example of an event condition is (activity,==,submit declaration),which occurs when a submit declaration activity is completed, while to select cases of declarations above 100 euros, we specify (>,e100) as a measure condition over a measure value that refers to a case attribute. Using instant and measure conditions, we can define the base measures supported by our approach. Definition 3 (Base measures). Base measures MB=MC∪M T∪M D, comprise three types: –Count measures MC=CI×(CM∪{⊥})are tuples that include an instant condition that specifies when to count, and an optional measure condition that is applied to the result of the count (⊥as the absence of a condition). –Time measures MT=CI×CI×(CM∪{⊥})are tuples that include two instant conditions specifying when the time measure starts and stops, respectively, and an optional measure condition (⊥as the absence of a condition). –Data measures MD=Uatt ×(CM∪{⊥})are tuples that include the attribute whose value we want to obtain, and an optional measure condition. For example, the count measure for ppi4 is ((activity,==,reject),(>, 0)),1the time measure for ppi3 is (begin, (activity,==,payment handled),⊥),andthe data measure for ppi7 is (amount, ⊥). Finally, we define aggregated measures, which expand the expressiveness of base measures with aggregation, group-by, and filtering options: Definition 4 (Aggregated measures). The set of aggregated measures MA=MB×U agg ×(Uatt ∪{None})×(CE∪⊥)is the set of tuples such that a=(b, agg, att, c)means that the aggregation function agg is applied over the base measure b, grouping by attribute att, and filtering the cases that meet condition c.Ifatt =None, this means that no grouping is applied; if c=⊥, this means that no condition is applied. For example, the full measure for ppi1 is (((activity,==,submit),(activity,== ,payment handled),⊥),average,None,⊥), i.e., the average time between the two activities, and for ppi7 we get ((amount, ⊥),sum,year,⊥), to group the data measure (sum of the amount attribute) per year. 4 Approach Figure 1depicts an overview of our approach. The input is a textual description of a PPI and the output is the result of evaluating this PPI against a given event log. The approach consists of four main steps. Step 1 focuses on the extraction of relevant entities from the textual PPI description (tackling challenge C1). Step 1(>, 0) is used to count the cases for which this activity happens at least once. From Text to Performance Measurement 271 2 matches the extracted entities against the contents of the event log in order to start establishing a measurable PPI definition (challenge C2). Then, for cases in which a user left out certain required information (challenge C3), Step 3 uses various heuristics to fill in the gaps and thereby complete the PPI definition. Finally, Step 4 uses the established definition in order to compute the desired PPI, thus directly measuring process performance for the event log. Fig. 1. Overview of our approach 4.1 Step 1: Entity Extraction In this first step, our approach takes a textual PPI description Pas input and extracts the entities necessary to establish a PPI definition. We first discuss the kinds of entities we extract, before describing the extraction technique itself, and the data augmentation strategy we used to compensate for the small size of the available training data. Entity Types. Our approach aims to extract a number of entity classes, which we each denote with a specific tag. The tags for base measures are: – Count measures: We use a count entity CE to capture what should be counted, e.g., requests for reimbursement or accepted orders. – Time measures: We use start and endpoints (TSE and TEE) when present in descriptions, e.g., “The amount of time until reimbursement” contains an end point (“reimbursement”). However, descriptions of time measures may also use a single entity to refer to the range from start to end, for which we use a TBE tag, such as for “The time to approve declarations”. – Data measures: We use the DMA tag to refer to the description of the attribute to be measured, such as the total amount in ppi7. On top of these classes for the measure types, we extract aggregation functions (AGR), such as average or maximum, group-by clauses (GBC), such as per year in ppi7, measure conditions, which are composed of an operator (CCI)anda measure value (MEV), such as above 100 euros in ppi6, and filters (FDE), such as pre-approved in the total amount of requests that are pre-approved. Extraction Technique. We apply a two-stage approach for entity extraction, using an annotated training dataset of PPI descriptions DT(see below) and a pre-trained language model [20] as a basis for both stages. In this paper, we use DistilBERT [18] as the pre-trained language model, but the proposal is independent of the language model used. 272 M. Resinas et al. In the first stage, we use text classification to categorize a PPI description Paccording to its base type (count, time, or data). To this end, we fine-tune the pre-trained language model, with a linear layer on top of its pooled output, using the gold-standard measure types of the training collection DT.Weusethe resulting fine-tuned model to infer the measure type of unseen PPI descriptions. In the second stage, we use token classification to identify the entities of interest in a description P. As a basis for this, we use the gold-standard tags of the descriptions in DT. Since entities can span multiple words, tags are assigned per chunk, e.g.: The\O,average\AGR, time between\O submission\TSE and\O payment of a declaration\TEE.2Based on these gold-standard tags, we finetune the pre-trained language model for token classification, using a linear layer on top of its hidden-states output. We do this separately for count, time, and data measures, so that we obtain language models that are specifically fine-tuned to extract information from a given measure type, as identified by the aforementioned text classifier. Given a textual PPI description P, we represent the output after token classification as a sequence ΦP\TP=φ1\t1,...,φ m\tm.Eachφi\tirepresents a chunk of text (φi) and its assigned tag (ti). Each chunk, φiconsists of one or more consecutive words from Pand each word is assigned to exactly one chunk. Data Augmentation. We only had a collection of 165 PPI descriptions available, including 129 from prior research [1]. However, the fine-tuning of language models requires a considerable amount of training data, especially when dealing with such diverse kinds of input and entities as in our work (challenge C1). We address this challenge through data augmentation. Specifically, we greatly extend the initial set of PPI descriptions with automatically generated ones. To this end, we handcrafted textual patterns based on the descriptions in the initial set, making sure that a wide variety of different patterns is included. For instance, a typical text pattern for time measures is [Agg] time from [cond1] to [cond2]. Then, we used Chatito3, a text generation tool, to generate distinct training phrases by combining all alternatives provided for each pattern. In this manner, we ended up with a total of 12,036 annotated descriptions in DT. Using this augmented set to train the aforementioned text and token classification techniques, our entity extraction step can deal with highly flexible input. 4.2 Step 2: Entity Matching We next set out to establish an actual measure MP, according to the concepts defined in Sect. 3. To illustrate this step, we use ppi1 as an example, whichStep1identifiesasatime measure with the tagged sequence ΦP\TP: average\AGR,submission\TSE,payment of a declaration\TEE. To establish MP, we start with the structure of an aggregated measure (b, agg, att, c), as defined in Definition 4, and expand its base measure bwith the structure of the identified measure type. Since ppi1 is a time measure, b’s structure is (s, e, (op, v)) (cf. Definition 3), yielding MP=((s, e, (op, v)),agg, att, c). 2Tag Oindicates that a chunk does not belong to any entity from the tag set. 3https://rodrigopivi.github.io/Chatito/. From Text to Performance Measurement 273 Overall Procedure. Given a tagged sequence ΦP\TP, each tagged chunk φ\t will be used to find a value for an element of MP. The correspondence between a chunk φ\tandanelementfollows from the tag t. For instance, average\AGR corresponds to agg,whereassubmission\TSE and payment of a declaration\TEE, respectively, correspond to the time measure’s start (s)andend(e). For a chunk φ\t,weuseamatching function match(φ\t)to identify the right value for its corresponding element in MP, from a target domain D.For instance, match(φ\AGR)identifies the value for its corresponding element agg from its domain, which is Uagg (cf., Definition 4). As detailed next, we propose six instantiations of match(φ\t), for different tags t. The general procedure is the same, though: match evaluates the similarity between the chunk φand elements of the target domain Dusing a similarity measure, returning the element with the highest score. Matching AGR Tags. Chunks with AGR tags correspond to aggregation functions, which means that the matching function’s target domain is Uagg = {avg, max, min, sum, perc}. Therefore, to match φ\AGR, we consider the similarity between φand each of the possible values of Uagg. Additionally, we also consider synonyms for each of them (e.g., average,mean,total average). This is formalized as match(φ\AGR)=argmaxd∈Uagg simcomb(φ, d),where simcomb combines both the syntactic and semantic similarity of text chunk φto the respective domain values dof the tag and its synonyms as follows: simcomb(φ, d)=w1semSim(φ, d)+w2synSim(φ, d) The semantic similarity (semSim(φ, d))ofφwith a domain value dis the average of the semantic similarity of dand its synonyms. This semantic similarity is the cosine distance between vectors obtained using an algorithm like word2vec4. The syntactic similarity (synSim(φ, d)) is determined using the mean of the wellknown Damerau-Levenshtein dl(φ, d)and the Jaro-Winkler j(φ, d)distances, where we consider the synonym of φwith the highest score. Matching CCI Tags. This tag corresponds to an operator (e.g., above or greater than). Therefore, its target domain is Uop. The approach followed is exactly the same as for aggregation functions: match(φ\CCI)=argmaxd∈Uop simcomb(φ, d). Matching DMA and GBC Tags. These tags corresponds to the attribute name used in data measures (e.g., amount in “the total amount paid per year”), and group by clauses (e.g., “orders per customer type”), respectively. For these tags, the function match again uses simcomb, this time to compare φto L’s attribute names in Uatt, which is their target domain. However, for the GBC tag, we restrict the target domain to a subset of Uatt, so that it only includes attributes with a relatively low number of distinct values (we use 15 as a guideline). The rationale is that we expect users to be interested in groupings with a limited number of categories (e.g., per customer_type), as opposed to grouping by attributes that have unique values per case (e.g., per order_ID). 4We use the en_core_web_lg model provided by spacy (https://spacy.io/). 274 M. Resinas et al. Matching CE and TBE Tags. The target domain of these tags is the set of instant conditions CIof the event log L, which are composed of three parts: attribute, operator, and value. Therefore, given a chunk φi\TBE like reimbursement in ppi3, its matched value is (activity,==,payment handled). We perform this matching at once, for just a single chunk φ, since users generally do not specify conditions in an attribute-operator-value manner, but rather use a shorthand. For instance, “reimbursement” does not explicitly state that this condition refers to an equals to operator and that the relevant attribute is an event’s activity. A challenge is that the target domain is huge (spanning numerous combinations of attributes and their values), so we apply three actions to reduce it. First, we only consider categorical attributes whose number of categories is lower than a threshold (we use 100). Second, we only consider the attributes whose value changes across the events of at least one case. The reason is that, if the attribute does not change, it is not useful to use it as a condition in count or time metrics. Finally, we restrict the operator of instant conditions to equals to,whereaswe support not equals to in the PPI completion step (Sect. 4.3). This restriction of the operator does not apply to the operator identified by the CCI tag. Let S⊆C Ibe the subset of conditions after applying these three actions. Then, to match a text chunk φto an instance condition, we define match(φ\t)= argmaxd∈S conSim(φ, d),wheret∈{CE,TBE}. Here we quantify the condition similarity conSim between φand a condition d=(a, ==,v)as follows: conSim(φ, d)=(1−watt)valSim(φ, a, v)+watt  vi∈dom(a) valSim(φ, a, vi) |dom(a)| The first part of the equation computes the value similarity (valSim) between the text chunk φand the attribute-value pair of the condition. The second part of the equation considers the average similarity between the chunk and all values of the attribute domain (dom(a)). This allows us to prioritize those attributes whose domain is closer to φ.watt represents the weight given to the latter. valSim(φ, a, v)itself is computed as follows: valSim(φ, a, v)=(1−wc)indSim(φ, v)+wcindSim(φ, a +v) Here, the first part of the equation computes the individual similarity (indSim) between the text chunk and the value of the attribute v, whereas the second part of the equation computes the same similarity but considering both the name of the attribute and its value (“<a><v>”). This is done to account for cases where the text chunk either omits or includes the name of the attribute. Finally, indSim(φ, v)can be computed using any established similarity measures. In this paper, we combine four similarity measures with a weighted sum: simcomb as defined above, simis, which uses a standard measure for bag-of-words- based similarity we applied in a previous work [1]. simemb, which uses the cosine distance between the sentence embeddings of the chunks [14]. simbert, which uses pre-trained natural language inference models as zero shot text classifiers [22]. Matching TSE and TEE Tags. These tags refer to the conditions that determine the beginning (TSE)andend(TEE) of a time measure such as “the duration From Text to Performance Measurement 281 language text and a database schema as input to generate an SQL query that can be directly computed on the database. An example of this is the work presented in [8], but many more can be found in a recent survey [10]. The second thread does not generate an intermediate query model, but uses deep learning to learn the appropriate output, given a natural language query and a database table. Some examples are [6,7]. Although these works address a similar problem to the one presented in this paper, the nature of processes and the event logs that collect their information differs significantly from that of databases, which prevents us from using the same approaches off-the-shelf. NLP Interfaces for Process Mining. Finally, there is developing interest in using NLP to facilitate process mining tasks. For instance, Kobeissi et al. [11] present an intent-based natural language interface to allow users to perform queries on an event log. However, their work focuses on queries about event related data, with performance queries being out of scope. In addition, [11] needs to be adapted for each event log, unlike our proposal, which is log-independent and does not require human intervention to hand-craft the training set for each new event log. Barbieri et al. [2] present an architecture to support a conversational, process mining oriented interface to existing process mining tools, but only focused on questions over process execution data. This preliminary work is extended in [3]. It introduces a taxonomy for natural language questions for process mining and also provides support to queries over process behavior and process mining analyses. However, their implementation and evaluation is performed with general questions applicable to any event log, leaving those associated with selected, domain-specific event logs for future work. 7 Conclusion In this paper, we presented a first approach to calculate process performance indicators against a given event log based on textual descriptions. Our work builds on a fine-tuned language model to extract relevant entities, tailored techniques to match these entities to the contents of an event log, as well as completion heuristics to deal with incomplete descriptions provided by users. The evaluation performed yielded promising results, although there are also clear open challenges. It is worth noting that the heuristics used for completing PPIs have been useful in improving the performance of our approach. In addition, the publicly available dataset we established for this work is valuable in itself, as it allows other researchers and us to continue advancing in this direction. In future work, we aim to improve both the scope and accuracy of our work. For this, we naturally aim to use the exponentially increasing potential of large language models (LLMs) such as GPT-4. A first direction, aiming for accuracy improvements, is to incorporate LLMs directly into our proposed approach by using their functionality to replace parts of our work that currently rely on other NLP technologies, such as the entity-extraction step and the computation of semantic similarity scores. Next to that, we aim to use the conversational capabilities of LLMs to facilitate interaction between a user and an approach 282 M. Resinas et al. such as ours, in order to guide users through the step-by-step definition of PPIs using textual input. In such an interactive setting, the agent can ask clarifying questions where appropriate and, furthermore, allow users to iteratively build up highly expressive performance measures, thus improving both the accuracy and scope of our work. Acknowledgments. We thank Maria Isabel Ramos and Javier Vilariño for their support in the implementation. References 1. van der Aa, H., Leopold, H., del Río-Ortega, A., Resinas, M., Reijers, H.A.: Transforming unstructured natural language descriptions into measurable process performance indicators using hidden markov models. Inf. Syst. 71, 27–39 (2017) 2. Barbieri, L., Madeira, E.R.M., Stroeh, K., van der Aalst, W.M.P.: Towards a natural language conversational interface for process mining. In: Munoz-Gama, J., Lu, X. (eds.) ICPM 2021. LNBIP, vol. 433, pp. 268–280. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-98581-3_20 3. Barbieri, L., Madeira, E.R.M., Stroeh, K., van der Aalst, W.M.: A natural language querying interface for process mining. J. Intell. Inf. Syst. (2022). https://doi.org/ 10.1007/s10844-022-00759-9 4. del-Río-Ortega, A., Resinas, M., Durán, A., et al.: Visual PPINOT: a graphical notation for process performance indicators. Bus. Inf. Syst. Eng. 61(2), 137–161 (2019) 5. van Dongen, B.: BPI challenge 2020 domestic declarations. https://doi.org/10. 4121/uuid:52fb97d4-4588-43c9-9d04-3604d4613b51 6. Eisenschlos, J.M., Gor, M., Müller, T., Cohen, W.W.: MATE: multi-view attention for table transformer efficiency (2021) 7. Herzig, J., Nowak, P.K., Müller, T., Piccinno, F., Eisenschlos, J.: TaPas: weakly supervised table parsing via pre-training. In: ACL, pp. 4320–4333 (2020) 8. Hui, B., et al.: Improving text-to-SQL with schema dependency learning. arXiv preprint arXiv:2103.04399 (2021) 9. Janiesch, C., Matzner, M.: BAMN: a modeling method for business activity monitoring systems. J. Decis. Syst. 28(3), 185–223 (2019) 10. Katsogiannis-Meimarakis, G., Koutrika, G.: A survey on deep learning approaches for text-to-SQL. VLDB J. 32, 905–936 (2023) 11. Kobeissi, M., Assy, N., Gaaloul, W., Defude, B., Haidar, B.: An intent-based natural language interface for querying process execution data. In: ICPM, pp. 152–159. IEEE (2021) 12. de Leoni, M., Mannhardt, F.: Road traffic fine management process. https://doi. org/10.4121/uuid:270fd440-1057-4fb9-89a9-b699b47990f5 (2015) 13. Popova, V., Sharpanskykh, A.: Modeling organizational performance indicators. Inf. Syst. 35(4), 505–527 (2010) 14. Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: EMNLP. ACL, November 2019 15. Resinas, M., del Río-Ortega, A., Ruiz-Cortés, A.: PPINOT computer and ppinot4py: Two libraries to compute process performance indicators. In: ICPM (Demo track) (2021) From Text to Performance Measurement 283 16. del Río-Ortega, A., Resinas, M., Cabanillas, C., Ruiz-Cortés, A.: On the definition and design-time analysis of process performance indicators. Inf. Syst. 38(4), 470– 490 (2013) 17. del Río-Ortega, A., Resinas, M., et al.: Using templates and linguistic patterns to define process performance indicators. Ent. Inf. Sys. 10(2), 159–192 (2016) 18. Sanh, V., Debut, L., Chaumond, J., Wolf, T.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019) 19. Vogelgesang, T., Ambrosy, J., Becher, D., Seilbeck, R., Geyer-Klingeberg, J., Klenk, M.: Celonis PQl: a query language for process mining. In: Polyvyanyy, A. (ed.) Process Querying Methods, pp. 377–408. Springer, Cham (2022). https:// doi.org/10.1007/978-3-030-92875-9_13 20. Wang, H., Li, J., Wu, H., Hovy, E., Sun, Y.: Pre-trained language models and their applications. Engineering (2022) 21. Wetzstein, B., Ma, Z., Leymann, F.: Towards measuring key performance indicators of semantic business processes. In: Abramowicz, W., Fensel, D. (eds.) BIS 2008. LNBIP, vol. 7, pp. 227–238. Springer, Heidelberg (2008). https://doi.org/10.1007/ 978-3-540-79396-0_20 22. Yin, W., Hay, J., Roth, D.: Benchmarking zero-shot text classification: datasets, evaluation and entailment approach. In: EMNLP, pp. 3914–3923 (2019)