TugaTAC Broker: A Fuzzy Logic Adaptive Reasoning Agent for Energy Trading

TugaTAC Broker: A Fuzzy Logic Adaptive Reasoning Agent for Energy Trading Thiago R.P.M. R´ubio(B), Jonas Queiroz, Henrique Lopes Cardoso, Ana Paula Rocha, and Eug´enio Oliveira LIACC/DEI, Faculdade de Engenharia, Universidade do Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal {reis.thiago,jonas.queiroz,hlc,arocha,eco}@fe.up.pt Abstract. Smart Grid technologies are changing the way energy is generated, distributed and consumed. With the increasing spread of renewable power sources, new market strategies are needed to guarantee a more sustainable participation and less dependency of bulk generation. In PowerTAC (Power Trading Agent Competition), different software agents compete in a simulated energy market, impersonating broker companies to create and manage attractive tariffs for customers while aiming to profit. In this paper, we present TugaTAC Broker, a PowerTAC agent that uses a fuzzy logic mechanism to compose tariffs based on its customers portfolio. Fuzzy sets allow adaptive configurations for brokers in different scenarios. To validate and compare the performance of TugaTAC, we have run a local version of the PowerTAC competition. The experiments comprise TugaTAC competing against other simple agents and a more realistic configuration, with instances of the winners of previous editions of the competition. Preliminary results show a promising dynamic: our approach was able to manage imbalances and win the competition in the simple case, but need refinements to compete with more sophisticated market. Keywords: PowerTAC ·Energy trading agents ·Smart electricity market ·Smart grids ·Fuzzy logic ·Power tariffs 1 Introduction The management of energy consumption and production is not only a customer concern, but however, a new trend characterised by the wide presence of distributed renewable power sources in low voltage grids. This factor is imposing new challenges for main energy generation and distribution companies. In this new scenario companies are not able anymore to predict energy demand, given limited visibility (small and distributed generator units are unknown), production volatility (weather uncertainty affects renewable energy generation) and consumption flexibility (caused by smart grid and home automation technologies that can control and shift loads to improve customer efficiency). c Springer International Publishing Switzerland 2016 M. Rovatsos et al. (Eds.): EUMAS 2015/AT 2015, LNAI 9571, pp. 188–202, 2016. DOI: 10.1007/978-3-319-33509-4 16 [email protected] TugaTAC Broker: A Fuzzy Logic Adaptive Reasoning Agent 189 Electricity markets at retail level can help to address grid energy generation and load balance challenges, providing economical incentives for customers to control and shift loads. Moreover, the available information also enables the consumption and production forecasts [1]. As a result of liberalisation programs, the new dynamics of electricity markets allow more complex market approaches, competition and indirect supply-demand regulation through energy tariffs [2]. Figure 1shows the different layers of a smart grid, highlighting tariff as the element which enables brokers to act on the customers layer and provide customer’s access to energy plans. Fig. 1. Tariffs as the link element between Market and Customers The energy brokers represent companies, intermediary trading entities responsible for providing tariffs to customers. Aiming to achieve higher market shares and profit, brokers compete for offering attractive tariffs based on the negotiated energy and prices. Electricity market simulation frameworks comprise important tools to test and validate different approaches and algorithms for brokers in a simulated and controlled environment. In this sense, this work presents an approach based on fuzzy logic to define competitive strategies for energy brokers in the market. The proposed approach was defined and tested on the PowerTAC simulation framework. The paper is organised as follows. Section 2gives an overview of the challenges on the Electricity Markets area and related work regarding the PowerTAC competition. Section 3presents our approach to create a competitive broker agent named TugaTAC and Sect. 4describes the preliminary results, obtained from running competition experiments, comparing our model with other approaches. [email protected] 190 T.R.P.M. R´ubio et al. We conclude Sect. 5pointing the advantages of using fuzzy logic in the tariff creation process and looking for improvements in future work. 2 Electricity Markets and Power Trading Agent Competition Electricity markets comprise commercial environments where energy is traded by several entities, such as generator companies, retailers, intermediary utilities, households, small and medium enterprises, electric vehicles owners and others. Energy is negotiated for different time slots and intervals, ranging from several minutes to months and can be negotiated through directly purchase transactions, auctions, or tariff contracts. Usually, electricity markets are separated in the wholesale electricity market, where retailers (brokers) negotiate large amounts of energy (MWh) with big distribution entities, such as generation companies or power plants, and the retail market, where small and medium customers negotiate small quantities of energy (kWh). The retail market dynamics directly influence the wholesale market and viceversa, since retailers define their prices based on their customer portfolio and wholesale market price, while wholesale market define their prices based on supply and demand principles. These features create dynamic environments with high financial risks that have been leveraged by the advent of smart grid and the use of all kinds of smart appliances and metering. Creating intelligent autonomous systems to safely and effectively operate in such environments requires tests and validation of the employed strategies and algorithms, before deploying them in real world scenarios. In this sense, there exist electricity market simulation frameworks, such as the PowerTAC [3]. The PowerTAC employs many robust models, based on real historical data, to simulate the wholesale market, the regulated distribution utility, and the customer population, composed by different kinds of customers, such as households, electric vehicles, and a variety of commercial and industrial entities. Some of them can also have energy production capabilities through solar panels or wind turbines, for example. The regulated distribution utility uses a market-based mechanism for balancing the energy supply and demand. 2.1 PowerTAC Broker Agents PowerTAC also comprises an electricity market competition1, where different teams are challenged to develop fully autonomous broker agents to operate between wholesale and retail markets. In order to simulate a more realistic environment, the simulation relies in different constraints, such as fees and periodic payments [4]. The broker agents act as self-interested companies, aiming to make high profits from energy negotiations on both supply and demand sides. In the real world, brokers would represent energy retailers, commercial or municipal utilities or even energy cooperatives [5]. 1http://powertac.org/. [email protected] TugaTAC Broker: A Fuzzy Logic Adaptive Reasoning Agent 191 As retailers, brokers need to define profitable tariff contracts to achieve bigger market share. Thus, brokers indirectly compete in the energy market by offering specific tariffs contracts for each kind of customers (production, consumption, storage) and specific type of energy source (solar, wind, thermal, etc.). Moreover, brokers should try to reach a balanced portfolio, i.e. trying to keep the amount of energy produced by customers close to the demanded energy, in order to reduce the dependency from wholesale-coming energy. Agents can use different tariff features in order to draw the customers’ attention, including fixed or dynamic price for kWh along the day, incentives for energy saving, bonus for sign-up, early withdrawal penalties, and monthly distribution fees. In fact, brokers can analyse information from different sources, such as customers, wholesale market and even weather. Such information enables predicting subscribers’ production and consumption, which can lead to the necessary actions to keep reduced imbalances, through tariffs for complementary types of customers. In simulation environments it is easy to analyse values and compose binary solutions as “if variable is greater than some value then do that”. This approach limits brokers coverage, creating crisp sets of possible actions. In real decision making scenarios, human brokers compose their solutions based on both numbers and conceptual analysis. Humans often interpret concepts, such as “high”, “low”, or “interested”, enabling a richer set of combination values for actions [6]. In energy markets and also in the PowerTAC competition, many of these conceptual values could be combined to design a tariff generator mechanism. 2.2 Tariff Selection Problem Since we are dealing with tariff composition, an important problem is how to design competitive and interesting tariffs that provide the conditions required by customers and yet, be profitable to brokers [3]. Customers want to select the best tariff based on self interest. For example, some customers prefer tariffs with timeof-use price while others could prefer fixed prices, and so on. Customers actively participate in the market by choosing new tariffs through periodic evaluation of publicly offered tariffs. In PowerTAC, customers are utility-based, which means that they choose the next action based on the calculated gain on doing so. Nevertheless, the utility function used in customer models include an aversion of change and complexity that can retard the changing for better tariff offers [7]. Accordingly to the PowerTAC specification [8], customers in the competition evaluate new offers with a higher frequency at the beginning of the simulation. They use a inertia model for the probability of not evaluating tariffs, calculated as Iathat depend on the number of tariff publication cycles (n) and a factor I∈[0,1] as seen in Eq. 1. Ia= (1 −2−n)I(1) The key part of customer evaluation is the calculation of the expected gain over maintaining the current contract. Our intention is to find a good approach that can attract customers’ attention and also have high utility. If no broker [email protected] 192 T.R.P.M. R´ubio et al. achieve this goal, customers will use default tariffs provided by the default broker, an agent that assure at least one option for customers. In PowerTAC, the utility of a given tariff Tiis computed as a function of perkWh payments pvi, periodic payments ppi, a one-time sign-up payment psignupi, a potential one-time withdrawal payment pwithdrawiin case the customer withdraws its subscription before the tariff’s contract minimum duration, and an inconvenience factor xito account for inconvenience of switching subscriptions, and of dealing with time-of-use or variable prices or capacity controls. The Eq. 2 describes this utility. ui=f(pvi, ppi, psignupi, pwithdrawi, xi) (2) On the other hand, the cost of using a default broker tariff depends on the consumption amount Ctdefault, the cost per-kWh (P vdefault) and the periodic payment P pdefault, as seen in Eq. 3. More details about PowerTAC models can be found in [8]. costdefault = de  t=0 (Ctdefault ∗P vdefault +P pdefault) (3) Many PowerTAC related works address this problem with different approaches, depending on the market type and tariff features focus, as seen in Table 1. Reddy et al. [9] created a model to predict the attraction probability of a specific tariff, given the broker’s portfolio. Liefers et al. [10] uses a Tit-ForTat strategy, copying and improving opponent’s tariffs. The CrocodileAgent [11] in the other hand, uses market properties as scarcity, balance and oversupply to generate the “most needed” tariffs at a given time. Table 1. Comparison of PowerTAC broker agents Broker Approach Market AgentUDE [12] Agressive fee manipulation Retail CwiBroker [13] Equilibrium in continuous markets Wholesale CrocodileAgent [11] Maximize profitability Wholesale Mertacor [14] PSO to estimate relevant features Wholesale TacTex [15] MDP to minimize costs Wholesale Default broker [16] Fixed (high) prices Retail Although some of the related works on PowerTAC retail market describe conceptual characteristics, none have considered modelling conceptual values in the calculations. We have seen that mapping features to values only helped competitors to interpret the market in a simplistic and rigid way. This motivates addressing the tariff creation problem with a different paradigm, the conceptual analysis. The goal is to create a tariff generator that could interpret and adapt linguistic concepts helping to easily define efficient tariffs. [email protected] TugaTAC Broker: A Fuzzy Logic Adaptive Reasoning Agent 193 3 TugaTAC Broker Agent In this work, we propose a strategy for developing PowerTAC agents based on fuzzy models. The proposed approach is called TugaTAC. It consists on updating tariffs using a conceptual model for agent’s interest on selling or buying energy. Depending on the production and consumption quantities coming from the customers who are subscribed to TugaTAC’s tariffs (portfolio), a fuzzy model determines the broker intentions and what it needs to do in order to improve the tariffs and attract the best profile of clients (consumers or producers) that could help reducing imbalances. Figure 2shows a simplified scheme of the TugaTAC reasoning mechanism, in which the market prices are combined with the values of energy production and consumption from the broker’s customer portfolio. Fig. 2. TugaTAC tariff composition and interactions with the environment 3.1 Fuzzy Conceptual Tariff Strategy for Retail Market The reasoning mechanism described in Fig. 2is highly conceptual and connects numerical values to abstract interest. Fuzzy systems are suitable models for this kind of approach. Fuzzy is an alternative for the traditional binary logic in which variables can present more than two values (true or false), usually presenting a continuous range between 0 (completely false) to 1 (completely true) [17]. In fuzzy logic instead of a complex mathematical formulation, the variables are described using conceptual values, e.g. a temperature variable could be specified as “cold”, “normal”, and “hot”. Our approach for tariffs is based on a fixed price tariff model, where customers pay the same price along the day for the kilowatt-hour. Therefore, this strategy focuses on the price value definition. For this, it is considered the fluctuation of the wholesale market price (clear price), which varies along the day according to demand and production. This way the tariff price is always above the clear price or otherwise the broker would lose money. Thus, we have defined two fuzzy models: one for selling energy and the other for buying, implemented and tested using the jFuzzyLogic API2. 2http://jfuzzylogic.sourceforge.net/. [email protected] 194 T.R.P.M. R´ubio et al. The fuzzy models are illustrated in Fig. 3for buy-interest and in Fig. 4 the sell-interest model. The fuzzification process establishes the correspondence between the input and output models. A set of IF-THEN fuzzy rules are defined in terms of the concepts defined. This comprises one of the advantages of using fuzzy logic, allowing adaptive configurations for brokers in different scenarios by easily changing rules as observed in Algorithm 1. Fig. 3. Fuzzy input variable for model buy-interest Finally, to create the tariffs we used an approach similar to [11]. On initial rounds customers tend to be more opened to new tariffs. Besides, most of the tariffs are published in this period. TugaTAC uses a Tit-For-Tat approach, copying competitors’ tariffs when a tariff for the same power type does not exist in its portfolio. If there is already some similar tariff, the fuzzy model is triggered to calculate a new value, in order to beat the offered conditions. IF buy-production IS high THEN definitely-interested IF buy-production IS high AND buy-consumption IS high THEN interested IF buy-production IS low AND buy-consumption IS high THEN not-interested IF buy-production IS medium OR buy-consumption IS high THEN not-interested IF buy-production IS medium AND buy-consumption IS medium THEN interested Algorithm 1. Fuzzy rule set example: buy-interest variable The resulting value of the fuzzy represent a multiplying factor for the prices (per-kWh) called interest, which represent the willingness to perform the action, as interest ∈[0,1]. When the interest is high, then multiplication will make the prices rise, otherwise the price will decrease. The evaluation of the new price values are verified in two equations, one for buying (to producers, Eq. 4) and other for selling (to consumers, Eq. 5). buying :pricenew =pricelast −(pricelast ∗buyinterest) (4) selling :pricenew =pricelast + (pricelast ∗sellinterest) (5) [email protected] TugaTAC Broker: A Fuzzy Logic Adaptive Reasoning Agent 195 Fig. 4. Fuzzy input variable for model sell-interest 3.2 Tariff Composition Mechanism The tariff composition module is responsible for creating and updating tariffs that are offered to customers. We used a similar approach as in [11]. The fuzzy model is triggered hourly for each energy type in order to calculate the updated tariff prices, as seen in Sect. 3.1. We also considered generic tariffs, such as production,consumption, and storage types. With this approach, the broker can cover more market possibilities offering a wide range of tariffs. 4 Evaluation Experiments on a competition environment bring some challenges regarding evaluation metrics and how to evaluate the performance of the broker. PowerTAC, as an open source distributed Multi-Agent System simulation platform, allows to configure a local server to run the simulation. In this work we have used the 2015 version for both server and client. The best way to evaluate the performance of TugaTAC is facing well consolidated agents, winners of the last competitions. If it presents good results under such scenarios, then it could be considered a competitive broker for real tournaments. Since the binary code for the ultimate PowerTAC finalists are available online, we could run the simulation with this exciting configurations. In fact, besides downloading the real PowerTAC competitors we instantiated another broker and called it ZucaTAC. ZucaTAC shares the same code of TugaTAC but has the fuzzy module disabled, updating the tariff prices with a random interest factor, useful for increasing the number of competitors without introducing other complex strategies as those presented by real competitors. Being a preliminary work, our broker is not yet tuned in order to fairly compete with the big ones, which have very complex reasoning architectures and include many other factors to tariff creation, as seen in Sect. 2.2. However, we wanted to check whether TugaTAC was able to win the competition in three evaluation experiments: 1. Experiment 1 - 1 vs DF: only one broker vs the default 2. Experiment 2 - 2 vs DF: 2 agents plus the default broker 3. Experiment 3 - 3 or more: 3 or more agents competing. [email protected] 196 T.R.P.M. R´ubio et al. The experiments consisted on running one complete simulated tournament and evaluate the results. The winner of the competition is the broker agent with the highest total profit. Our validation metrics are: the energy traded both in wholesale market and retail market, and the total profit at the end of the game. We analyse each one of the experiments and their results regarding these metrics and the dynamics through the simulations. 4.1 Experiment 1 - TugaTAC Against the Default Broker Firstly, we ran the simplest test: competing against the default broker. As explained on Sect. 2, the default broker guarantees that customers have at least one tariff option. In this case, if TugaTAC wins against the default broker, it means that our strategy at least makes sense. If something is wrong, e.g., if prices are not competitive, the results would show a big deficit with the bank. Figure 5presents the profit evolution during the simulation. TugaTAC won the competition with more than 2 million euros in cash, a significant difference against default broker’s profits. It seems that initially, the default broker had some advantage in the tariff publication period, being overcame in less than 10 h. Although it seemed to gather customers attention in the moment our tariffs were being adjusted. TugaTAC refinements in future versions will try to reduce this time. Fig. 5. Experiment 1 - Profit results [email protected]