Towards Revolutionized Smart Grids: An AI-Driven Broker for Improved Operational Efficiency

1 Introduction

A smart grid system refers to an advanced electricity delivery infrastructure to optimize electricity generation, distribution, and consumption. Each of the markets in the smart grid faces certain challenges, and our goal is to address these prominent challenges from the electricity broker's point of view.

In the wholesale market, the broker must devise a strategic approach for minimizing electricity procurement costs in day-ahead PDAs that enable the sale of electricity 24 hours prior to the delivery timeslot, necessitating planning for current and future auctions. Subsequently, electricity procured must be sold to customers in the tariff market via published tariff contracts. Tariffs should be such that they attract customers yet also be profitable to the broker. To remain competitive, tariff generation strategies need to be adaptive to periodically update tariffs, as opponents may adjust their tariffs accordingly. Additionally, the peak demands problem is one of the most prevalent problems in smart grids, where a broker gets heavily penalized if found guilty of contributing to peak demands, which is proportional to the broker's market share. In addressing peak demand issues, the broker aims to avoid heavy penalties through appropriate market share targeting, with alternative strategies involving demand response (DR)

∗Advisor: Dr. Sujit Gujar, IIIT Hyderabad

techniques to incentivize peak demand reduction among cus-

tomers. Below, we list down the specific research questions: RQ1: What should be the optimal PDA bidding strategy to

maximize the profits of the broker?

RQ2: What should be the optimal market share to effectively manage the trade-off between revenue and penalties?

RQ3: How tariff contract generation strategies should be designed to achieve and maintain an optimal market share?

RQ4: How to effectively use the DR technique to incentivize customers to reduce peak-demand scenarios?

Below, we address all the RQs stated above. A combination of our proposed strategies was deployed in our broker VidyutVanika [Chandlekar et al., 2023c] for PowerTAC 2022 and 2023 tournaments, where VidyutVanika emerged as champion in both tournaments with record-breaking margins.

2 Our Contributions

2.1 Bidding Strategies in theWholesale Market

Herein, we focus on addressing the RQ1. Specifically, we propose the following two bidding strategies for PDAs.

Strategy1: This work [Chandlekar et al., 2022b] leverages insights from game theory to conduct a Bayesian Nash Equilibrium analysis for single-buyer single-seller multi-unit double auctions, employing scale-based bidding strategies where the true valuation of a player is multiplied by a scaler to determine the player's bid/ask. Scale-based strategies are preferred due to their simplicity, particularly in real-time bidding scenarios like smart grids. As the complexity of auctions increases, equilibrium analysis becomes intractable. Thus, we model determining equilibrium bidding strategy as a reinforcement learning (RL) problem; specifically, we employ deep deterministic policy gradient (DDPG) for this strategy. First, we perform validation experiments to showcase that the generated strategy follows the theoretical equilibrium. Then, we extend the strategy for complex real-world auction scenarios, where the effectiveness of the strategy is demonstrated through experimentation in PowerTAC [Ketter et al., 2013] wholesale market, where our proposed strategy achieved the lowest procurement costs in various scenarios, showcasing its superiority over existing wholesale strategies.

Strategy2: This work [Chandlekar et al., 2022a] introduces a novel approach leveraging supplier knowledge in wholesale markets, particularly focusing on modelling the cost curve of a prominent power-generating company. The bidding strategy utilizes this knowledge of the supply curve to minimize procurement costs by identifying the lowest ask in auctions with the help of a demand forecaster that predicts the customers' demand. Then, the bid generator module participates in PDA across 24 future time slots, guided by the supply curve follower strategy to set appropriate bid limits. Results from the PowerTAC 2022 finals analysis demonstrate the effectiveness of the proposed strategy in achieving the lowest mean procurement price across various player configurations and weather locations.

2.2 Optimal Market Share Analysis

To address RQ2, we focus on optimal market share determination in the tariff market [Chandlekar et al., 2023d]. Our analysis reveals that obtaining a 100%market share is not optimal due to potential peak demand penalties. We utilise game theoretical analysis to determine an optimal market share for our broker by modelling the interactions in the markets as a zero-sum game against the opponents. Then, we design tariffgeneration strategies that achieve and maintain this optimal market share while aiming to create a balanced customer portfolio to mitigate peak-demand penalties (Sec. 2.3). Modelling the interaction between brokers as a zero-sum game to maximize the revenue difference between our broker and opponents helped us to maximize profits and achieve recordbreaking performance in the PowerTAC tournaments.

2.3 Adaptive Tariff Generation Strategies

In this section, our attention is directed towards RQ3 in which we propose the following tariff contract generation strategies. Strategy1: The proposed strategy [Chandlekar et al., 2022a] is in charge of maintaining market share within predefined bounds by improving active tariffs by considering all active tariffs present in the market. These predefined bounds are determined using the above-mentioned game theoretical analysis. We follow a heuristic-based strategy that compares our broker's current market share with the suggested optimal market share. Then, based on the market scenario and rigorous heuristics, the tariff contract will be updated. In simplified terms, if the current market share is below the optimal market share, then the proposed strategy would aim to publish a slightly cheaper tariff by looking at its wholesale cost and opponents' tariffs in order to increase the market share and vice versa. The proposed strategy was a major reason for our unparalleled performance in the PowerTAC 2022 tournament.

Strategy2: This strategy builds on a previous heuristicbased approach, aiming to replace heuristics with a learningbased method to achieve similar performance as the heuristic method [Chandlekar et al., 2023d]. This strategy, too, uses the game theoretical analysis to determine an optimal market share, then leverages Multi-armed Bandit (MAB) literature to adapt, achieve and maintain optimal market share. Particularly, the Exponential-weight algorithm for Exploration and Exploitation (EXP-3) for MAB-based learning helps us to train our proposed strategy to achieve and maintain the suggested level of market share by adapting to the market situation and revising the tariff contracts periodically. We validate the strategy in the PowerTAC 2023 tournament and demonstrate its ability to maintain the suggested market share effectively, thus generating higher revenue.

2.4 DR Technique for Peak Reduction

In addressing RQ4, we investigate the impact of DR incentives on customer behaviour regarding peak-time usage shifts [Chandlekar et al., 2023b; Chandlekar et al., 2023a]. We first model a function depicting reduction probability (RP) based on discounts, where each customer's behaviour is characterized by a reduction rate (RR). Then, an optimal algorithm is proposed for discount allocation when RRs are known, while a MAB-based online algorithm is suggested for unknown RRs scenarios, demonstrating sublinear regret in experimentation. The method is extended to realistic smart grid settings (PowerTAC) involving numerous customers and is validated by reducing peak demand penalties by 10%.

3 Ongoing and FutureWork

In our current work, we aim to develop sophisticated peak reduction methods through DR, incorporating additional budget constraints and providing theoretical guarantees on algorithmic regret. Additionally, we intend to devise a bidding strategy for a broker in the wholesale market employing Multiagent RL, enhancing the modelling of the dynamic market environment that includes strategic opponent brokers.