Optimizing Prosumer Policies in Periodic Double Auctions Inspired by Equilibrium Analysis

1 Introduction

Auctions play a crucial role in the real world, serving as dynamic marketplaces wherein the value of a commodity gets determined by supply and demand in real-time [Wikipedia contributors, 2023]. Double auctions are the most prominent type of auction, with a trade volume of more than a trillion dollars daily in stock exchanges [Parsons et al., 2011] and energy markets [Ketter et al., 2020]. Such auctions involve bids from the buyers and asks from the sellers; these bids and asks are submitted as a tuple comprising the desired unit price for the commodity and the intended number of units for procurement (or sale). Subsequently, the auction mechanism determines each participant's clearing price and quantity.

A periodic double auction (PDA) is a setup wherein buyers and sellers engage in a (finite) sequence of auctions to trade certain units of a commodity. For example, a buyer with the intention of procuring certain units of a commodity will engage in multiple (but finite) trades with the seller to satisfy her desired procurement target. These types of auctions are prevalent in energy markets wherein a power generating company (GenCo) that sells energy in bulk is the prominent seller and retail brokers are the buyers. In addition, PDAs are also used to model the call auctions in financial markets [Constantinides and Cartlidge, 2021]. The PDA setup allows buyers and sellers to trade energy periodically until a few hours before delivery. As players need to participate in a series of auctions, devising a bidding strategy that caters to current as well as future auctions is a challenging problem. Considering that energy trades in a smart grid setup enable trades worth 1000 𝑇𝑊ℎ of energy, resulting in daily transactions of more than 3 billion dollars just in Europe [Disbrey and Zeier, 2020], it is prudent to design efficient bidding strategies on behalf of a market participant to bring in cost optimization and ecosystem efficiency. To this end, we model PDAs as a finite horizon Markov game and propose equilibrium solutions to aid in devising efficient bidding strategies.

Developing efficient bidding strategies depends mainly on clearing and payment rules of the auction mechanism. Although equilibrium solutions for double auctions have been studied under various payment rules [Wilson, 1992] including some existence results for average clearing price rule [Satterthwaite and Williams, 1989], multi-buyer [Ghosh et al., 2020] and multi-item settings [Chandlekar et al., 2022b], all these works involve simple double auctions and hence may not be applicable for PDAs as devising bidding strategies in PDA involves sequential decision-making. Some recent works do model PDAs as a Markov game and use techniques from reinforcement learning such as multi-agent Q learning [Rashedi et al., 2016], multi-agent deep Q network [Ghasemi et al., 2020], deep deterministic policy gradients [Du et al., 2021; Chandlekar et al., 2022b]. Nevertheless, many of these works involve single-sided auctions 1 and involve numerical simulations rather than analytical solutions. A recent work [Manvi and Subramanian, 2023] did propose an equilibrium solution by modelling the PDA as a complete information Markov game. However, the work is limited to cases where the clearing mechanism is the average clearing price rule (ACPR) and supply by wholesale suppliers is adequate. Furthermore, the players involved in the auction were only buyers, not the prosumers. Moreover, the said work does not provide any policy for a realistic auction setting, which involves incomplete information.

1 Auctions wherein only buy or sell bids are placed

Our work overcomes the abovementioned limitations by providing equilibrium strategies for prosumers (not just buyers) for any general uniform clearing rule satisfying certain properties. We then propose a novel bidding strategy MPNEBBS inspired by this analytical equilibrium solution that approximately reconstructs the supply curve from past auction information to place bids in the market. Our distinct contributions are as follows,

1. We consider a Markov game framework wherein the retail market brokers (net buyers or prosumers) can also participate in small-scale selling, enabling retailers with solar panels and EVs to sell energy along with buying.
2. We define the properties of the uniform clearing rule of the double auction, which encompasses various clearing mechanisms prevailing in diverse auction setups across different geographies.
3. We propose novel Markov perfect Nash equilibrium (MPNE) solutions when prosumers compete to procure the required commodities. The proposed solutions are valid for any uniform clearing mechanism of the PDA with the above mentioned properties.
4. Based on the solution concept derived for the complete information setting, we propose a novel bidding strategy, MPNE-BBS, to work in the incomplete information setup where the supply curve and information regarding the demand requirement of other buyers are not known.
5. We then demonstrate the efficacy of the MPNE-BBS algorithm against several baseline and state-of-the-art bidding strategies deployed in the close-to-real-world energy market simulator PowerTAC.

2 RelatedWork

Some of the early work in the double auction setting was devoted to obtaining equilibrium solutions for single buyer and single seller with a uniform distribution of valuations [Chatterjee and Samuelson, 1983]. [Satterthwaite and Williams, 1989] attempt to find non-trivial equilibria and show the existence of a multiplicity of equilibria for the 𝑘-double auction for a generic class of market participants' valuations. They propose the equilibrium strategies in the form of differential equations and then examine the efficiency of the proven equilibrium. A work [Krausz and Rieder, 1997] considers analytical solutions for a two-player zero-sumMarkov game of incomplete information; whereas this work considers the multiplayer general-sum game. [Vetsikas, 2014] proposes equilibrium strategies for multi-unit sealed bid auction for 𝑚 𝑡ℎ and (𝑚+1) 𝑡 ℎ price sealed bid auction, which differ from 𝑘-double auctions in clearing rules.

PowerTAC [Ketter et al., 2020] is a widely adopted platform to validate bidding strategies in PDAs and has a vast literature on bidding strategies. Various works have proposed Markov Decision Process (MDP) based strategies; for instance, [Urieli and Stone, 2014]'s MDP-based strategy was inspired by Tesauro and Bredin's bidding strategy [Tesauro and Bredin, 2002], which they solve using dynamic programming. Urieli and Stone's strategy was improved upon by [Ghosh et al., 2020], where the authors also provide equilibrium analysis for single-item single-shot double auctions. [Chandlekar et al., 2022b] present an analytical equilibrium solution for single-shot multi-unit auctions to design a DDPG-based bidding strategy. [Kuate et al., 2013] also proposed an MDP-based bidding strategy to determine the bid quantity and use of Non-Homogeneous Hidden Markov Models (NHHMM) to determine bid prices. Additionally, some works have also adopted a Monte Carlo Tree Search (MCTS) framework to device bidding strategies for PDAs [Chowdhury et al., 2018; Orfanoudakis et al., 2021].

At best, all the strategies mentioned above involve equilibrium analysis of single-shot double auctions. Hence, the analysis may not be readily extended to the PDA setting which is a multi-shot auction and the current work bridges this gap.

3 Market ClearingMechanism

We begin by describing the market clearing mechanism of a double auction. Consider a group of 𝑁 prosumers who want to procure multiple units of a commodity from a group of sellers by participating in a PDA having 𝐻 rounds. At any round ℎ ∈ [𝐻] 2 of the PDA, a prosumer 𝑏 ∈ [𝑁] has 𝑄 𝑏,ℎ + and 𝑄 𝑏,ℎ - units of a commodity to buy and sell, respectively. We let 𝑄 𝑏,ℎ + ,𝑄 𝑏,ℎ - to be unique ∀𝑏 ∈ [𝑁] with 𝑄 𝑏,ℎ + > 𝑄 𝑏,ℎ - . The set of buy bids is denoted as B 𝑏,ℎ + with 𝐵 𝑏,ℎ + elements and the sell bids 3 are denoted by B 𝑏,ℎ - with 𝐵 𝑏,ℎ - elements. The buy bids are the pair of price and quantity denoted as (𝑝 𝑏,ℎ 𝑖,+ , 𝑞 𝑏,ℎ 𝑖,+ ), 𝑖 ∈ [𝐵 𝑏,ℎ + ], where price and quantity are bounded by 𝑝max and 𝑄 𝑏,ℎ + respectively. Similarly, the sell bids are denoted as (𝑝 𝑏,ℎ 𝑛,-, 𝑞 𝑏,ℎ 𝑛,-), 𝑛 ∈ [𝐵 𝑏,ℎ - ], with 𝑝max and 𝑄 𝑏,ℎ - as price and quantity upper bounds. As a result, the total outstanding demand in round ℎ ∈ [𝐻] from all 𝑁 prosumers is denoted as 𝑄 𝒟,ℎ = � 𝑏∈[𝑁 ] 𝑄 𝑏,ℎ + .

The wholesale sellers, on the other hand, are represented by a consolidated supply curve with 𝐿 ℎ asks expressed as L ℎ = {(𝑝 ℎ 𝑖 , 𝑞 ℎ 𝑖 ) | 𝑖 ∈ [𝐿 ℎ ]}, with 𝑝 ℎ 𝑖 ∈ [0, 𝑝max] and 𝑞 ℎ 𝑖 ∈ [0, 𝑞max] as the price and quantity components of the 𝑖 th ask (𝑝 ℎ 𝑖 , 𝑞 ℎ 𝑖 ), with 𝑝max and 𝑞max as suitable upper bounds. Accordingly, the total supply provided by wholesale sellers, at round ℎ, is denoted as 𝑄 𝒮,ℎ = � 𝑚∈[𝐿 ℎ ] 𝑞 ℎ 𝑚. Now, the overall supply at round ℎ of the PDA is given by ˆ𝑄 𝒮,ℎ = 𝑄 𝒮,ℎ +𝑄 𝒮,ℎ - , where 𝑄 𝒮,ℎ - = � 𝑏∈[𝑁] 𝑄 𝑏,ℎ - is the supply from the prosumers. Finally, at the round ℎ, the combined asks and sell bids of the wholesale suppliers and the brokers is expressed as ˆL ℎ = {( ˆ 𝑝 ℎ 𝑖 , ˆ 𝑞 ℎ 𝑖 ) | 𝑖 ∈ [𝐿 ℎ + 𝐵 ℎ - ] }.

The market regulator at each round ℎ, collects the elements of ˆL ℎ and B ℎ + and uses a clearing rule to produce a clearing price(s) and cleared quantities. In this work, we focus on the uniform clearing rule, where all the cleared bids and asks have the same clearing price. The cleared price for round ℎ is called the market clearing price (denoted as 𝜆 ℎ ) and the total quantity cleared for all the 𝐵 ℎ + bids at round ℎ is denoted as 𝑄 ℎ . To further elaborate the clearing process, we assume, without loss of generality, that the elements of the set ˆL ℎ (B ℎ + ) are sorted in increasing (decreasing) order of the price component. If the price components of the elements are equal, then ˆL ℎ and B ℎ + are sorted in decreasing order of the quantity component. If both price and quantity components of certain elements are equal, then the ordering between them is arbitrary. More concretely, we consider the uniform clearing rules, which have properties defined in Definition 1.

2 For any integer 𝐾, we denote [𝐾] as the set {1, . . . , 𝐾}.

3 Sell bids are prosumer's asks.

Definition 1. Given

• ( ˆ 𝑝 ℎ 𝑑 , ˆ 𝑞 ℎ 𝑑 ) ∈ ˆL ℎ as the last cleared ask and (𝑝 𝑏,ℎ 𝑙,+ , 𝑞 𝑏,ℎ 𝑙,+ ) ∈ B ℎ + as the last cleared buy bid at round ℎ.
• 𝛼 𝑏,ℎ 𝑖 with 𝑖 ∈ [𝐵 𝑏,ℎ + ] and 𝛽 𝑏,ℎ 𝑘 with 𝑘 ∈ [𝐵 𝑏,ℎ - ] as the cleared buy and sell bid quantities of a prosumer 𝑏 ∈ [𝑁] at round ℎ.

The properties of the considered uniform clearing rules are defined as follows.

• The uniform clearing price 𝜆 ℎ at round ℎ is a scalar value that lies in the interval [ ˆ 𝑝 ℎ 𝑑 , 𝑝 𝑏,ℎ 𝑙,+ ].
• The total market cleared quantity at round ℎ is,

• Buy bids that are higher than the last cleared buy bid are fully cleared. That is , 𝛼 𝑏,ℎ 𝑚 = 𝑞 𝑏,ℎ 𝑚,+, if 𝑝 𝑏,ℎ 𝑚,+ > 𝑝 𝑏,ℎ 𝑙,+ .
• Buy bids that are lower than the last cleared buy bid are not cleared. That is, 𝛼 𝑏,ℎ 𝑚 = 0, if 𝑝 𝑏,ℎ 𝑚,+ < 𝑝 𝑏,ℎ 𝑙,+ .
• Buy bids that are equal to the last cleared buy bid are cleared as 𝛼 𝑏,ℎ 𝑚 = 1 |𝐵𝑏=𝑙 | � 𝑄 ℎ - �|𝐵𝑏>𝑙 | 𝑗=1 𝑞 𝑏,ℎ 𝑗 ,+ � , where |𝐵𝑏=𝑙 | and |𝐵𝑏>𝑙 | denote the number of bids that are equal and higher than the last cleared bid, respectively.
• Conversely, the asks and sell bids from ˆL ℎ are cleared such that the cheaper asks are given the higher priority.

In practice, regulators deploy a variety of clearing rules that satisfy the properties defined in Definition 1. Popular examples include merit order dispatch [Taylor, 2015] and 𝑘-double auctions [Angaphiwatchawal et al., 2021]. The merit order dispatch is a mechanism where the market operator clears the market by maximizing the area between supply (asks) and demand curves (bids). Specifically, market clearing is posed as an optimization problem, and the (primal and dual) solutions to this optimization give the cleared quantities and clearing price. The clearing price for the 𝑘-double auction (where 𝑘 ∈ [0 ; 1]) is given by 𝜆 ℎ = 𝑘 · ˆ 𝑝 ℎ 𝑑 + (𝑘 - 1) · 𝑝 𝑏,ℎ 𝑙,+ . Furthermore, the average clearing price mechanism (ACPR) is a special case of 𝑘-Double auction with 𝑘 = 0.5, and its clearing price is given as the average of the last cleared ask and bid. For further details about the clearing mechanism please refer to [Manvi et al., 2024].

4 The Markov Game Structure

Consider a Markov game [Zhang et al., 2023] with a finite horizon to model the PDA with 𝑁 prosumers. Specifically, let G = ⟨𝑁, 𝑆, 𝐴, 𝐶, 𝑃, 𝐻⟩ denote the Markov game with 𝑁 as the number of players (prosumers), 𝑆 as the state space, 𝐴 as the joint action space, 𝐶 = {𝐶 𝑏,ℎ | 𝑏 ∈ [𝑁], ℎ ∈ [𝐻]} as the cost functions for the players, with 𝐶 𝑏,ℎ denoting the cost function for the player 𝑏 at round ℎ, 𝑃 : 𝑆 × 𝐴 → 𝑆 as the transition probability and 𝐻 as the length of the horizon.

More concretely, the state space 𝑆 is defined as the set of wholesale suppliers' asks L ℎ , brokers' demand Q ℎ + and brokers' supply Q ℎ - and thus 𝑆 is {L ℎ , Q ℎ + , Q ℎ -}. Here, Q ℎ + and Q ℎ - are sets of brokers' demand 𝑄 𝑏,ℎ + and supply 𝑄 𝑏,ℎ - for all 𝑏 ∈ [𝑁]. Note that since we are working with the complete information setting, the wholesale suppliers' asks, brokers' demand and brokers' supply are known to all the brokers; hence, the transition of the states is deterministic. The joint action space 𝐴 = ×𝑏∈[𝑁] 𝐴 𝑏 is a product space of all the prosumer's action space 𝐴 𝑏 , where 𝐴 𝑏 = ∪ℎ∈[𝐻] (B 𝑏,ℎ + × B 𝑏,ℎ - ) is the product space of the all the buy bids B 𝑏,ℎ + and sell bids B 𝑏,ℎ - over ℎ ∈ [𝐻]. The cost function 𝐶 𝑏,ℎ : 𝑆 × 𝐴 𝑏 →R for a player 𝑏 is given by

where 𝑎 ℎ = (𝑎 𝑏,ℎ , 𝑎 -𝑏,ℎ ) ∈ 𝐴 denote the joint action, 𝑎 -𝑏,ℎ denote the actions taken by players other than 𝑏, 𝛼 𝑏,ℎ 𝑛 are cleared quantities for player 𝑏's buy bids and 𝛽 𝑏,ℎ 𝑚 are the cleared sell bids. The constant Υ ≥ 0 is the balancing price required to buy the quantity 𝑄 𝑏,𝐻+1 + outside of the PDA at time 𝐻 + 1.

A state 𝑠 ℎ at round ℎ, when a joint action 𝑎 ℎ ∈ 𝐴 is taken, transitions to a state 𝑠 ℎ+1 = {L ℎ+1 , Q ℎ+1 + , Q ℎ+1 - } at round ℎ + 1. Here, L ℎ+1 represents the uncleared asks at round ℎ + 1 and {Q ℎ+1 + , Q ℎ+1 - } are the updated demand and supply respectively of the prosumers after accounting for the cleared quantities at round ℎ. In addition, the sequence of transitions 𝑠 1 , 𝑎 1 , 𝑠 2 , 𝑎 2 , . . . , 𝑠 𝐻 , 𝑎 𝐻 , 𝑠 𝐻+1 , which starts at 𝑠 1 and ends at 𝑠 𝐻+1 , is denoted as a trajectory 𝜏 of the Markov game. Here, the trajectory 𝜏 induces a sequence of costs 𝐶 𝑏,1 , . . . , 𝐶 𝑏,𝐻 , 𝐶 𝑏,𝐻+1 for each player 𝑏 ∈ [𝑁]. A player 𝑏 will choose an action 𝑎 𝑏,ℎ ∈ 𝐴 𝑏 at each of the state 𝑠 ℎ she visits and the collection of actions taken at each round ℎ is denoted by a Markov policy 𝜋 𝑏 = {𝜋 𝑏,ℎ : 𝑆 → 𝐴 𝑏 | ℎ ∈ [𝐻]}. We further let 𝜋 = (𝜋 𝑏 , 𝜋 -𝑏 ) as the joint policy that includes player 𝑏's policy 𝜋 𝑏 along with the policies of players except 𝑏, denoted as 𝜋 -𝑏 . Moreover, we let Π 𝑏 denote the policy space of player 𝑏 and Π = ×𝑏∈[𝑁]Π 𝑏 denote the joint policy space. Importantly, to capture the cost of acquisition, we define a value function 𝑉 ℎ 𝜋 : 𝑆 → R of a joint policy 𝜋 ∈ Π at round ℎ in Equation (1).

More specifically, the value function is given by

�

�

For the game G, we define MPNE [Yang and Wang, 2020] using the value function in Definition 2.

Definition 2. Given the game G of 𝐻 horizon with 𝑁 players, a joint policy 𝜋∗ = (𝜋 𝑏 ∗ , 𝜋 -𝑏 ∗ ) is an MPNE if ∀𝑏 ∈ [𝑁], ∀𝑠 ∈ 𝑆, ∀ℎ ∈ [𝐻] and ∀𝜋 𝑏 : 𝑆 → 𝐴 𝑏 , we have

Having described the Markov game framework, we now proceed to develop MPNE solutions in the next section.

5 Equilibria of the Markov Game

We begin by focusing on the case where the prosumers 𝑏 ∈ [𝑁] are restricted to place at most one buy bid and one sell bid. In addition, first, we consider that the supply from the bulk sellers is enough to satisfy the demand, that is 𝑄 𝒮,ℎ ≥ 𝑄 𝒟,ℎ . Later, we consider the inadequate supply case, where the equilibrium solution differs from the adequate supply case. Note that it is essential to consider the inadequate supply case since GenCos may not always be able to produce the supply needed to satisfy the expected demand. Furthermore, in [Manvi et al., 2024], an incremental case of prosumers placing multiple buy and sell bids is considered.

5.1 Adequate Supply Case

Let us define a few entities that will help to illustrate the proposed MPNE policy. First, denote 𝑄 𝒟-𝑏,ℎ = 𝑄 𝒟,ℎ - 𝑄 𝑏,ℎ as the demand of players excluding the player 𝑏 ∈ [𝑁] at round ℎ. Second, let 𝑢ℎ be the lowest index of an ask in the sorted set L ℎ such that the supply up to the first 𝑢ℎ asks is greater than the quantity 𝑄 𝒟,ℎ - 𝑄 𝒮,ℎ - . That is, 𝑢ℎ = argmin𝑖 (𝑄 𝒟,ℎ -𝑄 𝒮,ℎ - < � 𝑖 𝑚=1 𝑞 ℎ 𝑚). Similarly, let 𝑣 𝑏 ℎ be an index of the sorted set L ℎ such that the supply up to first 𝑣 𝑏 ℎ asks is greater than 𝑄 -𝑏,ℎ -𝑄 𝒮,ℎ - . That is,

Finally, define an index of L ℎ as 𝑧ℎ = max{𝑣 1 ℎ , 𝑣 0 ℎ } and let 𝜙 ℎ be the player who bids at a price 𝑝𝑧ℎ , where 𝑣 0 ℎ is 𝑢ℎ-(𝐻-ℎ) and 𝜙 ℎ is defined as,

MPNE Policy for Adequate Supply Case: The joint policy 𝜋∗ suggests that the player 𝑏 needs to check 𝑄 𝑏,ℎ > 𝑞𝑢ℎ , that is, whether her selling quantity is greater than the ask quantity at the index 𝑢ℎ (of L ℎ ). If yes, then the player places sell bid 4 at price 𝑝 𝑏,ℎ -,∗ = 𝑝𝑢ℎ with quantity 𝑞 𝑏,ℎ -,∗ = 𝑄 𝑏,ℎ - . However, if 𝑄 𝑏,ℎ ≤ 𝑞𝑢ℎ the prosumer places sell bid at price 𝑝 𝑏,ℎ - = 𝑝𝑢ℎ - 𝜖, where 𝜖 > 0 is a small constant. Furthermore, the player has to check the condition 𝑏 = 𝜙 ℎ and if the constraint is satisfied, then the player bids at buy bid price 𝑝 𝑏,ℎ +,∗ = 𝑝𝑧ℎ with quantity 𝑞 𝑏,ℎ +,∗ = 𝑄 𝑏,ℎ + . However, when 𝑏 ≠ 𝜙 ℎ , the player places a buy bid price 𝑝 𝑏,ℎ +,∗ = 𝑝max and quantity bid equal to her requirement. The MPNE policy 𝜋 𝑏,ℎ ∗ with buy and sell bids (𝑝 𝑏,ℎ +,∗ , 𝑞 𝑏,ℎ +,∗ , 𝑝 𝑏,ℎ -,∗, 𝑞 𝑏,ℎ -,∗) is given in Equation (5). Note that the policy 𝜋 𝑏,ℎ ∗ considers both sell bids and buy bids of the prosumer, unlike [Manvi and Subramanian, 2023], where only buy bids are considered.

4 Bid indices omitted since there is only one sell and one buy bid.

Having explained the candidate policy 𝜋∗, the value function of the joint policy 𝜋 𝑏,ℎ ∗ at round ℎ for player 𝑏 ∈ [𝑁] at state 𝑠 ℎ ∈ 𝑆 is given in Equation (6) using the properties of the uniform clearing mechanism defined in Definition 1.

EquilibriumAnalysis: We demonstrate that the candidate policy in Equation (5) is indeed MPNE policy in the space of all deterministic policies. More specifically, we show that, for all players 𝑏 ∈ [𝑁], for all state 𝑠 ℎ ∈ 𝑆, for all round ℎ ∈ [𝐻] and for any deterministic policy 𝜋 𝑏 , Equation (2) is satisfied. To this end, we consider the value function for all possible deviations where the sell bid deviations are tabulated in Table 1. Furthermore, the buy bid deviations are exactly the same as in Table 1 (except for the notation from 𝑝 𝑏,ℎ - to 𝑝 𝑏,ℎ + ). Note that, by assumption, prosumers cannot buy more than they require and sell more than they have; hence, only five sell and buy bid deviations are possible. Finally, the combined deviations 𝜋 𝑏 ∈ Π 𝑏 is the cartesian product of the sell bid deviations and the buy bid deviations 5 . In the sequel, we provide a preliminary result in the Lemma 1. The first part of Lemma 1 provides a property of the clearing price 𝜆 ℎ for a uniform clearing rule, which satisfies Definition 1. The second part of Lemma 1 provides a condition on the cost of balancing outside the horizon of the PDA.

Lemma 1. Given that the supply curve from bulk sellers across rounds ℎ ∈ [𝐻] is constant and the clearing mechanism satisfies the properties in Definition 1, we have the following.

1. The clearing price at ℎ satisfy 𝜆 ℎ ≥ 𝜆 ℎ+1 for ℎ ∈ [𝐻-1].
2. The condition on the balancing price to buy outside the auction at 𝐻 + 1 denoted by Υ is given by Υ > 𝛾 · 𝑝max, where 𝛾 > 1.

5 In total, 25(5 × 5) deviations are possible.

We now move to the Theorem 1, which shows that the value function of the deviated policies is greater than the candidate policy's value function.

Theorem 1. Given the conditions in Lemma 1 hold, we have the following.

1. The sell bid deviations 𝜋 𝑏 ∈ Π 𝑏 of player 𝑏 with any buy bids 𝑝 𝑏,ℎ + ∈ [0; 𝑝max] and 𝑞 𝑏,ℎ + ∈ [0;𝑄 𝑏,ℎ + ] satisfy (2).
2. The buy bid deviations 𝜋 𝑏 ∈ Π 𝑏 of player 𝑏 with any sell bids 𝑝 𝑏,ℎ - ∈ [0; 𝑝max] and 𝑞 𝑏,ℎ - ∈ [0;𝑄 𝑏,ℎ - ] satisfy (2).

Proof. Please refer to [Manvi et al., 2024].

□

5.2 Inadequate Supply Case

We now consider the inadequate supply, (that is, 𝑄 𝒮,ℎ < 𝑄 𝒟,ℎ ). To provide the MPNE for this case, we modify the definitions of 𝑢ℎ, 𝜙 ℎ , 𝑣 𝑘 ℎ and 𝑧ℎ provided in the adequate supply case. First, we assign 𝑢ℎ = 𝐿 ℎ as the value. Second, 𝜙 ℎ is given as

Third, when 𝜙 ℎ = 𝑁, we set index 𝑣 𝑁 ℎ as

Finally, 𝑧ℎ when 𝜙 ℎ = 𝑁 is defined as 𝑧ℎ = max{𝑣 𝜙 ℎ , 𝑣 0 ℎ }. However, if 𝜙 ℎ < 𝑁, then 𝑝𝑧ℎ = 𝑝max.

MPNE Policy for Inadequate Supply Case: The buy bids mentioned in Equation (5) now use modified definitions, whereas the sell bids change to bid at a price 𝑝 𝑏,ℎ - = Υ-𝜖 and bid quantity 𝑞 𝑏,ℎ - = 𝑄 𝑏,ℎ - . In [Manvi et al., 2024], we provide a Lemma that shows the policy is indeed MPNE.

6 Experimental Evaluation

In the previous sections, we presented analytical equilibrium solutions for PDAs using a Markov game framework. In doing so, we had a complete information setup wherein the players (prosumers) have knowledge of the supply curve and the demand information of other players at each round of the PDA. In this section, we use the equilibrium solutions (5) obtained in the previous section to propose a bidding strategy MPNE-BBS for the incomplete information case where a buyer has neither the knowledge of the demand requirement of other players nor complete information about the supply curve. We start by providing the algorithm for MPNE-BBS, followed by numerical experiments to showcase the efficacy of the proposed strategy. The numerical experiments are conducted on the wholesale market module of the Power Trading Agent Competition (PowerTAC) [Ketter et al., 2020].

PowerTAC: PowerTAC is an efficient and close-to real-world smart grid simulator that models all the crucial elements of a typical smart grid system, including GenCo and energy brokers acting as prosumers. During the simulation (or game), which typically lasts for 60 simulation days, an energy broker has to compete against several other brokers. To test our bidding strategy, we focus on PowerTAC's wholesale market PDAs, where the energy broker plays a crucial role in buying/selling energy. These PDAs are day-ahead auctions in which the broker can purchase energy 24 hours ahead of the delivery timeslot by participating in a total of 24 auctions at an interval of every hour. PowerTAC PDA employs ACPR and uniform pricing rules for clearing and payments. To bid in PowerTAC PDA, a broker must submit the bid price and quantity (decided based on demand forecasts). For determining the bid price, a broker may use the information available from the server, which includes the market-clearing price, its own cleared quantity, net cleared quantity and orderbook information. Orderbook includes an anonymized list of uncleared asks and bids, while the knowledge about the other brokers' cleared bids/asks is kept hidden. Failing to acquire the required quantity from the wholesale market, a broker has to purchase the remaining quantity at balancing market prices, which are typically high and serve as a penalty for a broker for causing imbalance. The simulation also includes a buyer called MISO that procures energy for a region that contains retailer users not serviced by the main brokers of the PowerTAC setup. The MISO buyer's energy requirement is almost ten times the PowerTAC retail market's energy requirements and substantially affects the clearing prices. The MISO buyer purchases all of its estimated demand in the first round of the PDA and any excess procurement is sold in the subsequent rounds. The MISO always places a market order to buy or sell energy in any round of the PDA. The GenCos are the primary sellers in the market that follow a quadratic cost function to decide the ask prices. Refer to PowerTAC specifications [Ketter et al., 2020] for more information.

MPNE-BBS Algorithm: We propose an algorithm MPNE-BBS for the incomplete information setting of PowerTAC inspired by the equilibrium analysis on the complete information setting. As our proposed MPNE-BBS bidding algorithm uses some design ideas from VV21, we first describe the design of VV21. It models the cost supply curve of the GenCos from the uncleared ask information data available from the simulator. The idea is to locate the price corresponding to the broker's bid quantity (requires demand forecast of broker's and market's demands) on the supply curve, treat that price as the upper bound on the limit-prices, and place multiple bids below that price. The reason for placing multiple bids below the chosen upper bound is that it aims to procure the majority of the quantity from the asks of the MISO buyer and other prosumers in the market and treats GenCo as the sup-

Algorithm 1 MPNE-BBS

1: totalDmd[] ←netDmdPredict(currentTime) 2: 𝑄 𝑏,ℎ [] ← indvDmdPredictor[currentTime] 3: for hour in [1, . . . , 23] do 4: futureTime ← currentTime + hour 5: unclearedAsks[] ←Auction(currentTime-1,futureTime) 6: if unclearedAsks is not empty then 7: 𝑄 𝒮,ℎ ← sum(unclearedAsks.q) 8: 𝑄 𝒟,ℎ ← totalDmd[futureTime] 9: if 𝑄 𝒮,ℎ ≥ 𝑄 𝒟,ℎ then 10: 𝑝𝑢ℎ ←min 𝑝𝑟 s.t 𝑄 𝒟,ℎ ≤ � 𝑟 𝑖=1 unclearedAsks.𝑞𝑖 11: 𝑄 𝒟-𝑏 ,ℎ ← 𝑄 𝒟,ℎ - 𝑄 𝑏,ℎ [futureTime] 12: 𝑝 𝑣 𝑏 ℎ ←min 𝑝𝑟 s.t 𝑄 𝒟-𝑏,ℎ ≤ � 𝑟 𝑖=1 unclearedAsks.𝑞𝑖 13: else 14: 𝑝𝑢ℎ ← 𝑝 𝐿 ℎ , 𝑝 𝑣 𝑏 ℎ ← 𝑃 𝐿 ℎ 15: end if 16: 𝑣 0 ℎ ← max{1, 𝑢ℎ - hour + 1} 17: estBidPrice = max{𝑝 𝑣 0 ℎ , 𝑝 𝑣 𝑏 ℎ } 18: else 19: clearedPrices[] ← Auction(𝑡,𝑡+hour) ∀𝑡 < currentTime 20: estBidPrice = max{cleredPrices} 21: end if 22: if (currentTime is far from futureTime) then 23: minP = 𝛼 𝑓 * estBidPrice; maxP = 𝛽 𝑓 * estBidPrice 24: else 25: minP = 𝛼𝑐 * estBidPrice; maxP = 𝛽𝑐 * estBidPrice 26: end if 27: Sample 𝑃 prices, 𝑝 𝑏,ℎ 𝑖 ∼ U[minP,maxP] 28: Distribute 𝑄 𝑏,ℎ uniformly across 𝑃 prices, 𝑞 𝑏,ℎ 𝑖 ←𝑄 𝑏,ℎ /𝑃 29: bidList ← (𝑝 𝑏,ℎ 𝑖 , 𝑞 𝑏,ℎ 𝑖 ) ∀ 𝑖 ∈ {1, 2, ..., 𝑃} 30: Auction(currentTime,futureTime) ←bidList 31: end for

plier in the last resort. The supply curve of the GenCo also has an element of randomization between rounds of the PDA; hence, placing multiple bids in a range-bound manner helps the agent procure energy at a lower price. More details of the VV21 strategy can be found in [Chandlekar et al., 2022a].

On the other hand, MPNE-BBS decides the price on the supply curve by using the forecasts of the total demand and the broker's demand to determine the indices 𝑢ℎ and 𝑣 𝑏 ℎ on the uncleared asks list (number of uncleared asks to satisfy the demand). Using these indices, it estimates the best possible limit-price, which follows the analytical solution presented in Section 5. After deciding the price, it places multiple bids similar to VV21 to work around the supply curve's randomness. Unlike VV21, MPNE-BBS aims to purchase most of the quantity from GenCo.

As shown in the algorithm, MPNE-BBS takes the current bidding timeslot as input, uses the 24th hour (the first opportunity) of each auction to observe the uncleared asks, and places a list of bids for each of the next 23 hours. For each of these future hours (futureTime), the algorithm queries for the list of uncleared asks from past auction data. These asks are a list of price and quantity tuples (𝑝𝑖 , 𝑞𝑖) sorted in increasing order of prices. Then, based on the list of unclear asks, the algorithm estimates the bid price (estBidPrice) by utilising the knowledge of the market and its own demand forecasts. The bid price estimation approach in lines 4 to 21 of algorithm 1 is inspired by the equilibrium solution presented in Equation (5). As the buyer has 24 opportunities to procure the required demand, thus can afford to take risks during the initial rounds and play conservatively in the last few rounds, as shown in lines 23 to 27. The hyperparameters 𝛼 𝑓 , 𝛽 𝑓 , 𝛼𝑐 and 𝛽𝑐 can be decided based on the risk level of the buyer. After sampling 𝑃 prices uniformly, the buyer's required quantity is uniformly divided into 𝑃 bids and submitted for clearing.

Benchmark: Below, we briefly describe the state-of-theart as well as baseline bidding strategies of PowerTAC PDAs that are used in the experiments to compare the performance of MPNE-BBS. We use six bidding strategies to compete against MPNE-BBS, namely, VidyutVanika21 (VV21) [Chandlekar et al., 2022a], Deep-DeterministicPolicy-Gradient-based Bidding Strategy (DDPG) [Chandlekar et al., 2022b], SPOT [Chowdhury et al., 2018], VidyutVanika18 (VV18) [Ghosh et al., 2020], Zero Intelligence (ZI) and Zero Intelligence Plus (ZIP) [Cliff, 1997]. DDPG and VV18 are scale-based bidding strategies implemented using RL algorithms. SPOT is built on top of Monte Carlo Tree Search (MCTS). ZIP is a heuristic strategy that keeps a profit margin while bidding and the ZI is a randomized approach to placing bids by sampling prices from a uniform distribution. The reader is referred to Appendix 6 for more details about these strategies.

Experimental Setup: To test the efficacy of MPNE-BBS against the above-listed bidding strategies, we perform two sets of experiments on the PowerTAC platform, one that includes the MISO buyer and the other without it. In these experiments, we compare the unit purchase costs of the brokers in the wholesale market, a strategy having a lower purchase cost is preferable. In Set-1, we play all-player games that include the above six bidding strategies and MPNE-BBS, along with MISO buyer in the market. We ask all the players except MISO to procure a fixed demand requirement for each timeslot; the demand requirement is the same for all the brokers for a fair comparison. Set-1 is further divided into four configurations depending on the demand requirements of the brokers, namely, low-demand, mid-demand, high-demand and extreme-demand. Similarly, in Set-2, we remove the MISO buyer from the market and repeat the same experiments for all four demand levels. We play 10 games for each configuration in each set (Figures 1 and 2). Additionally, we play twoplayer games between MPNE-BBS and all the other strategies; that is, we play 10 games between MPNE-BBS and VV21, 10 games between MPNE-BBS and ZI, and so on. We perform these experiments for both the sets and all four demand levels as mentioned above (Tables 2 and 3).

Results: As shown in Figure 1, MPNE-BBS is one of the best-performing bidding strategies in terms of wholesale cost as it achieved close to the best or second-best cost among the seven brokers across variable demand levels. Specifically, its wholesale cost is close to the best wholesale cost in the market for extreme demand level (only 6% away from the best) and close to second-best for high and mid demand levels (4.7% and 3.4% difference, respectively), with only VV21 performing better. The performance is more prominent in Set-2; as shown in Figure 2, it achieves a wholesale cost very close to the best wholesale cost in the market across all demand levels and consistently outperforms DDPG and VV21.

The results of the 2-Player experiments in Tables 2 and 3 show the wholesale cost of the opponent strategy relative to MPNE-BBS, a value more than 1 would indicate MPNE-BBS is the superior bidding strategy among the two. Particularly, it matches VV21, which is the best strategy in the literature and achieves similar wholesale cost as VV21 in all demand levels, with and without MISO buyer. It even outperforms VV21 for the extreme demand level where VV21's wholesale cost is 1.18 times higher than MPNE-BBS's cost. Overall, the results show that it achieves superior performance against each opponent and for almost all demand levels. This set of experiments aims to validate the efficacy of MPNE-BBS against several different state-of-the-art strategies having different bidding patterns. Furthermore, we performed the same experiments for 3-Player and 5-Player games. On an average, the relative costs of best and worst opponent strategies were 1.13 and 1.87, respectively in 5-player without MISO games; 1.0 and 2.34, respectively in 5-player with MISO games. Similarly, for 3-player games, these numbers are 1.27 and 1.47, respectively for best and worst opponent strategies without MISO; 0.98 and 1.19, respectively with MISO. Thus, the simulation results show that the MPNE-BBS achieves significant performance improvements against the best state-ofthe-art bidding strategy across various number of and players in a game for various market and demand scenarios.

7 Conclusion

In this paper, we proposed equilibrium strategies for prosumers involved in a PDA to buy and sell commodities. We modelled the PDA as a Markov game with prosumers as players and derived equilibrium solutions for the complete information setting when the players are aware of the supply curve and the outstanding demand requirement of other players at every round of the PDA. Specifically, we derived MPNE solutions for the setting when the players compete to procure required commodities. Thereafter, the proposed MPNE solutions were used to design a bidding algorithm called MPNEBBS for a more practical setup and its efficacy was demonstrated using the PowerTAC simulator test-bed against several state-of-the-art algorithms. In future work, we hope to extend the analysis of equilibrium solutions to the incomplete information setting by modelling the PDA as a partially observable stochastic (Markov) game and compare its effectiveness against the algorithm proposed in this work.