Incentivizing Intelligent Customer Behavior in Smart-Grids: A Risk-Sharing Tariff & Optimal Strategies
Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. Pages 380-385.
Abstract
Current electricity tariffs for retail rarely provide incentives for intelligent demand response of flexible customers. Such customers could otherwise contribute to balancing supply and demand in future smart grids. This paper proposes an innovative risk-sharing tariff to incentivize intelligent customer behavior. A two-step parameterized payment scheme is proposed, consisting of a prepayment based on the expected consumption, and a supplementary payment for any observed deviation from the anticipated consumption. Within a game-theoretical analysis, we capture the strategic conflict of interest between a retailer and a customer in a two-player game, and we present optimal, i.e., best response, strategies for both players in this game. We show analytically that the proposed tariff provides customers of varying flexibility with variable incentives to assume and alleviate a fraction of the balancing risk, contributing in this way to the uncertainty reduction in the envisioned smart-grid.
- [Borenstein et al., 2002] Severin Borenstein, Michael Jaske, and Arthur Rosenfeld. Dynamic pricing, advanced meter- ing, and demand response in electricity markets. 2002.
- [Braithwait et al., 2007] Steven Braithwait, Dan Hansen, and Michael OSheasy. Retail electricity pricing and rate design in evolving markets. Edison Electric Institute, pages 1-57, 2007.
- [Fadlullah et al., 2011] Z.M. Fadlullah, Y. Nozaki, A. Takeuchi, and N. Kato. A survey of game theoretic approaches in smart grid. In Wireless Communications and Signal Processing (WCSP), 2011 International Conference on, pages 1-4, Nov 2011.
- [Fang et al., 2012] Xi Fang, Satyajayant Misra, Guoliang Xue, and Dejun Yang. Smart grid the new and improved power grid: A survey. Communications Surveys Tutorials, IEEE, 14(4):944-980, Fourth 2012.
- [Ferguson, 1967] Thomas S Ferguson. Mathematical statis- tics: A decision theoretic approach, volume 1. Academic press, 1967.
- [Geary, 1935] R. C. Geary. The ratio of the mean deviation to the standard deviation as a test of normality. Biometrika, 27(3/4):310-332, 1935.
- [Herter and Wayland, 2010] Karen Herter and Seth Way- land. Residential response to critical-peak pricing of elec- tricity: California evidence. Energy, 35(4):1561 -1567, 2010. Demand Response Resources: the {US} and In- ternational ExperienceDemand Response Resources: the {US} and International Experience. [Hoogland et al., 2015]
- Jasper Hoogland, Mathijs De Weerdt, and Han La Poutré. Now, later, or both: A closed-form optimal decision for a risk-averse buyer. In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, AAMAS '15, pages 1769-1770, Richland, SC, 2015. International Foundation for Autonomous Agents and Multiagent Systems.
- [McFadden, 1975] Daniel McFadden. The revealed prefer- ences of a government bureaucracy: Theory. The Bell Journal of Economics, 6(2):401-416, 1975.
- [Nair et al., 2014] Jayakrishnan Nair, Sachin Adlakha, and Adam Wierman. Energy procurement strategies in the presence of intermittent sources. SIGMETRICS Perform. Eval. Rev., 42(1):85-97, June 2014.
- [Ortega and Braun, 2011] Daniel Alexander Ortega and Pe- dro Alejandro Braun. Artificial General Intelligence: 4th International Conference, AGI 2011, Mountain View, CA, USA, August 3-6, 2011. Proceedings, chapter Information, Utility and Bounded Rationality, pages 269-274. Springer Berlin Heidelberg, Berlin, Heidelberg, 2011.
- [Oruc et al., 2012] Sertac Oruc, Ashish Pandharipande, and S Cunningham. An electricity market incentive game based on time-of-use tariff. In Fourth Workshop on Game Theory in Energy, Resources and Environment, Montreal, Canada, 2012.
- [Owen and Ward, 2010] Gill Owen and Judith Ward. Smart tariffs and household demand response for great britain. Sustainability First, London, page 2010, 2010.
- [Pettersen et al., 2005] Erling Pettersen, Andrew B. Philpott, and Stein W. Wallace. An electricity market game between consumers, retailers and network operators. Decis. Sup- port Syst., 40(3-4):427-438, October 2005.
- [Pipattanasomporn et al., 2009] M. Pipattanasomporn, H. Feroze, and S. Rahman. Multi-agent systems in a distributed smart grid: Design and implementation. In Power Systems Conference and Exposition, 2009. PSCE '09. IEEE/PES, pages 1-8, March 2009.
- [Rohjans et al., 2010] S. Rohjans, M. Uslar, R. Bleiker, J. Gonzalez, M. Specht, T. Suding, and T. Weidelt. Survey of smart grid standardization studies and recommenda- tions. In Smart Grid Communications (SmartGridComm), 2010 First IEEE International Conference on, pages 583- 588, Oct 2010.
- [Roozbehani et al., 2010] M. Roozbehani, Munther Dahleh, and S. Mitter. Dynamic pricing and stabilization of supply and demand in modern electric power grids. In Smart Grid Communications (SmartGridComm), 2010 First IEEE In- ternational Conference on, pages 543-548, Oct 2010.
- [Roozbehani et al., 2012] M. Roozbehani, M.A. Dahleh, and S.K. Mitter. Volatility of power grids under real-time pric- ing. Power Systems, IEEE Transactions on, 27(4):1926- 1940, Nov 2012.
- [Simon, 1972] Herbert A Simon. Theories of bounded ratio- nality. Decision and organization, 1(1):161-176, 1972.
- [Sutton and Barto, 1998] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction, vol- ume 1. MIT press Cambridge, 1998.
- [Vinyals et al., 2014] Meritxell Vinyals, Valentin Robu, Alex Rogers, and Nicholas R. Jennings. Prediction-of-use games: A cooperative game theory approach to sustain- able energy tariffs. In Proceedings of the 2014 Interna- tional Conference on Autonomous Agents and Multi-agent Systems, AAMAS '14, pages 829-836, Richland, SC, 2014. International Foundation for Autonomous Agents and Multiagent Systems. [Vytelingum et al., 2010]
- Perukrishnen Vytelingum, Thomas D. Voice, Sarvapali D. Ramchurn, Alex Rogers, and Nicholas R. Jennings. Agent-based micro-storage management for the smart grid. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: Volume 1 -Volume 1, AAMAS '10, pages 39-46, Richland, SC, 2010. International Foundation for Autonomous Agents and Multiagent Systems.
References were extracted automatically from the PDF and may contain errors.
Full text extracted automatically from the PDF; figures, tables and equations may differ. The PDF is the version of record.
1 Introduction
Energy systems are in transition towards more sustainable generation portfolios, which must be matched with more flexible demand [Rohjans et al., 2010; Fang et al., 2012]. Many potential and existing problems that the main power grid is facing are connected to the need for continuous balance and the increasing peak demand, both essential in determining the resulting system efficiency and eventually the costs of electricity. Maintaining balance becomes more challenging in face of generation from natural resources such as the sun and wind, which are subjects to stochastic availability. Such stochastic fluctuations and especially deviations from predictions may be matched with expensive fast ramping conventional generators, e.g., gas turbines, which are otherwise only used for matching the peak demand. The balancing power of such quickly adjustable generators is traded on balancing markets. Due to the high marginal costs, the main strategy to control costs is to avoid the need to purchase balancing power by reducing deviations from energy demand and supply predictions, and thus reducing uncertainty.
In the current electricity system, retailers are the balancing responsible parties, pooling customers into larger portfolios. Electricity retailers are facing high risks of balancing market participation due to the volatility of reserve power prices for balancing. At the same time, the penetration of distributed renewable energy sources drives the increasing adoption of flexibility by retail customers, which may be micro-grids, energy cooperatives, prosumers, and plain consumers. Local renewable power generation implies higher risks of energy shortages or overproduction, since generation is volatile and locally highly correlated. Customers may use their flexibility, e.g., from storage, primarily to their own interest rather than in the interest of the retailers' balancing needs [Vytelingum et al., 2010].
Most existing electricity tariffs by electricity retailers, especially in Europe, do not provide incentives for intelligent behavior by customers, precluding flexible customers from assuming some of the high costs related to the participation in the balancing markets. Customers can only subscribe to tariffs given their amount of flexibility and local generation with flat or day-night tariffs that may exhibit different costs in relation to the capacity of their flexibility as well as their scale. Dynamic pricing is a means to encourage favorable changes in demand patterns by the customers [Borenstein et al., 2002; Roozbehani et al., 2010]. Time of use (ToU), critical peak price (CPP), and real-time pricing are some of the pricing schemes used to stimulate favorable customer behavior in different pilot studies [Owen and Ward, 2010], e.g., in Ontario or California. However, dynamic pricing approaches may introduce disruptive and unfavorable market behavior [Roozbehani et al., 2012; Herter and Wayland, 2010], and thus planning and ahead prices are required [Braithwait et al., 2007].
We consider a multi-agent system, where a buyer agent wants to purchase an uncertain quantity of a continuously divisible good from a seller agent. We refer to the buyer and the seller as the customer and the retailer respectively. We present the risk-sharing tariff, a novel approach to incentivize intelligent customer behavior (uncertainty reduction by the demand side) by giving customers the choice to assume balancing risk of the retailer. We consider settings where one customer has a direct or a representative influence on the balancing requirements of the retailer. This is the case in:
- Service level agreements (SLAs) formally define an agreement between a service provider and the service
user, specifying the service and its characteristics, e.g., quality, risk. In the context of electricity markets we interpret SLAs as a direct extension of conventional electricity tariffs: While current electricity tariffs ensure delivery (100% quality) and a fixed kWh price (0% risk), SLAs may provide customer choice, such as assuming parts of the balancing risk, as discussed in this paper. Such SLAs may be better suited for decentralized trading of electricity between small-scale producers and individual customers.
- Highly correlated demand can be the result of similar behavior of customers, influenced for instance by weather conditions in specific locations. The higher the correlation, the closer the deviations of one customer to the deviations of other customers, i.e., changes in the consumption behavior of one customer predicts the same change in the behavior of other customers. Therefore, the portfolio distribution may closely resemble the demand distribution of an individual customer for any specific location.
III. Local balancing: Current market-based balancing strategies do not consider the location of the customer, while it is in the retailer's own interest to balance customers locally. This can lower the costs corresponding to energy losses, transportation costs, and network load.
We formalize the interaction between the retailer and the customer as a two-player game (Section 3), and we study optimal strategies for both players. We define a two-step payment scheme, where the customer pays for its expected demand and later pays for any imbalances (Section 3.2). We show that the proposed tariff provides variable incentives and elicits intelligent behavior by the customer (Section 3.3). We further demonstrate the existence of Nash equilibria in this game, assuming that the retailer has access to the private costs of the customer (Section 4). Last, we discuss the concept of bounded-rationality and show that retailers may offer larger incentives to bounded-rational customers (Section 4.1).
2 RelatedWork
Challenging problems are arising with the transition from the current energy system to the envisioned smart-grid [Fang et al., 2012]. These problems provide a fertile ground for tools like game-theory and multi-agent systems to study situations with more than one stakeholder [Fadlullah et al., 2011; Pipattanasomporn et al., 2009]. The conflict of interest between the retailer and the customer has been formalized as a non-cooperative game with regards to ToU tariffs [Oruc et al., 2012] and incentives provided to customers for loadshifting [Pettersen et al., 2005]. Similarly, we formalize a game between the retailer and the customer to study incentives for intelligent customer behavior. In line with related literature, which studies optimal procurement strategies for the retailer under the presence of either uncertain demand [Nair et al., 2014] or uncertain prices [Hoogland et al., 2015], we assume a two-step market setting, where the prices are fixed but the demand is uncertain. In the closest state-of-the-art work a "prediction-of-use" tariff is proposed, where customers are charged both on their predicted consumption, but also on their deviations from this prediction [Vinyals et al., 2014]. In the same fashion, we propose a two-step parameterized payment, where the customer precommits and prepays for its expected demand, and later pays for any deviation between the observed and the anticipated load.

To the best of our knowledge, this is the first gametheoretical study that considers incentives for intelligent customer behavior, giving customer the choice of how much risk to take from the retailer.
3 The Risk-Sharing Game
We capture the strategic interactions between the retailer and the customer in a two-player game. Figure 1 illustrates the extensive form representation of the risk-sharing game, showing the time sequencing of the actions. We consider a twostep market. The retailer first procures electricity in the ahead market with the unit price p and later pays for any absolute 1 deviation, between the observed demand of the customer and the procured quantity in the ahead market, in the balancing market with the unit price p0 > p. The prices p↪ p0 are determined by an exogenous process and cannot be influenced by the retailer (price-taker). Let x denote the random variable and fx the probability distribution function (PDF) of the customer's demand. We consider the distribution fx as the default behavior by the customer. The distribution fx is known to both players, since it can be observed, e.g., by smart-meters, and can be approximated given enough observations. Therefore, the proposed tariff requires the customer to precommit to and prepay the quantity bc = Ef [x], which is equal to the anticipated consumption.
The retailer, based on the customer's demand distribution fx, procures the quantity br in the ahead market. Any absolute deviation, between the quantity br and the observed consumption x of the customer, is balanced by the retailer in the balancing market. We consider the expected balancing costs as the balancing risk for the retailer [Ferguson, 1967], which is equal to Ef [♣br - x♣]p0 , assuming a direct influence of the customer to the balancing needs of the retailer (Cases I-III in Section 1).
1 In practice, both power excess and shortages can result in the increase of balancing costs for the retailers, since they may be charged for the deployment of upwards or downwards regulation power by the TSO.
In current electricity systems the retailer holds all the risk of balancing supply and demand. However, in the risksharing tariff, the balancing risk can be shared between the retailer and the customer. Let ⌧ 2 [0↪ 1] denote the share of risk that remains with the retailer and (1 - ⌧) the share of risk that is assumed by the customer. The risk-sharing tariff comprises two price functions: The precommitment price pc(⌧) for the quantity bc, which we assume equal to the anticipated load (bc = Ef [x]), and the imbalance price p0c (⌧) for any absolute deviation ♣bc -x♣ from the anticipated load. The retailer decides the price functions pc(⌧) and p0c (⌧), based on br and the uncertainty of the demand given the distribution fx. The customer then chooses the risk share ⌧ to be covered by the retailer.
The utilities Ur↪Uc for the retailer and the customer respectively can be determined after the observed consumption x. Let T denote the payment from the customer to the retailer and Cm the market costs of the retailer. The utilities can be written as: Ur = T - Cm↪ Uc = -T . Analytically,
is the utility of the customer, including the cost for the precommitted quantity bc and the cost for absolute deviations from the anticipated load. Similarly,
is the utility of the retailer, which is equal to the payment by the customer deducting the market costs of the retailer.
We described the risk-sharing game between the retailer and the customer, defining the utilities for both players. We can generalize and say that the risk-sharing tariff approximates the current retail flat tariff situation when no risk is assumed by the customer (⌧ = 1). Let x be the consumption of the customer and N the number of payments during one year from the customer to the retailer under the current flat tariff market. Given the law of large numbers we know that for large N, PN x ⇡ NEf [x] holds. Thus, the total payment of the customer approximates the payment under the risk-sharing tariff when the retailer retains all the risk (⌧ = 1).
3.1 Optimal Quantity of Procurement
After the prices p↪ p0 and the distribution fx are determined, the retailer procures the quantity br in the ahead market. In this section, we compute the optimal procurement b⇤r that maximizes in expectation the utility of the retailer in (2). Let Ufr denote the expected utility of the retailer with respect to the distribution of the uncertain demand fx.
The price functions pc(⌧)↪ p0c (⌧) are free parameters, since they will determine the profit. We treat the price functions as independent of br and therefore we minimize the market costs Cm of the retailer.
Lemma 3.1 The first derivative of the expected utility of the retailer in (3) with respect to br is:
where Fx is the cumulative distribution function (CDF) of the random variable x.
Theorem 3.2 The quantity b⇤r maximizes the expected utility of the retailer.
where F -1 x is the inverse cumulative distribution (ICDF) function.
Proof Equation 5 follows from d dbr Ufr = 0. The expected utility of the retailer is a strictly concave function: d2 db2r Ufr = -2p0 fx(br) < 0. Therefore, b⇤r is indeed the unique optimum.⌅
Note, for any given p0 > p, the quantity b⇤r is lower than the expected demand due to the absolute imbalance quantity.
3.2 Determining the Price for Risk-Sharing
In this section, we define the requirements and the properties of the risk-sharing tariff and we propose how to choose the price functions. An important requirement for the price functions pc(⌧)↪ p0c (⌧) is that the expected utility of the retailer for any given ⌧ 2 [0↪ 1) should be greater or equal to the expected utility when ⌧ = 1. Analytically,
where ' denotes an extra profit for the retailer per expected unit of consumption. ' is approaching business costs in a perfect competition and arbitrarily large values in a monopoly.
Given the requirement in (6) and using (3), we derive the following inequality:
To find functions pc(⌧)↪ p0c (⌧) that satisfy the above inequality, we define the minimum imbalance price function:
which is equal to the price the customer would pay by participating in the balancing market for its share (1 - ⌧) of balancing risk. Since pc(⌧) is a free choice, we propose the minimum ahead price function that satisfies (7) when replacing p0c (⌧) with (8):
We will proceed to show that this proposed price function guarantees the minimum profit margin ' for the retailer.
Theorem 3.3 Any tariff (pc(⌧)↪ p0c (⌧)), using pc(⌧) as defined in (9) and satisfying p0c (⌧) ≥ %0c (⌧)↪ 8⌧ 2 [0↪ 1], and p0c (1) = 0, satisfies (6).
Proof Note that pc(⌧) has been defined such that U % r (⌧) = Ur(1), if p0c (⌧) = %(⌧). For any p0c (⌧) that satisfies p0c (⌧) ≥ %(⌧ )↪ 8⌧ , U p r (⌧ ) ≥ U % r (⌧), since only the profit from the term p0c (⌧)Ef [♣bc -x♣] increases while all other terms are fixed. ⌅
The function p0c (⌧) refers to the price per unit for any absolute deviation of the customer's consumption given the choice of ⌧. We propose p0c (⌧) to embrace some additional desired properties with regards to the ability of the customer to reduce its demand uncertainty.

Consider a customer that can alter its demand distribution fx ! gx, such that Eg[♣bc -x♣] Ef [♣bc -x♣]. We define gx as the demand response of the customer. We propose a tariff that additionally imposes the constraint Eg[x] = Ef [x]. Let ⌧ ⇤ (gx) denote the risk that maximizes the utility of the customer under gx. The following two properties are common sense conditions for demand response tariffs.
Property 1. No demand response, no risk incentive: If Eg[♣bc - x♣] = Ef [♣bc - x♣] then ⌧ ⇤ (gx) = 1.
We propose the following imbalance price function that satisfies the above properties (Section 3.3) under # > 0.
where Φ(⌧) denotes the penalty that is equal to the discount in the precommitment price the retailer offers, Φ(⌧) = pc(1) - pc(⌧). The parameter # 2 R+ scales the penalty term Φ(⌧). Figure 2 illustrates the shape of the price functions pc(⌧)↪ p0c (⌧), computed for fx = N(0▷15↪ 0▷1)↪ truncated to x 2 [0↪ 0▷79], p = 0▷1, p0 = 0▷5, ' = 0▷02, # = 1.
The tariff composed of pc(⌧) and p0c (⌧) guarantees a minimum acceptable utility for the retailer, which is equal to the current flat tariff situation (⌧ = 1). The imbalance price function p0c (⌧) proposed in (10) also satisfies desirable properties with respect to the upcoming discussion, associated with the strategy of flexible customers.
3.3 Optimal Strategies for Flexible Customers
Demand response in electricity systems refers to the ability of customers to adjust their demand behavior in response to financial incentives provided by electricity providers. In this paper, we interpret demand response as the ability of the customer to reduce the uncertainty of its demand. Let ∆ denote the action of the customer, which affects the distribution of the demand fx, such that the observed consumption x is sampled from the new distribution gx. Recall that the expected demand remains the same Eg[x] = Ef [x] and the expected absolute deviations may become lower Eg[♣bc - x♣] Ef [♣bc - x♣]. Let C∆(gx) , C∆(fx ! gx) denote the costs associated with reducing the uncertainty, e.g., capturing customer's discomfort or costs of smart devices and batteries.

We show that for any distribution gx there is a unique ⌧ ⇤ 2 [0↪ 1] that maximizes the expected utility of the customer. Let Ugc denote the expected utility of the customer with demand response ∆ and resulting demand distribution function gx.
Lemma 3.4 Under pc(⌧) as in (9) and p0c (⌧) as in (10), the first derivative of the expected utility of the customer in (11) with respect to ⌧ is:
where af = Ef [♣bc - x♣], and ag = Eg[♣bc - x♣].
Theorem 3.5 The quantity ⌧ ⇤ maximizes the expected utility of the customer for any given gx and C∆.
where [x]hl =max(l↪min(h↪ x)).
Proof Equation 13 follows from d d⌧ Ugc = 0. The utility function of the customer with regards to the risk assumption ⌧ is strictly concave, since d2 d⌧ 2Ugc = -2 # bc agafp0 < 0. Therefore, ⌧ ⇤ is the unique optimum. ⌅
Under the assumption of a cost-free demand response model, i.e., C∆(·) = 0, we will proceed to show that a customer with uncertain demand response has incentives to participate in the risk-sharing tariff contributing its maximum available demand response. Consider the distribution ux such that:
where ux provides a threshold ability of the customer to reduce the expected absolute deviation of the demand.
Theorem 3.6 For a customer with uncertain demand response gx that can at least reduce the uncertainty of its demand to the level of ux, such that (14) holds, Ugc (⌧ ⇤ (ux)) > Uuc (⌧ ⇤ (ux)) > Ufc (1) and 8⌧ ≥ ⌧ ⇤ (ux) : Ugc (⌧ ) ≥ Ufc (1).
Proof The inequality in (14) implies that the imbalance payment in (11) follows the same ranking, as it is the product of the unequal expectations with the identical imbalance price p0c (⌧ ⇤ (ux)). Since all other terms remain the same, this directly induces the inequalities of the resulting utilities stated in the theorem. ⌅

Theorem 3.6 implies that a customer with uncertain demand response gx, that is bounded by ux, can only benefit by contributing all available demand response. Furthermore, any choice of ⌧ ≥ ⌧ ⇤ (ux) ensures a lower bound for the utility of the customer.
For the remainder of this paper, we assume that both gx↪ fx are normal distributions with θg = θf , and standard deviations σf and σg 2 (0↪ σf ] respectively. For this restricted case, we will apply a simplified notation. The optimal strategy for the flexible customer is denoted by ⌧ ⇤ (σg), similarly the cost by C∆(σg). Figure 3 presents the function ⌧ ⇤ (σg) for different values of # in (10). For # ⇠ 0, the utility of the customer becomes a linear function that is monotonically increasing in ⌧ when σg = σf . Thus, the optimal choice of the customer becomes ⌧ ⇤ (σf) = 1, and ⌧ ⇤ (σg < σf) = 0. For # ⇠ 1, the optimal choice of the customer is to assume no risk (⌧ ⇤ (σg) = 1↪ 8σg 2 (0↪ σf ]), since the penalty term Φ in (10) is infinitely scaled.
In this section, we derived the optimal strategy ⌧ ⇤ (gx) of the customer. We showed how the choice of the parameter # by the retailer can influence the optimal strategy of the customer. Furthermore, we demonstrated by Theorem 3.6 that the risk-sharing tariff is attractive to customers with uncertain demand response.
3.4 Comparison of the Utilities
We compare the expected utilities of both players, again under the assumption of a cost-free demand response model, i.e., C∆(σg) = 0↪ 8σg 2 (0↪ σf ]. Figure 4 illustrates the expected utilities of both players. Let the tuple (U0c ↪U0r ) illustrate the point in the utility space that represents the current flat tariff situation, i.e., σg = σf and ⌧ = 1. Each line segment in the figure represents the utility tuples given a specific demand response σg and varying ⌧ . The empty circles represent the utility tuples when the customer chooses to assume no risk (⌧ = 1). In such a case, demand response only yields benefits to the retailer. On the contrary, filled circles represent the utility tuples when the customer chooses to assume the full share of risk. Increasing the risk assumption (moving across the line segments from ⌧ = 1 to ⌧ = 0) requires a certain level of demand response to be profitable for the customer. For high demand response (low σg), it results in the utility increase for the customer. For low demand response (high σg), only the retailer benefits from the decreasing uncertainty of the demand. Reduced uncertainty in the demand side can contribute to the improved social welfare (sum of the players' utilities) through the risk-sharing tariff.

Theorem 3.6 can also be illustrated using Figure 4. Note that for normal distributions, E[♣x - θ♣] = σp2◁⇡ [Geary, 1935]. Hence, σg < σu < σf implies that the inequalities in (14) hold. According to Theorem 3.6, it follows that Ugc (⌧ ⇤ (σu)) > Uuc (⌧ ⇤ (σu)) > Ufc (1). Intuitively, the customer can increase its utility by switching from σu to σg, or more generally by switching from ux to gx.
4 Nash Equilibrium Strategies
In this section, we study the Nash equilibria (NE) of the risk-sharing game. Where necessary, we make the dependence of utilities on both strategies more explicit by using notation Uc(⇡r↪ ⇡c) and Ur(⇡r↪ ⇡c), where ⇡r↪ ⇡c denote the strategies of the retailer and the customer respectively. NE are pairs of strategies (⇡⇤ r ↪ ⇡⇤ c ), such that Uc(⇡⇤ r ↪ ⇡⇤ c ) ≥ Uc(⇡⇤ r ↪ ⇡c)↪ 8⇡c and Ur(⇡⇤ r ↪ ⇡⇤ c ) ≥ Ur(⇡r↪ ⇡⇤ c )↪ 8⇡r. Let C∆(σg) ≥ 0↪ 8σg 2 (0↪ σf) be an arbitrary cost model for demand response and C∆(σf ) = 0, i.e., cost without demand response is zero. We assume that the demand response cost model is known by the retailer.
First, we define the strategies of the two players. For the retailer, the only free choice is the scalar # that parameterizes the proposed tariff in (10). The strategy of the retailer is denoted by ⇡r = #. For the customer, the strategy ⇡c = (⌧↪ σg) refers to the choice of risk ⌧ and demand response σg. Furthermore, the strategy includes the credible threat of returning to the flat tariff without any demand response, ⇡threat c = (1↪ σf), if the utility drops below the reference utility U0c . The threat is possible due to the action sequence indicated in Figure 1 and credible since the threat strategy outperforms the protected equilibrium strategy. Given any C∆(σg) we know from Theorem 3.5 that for any given strategy ⇡r there always exists a strategy ⇡⇤ c that maximizes the expected utility of the customer: ⇡⇤ c = (⌧ ⇤ (σ⇤ g ) ↪ σ⇤ g = arg maxσg [Ugc (⌧ ⇤ (σg))]).
Figure 5 illustrates the utilities of the two players, computed using the quadratic demand response cost model C∆(σg) = w♣σf - σg♣2 , for w = 10. Each curve corresponds to one of the following three retailer strategies: ⇡⇤ r = #⇤ , ⇡↵ r = (# ≥ #⇤ ), ⇡β r = (# < #⇤ ). The utility tuples along each curve correspond to the customer strategies ⇡c = (⌧ ⇤ (σg)↪ σg). The curves start from the utility tuple

(U0c ↪U0r ) denoted by the empty circle, where the customer does not assume any risk, and hence no demand response, i.e., ⇡c = (1↪ σf). The curves end where ⇡c = (0↪ 0) denoted by the filled circles. Note that the strategy ⇡⇤ r = #⇤ that maximizes the utility of the retailer (solid curve), yields Ugc (⌧ ⇤ (σ⇤ g )↪ σ⇤ g ) = Ufc (1↪ σf ) for the customer. The utility of the customer of using demand response (star on solid curve) becomes equal to the utility without demand response (open circle).
Theorem 4.1 The strategy pair I, (⇡⇤ r ↪ ⇡⇤ c = (⌧ ⇤ ↪ σ⇤ g )), and the set of pairs II, (⇡↵ r ↪ ⇡threat c = (1↪ σf)), are the only two types of NE in the risk-sharing game.
Proof Any positive change " in the strategy of the retailer, such that # #⇤ + " (e.g., ⇡↵ r ), will cause the customer to adopt the strategy ⇡threat c , since Ugc (⇡⇤ c ↪ #) < U0c , leading both players to the utility pair (U0c ↪U0r ). On the other hand, any negative change " (e.g., ⇡β r ) will directly reduce the retailer's income. Hence, the retailer has no incentive to deviate from I. In addition, the customer strategy is the best response by definition, and thus the customer has no incentive to deviate either, making I a NE. Now, consider any strategy pair (⇡β r ↪ ⇡c). The retailer can gain by deviating from this strategy pair by increasing #, making sure that no equilibrium containing ⇡β r exists. Finally, consider any pair (⇡↵ r ↪ ⇡threat c ). Since Ur is unaffected by # given ⌧ ⇤ (⇡↵ r ) = 1, providing no incentive to apply demand response, no player can gain by deviating unilaterally. Hence, each pair of strategies in the set II is a NE. ⌅
We showed the existence of two types of NE within the risk-sharing game, where (⇡⇤ r ↪ ⇡⇤ c = (⌧ ⇤ ↪ σ⇤ g )) Pareto dominates (⇡↵ r ↪ ⇡threat c = (1↪ σf)) and therefore is favorable for both players.
4.1 Bounded-Rational Customer
The concept of bounded rationality [Simon, 1972] assumes that agents, automated or not, do not behave as perfectly rational decision-makers, bounded by imperfect information and their limited capacity.
Customer behavior can be modeled using the bounded rationality paradigm [McFadden, 1975]. Customers do not always subscribe to the cheapest tariff but the probability of doing so is high. Softmax is a function that can model the decision-making of such an agent [Ortega and Braun, 2011; Sutton and Barto, 1998] with the irrationality parameter λ: p(⇡k c ) = exp(Uc(⇡k c )◁λ)◁ P⇡ic exp(Uc(⇡ic )◁λ), where p(⇡k c ) is the probability of the customer to use strategy ⇡k c . We apply this function to a discrete sampling of the continuous parameter σg to probabilistically mix between strategies ⇡c = (⌧ ⇤ (σg)↪ σg)↪ σg 2 (0↪ σf ]. Figure 6 illustrates the numerical approximation of the optimal choice of #⇤ of the retailer and the utility tuples under various degrees of irrationality λ. For λ = 10-4 , #0 approximates the value of #⇤ computed earlier for the rational customer, since for low λ Softmax is approximating the rational (greedy) strategy selection. Beyond λ ⇡ 10-2 , the retailer starts increasing ✓⇤ to infinity as the customer becomes random, resulting in the infinite increase in the utility of the retailer at the cost of the customer. Note that λ⇤ = 4▷27·10-3 maximizes the utility of the customer, yielding a larger utility than the utility the customer receives in NE. We can deduce that larger incentives for demand response will be offered if the retailer believes to be facing bounded-rational agents.
This result has implications for implementing automated tariff selection algorithms. In particular, the irrationality parameter λ⇤ can be seen as in equilibrium with #⇤ and suggests that automated strategies may gain utility by adopting probabilistic softmax selection.
5 Conclusion
In this paper, we proposed a tariff where the balancing risk can be shared to incentivize intelligent customer behavior. We defined a formal game between the retailer and the customer in settings where the customer has a direct influence on the balancing requirements of the retailer (Section 3). We showed analytically that the proposed tariff is acceptable for both the retailer (Section 3.2) and the customer (Section 3.4). We further studied best response strategies that are computable as presented in Sections 3.1 and 3.3. We showed how social welfare is improved (Figure 4) due to the uncertainty reduction resulting from the intelligent customer behavior, and we provided arguments (Theorem 3.6) why the proposed tariff elicits all freely available demand response. In Section 4, we showed the existence of NE within the risksharing game and illustrated them with computations. Last, we showed that bounded rationality can be a valuable concept when implementing automated tariff selection schemes (Section 4.1). The proposed tariff provides a broad basis for future extensions, e.g. relaxing assumptions about the precommitment quantity bc of the customer, or the exogenous and known market prices p↪ p0 .
Acknowledgments
This work is part of the research programme Uncertainty Reduction in Smart Energy Systems (URSES) with project number 408-13-012, which is partly financed by the Netherlands Organisation for Scientific Research (NWO).
@inproceedings{ijcai2016p61,
title = {Incentivizing Intelligent Customer Behavior in Smart-Grids: A Risk-Sharing Tariff & Optimal Strategies},
author = {Methenitis, Georgios and Kaisers, Michael and La Poutré, Han},
booktitle = {Proceedings of the Twenty-Fifth International Joint Conference on
Artificial Intelligence, {IJCAI-16}},
publisher = {AAAI Press / International Joint Conferences on Artificial Intelligence},
editor = {Subbarao Kambhampati},
pages = {380--385},
year = {2016},
month = {7},
url = {https://ijcai.org/proceedings/2016/61},
}
TY - CPAPER TI - Incentivizing Intelligent Customer Behavior in Smart-Grids: A Risk-Sharing Tariff & Optimal Strategies AU - Methenitis, Georgios AU - Kaisers, Michael AU - La Poutré, Han T2 - Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-16 PB - AAAI Press / International Joint Conferences on Artificial Intelligence PY - 2016 SP - 380 EP - 385 UR - https://ijcai.org/proceedings/2016/61 ER -Copied
Machine-readable versions of this paper, for tools and AI agents: