Long-term issues with the Energy-Only Market design in the context of deep decarbonization

There has been fierce controversy in the literature over the long-run efficiency of the energy-only market (EOM) design ever since its inception. In this paper, we provide novel insights to illuminate this historical controversy, and we revisit it with a focus on contemporary issues and the profound changes brought about by the energy transition. Specifically, we develop an analytical and modeling framework to quantitatively investigate how EOM outcomes hinge on the underlying behavioral, informational and structural assumptions. We apply our framework to a case study calibrated on Californian fundamentals that captures the key features of energy systems under deep decarbonization. We characterize how EOM outcomes can substantially deviate from the long-run optimum as soon as one assumption is relaxed compared to theoretical requirements. This leads to pathways with higher electricity prices, lower security of supply and delayed decarbonization. In particular, we highlight how market price signals alone are prone to a dynamic entry-exit coordination problem between investment in low-carbon assets and the phaseout of fossil-fired assets. This calls for a market design reform to complement price signals that accounts for realistic assumptions.


Introduction
The energy transition poses mounting challenges to energy systems across the world with various interrelated facets including decarbonization, renewables integration, energy efficiency improvement and electrification.In practice, these ambitious aspirations translate into different targets at different horizons.For instance, Senate Bill 100 in California mandates targets of 60% and 100% zero-carbon electricity retail sales to end-use customers by 2030 and 2045, respectively (California State Senate, 2018).The European Union Green Deal and fit-for-55 policy package are another case in point, with the 2030 target of cutting greenhouse gas emissions by 55% below 1990 levels and the objective of a net-zero economy by mid-century (European  Commission, 2019, 2022).
In the electricity sector, generation expansion planning (GEP) models are often used to explore cost-efficient pathways to achieve decarbonization objectives.There is a rich literature discussing the underlying optimization techniques and technological assumptions (e.g., integration of renewable energy sources, representation of short-term operations and seasonal storage) in such analyses (e.g., Alimou et al., 2020;  Abdin et al., 2022).There are also numerous applications that provide key insights on decarbonization pathways in given jurisdictionse.g., CPUC (2019) for California or RTE (2021) for France-or on the role of specific technologies-e.g., hydrogen in Schulthoff et al. (2021).This class of models offers a normative framework that is widely used by academics, regulators, agencies, consultancies, investors and market participants alike.
A crucial assumption of these models is that they take the perspective of a benevolent social planner.While this has the advantage https://doi.org/10.1016/j.eneco.2024.107418Received 26 July 2023; Received in revised form 4 February 2024; Accepted 23 February 2024 A. Lebeau et al.   of yielding normative results that hold independently of the institutional or market framework, this turns into a blind spot when the latter is to be studied and scrutinized.In particular, there is increasing concern that the energy-only market (EOM) design, 2 which has been held up as the target design model in many jurisdictions since the beginning of the deregulation era, may fall short of supporting necessary investments to deliver decarbonization, reliability and affordability objectives in a cost-efficient and timely manner (e.g., Roques and Finon,  2017; Newbery, 2018; Blaquez et al., 2020; Joskow, 2022; Keppler  et al., 2022).
The EOM paradigm rests on the principle that socially optimal long-term entry and exit decisions can be decentralized in competitive markets.Theoretical foundations can be traced back to Arrow  and Debreu (1954) in a general framework and to Caramanis (1982)  for a seminal application to the electricity industry. 3They are based on the equivalence between optimality conditions of private agents in a perfectly competitive market and those of a benevolent social planner (also assuming private and social discount rates are equal).From an investor perspective, this notably translates into the equality between inframarginal rents and fixed costs at the optimum.Although commonly presented for a representative year with annualized costs under certainty (e.g., Joskow and Léautier, 2021), this result can be extended to a multi-year, stochastic framework by equating the expected discounted sum of inframarginal rents with total fixed costs (e.g., Abada et al., 2017; Poncelet et al., 2019).Yet this equivalence holds under ''several strong simplifying hypotheses'' (Rodilla and Batlle, 2012)  or ''demanding conditions'' (Newbery, 2018) that include perfect information, full rationality, risk neutrality, and complete markets for risk trading. 4hether or not the tenability of these assumptions would compromise the practicability of the EOM paradigm has spurred heated debates in the community since its very inception. 5At the dawn of the deregulation era, Littlechild (1988) observed the disagreement, noting that ''mathematical models designed to prove that spot pricing is socially optimal are unpersuasive''-referring to the aforementioned ''MIT models'' of Caramanis (1982) and Schweppe et al. (1988).More skeptical of the purported long-run efficiency of spot markets, others like Westfield  (1988) quite remarkably foreshadowed detrimental impacts on the cost of capital that are front and center in current debates (e.g., Peluchon,  2021; Gohdes et al., 2022; Neuhoff et al., 2022; Newbery, 2023). 6Even 2 The traditional 'energy-only' terminology used in the literature can sometimes be misleading as it may include a complete sequence of short-term markets (e.g., for adjustment and ancillary services) as well as derivatives (e.g., futures contracts) markets.In this paper, we use this terminology to refer to fully deregulated market designs based on short-term wholesale markets and associated derivatives markets. 3It is worth recalling that Boiteux (1949, 1960) established seminal results on marginal electricity pricing for a regulated utility under the assumption of an optimal investment policy: ''Provided there is an optimal investment policy, short-term pricing is also long-term pricing, and there is no longer any contradiction between the two''.Similarly, Schweppe et al. (1988) also considered an energy marketplace in a regulated environment.Acknowledging the investment decentralization result of their co-author (Caramanis), Schweppe et al. wrote  ''The spot price based energy marketplace is designed to operate in a regulated environment [...].This chapter only presents a set of basic ideas [...].Since the advantages and disadvantages have not been quantified, we are not advocating deregulation (i.e., we do not know whether there is 'a lady or a tiger' behind the door)''. 4These aspects are discussed in detail in Section 4.1.1.For instance, even when markets are complete, risk averse investors evaluate a risk-adjusted expectation of future inframarginal rents against total fixed costs. 5This prolonged a similar controversy in the public utility pricing literature on the conditions for the equivalence between short-and long-run marginal cost pricing, see e.g.Andersson and Bohman (1985) and Section 4.1.
6 More generally on long-run efficiency, Westfield (1988) warned that ''spot markets for electric power will not perform the miracles that perfect markets perform though the theoretical controversy never settled, political impetus was a critical driver for the liberalization of the industry (e.g., Joskow and  Schmalensee, 1988; Léautier, 2019).The implications of deregulation for long-run efficiency also took a long time to materialize in practice, notably because wholesale markets were implemented in relatively mature power systems with a stable or contracting demand and little need for new investments.The main focus was on short-run efficiency, i.e. on harnessing market forces to ensure an efficient use of the existing fleet (e.g., Pollitt, 2021; Cicala, 2022). 7ver the last decade or so, this controversy has been reinvigorated in three phases with new variations.First, as documented in Bublitz  et al. (2019), security of supply and 'missing money' in liberalized markets gradually became a focus of attention, with debates on the need for and design of capacity remuneration mechanisms.Second, as documented in Keppler et al. (2022), the energy transition shed a new light on the debates due to the required profound changes in energy system structures and generation mixes.In particular, the sheer scale and speed of necessary low-carbon investments, their specificities and capital-intensiveness, and various externalities (e.g., learning spillovers, social and industrial preferences) generate a market design and regulatory conundrum.While these problems are now rather well delineated at a conceptual level, there is scant literature offering quantitative insights. 8Last but not least, the ongoing energy crisis further exposed pre-existing design shortcomings and initiated a new wave of market reforms, also putting the question of affordability at the core of the debates (e.g., Fabra, 2023; Schittekatte and Batlle, 2023).
The objective of this paper is to provide novel quantitative elements to illuminate this historical controversy on electricity market design and to revisit it with a focus on contemporary issues and profound changes brought about by the energy transition.Specifically, we develop an analytical and modeling framework to quantitatively investigate how EOM outcomes are sensitive to theoretical assumptions and characterize how they deviate from the long-run optimum when these assumptions are relaxed.We apply our framework to a case study capturing the key features of energy systems under deep decarbonization, which differ from those when short-term markets were introduced and call for specific investigation.More precisely, we make three contributions to the literature.
As a first contribution, we provide an analytical framework to unpack the underlying assumptions that govern the efficiency of an EOM design in the long run.Specifically, we clearly delineate the behavioral, informational and structural assumptions that are conducive to an optimal energy mix in a pure EOM.In particular, we emphasize the dynamic nature of entry-exit decisions and the crucial roles of risks, hedging and anticipations.Compared to the existing literature that has looked into these assumptions at a conceptual level (e.g., Joskow,  2008; Rodilla and Batlle, 2012; Newbery, 2018) or with a focus on specific assumptions (e.g., Kraan et al., 2019; Fraunholz et al., 2023;  Tao et al., 2021, 2023), our framework is synthetic and unified.This in turn allows us to relax assumptions separately in order to isolate and compare the effects of doing so.
in the economic theory textbook.Many of the gains achievable through centralized coordination will be lost ''.
7 Back then the context and policy objectives were markedly different from those that prevail today for the energy transition.The focus was on improving the efficiency of mature systems and on replacing old coal-fired plants with modern gas-fired plants as peaking units (financing needs were modest because of CCGT's low fixed costs relative to variable costs).See also Section 5 in Keppler et al. (2022) for a short historical perspective and discussion.
8 To our knowledge, there are two exceptions: (Kraan et al., 2019), who find that an EOM does not give sufficient and stable investment signals to sustain a renewable, reliable and affordable power system; Zimmermann and  Keles (2023), who find that carbon neutrality cannot be achieved only through market-based investments in France.
As a second contribution, we develop a modeling framework combining optimization and simulation models in a novel way in order to operationalize the above analytical framework.Specifically, our core simulation model uses System Dynamics (SD) as a modeling approach to consider a representative agent and capture the aggregate market impact of relaxing each assumption.Compared to the literature that has used this or similar modeling approaches to analyze agent behavior and market design issues (see Section 2.1 for a detailed literature review), our SD market simulation model has key distinguishing features.First, it has a linkage with a GEP model that can be used to define the anticipations of the representative agent. 9The outcomes of the GEP model are also used as a normative benchmark against which the performances of simulated EOM outcomes are evaluated.Second, it also has a linkage with an optimization-based merit-order dispatch model to represent short-term operations and determine future wholesale electricity prices that are the only source of remuneration for assets in an EOM.Third, it solves for both investment and retirement decisions simultaneously in conventional, renewable and storage technologies.
As a third contribution, we combine these two frameworks and provide quantitative illustrations in the context of the energy transition.Specifically, we build a case study on the basis of a stylized representation of Californian fundamentals over 2025-45.We select the Californian system because of public data availability and reliability, and because it presents the key characteristics of power systems under decarbonization in a relatively simple setting.10This facilitates the modeling and the interpretation of the results, whose identified trends are relevant for all power systems in transition.These characteristics are a commitment to eliminate emissions that requires massive investments in non-dispatchable renewable energy sources along with storage solutions, an existing fleet with a sizable share of fossil-fired plants that will partly be phased out before the end of their lifespans, and an increasing demand at wholesale level driven by electrification.
Overall, our quantitative results illustrate how the theoretical assumptions needed for an EOM to deliver the long-run optimum can hardly be met in practice.In particular, we highlight the high level of informational and computational complexity associated with rational decision-making and optimal anticipations of all relevant market fundamentals and future entries and exits.Our results also reveal that EOM outcomes can substantially deviate from the optimum when we introduce limited anticipation sophistication or risk aversion that affect expected revenue streams and asset profitability.11This tends to increase electricity prices, decrease security of supply and delay decarbonization.A crucial new insight is a coordination problem between investment in low-carbon assets and the phaseout of fossil assets that hinders the energy transition.For instance, under risk aversion, our results go beyond the standard under-investment result established in the literature.Specifically, risk aversion results in delays in both new investment and fossil phaseout that mutually reinforce one another, and thus delay decarbonization further.
The remainder proceeds as follows.Section 2 presents the modeling approach in relation with the literature and describes the modeling framework, notably the SD model that simulates EOM outcomes.Section 3 presents our case study, describes the model calibration and characterizes the long-run optimum obtained with the GEP model.Section 4.1 presents our analytical framework, examines the assumptions under which the SD-EOM model replicates the GEP optimal outcomes, and discusses the impacts of unit indivisibility.Section 4.2 explores how simulated EOM outcomes deviate from the optimum when we relax these assumptions.Section 4.3 summarizes our results and offers implications for policy and market design.Section 5 concludes and outlines how our modeling framework can be extended to assess and compare alternative designs of long-term contracting mechanisms currently contemplated as part of undergoing market reforms.

Modeling framework
In this section, we first provide an overview of our modeling framework and place it within the related literature.We next describe its two constituent models.Models were developed and coded in Python, and the open-source codes are provided here: GitHub/ANTIGONE.

Literature review and model overview
Classes of models in the literature.For decades, electricity economists and engineers have used a rich toolbox of complementary approaches to model and get to grips with long-term power system issues.The various modeling options are generally classified into three categories, namely optimization, equilibrium, and simulation models (e.g., Ventosa  et al., 2005; Creti and Fontini, 2019) with distinct and complementary areas of relevance: • Optimization models are the original and traditional approach to modeling the evolution of energy systems.The so-called generation expansion planning (GEP) models typically take the perspective of a central planner that seeks to determine the socially optimal capacity development plan (i.e., that which minimizes system-wide investment and operating costs) given a variety of constraints (e.g., a cap on carbon emissions), see Kagiannas  et al. (2004) for a historical perspective.Over time, GEP models have notably been extended to stochastic frameworks and are still widely used today to analyze decarbonization pathways for energy systems, see Weber et al. (2021) for a recent review.• Equilibrium models simultaneously solve individual profit maximization problems for different types of agents (e.g., producers with different technologies, intermediaries, consumers), finding equilibrium solutions where no agent is better off deviating unilaterally (e.g., Fan et al., 2012).These models typically feature uncertainty and risk aversion (e.g., Ehrenmann and Smeers, 2011;  Abada et al., 2017; Mays et al., 2019; Mays and Jenkins, 2022)  or imperfect competition (e.g., Hobbs and Pang, 2007; Acemoglu  et al., 2017).• Simulation models can represent different decision-making rules (i.e., beyond profit maximization) and degrees of agent's sophistication and rationality.There are two broad types of simulation models: The first is agent-based modeling (ABM) that can feature heterogeneous agents.The second uses system dynamics (SD) and typically considers representative agents.dos Santos and Saraiva (2021) and Tao et al. (2021) (resp.Teufel et al. (2013)  and Ahmad et al. (2016)) provide useful reviews of ABM (resp.SD models) applied to energy systems.
Optimization models abstract the market realities away, and as such, they provide insightful and normative results.In fact, linear/ convex GEP model results can be interpreted as the outcomes from perfectly competitive markets with fully rational and informed agents.While this constitutes a useful benchmark, it does not capture the market environment in which agents trade and invest, the individual decision-making process of market participants (possibly with bounded rationality, information or foresight) or the sequentiality of discrete investment/divestment decisions over time (since all time steps are solved simultaneously) -see also Section 4.1.1.
Equilibrium models allow modelers to represent and assess the impacts of the market structure and heterogeneous agents and behaviors.This notably endogenizes key decisions and model variables (e.g., risk trading and the associated cost of capital).Yet these models rely on solvers whose results do not lend themselves to a straightforward interpretation of the mechanisms leading to the equilibrium, and they are typically solved in steady state, which does not unveil the dynamic nature of investment/divestment decisions.Additionally, by design these models cannot account for out-of-equilibrium situations, which are acknowledged to be common and deserve more attention (e.g., de Vries and Heijnen, 2008; Léautier, 2019).
Compared to equilibrium models, simulation models give more latitude in making explicit assumptions about agents' rationality, information and foresight levels, and in representing out-of-equilibrium situations.Arguably, this strength may also be a weakness, in that assumptions must be clearly spelled out and articulated with one another in order to arrive at sensible modeling results.Although SD models were first developed and applied to the electricity sector (e.g., Ford, 1983; Bunn and Dyner, 1996), both ABM and SD models have since then been widely used, notably to study capacity markets and more recently other market design issues in the context of the energy transition-see inter alia (Keles et al., 2016; Kraan et al., 2019;  Fraunholz et al., 2021, 2023; Tao et al., 2021, 2023; Anwar et al.,  2022) for ABM models, as well as Petitet et al. (2016, 2017), Ousman  Abani et al. (2018), Rios-Festner et al. (2019, 2020), Tang et al. (2021)  and Pourramezan and Samadi (2023) for SD models.
Modeling approach in this paper.We develop a modeling framework consisting at its core of an SD model that we complement by a GEP model.Given that we aim to assess the impacts of relaxing the assumptions underpinning the optimality of long-term decision-making in a canonical energy-only market (see Section 4.1.1),the choice of an SD model proceeds in two steps: (1) We first opt for a simulation model because we wish to explicitly represent and vary the assumptions about investor behavior (i.e., rationality, foresight, information, risk aversion) as well as quantify and describe the temporal dynamics of the energy transition.
(2) We next select an SD model to focus on a representative investor and isolate the impacts of relaxing said assumptions at an aggregate level-i.e., possibly capturing the resultant of different agents' decisions, but without formally accounting for heterogeneous agents and behaviors.
As discussed above, great care will be taken to motivate and delineate our assumptions about investor behavior and describe how we implement them in the following.To name but one key issue at this stage, Tao et al. (2021) and Fraunholz et al. (2021) note how simulation model results can be sensitive to (long-term) price projection methods, and in particular to the way future capacity developments are anticipated and impact future price formation -and in turn govern investment decisions -thus exemplifying an issue of endogeneity we will come back to in Section 2.2.This is the main reason why we complement our modeling framework with a GEP model.That is, we feed the SD model with two relevant outcomes from the GEP model (see Fig. 1).First, some information about future capacity developments from the GEP model can be used in the SD model when the representative investor forms future price and revenue anticipations (see Section 2.2).Second, because we do not explicitly model the market for carbon emissions, the shadow price associated with the annual constraint on emissions from the GEP model is used as an exogenous carbon price signal in the SD model (see Section 3).Finally and intuitively, the GEP model also constitutes a valuable normative benchmark for optimal capacity developments against which we will assess the (deviations in) outcomes resulting from the SD model. 12he GEP model is a standard constrained pluriannual costminimization problem. 13In short, its objective function is the expected net present value of total systems costs (operating costs + investment costs + cost of rationing non price-responsive consumers at the value of lost load).Decision variables include short-term operations and longterm entries and exits.Several constraints govern the market clearing, various generation and storage asset operations, asset fleet dynamics, and carbon emissions.The different technologies we represent are described in Section 3. We now turn to the description of the SD model.

System dynamics market model
The SD market model simulates the evolution of the generation mix as the result of successive market outcomes and stepwise investment and decommissioning decisions over time.Compared to intertemporal optimization or equilibrium problems that are generally processed by a numerical solver computing optimal values for all current and future decision variables simultaneously, this approach explicitly unpacks both the agents' decision-making process and the dynamics of capacity development.While this has the potential to capture important behavioral and transitory effects (e.g., path dependency), it also entails that we have to explicitly address the issue of endogeneity (i.e., circularity or co-determination) between current and future decisions, that is the formation and adjustment of (long-term) anticipations.
Specifically, individual investment and decommissioning decisions are determined endogenously and sequentially each year of the simulation period on the basis of an economic profitability assessment for a given set of investor behavior assumptions and range of future market outcomes.The SD model is composed of three interconnected modules to enable this assessment: First, an anticipation module that produces reference scenarios for market fundamentals over time.Second, a dispatch module that generates market prices as well as generation and storage patterns in future years for given reference scenarios.Third, a decision module that makes annual investment and retirement decisions using information about costs and revenues from the first two modules.The outcome of the decision module is then fed into the anticipation module, adjusting reference scenarios and initiating an iterative loop.The three modules and their linkages are graphically shown in the causal-loop diagram in Fig. 2. We describe them in turn below.

Anticipation module
The anticipation module produces reference scenarios on the basis of two types of long-term market fundamentals that govern current investment and decommissioning decisions.First, some parameters such as future demand, fuel prices, and carbon prices are set and calibrated exogenously (see Section 3.1).Second, anticipations must also be formed on future endogenous variables (i.e., future investment and decommissioning decisions that define the generation mix in the long term) which affect investment and decommissioning decisions today, and vice versa.This section focuses on the formation of the second type of anticipations over a 'prospective horizon'; namely, the time period over which investment projects assessed in the current year of the simulation operate and are remunerated (and equivalently for retirement projects).Relatedly, note that the model embeds build times that are set to zero in this paper. 14That is, we assume that there is no investment delay and assets start operation as soon as the investment decision is made.This assumption facilitates the capacity adjustment process for merchant entry-exit decisions in an EOM and entails that our results should be seen as an optimistic upper bound estimate of EOM performance.
As a (rational) way of dealing with deep uncertainty and cognitive limitations associated with their long-term decision-making, investors may use heuristics or rules of thumb to alleviate associated computational complexity and informational requirements (e.g., Simon, 1955;  Baumol and Quandt, 1964).For instance, heuristics can be utilized to forecast future relevant factors (Brock and Hommes, 1997), such as backward-looking adaptative expectations (Cepeda and Finon, 2011).Alternatively, simplified models considering only variables of firstorder importance can be built (Gabaix, 2014), information that is costly to obtain and process can be ignored (Reis, 2006), and planning horizons can be truncated and sliding (Quemin and Trotignon, 2021).Moreover, limited sophistication in forecasting future entries and exits over the whole prospective horizon can be justified by the fact that anticipating other agents' decisions is complex, especially without complete long-term markets that may allow for the coordination of agents towards the first-best outcome (e.g., Williamson, 1975; Van Huyck  et al., 1990; Felder, 2002).
For expositional clarity, we consider two polar cases of anticipation sophistication labeled 'static' and 'dynamic' anticipations.This notably allows us to avoid using in-between ad-hoc anticipation heuristics or rules that would make our results dependent on arbitrary modeling choices.The two cases are graphically illustrated in Fig. 3 for a stylized example of a fossil technology phaseout.
Static anticipation.The first and simplest case consists in not considering any future decisions throughout the prospective horizon.That is, at a given point in time, the existing fleet is maintained online with no new entries until existing assets reach the end of their lifespans and retire without early economic exits.This simplifying assumption of myopia, albeit somewhat extreme, is often (implicitly) made in the related literature (e.g., de Vries et al., 2013; Chen et al., 2018).In Fig. 3, in any given year of the simulation (here 2025), before any investment or decommissioning decision is made on that year, installed capacity 0 inherited from the previous year is prolonged into the future to form the anticipated trajectory A .Note that this trajectory is constant as long as no unit reaches the end of its lifespan (as is the case here until 2028).Dynamic anticipation.The second case introduces sophistication in anticipating the evolution of the generation mix over time.In line with standard prospective analysis methodology utilized in practice to inform investment and retirement decisions, the GEP model is leveraged to provide an optimal capacity development plan. 15Specifically, in a given year, the GEP model is run from the current state of the fleet, yielding optimal capacity trajectories by technology until the end of the prospective horizon.Then, the anticipated evolution of the generation mix is defined as per the optimal yearly capacity changes beyond the current year, while entry and exit decisions for the current year are left for the decision module to determine.This guarantees that in the benchmark case (i.e., assuming perfect information, rationality and risk neutrality) the decision module arrives at the optimal decisions for the current year that are congruent with the entire optimal capacity development plan derived from the GEP. 16This also constitutes a conservative assumption when assessing the deviations induced by moving away from the benchmark case (Section 4).In Fig. 3, the GEP model run from the initial conditions 0 yields the full optimal trajectory 1 , and the dynamic anticipation of future entries and exits B is obtained by only keeping optimal decisions beyond the current year. 17oreover, the anticipation module is designed to accommodate two types of deviations from the above perfect dynamic anticipation case.First, we can relax the perfect information assumption and introduce various biases in the anticipation of future fundamentals, regarding either the parameters used as inputs in the GEP model (e.g., demand level, technology costs) or its outputs (e.g., carbon price, capacity trajectories).The introduction and choice of these biases is arbitrary, but this should be seen as a complement to our analysis that resorts to two polar anticipation cases.Second, we can choose the frequency at which the GEP model is called to update the anticipation of future entries and exits.While annual updates would be ideal, in practice the associated cost and complexity of this exercise may only warrant an update every couple of years or more.18

Dispatch module
The SD model embeds an economic dispatch module to represent short-term operations and market prices.This type of linkage, first introduced in Dyner et al. ( 2011), is preferred over the 'revenue curve ' (e.g., Ousman Abani et al., 2018) and 'scarcity rents curve' (e.g., Kraan et al., 2019) approaches that exogenously define a direct relation between the level of capacity and assets' revenues.Although computationally heavier, our approach allows for a more accurate representation of key features of decarbonized power systems (notably the time variations of weather-dependent generation and the dynamics of storage), and consequently of the price and revenue distributions.Below we first describe the module's structure and main assumptions, and then specify when and how it is utilized within the SD model workflow.
The dispatch model is formulated as a standard short-term costminimization problem whose objective function is the total operating cost; i.e., the sum of variable costs and cost of rationing price-inelastic consumers (set at the VoLL).We consider an hourly resolution and to alleviate the computational burden, we divide the annual problem into sub-problems with a rolling horizon and a 'look-ahead interval' (i.e., a final or continuation period that is included in each sub-problem whose solutions are discarded in the current sub-problem but utilized in the subsequent sub-problem).We set these parameters to 1 month and 24 h, respectively, and storage assets are dispatched over these optimization steps assuming perfect foresight.
Implicitly, this representation of short-term operations assumes that the sequence of short-term markets is frictionless and yields an optimal outcome. 19This important assumption is deliberate, as we wish to zero in on the long-term aspects of a canonical energy-only market that would still prevail if current short-term markets were improved through more integration and finer granularity.For the same reason as well as for consistency with the case study presented in Section 3, we keep the dispatch model as parsimonious as possible.Although this version A. Lebeau et al.   of the dispatch model does not represent short-term uncertainties, seasonal storage, ancillary services and grid congestion, its generic and modular implementation is amenable to such developments and refinements.
Finally note that the dispatch module is called and run in two different places in the SD model: First, it is primarily utilized to convert long-term fundamentals from the anticipation module into future market prices and revenues as well as generation, storage charging and renewable curtailment patterns.This dispatch is dubbed ''virtual dispatch'' (Tao et al., 2021) as it is run for future dates on the basis of different anticipations of the future state of the system.Second, once investment and decommissioning decisions are made in a given year, the dispatch module is run to simulate the (actual, not virtual) shortterm market outcomes for that year, before moving on to the following year.This final run is notably used as a basis to compute different metrics in Section 4.

Decision module
The decision module consists of a loop that considers all possible investment and retirement decisions and iteratively selects the most profitable one at each step in the simulation until none is left.Specifically, economic profitability is assessed on an annual basis from a representative investor perspective using a Net Present Value (NPV) criterion per asset, possibly adjusted for risk aversion. 20The NPV approach is a well-established tool in the literature to appraise and compare the economic profitability of assets available for investment and retirement. 21The representative investor perspective is at the core of the SD approach and has been motivated above (Section 2.1).
Decision criteria and loop.The NPV associated with each decision is computed using relevant costs (i.e., avoidable costs) and market revenues over a certain time horizon that all depend on the nature of the decision or the underlying asset (e.g., investment or closure, existing or new asset).
• Costs: We consider two types of fixed costs, the investment cost (CAPEX) and fixed Operations and Maintenance (O&M) costs.CAPEX is a sunk cost that cannot be recovered when an existing unit is retired.By contrast, fixed O&M cost is due when the unit is in operation but can be saved by decommissioning it.Some technologies also have a variable operating cost, that is fuel and carbon costs for thermal assets or charging cost for storage assets.• Revenues: Annual revenues accruing to all assets over their entire lifespans are computed using the hourly virtual short-term dispatch module (Section 2.2.2) with the long-term fundamentals 20 The per-asset NPV approach does not capture synergies across assets that could affect investment profitability and decision (e.g., through risk pooling) when considered from a portfolio perspective (e.g., Roques et al., 2008; Tietjen  et al., 2016).Here, we employ a per-asset NPV calculation for computational simplicity and because a portfolio-based calculation does not fit well with our representative investor approach (the portfolio would coincide with the entire asset fleet).We note this may provide an upper bound on entry-exit distortions relative to the first best as the portfolio-based calculation would better approach the complete market setting.Yet we believe this is without loss of generality for our results -if not the relevant situation to considernotably as Abada et al. (2017) argue that assessing investments in isolation is in line with the currently prevailing project finance approach and ''probably best fits the current market situation where long-term hedging possibilities essentially do not exist ''. 21Although there is an option value in deferring decisions to invest in new assets under uncertainty and investment irreversibility (e.g., Dixit and Pindyck,  1994; Rios-Festner et al., 2019), our approach makes no arbitrage between investing now or a few years later (i.e., no real-options valuation).However, we implement a procedure that captures some optionality for decommissioning decisions (see below).from the anticipation module (Section 2.2.1) as inputs. 22What matters is the stream of net revenues, that is short-term market revenues minus variable costs.
• Horizon: The profitability of a potential investment in a given year must be assessed over its whole lifetime.By contrast, the timing of a potential retirement decision is more complex, as temporary losses in the short term can be offset by larger gains in the long term. 23To capture this, we implement a simplified procedure whereby retirement occurs only if revenues do not cover fixed O&M costs both in the current year and over the asset's remaining lifespan.
The NPVs of all potential decisions in a given year are computed as described above and the investor picks up the one with the highest NPV per megawatt of capacity in absolute terms.The decision-making process thus places investment and retirement decisions on an equal footing and treats both types of decisions simultaneously. 24Once a decision is made, the asset fleet is adjusted for the corresponding capacity addition or withdrawal, which affects the economic profitability of all other units -be they installed or under consideration.That is, at each iteration of the loop, expected market revenues and NPVs of all assets are updated on the basis of the iterative evolution of the asset fleet.Importantly, the profitability of all previous decisions made in the current year is reassessed at each iteration: if an earlier decision becomes unprofitable because of some following decisions, it is called off; and only those decisions that stay profitable until the end of the iterative loop become firm and effectively materialize.
Additionally, note that we implement a standard modeling artifact to account for those years in the profitability assessment that extend beyond the simulation period.Specifically, we assume market revenues earned in the last year of the simulation period are duplicated and repeated over the following years until the entire asset lifetime is covered in the profitability assessment.Because the 'edge effects' induced by this artifact become increasingly prevalent as the end of the simulation period nears, our interpretation of the simulation results in Section 4 will essentially focus on the time window where they are less distorted (i.e., in the first part of the simulation period). 25inally, the iterative loop in a given year terminates when one of the two following conditions is met: either there is no profitable decision left, or a given state of the asset fleet (i.e., the number of units per technology, which is stored after each iteration) is reached for the second time-see Appendix C for an algorithmic description of the loop.The second condition preempts infinite back and forth and addresses indeterminacy issues due to unit indivisibility.Indeed, Appendix D shows that the long-term profit of a marginal investment in a given technology for a given state of the asset fleet can be interpreted as the total system cost function's gradient component with respect to installed capacity for this technology.Because this function is continuous and convex by assumption, the tâtonnement process implemented through Appendix G.2 shows that our results are robust to the choice of an alternative modeling artifact whereby the years beyond the simulation period are truncated (i.e., not accounted for in the profitability assessment).
this iterative procedure belongs to the class of 'steepest coordinate descent' methods, which ensure convergence close to the optimum (closeness depends on the capacity step size that reflects investment lumpiness, see Section 4.1.2).
Uncertainty and risk preferences.We adopt a simple approach to introducing uncertainty around the reference scenario produced by the long-term anticipation module (Section 2.2.1).We follow Neuhoff et al.  (2022) and bypass the explicit modeling of multifaceted uncertainty, e.g. on demand, commodity prices, regulatory changes and so forth. 26hat is, we consider an aggregate risk that bears directly on the discounted sum of net revenues for a given asset that we compute based on a run of the virtual dispatch module (Section 2.2.2) in the reference scenario, denoted r.Then we add exogenous noise around these endogenous, asset-specific r values.In our main case, we assume that asset-specific net revenues are uniformly distributed between 0 and 2r. 27n the face of uncertainty, we consider that the representative investor can exhibit different degrees of risk aversion, including risk neutrality.There are many reasons why investors and firms (or firms acting on behalf of investors) de facto behave as if they were risk averse, resulting in a higher utility from more stable profits.These include, inter alia, hedging demand, corporate risk management policies (e.g., financial and operational constraints) or costs associated with financial distress (e.g., Froot et al., 1993; Bessembinder and Lemmon,  2002; Willems and Morbee, 2010; Acharya et al., 2013; Jagannathan  et al., 2016).Additionally, in the electricity industry where assets are often capital-intensive with long lifetime, investment decisions are the result of careful profitability assessments, and one may intuitively expect risk aversion to prevail (e.g., Vázquez et al., 2002; Neuhoff and  de Vries, 2004; Abada et al., 2019).
There are different approaches to representing risk aversion, including risk-adjusted discount factors (e.g., in the spirit of the CAPM), coherent risk measures (e.g., a linear mixture of expected surplus value and conditional value-at-risk), and concave utility functions.Although the first two approaches allow for a more detailed and state-of-theart analysis of risk impacts, they deploy a heavier machinery than the third approach that is irrelevant given our rather crude representation of uncertainty.We thus assume that the investor preference for stable and secure profits is described by a concave von Neumann-Morgenstern utility function  .Additionally, as is standard in the related literature (e.g., Petitet et al., 2017; Fraunholz et al., 2023), we consider that risk aversion applies directly on the (distribution of the) asset-specific discounted sum of net revenues. 28e choose a functional form that satisfies the property of constant relative risk aversion.This property ensures that the coefficient of risk aversion does not vary with the economic value of the decision under consideration, which typically is of a different order of magnitude for investments and retirements.Specifically, following Petitet et al.  (2017),  is defined by with  the coefficient of (constant relative) risk aversion,  the random discounted sum of net revenues, and r = E{} by definition. 29Under risk neutrality ( = 0), the investor considers the mean of the net revenues distribution E{ ()} = E{} = r when computing the NPV of a given asset.Under risk aversion ( > 0), the investor considers the certainty equivalent of net revenues  ⋆ , that is the certain revenue that yields the same utility as the expected utility over the random distribution of revenues,  ( ⋆ ) ≡ E{ ()}.The certainty equivalent  ⋆ is decreasing with  and tends to r in the limit as  goes to zero.Analytical details are relegated to Appendix E.

Case study
In this section, we introduce and document our Californian case study.We first describe the fundamentals used to calibrate the model (Section 3.1) and then present the optimal simulation results derived from the GEP model (Section 3.2).All input data are available here: Zenodo/Dataset.Before we proceed, a short discussion on market design is in order.In practice, the Californian system is composed of a nodal electricity market with a soft offer cap at 1000 $/MWh, a mandatory resource adequacy requirement with no formal capacity market, an emissions trading system (ETS), and a renewable portfolio standard (RPS) program.However, we intend to leverage our stylized case study to represent a canonical energy-only market (EOM).Bearing in mind that our study is for illustrative purposes, we thus make several simplifications in terms of implementation.
We consider a zonal market with an hourly resolution over a 20year time period, focusing only on wholesale electricity (i.e., ancillary services are outside the scope of this paper).There is no offer price cap and the single hourly price can go up to the VoLL set at 15,000 $/MWh.Moreover, we model an isolated system (no interconnection) and do not represent the internal network ('copper plate' assumption). 30Although we do not formally account for the resource adequacy requirement, we set load scenarios in line with the 'one-in-ten' regulatory criterion (e.g., Pfeifenberger et al., 2013; CPUC, 2022; CAISO, 2023).Likewise, we do not represent the RPS program, but note that CPUC (2019) found the RPS constraint to be non-binding (the associated shadow price is zero) and decarbonization to be driven only by the constraint on emissions.Finally, the ETS is not explicitly modeled but we endogenously set annual carbon prices as the optimal shadow prices associated with annual emission targets.In sum, this stylized setup allows us to capture the essence of EOM outcomes and to purposely assess how they hinge on investor behavior assumptions. 31. Lebeau et

Calibration
We use and adapt data from three open-data sources: the integrated resource planning (IRP) exercise by the California Public Utilities Commission (CPUC, 2019), Ninja Renewables (Ninja Renewables, 2021), and historical data from the California Independent System Operator (CAISO, 2018, 2019).Below, we describe how we calibrate demand, supply and decarbonization parameters for both the optimization (GEP) and simulation (SD) models.
Demand.We consider that demand is fully inelastic to price variations and does not adjust to installed capacity.While this simplifies the model and its calibration, note that this is innocuous for the EOM performance we aim to appraise.Indeed, price-inelastic demand does not fundamentally alter the peak-load pricing logic inherent to the EOM (e.g., Joskow and Tirole, 2007; Joskow and Léautier, 2021). 32ere, gross load is exogenously given and assumed to increase linearly between two given data points given by CPUC (2019), namely 250 TWh in 2025 and 360 TWh in 2045. 33Similarly, we assume that distributed solar generation is increasing linearly between two given data points, 29 TWh in 2025 and 66 TWh in 2045, and we subtract it (on an hourly basis, see below) from the gross load to obtain the grid load.The result is graphically depicted in Fig. 4.
Next, we convert the above annual amounts into hourly time series.To do so, we start from two historical years -2018 and 2019for which we obtain hourly load from CAISO (CAISO, 2018, 2019)  and hourly capacity factors for wind and solar from Ninja Renewables (Ninja Renewables, 2021).These constitute our two representative scenarios capturing 'short-term' uncertainties (i.e., which resolve as real time nears).Working with historical data allows us to infer and then utilize realistic correlations between various sources of uncertainty such as load and weather.We then set the hourly load profile so as to reflect the 'one-in-ten' capacity adequacy criterion that applies in California.We proceed in five steps: First, we normalize both time series.Second, we set 2018 as the representative year for the one-inten-years peak and 2019 as the representative average year.Third, we scale the normalized 2018 profile homothetically in order to have a demand peak that is 15% larger than that in 2019.Fourth, we give a 10% and 90% probability to the processed 2018 and 2019 load profiles, respectively.Fifth, we finally scale the two profiles so that when weighted by their respective probabilities they sum up to the annual amounts shown in Fig. 4.
Supply.There are two types of technologies: 'exogenous' technologies, whose installed capacities are exogenously given and in line with planned evolution over time (e.g., by mandate or regulation), and 'endogenous' technologies for which entry and exit decisions are explicitly modeled.
Appendix G.3 provides a sensitivity with a flat load showing that our qualitative results are unchanged.
The set of 'exogenous' technologies consists of Combined Heat and Power (CHP), nuclear, existing and planned wind and solar (with shorthand 'E&P'), as well as geothermal, biomass and small hydro, which we group into one category (labeled 'Other RES'). 34Hourly availability factors for these technologies are set on the basis of data from CPUC (2019), which we translate into an hourly resolution using Ninja Renewables (2021) if necessary.Fig. 5 depicts the evolution of generation volumes over time for these technologies.
The set of 'endogenous' technologies is further divided in two groups on the basis of decisions available to the representative investor: existing fossil-fired dispatchable technologies -Peaker and Combined Cycle Gas Turbine (CCGT) -can only be decommissioned (i.e., no new investment is possible), whereas Solar and Storage technologies can also be invested into.The storage technology we consider is generic and has the characteristics of a lithium-ion 6-hour Battery Energy Storage System (BESS) with a 85% roundtrip efficiency.The technical and economic parameters of these four technologies are given in Table 1.
We make several realistic or mild assumptions to streamline the simulations and their interpretations.That is, they allow us to isolate the impacts of varying investor behavior assumptions and are largely inconsequential for the qualitative nature of our results.First, all 'endogenous' technologies have a common weighted average cost of capital of 8%. 35Second, they have a common lifespan of 25 years that is longer than the simulation horizon.Third, investments are realized with no build time (i.e., new capacities are built and start operations on the year the investment decision is made).Fourth, in our central scenario, all technologies have a common investment/retirement step size of 500 MW. 36Taken together, these assumptions essentially guarantee that investment and retirement decisions are not structurally tilted towards specific technologies.Taken individually, each assumption is mild and simplifies the anticipation and decision modules introduced in Section 2.2.37Although the first three assumptions have relatively straightforward and innocuous implications (e.g., the higher the WACC, the lower the investment volume), Section 4.1.2provides a sensitivity analysis for the fourth assumption as a basis for a general discussion of capacity unit discreteness/indivisibility for modeling outcomes.
Finally, we set the initial conditions -that is, the capacities installed in 2025 -so as to start the simulations from an equilibrium state that is congruent with the anticipation and decision modules described in Section 2.2.Specifically, CCGT, Peaker and Storage capacities are determined by running the GEP model for the year 2025 alone-they amount

Note:
The letters D and I refer to decommissioning and investment, respectively.For simplicity, Solar's variable cost and carbon intensity are assumed to be zero.In addition, all technologies are assumed to have no build time, the same weighted average cost of capital (8%), and the same lifespan (25 years) that extends beyond the simulation duration.
a Indicates average values over the simulation period and the symbol.
-Denotes parameters whose measurement is not straightforward due to yield and intertemporal use issues.
to 11, 19 and 10 GW, respectively.'Endogenous' Solar capacities for 2025 are initialized to zero since the existing and planned (E&P) fleet is already accounted for in the set of 'exogenous' technologies.
Decarbonization.We consider that the emission externality associated with electricity production is fully internalized through an annual carbon price signal that we endogenously set as the optimal shadow price associated with annual emission constraints derived from our GEP model.In doing so, we follow CPUC's modeling approach which considers annual emission targets without intertemporal flexibility.Specifically, we assume that the annual cap on emissions is decreasing linearly over time and we calibrate it with a linear interpolation between 2025 and 2045 emission levels.We compute the former as the emission level resulting from the initial brownfield fleet (31 MtCO 2 ) while we get the latter from CPUC (12 MtCO 2 ).Fig. 7 depicts the resulting emissions cap trajectory (left panel) and associated optimal price signal (right panel).
In our setup, the shadow price associated with the emission constraint is equivalent to the equilibrium price that would arise in a perfectly competitive allowance market (Montgomery, 1972).Since we do not explicitly model the ETS, we abstract away from intertemporal flexibility through banking, which may induce second-order changes in emission and price levels (e.g., Rubin, 1996; Schennach, 2000), as well as from price containment mechanisms, which may have an impact on investment decisions (e.g., Burtraw et al., 2022; Cason et al., 2023).Yet because we set the carbon price at its optimal values resulting from the GEP model, the deviations from cost-efficient entry-exit decisions that we study in Section 4.2 will largely be driven by investor behavior assumptions. 38

GEP results (optimal trajectories)
We run the GEP model with continuous capacity adjustment to determine the optimal capacity trajectories for 'endogenous' technologies (Figs.6-7).Regarding new developments, Solar and Storage reach 84 and 59 GW of installed capacity in 2045 respectively with a quasi-linear 38 See Ruhnau et al. (2022) for a review and comparison of carbon price impacts in electricity market models.trend. 39Regarding existing assets, Peaker capacity is reduced by 6 GW (down to 13 GW), whereas CCGT capacity remains unchanged.The annual emissions constraint is satisfied and binding every year with the shadow price of carbon rising over time (399 $/tCO 2 in 2045). 40nterestingly, note that only a few hours of load rationing occur in the first years and that they vanish afterwards, even as decarbonization progresses (Fig. 9, dashed line).This adequacy result is specific to our case study and is driven by the interplay between the massive buildout in both new Solar and Storage capacities driven by decarbonization targets, the presence of a brownfield fossil fleet with sunk CAPEX and relatively low fixed OPEX to be covered to remain economically viable, and a high value for the VoLL. 41ne may rightly wonder how Peakers are able to recover their costs in such conditions, i.e. in the absence of inelastic load rationing events with prices reaching the VoLL.One must first recognize that the formation of electricity prices (i.e., the system's hourly marginal values of electricity) in carbon-constrained systems with high shares of renewables and storage is complex and ''determined dynamically by demand and intertemporal storage decisions, breaking the static logic of 'merit order' with dispatchable generation'' (Ekholm and Virasjoki,  2020).In particular, prices can settle above the highest conventional generator's variable costs because of storage's roundtrip efficiency and intertemporal arbitrage, possibly forming ''price plateaus''. 42Moreover, prices can factor in long-term cost components when capacity additions and retirements are endogenous in the model (e.g., Mallapragada et al.,  2023).In this context, all conventional generators (including Peakers) pocket sufficient inframarginal rents to recover their fixed costs. 39This trend is essentially driven by the input data and foreseen evolution of load over time (Fig. 4). 40As a quick sanity check, we compare our results to those of CPUC: Solar and Storage capacities amount to 64.3 and 50.8 GW in 2045 respectively; Peaker and CCGT capacities are reduced by 4.2 and 1.8 GW over 2025-45, down from 8.6 and 16.2 GW respectively; and the carbon price is found to reach 403 $/tCO 2 in 2045.Although there are some quantitative deviations, our modeling results for a stylized system are qualitatively very comparable overall. 41This adequacy result is again in line with CPUC's where the shadow price associated with the Reserve Margin constraint is 63 $/kW-yr in 2026, 0 $/kW-yr in 2030 (i.e., the constraint is not binding) and 1 $/kW-yr in 2045. 42Recall that storage units are optimized over a one-month window with perfect foresight.Running the GEP model allows us to illustrate numerically how, in line with theoretical principles, cost recovery is ensured for all assets with no 'excessive' rents.Specifically, we compute the Cost Recovery Ratio (CRR), i.e. the ratio of net market revenues (price -variable costs) to fixed investment and operational costs. 43GEP column in Table 2 shows that each endogenous technology recovers exactly 100% of its costs.Recall that the economic viability of the initially existing (brownfield) assets only requires fixed O&M costs to be recovered (respectively 20 and 30 USD/kW-yr for peaker and CCGT) as CAPEX is sunk.This explains why existing Peakers recover only 32% of their fixed costs, which corresponds to the share of their fixed OPEX.Since Peakers are at the margin regarding total capacity -i.e., Peaker capacity is adjusted downward against new capacity additions -retained units just break even, exactly recouping their fixed OPEX with no extra rent to cover their CAPEX.By contrast, infra-marginal CCGTs recover 91% of their total fixed costs, which is sufficient to cover the 20% share of fixed OPEX but also to recoup a certain amount of CAPEX (albeit not in its entirety).

Results and discussion
In this section, we present our simulation exercise and discuss the results.First, we examine the conditions under which the SD energyonly market (EOM) model is able to replicate the optimum as defined 43 Formally, with the notations of Appendix A, for a given vintage of a technology  invested in year , one has , +  , .
Table 2 shows the CRR averaged over vintages, but note that CRR , = 100% holds for all vintages  for endogenous technologies.In Section 4.2, we compute and analyze CRR , across vintages.
in Section 3.2, and we discuss the impacts of unit indivisibility.Second, we explore how the simulated EOM outcomes deviate from the optimum when we relax these conditions.
Our quantitative analysis is based on the following indicators: • Capacity trajectories to assess how investments and retirements for each technology compare with the optimal ones over time, Since some simulation or sensitivity cases are computationally demanding (notably as we reduce the investment/retirement step size, see Section 4.1.2),we run simulations and present results only over the 2025-35 horizon.Yet note that this is largely innocuous for the validity of our analysis beyond this window given the relative 'monotonicity' of our case study, as evidenced by the quasi-linear trends for capacity trajectories in the optimal case over the 2025-45 horizon (Fig. 6).

Definition of idealistic assumptions
As discussed in the Introduction, it is often regarded as wellestablished that the long-run optimum can in principle be decentralized through competitive market prices.Although the prerequisites are regularly stated in general and concise terms (e.g., 'perfect markets', 'Arrow-Debreu economy'), some authors including Rodilla and Batlle  (2012), Newbery (2018) and Joskow (2008, 2022) establish more detailed lists that we transcribe below 44 : 1. Agents (i.e., buyers and sellers) are anonymous, atomistic and fully rational (i.e, price-taking and non-strategic behavior); 2. Agents have convex cost and utility functions (no nonconvexities, no economies of scale); 3. Capacity, generation and consumption levels can be adjusted continuously (no lumpiness); 4.There is perfect information with well-informed agents and no asymmetries; 5.There is a complete set of markets covering all relevant contingencies and over all relevant timescales, including markets for insurance; 6. Agents have risk-neutral preferences.
Let us translate these canonical assumptions to our modeling framework.The first is satisfied by construction since we consider one representative investor who behaves non-strategically and makes investment and retirement decisions on the basis of a competitive profitability assessment.The second is met by ruling out cost non-convexities and because we do not represent the demand side formally.The third does not hold since capacity increments and decrements are discrete by implementation in the SD model.As capacity indivisibility also holds in practice, its impacts on market outcomes are analyzed and discussed below (Section 4.1.2).
Next, because the decision-making process crucially hinges on available information, we break down the fourth assumption into two sub-assumptions: A1. Agents have perfect information on all exogenous parameters over the whole horizon (e.g., demand, generation costs).In particular, the carbon price is assumed to coincide with the optimal shadow price from the GEP model (see Section 3.1).A2.Agents have perfect information on endogenous future investments and retirements (see the anticipation module defined in Section 2.2.1).
Finally, we merge the fifth and sixth assumptions into a single one.This serves as an alternative equivalent formulation to the fifth assumption in our model that does not formally represent long-term and insurance markets (which are acknowledged to be incomplete or 'missing' for electricity in e.g., Rodilla et al., 2015; Newbery, 2016;  Abada et al., 2019; Keppler et al., 2022; Wolak, 2022).Specifically, we follow Newbery (2018) who argues -drawing on Newbery and Stiglitz  (1981) -that the theoretical requirement of market completeness can be substituted by the combination of rational expectations and riskneutrality.In our framework, the merged assumption below thus allows us to capture both risk aversion and market incompleteness:

A3. Agents have risk-neutral preferences.
As explained in Section 2.2, when A1, A2 and A3 hold, the representative investor behaves rationally with perfect information, fully anticipates its future decisions and understands the interplay with its current decisions, and implements the optimum in the spirit of a rational expectations equilibrium.Before relaxing A1, A2 or A3, we turn to the issue of capacity indivisibility.

EOM outcomes with capacity indivisibility
To explore the impacts of discrete vs. infinitesimal capacity unit sizes, we run sensitivities with respect to the unit step size around the reference of 500 MW used in the main analysis (Section 4.2).Specifically, we assume that A1-A3 hold and consider step sizes of 250, 44 These authors do not claim to be exhaustive, and neither do we.750 and 1000 MW.These should not necessarily be seen as the typical sizes of single investment projects -especially for renewables which are usually smaller -but should rather be interpreted as the volumes of disclosed projects that induce market participants to update their longterm anticipations and adjust their investment and retirement decisions accordingly, our subject of study in this paper.
Fig. 8 depicts the simulated capacity trajectories.Intuitively, the smaller is the capacity step size, the closer the simulated trajectories for an EOM are to the optimal GEP solution on average.As the step size increases, the deviation from the optimum is characterized by a delay in Solar and Storage investments and in the fossil fleet phaseout.Indicators given on average in Table 2 or on an annual basis in Fig. 9 further reveal that a larger step size is conducive to larger total system cost, wholesale prices, emission levels and loss of load. 45ndivisibility issues in the electricity industry have long been recognized and analyzed in the literature, already in the seminal contributions by Boiteux (1949, 1960) and Williamson (1966) as well as in ensuing discussions (e.g., Andersson and Bohman, 1985).Yet they address these issues in a regulated utility environment and mainly focus on practical pricing policy considerations.More recently, Keppler (2017) reviews the literature and discusses the implications of indivisibility in a market context, observing that it leads to underinvestment when coupled with inelastic demand.Our results add to this literature by illustrating how indivisibility also hampers the joint dynamic of fossil phaseout with investment in renewable and storage units to achieve decarbonization.
In fact, this reveals a coordination (or circularity) issue between new entrants and existing (and possibly exiting) assets.Recall how the SD model's decision module proceeds, gradually selecting the most profitable available asset entry or exit up to an end state characterized by a zero-profit condition (Section 2.2.3).Incentives thus decrease in size as the iterative process progresses, and coordination issues start to materialize as the end state nears.For instance, in the neighborhood of optimal capacity, a potential new entry without a simultaneous exit typically leads to over-capacity that deters the actual entry decision.This is notably true for Storage that has a large contribution during scarcity hours if Peakers are not retired in a coordinated way.In turn, this further affects Solar that has to be associated with Storage to mitigate the revenue cannibalization effect.
Importantly, the materiality of this effect depends on the sensitivity of the entry and exit signals near the optimum rather than on the capacity step size relative to the overall size of the system.As mentioned in Section 2.2.3, the long-term profit of a marginal investment in a given technology can be interpreted as the total cost function's gradient component with respect to the installed capacity for this technology.Because these gradients are steep and asymmetric in the neighborhood of the optimal capacity, this effect remains tangible even in relatively large systems. 46As is the case here, this effect translates into extreme price sensitivity around optimal capacity, with prices jumping from a few hundreds to a few thousands $/MWh in some peak hours.This echoes previous results in the literature highlighting that the EOM is intrinsically prone to ''erratic'' (Cramton and Stoft, 2005)  price movements or ''discontinuity'' (Kraan et al., 2019).
It is worth making one final observation regarding computational time, which increases steeply as capacity step size decreases.Specifically, in our setup, reducing step size from 1000 (resp.500) to 500   (resp.100) MW increases computational time by a factor of 3 (resp.5).
In a loose sense, this illustrates how the tâtonnement process is computationally demanding, especially as capacity step size gets smaller to approach GEP-style optimization with continuous capacity adjustment.

EOM outcomes with relaxed assumptions
Let us now relax assumptions in turn and separately, keeping the capacity step size constant at 500 MW, and compare simulated EOM outcomes with those in our reference case where A1-A3 jointly hold.For brevity, the case where we drop A1 (Case 1) is relegated to Appendix F. Indeed, the impacts of downward biased anticipations of future carbon prices in Case 1 are qualitatively similar to those when we consider risk aversion and drop A3 below (Case 3).Table 3 contains the definition of the different cases and the associated assumptions.

Case 2: Biased anticipation of future entries and exits (only A2 does not hold)
In Case 2, we drop A2 while retaining A1 and A3.That is, we consider that investors and asset owners make incorrect anticipations about future investment and retirement decisions.Specifically, we consider three cases.In the first case, labeled 'static', there is no anticipation of future entries and exits (see Section 2.2.1).In the other two cases, future entries and exits are anticipated but in contrast to the perfect reference case, anticipations deviate from the optimal capacity trajectories: in the 'overestimate' (resp.'underestimate') case, we skew positively (resp.negatively) the optimal future dynamics of Solar and Storage development, all else being equal.For illustration, we choose a + (resp.-) 40% factor bias on annual capacity additions. 47he 'static' and 'underestimate' cases exhibit faster development for Solar and Storage early on relative to the perfect anticipation case (Fig. 10).This is because the anticipation of no or lower future capacity additions magnifies expected future market revenues and in turn the incentive to invest today.That is, future capacity additions that will have a dampening effect on the whole price distribution and number of scarcity hours are not accounted for in full when assessing future asset earnings.As a result, cost recovery is lower than initially expected and than in the perfect case for all assets, and realized total costs are also higher (Table 4).This situation also illustrates a coordination issue across investors that is inherent to an EOM (e.g., implicit herding behavior when investment conditions are good overall, difficulty to anticipate competitors' investment decisions) and possibly conducive to boom-and-bust cycles (e.g., Arango and Larsen, 2011; Hill, 2021). 48ver-investment is particularly salient for the first investment vintages (i.e., in the first years of the period) while the deviation is then gradually reduced over time with capacities being close to their optimal levels in the final year (Fig. 10).Note that this convergence results from a modeling edge effect due to the model's finite horizon (see Section 2.2.3): as the end of the simulation horizon nears, ever less future decisions have to be anticipated, which mechanically reduces 48 Despite initial over-investment, a cycle does not emerge in our case study, notably because demand is structurally growing over time.This strongly mitigates potential under-profitability of assets normally linked to over-investment, and reinforces a natural asymmetry between investment and retirement decisions -the former is evaluated against both fixed OPEX and CAPEX whereas the latter only against fixed OPEX (i.e., economic conditions must turn out to be asymmetrically worse than expected to justify decommissioning).Finally note that excess entry occurs here although we neglect build times.In practice, build times exacerbate investment coordination failure and the associated 'contagion' effect (as it takes some years for new plants to come online and to alter fundamentals). the impact of the anticipation bias.Hence, our results can only be meaningfully interpreted in the beginning of the simulation period.
Although carbon emissions are reduced early on as a by-product of initial over-investment in low-carbon assets (Fig. 12), we emphasize that this leads to higher system costs and creates a risk of economic unsustainability further down the road (as the first investment vintages turn out not to recoup cost in full due to biased anticipation, see Fig. 11).Symmetrically, the 'overestimate' case exhibits slower development for Solar and Storage early on relative to the perfect anticipation case (Fig. 10).Anticipating inflated capacity additions in the future reduces expected future prices and weakens investment incentives today, hence the under-investment.But because actual capacity development happens to be lower than anticipated, cost recovery ratios are well above 100% for new assets (Fig. 11).At the same time, under-investment threatens security of supply (the LOLE reaches close to 10 h/yr) and drives up prices (Table 4).Under-investment is also not conducive to reducing emissions, jeopardizing decarbonization targets.

Case 3: Risk aversion (only A3 does not hold)
In Case 3, we drop A3 while retaining A1 and A2.That is, we consider that investors and asset owners are averse to risk about future revenues and apply the certainty-equivalent decision-making criterion presented in Section 2.2.3. 49Specifically, we consider that future market revenues  are uniformly distributed between 0 and 2r. 50Because we lack empirical guidance to discipline the selection of a relevant risk aversion coefficient , we follow Petitet et al. (2017) and consider a range of integer values for  between 0 and 3, where  = 0 coincides with risk neutrality whereas  = 3 is deemed to capture a high degree of risk aversion. 51As we will see, the value of  affects our quantitative results monotonically, hence not their qualitative nature.
Capacity trajectories in Fig. 13 exhibit a pattern of under-investment in Solar and Storage in conjunction with a delay in fossil phaseout.Risk aversion also has a noticeable impact on total system cost, ranging from + 3% ( = 1) to + 14% ( = 3).Closely looking into the central case  = 2 shows that the delay in investment and retirement yields net savings on fixed costs (both fixed OPEX and CAPEX) of -18%, but increased fossil generation hikes total variable cost by 20% and the cost of rationing rises by + 4%, totaling a net total cost increase by circa 7%.
Table 5 shows other important insights for an EOM under risk aversion.First, decarbonization is delayed.This is because demand, which is rising over time because of electrification in our case study, remains in an inadequate part served by fossil units instead of low-carbon units.Second, security of supply deteriorates, with both under-investment and the LOLE increasing with the degree of risk aversion (LOLE ≥ 10 h/yr for  ≥ 2).Third, price spikes (including hours of VoLL pricing) entail significant financial transfers between consumers and producers.Higher prices for consumers also result from risk premiums that producers need to secure to run operations.
The effect of risk aversion has been studied in the literature with numerical models, essentially with a focus on security of supply and under-investment, and in turn on the scope and design of capacity 49 Recall that our approach to modeling risk aversion is in part grounded on the absence of sufficient, adequate hedging instruments (see Section 4.1.1).
50 Appendix E shows that our results are qualitatively unaltered when we change the variance and the type of the probability distribution.One may also conjecture that our results would be amplified if we considered stronger forms of uncertainty and corresponding preference representation theorems.Deep uncertainty indeed prevails in the long run, especially in the context of decarbonization where the long-term energy mix, market conditions and price distributions remain largely elusive for now (e.g., Keppler et al., 2022).
51 Following an applied study by RTE (2018) for France, we consider  = 2 as a central value.remuneration mechanisms (e.g., Petitet et al., 2017; Ousman Abani  et al., 2018; Tao et al., 2020; Fraunholz et al., 2023).Our results support previous findings and add to the literature on decarbonization aspects.Specifically, under-investment in low-carbon assets resulting from risk aversion hinders the existing fossil fleet phaseout which, especially when combined with growing demand for electricity, tends to delay decarbonization further. 52

Summary of modeling results and policy implications
Practical limits of price-based coordination.Several conclusions emerge from our modeling approach and results.First, the implementation and unpacking of the EOM framework clarify the assumptions needed to reproduce the optimal long-run outcome from a GEP model.By contrast, these theoretical assumptions and their practical implications are often implicit or remain elusive in the literature.Notably, we highlight the high level of informational and computational complexity associated with optimal anticipations of all relevant future market fundamentals and future entry and exit decisions.Assuming optimal anticipations, rational decision-making and risk neutrality, we further illustrate how the convergence of the EOM outcomes towards the optimum occurs only when the capacity step size is small enough -ideally infinitely smalland highlight the issue of lumpy investment and retirement decisions.While the total system cost is not very sensitive to unit indivisibility, this causes a coordination issue between new low-carbon investments and the phaseout of fossil assets that affects carbon emissions more markedly.
52 One may also conjecture that capturing the feedback loop between price and demand dynamics in the long run (not represented in our framework) would exacerbate the shortcomings of the EOM for decarbonization.For instance, high or volatile electricity prices could be detrimental to electrification, either directly by deterring investment in electrical equipment or indirectly by limiting public support if low-carbon generation investment does not keep pace.In the second part of our analysis, we relax these assumptions in isolation and find significant deviations (see Table 6 for a summary of quantitative indicators).Intuitively, the direction of the deviation in investment depends on the direction of the bias in the anticipation of future market entries-an upward bias relative to the optimum leads to a downward bias in anticipated market revenues that delays investment decisions, and vice versa.Moreover, on top of the standard result of under-investment when investors are risk averse, our joint modeling of entry and exit decisions shows how decarbonization risks being delayed if the existing fossil fleet is not pushed out of the market economically by low-carbon entrants in due time w.r.t.targets or cost-efficient pathways.
Potential benefits of complementary quantity-based coordination.Our modeling framework constitutes a good basis for extensions to explore changes in market design that could improve long-run coordination and risk-sharing mechanisms (e.g., Joskow, 2022; Keppler et al., 2022).Different add-ons such as a long-term contracting module could be plugged into our core model, of which a variety of designs could be assessed and compared.To illustrate, let us consider the case where assumptions A1-A3 hold and the optimal fossil phaseout is exogenously enforced, although we do not specify at this stage through which mechanism it is implemented.A cursory examination of the results in Fig. 14 shows that capacity trajectories are closer to the optimum.Importantly, this holds not just for Peakers whose phaseout is exogenously driven, but also for Solar and Storage.This shows how explicit coordination through quantities has the potential to complement implicit coordination through prices and improve on market efficiency in the long run.
Of course, the extent to which this potential can be tapped into depends on the implementation mechanism.One may for instance think of a situation where the regulator steers the decommissioning trajectory through an auction scheme, similarly to what is done in Germany (e.g., Tiedemann and Müller-Hansen, 2023).But various alternatives could also be investigated, notably regarding the form of compensation payments awarded in the auction or the type of planning used to define the phaseout path enforced by the regulator-in a bid to avoid the pitfall of substituting 'imperfect' market mechanism by 'imperfect' regulation.Similar crucial design options also exist in the case of long-term contracting mechanisms for investment, be they government-backed contracts issued through public auctions or private contracts stimulated through the provision of public guarantees or through an obligation on retailers-see CEPR ( 2023) and the Conclusion for short reviews.

Conclusion
In these times of renewed debates on electricity market design in the context of decarbonization, the EOM has often been criticized for various, at times dubious reasons.For instance, the increasing share of generation with (near) zero short-term marginal cost has been blamed for reducing prices and (expected) asset profitability.However, this merit-order effect is either transitory (Antweiler and Muesgens, 2021)  or caused by inadequate policy choices (Brown and Reichenberg, 2021)  rather than reflective of a limitation inherent to the EOM.In fact, price distributions are more likely to change in shape than to be lowered on average (e.g., Ekholm and Virasjoki, 2020; Mallapragada et al., 2023).Rather, our results suggest that a key issue with the EOM paradigm is that long-run efficiency holds only under a set of idealistic assumptions.When one of these preconditions is not met, entry-exit coordination through market price signals alone is insufficient and deviations from the optimum occur (see Section 4.3 for a summary).This notably leads to higher electricity prices, lower security of supply, and delayed decarbonization w.r.t.cost-efficient transition pathways.
Our paper offers several promising avenues for future research.First, our modeling framework can be enriched by activating or developing new features, such as build times or a finer-grained representation of long-run uncertainties.Second, it can also be extended in various ways to inform ongoing policy debates about electricity market design reform options.For instance, assessing and comparing design alternatives for a long-term contracting module -as delineated e.g. in Joskow (2022), Keppler et al. (2022), Wolak (2022) and Fabra (2023) -would be particularly relevant.This includes various approaches to auction design (e.g., Iossa et al., 2022; Fabra and Montero,  2023), contract design (e.g., Billimoria and Simshauser, 2023; Newbery, 2023; Schlecht et al., 2024) and planning (e.g., Corneli, 2020;  Anderson and Zachary, 2023).In this respect, accounting for realistic behavioral, informational and structural assumptions will be of the essence.

CRediT authorship contribution statement
where (B.5) tracks the number of units per technology over time and (B.6) imposes that each endogenous investment can be associated with a decommissioning decision during its lifespan.
A. Lebeau et al.Each decision variables can be constrained in an ad-hoc manner with an upper/lower bound or with a specific value.This feature is used to model the existing fleet for which  can be fixed at the beginning of the simulation (the initial fleet described in Section 3) and  + can be constrained to 0 afterwards if the technology is not available for new developments.

Appendix C. Description of the decision algorithm
This Appendix sketches the structure of the investment and decommissioning decision algorithm for the representative agent in our System Dynamics market simulation module.

Appendix D. Convergence of the simulation model
This Appendix lays out some theoretical considerations regarding the convergence of the simulation model.For clarity and without loss of generality, we consider a simplified setup with one representative year, one weather scenario, continuous capacity adjustments, full availability, no storage and no fixed O&M costs.This simplified case helps build Combining the above equations, it comes where   is the long-term profit.We thus see that the normalized long-term profit calculated for a given technology in an out-ofequilibrium state -in terms of installed capacity -corresponds to the Lagrangian functional's gradient component with respect to the capacity installed for this technology.Therefore, the iterative procedure underpinning the simulation model in Appendix C can mathematically be interpreted as a 'steepest coordinate descent' algorithm in an ideal case (see Boyd and Vandenberghe, 2004; Nesterov, 2012).While it is not the most computationally-efficient approach to solve this linear problem, it does have the merit to be meaningful from an economic point of view as it simulates the tâtonnement (groping) process of merchant entry-exit decisions.In practice, its level of accuracy depends on the number of iterations, the step size and the lower bound of the absolute value of the first derivative (i.e., the Lipschitz constant) that can be high (and asymmetric) close to the equilibrium, notably because of the high value of the VoLL.
Importantly, the GEP model and the optimization solver we use to run it allow us to determine the optimal outcome that can in principle be obtained with our simulation model.In Section 4.1.2,when all assumptions A1-A3 jointly hold, we can verify that total system costs decrease with the capacity step size (see Table 2) and are always close the optimum-namely, +0.34% with a 250 MW step size (the smallest one we manage to run with practically reasonable computational time) and +0.46% with a 500 MW step size (the reference case).Additionally, note that by the same token as in Case 2 (Section 4.2.1),all the capacity trajectories converge towards the reference case with unbiased carbon price anticipation at the end of the horizon due to a factitious edge effect (i.e., anticipations are by construction less and less biased the nearer the end of the simulation period due to the modeling artifact whereby the last year of the simulation period is repeated until assets' lifetimes are covered in whole).To illustrate this further, whereas emissions are equal across all CAGRs on the last year of the simulation period (because the installed asset fleet and market conditions are the same), the delay induced by a lower CAGR results in higher emissions over the whole period (Table F.1).
Finally, the cost recovery analysis reveals extra revenues for all asset types, which are increasing with the size of the anticipation bias (i.e., decreasing with the CAGR).This is because the realized carbon price happens to be higher than anticipated, which increases the realized price of electricity on average and is overall economically beneficial across the whole fleet.Intuitively, Fig. F.2 shows that this effect is more pronounced early on in simulation period (i.e., when a given anticipation bias has a greater impact on entry and exit decisions, all else being equal).

Appendix G. Robustness checks
This Appendix provides three sensitivity checks to ensure that our results are robust to two modeling assumptions-namely, the frequency of anticipation updates with the GEP module (G.1) and the treatment of years beyond the simulation horizon (G.2) -and to one specific characteristic of our case study -namely, the shape of the long-term load trend (G.3).

G.1. Frequency of anticipation updates
Here, we assess the impact of the update frequency of long-term entry-exit anticipations with the GEP module (Section 2.2.1).Specifically, we compare simulated capacity trajectories without anticipation updates (as is the case in the main text) and with biennial anticipation updates (based on a new run of the GEP starting from the simulated contemporaneous state of the asset fleet) in the reference case where assumptions A1 to A3 jointly hold with a capacity step size of 500 MW.Results are shown in Fig. G.1 and exhibit second-order absolute differences, respectively 0.31%, 0.01% and 2.89% for Peakers, Solar and Storage in relative terms on average across years.

G.2. Treatment of years beyond the simulation horizon
Here, we assess the impact of the modeling artifact used to deal with those years beyond the simulation horizon.Specifically, we compare simulated capacity trajectories with duplication of the anticipated net revenues earned in the last year of the simulation period (as in the main text, see Section 2.2.3) vs. truncation of the years beyond the simulation horizon (i.e., de facto assuming that total net revenues exactly offset amortized fixed costs over those years).Fig. G.2 shows the results in the reference case where assumptions A1-A3 jointly hold with a capacity step size of 500 MW.Differences are small, respectively 0.71%, 0.14% and 0.30% for Peakers, Solar and Storage in absolute relative terms on average.Moreover, as a sanity check, we observe that both methods yield the same outcome in the final year where they are equivalent by construction.

G.3. Increasing vs. flat load trend
Here, we assess the impact of the shape of the long-term load trend with different capacity step sizes.Specifically, we compare simulated capacity trajectories with a linearly increasing load as calibrated in Section 3.1 (Fig. 4) vs. flat (Fig. G.3) load over time.Quantitatively speaking, the overall picture is of course fundamentally different with a flat load from that with a relatively strong increase in load pushed by electrification as in the main text.In particular, the overall volume of optimal investment is lower for each technology (12 vs. 30 GW for Solar, 9 vs. 24 GW for Storage).Qualitatively speaking, however, our qualitative results are unaltered with a flat load.Specifically, when assumptions A1-A3 jointly hold, simulated capacity trajectories are closer to the GEP-optimal ones as we reduce the capacity step size

Fig. 1 .
Fig. 1.Modeling framework overview.Note: The linkage between the GEP and SD models represented by the orange arrow is optional.

Fig. 2 .
Fig. 2. Causal-loop diagram of the SD market model.Note: Elements in blue are endogenously determined and updated at each iteration of the loop delineated by blue arrows.Inputs in orange can be taken from the GEP model, whereas those in black are exogenously calibrated.
A.Lebeau et al.
Carbon emissions to track the progress of decarbonization, • Total cost as an overall cost efficiency indicator, • Cost recovery ratios (CRR) as technology-specific capacity remuneration indicators, • Average marginal cost, interpreted as an average price, as an affordability indicator, • Loss of load expectation (LOLE) as a system-wide capacity adequacy indicator.

Fig. 8 .
Fig. 8. Capacity trajectories with different capacity step sizes (EOM with A1, A2 & A3).Note: Although capacity trajectories are not perfectly ordered by capacity step size (they intersect for some years), the ordering holds on average (in terms of distance to the GEP trajectory).Beware of the scale for Peakers.
A.Lebeau et al.

Fig. E. 2 .
Fig. E.2.Certainty equivalent to expected revenue ratio under different modeling assumptions.
A.Lebeau et al.

Fig. G. 3 .
Fig. G.3.Capacity trajectories with a flat load, risk neutrality and different capacity step sizes.

(
Fig. G.3).Additionally, dropping A3 (i.e., assuming risk aversion), the results shown in the main text are magnified since the fossil fleet is not phased out and no new investments are triggered (Fig. G.4).

Table 1
Technical and economic parameters for 'endogenous' technologies.

Table 3
Definition of studied cases and associated assumptions.

Table 6
Summary of average indicators across simulations (range of values is provided).

Table A
Power capacity of technology  [MW/unit]  ,,,ℎ Production of technology  in year , weather scenario  and hour ℎ [MW]  ,,,ℎ Power charged into technology  in year , weather scenario  and hour ℎ [MW]  ,,,ℎ State of charge of technology  in year , weather scenario  and hour ℎ [MWh] ∈ ∑ ℎ∈   ⋅  ,,ℎ ≤   .(B.7) , and its logic and resolution extend to more complex cases.If we consider the representative year as a steady state, our optimization problem can be interpreted as a long-term cost-minimization problem that writesmin {  },{ ,ℎ },{ ℎ } (  ,  ,ℎ ,  ℎ ) = ∑  ⋅  ℎ ( ℎ ),and ,ℎ ≤  ⋅   ( ,ℎ ),  ,ℎ ≥ 0 ( ,ℎ ),  ℎ ≥ 0 ( ℎ ),   ≥ 0 (  ),where the variables within parentheses denotes the constraints' dual variables.For any given positive values of {  }  > 0 (i.e., not necessarily optimal), the gradient component with respect to   of the Lagrangian function (of all primal and dual variables) is given by If we minimize  for these given {  }  (i.e., we solve the dispatch problem for a given capacity vector), we can define the set of hours  +  where technology  is infra-marginal and we get from the dispatch problem's KKT conditions intuition∈  ,ℎ =  ( λℎ −    ).