Document Type



Doctor of Philosophy


Industrial Engineering

First Adviser

Defourny, Boris


This thesis concerns distributionally robust Markov decision processes for multistage stochastic programming problems. Despite recent advancements in algorithms for multistage stochastic programming problems and improvements in high-performance computing, different kinds of constraints still challenge the use of stochastic programming. The focus of this thesis is to design Markov decision processes and propose algorithms to solve the multistage generation capacity expansion model for the sake of economy on computation and fewer requirements on the properties of the problem compared to stochastic programming. It provides quantitative models and efficient algorithms to support decision-makers with rigorous and robust suggestions that capture the uncertainties in generation planning problem.The first problem of this dissertation is concerned with a multi-objective approach to the management of Opt-in distributed energy resources. Loads, dispatchable resources, non-dispatchable resources, and energy storage are modeled in this problem. Uncertainty in load and non-dispatchable generation is considered using scenarios. Both the equity of network access among agents over time and the overall deviation from the injection schedule are considered in the objective. Numerical results are provided to illustrate the tradeoff between these two objectives and choices among dispatchable and non-dispatchable resources.The second problem of this thesis is concerned with the impact of gas network disruptions on dual-firing power generation and the value of introducing dual-firing power generation. Uncertainties in gas price, oil price, electricity price are considered and the log-prices are represented by mean-reverting processes. A stochastic optimization model is formulated to analyze the value of dual-firing generator asset and the sensitivity of the value of the dual-firing generating unit to the gas network availability parameters. An approximate dynamic programming structure is applied efficiently to solve the Markov decision processes optimization model in seconds. An equivalent formulation in the framework of multistage stochastic optimization with structural results, lower bounds and upper bounds on the value function are obtained.In the third problem considered in this thesis, a closed-form solution for the good-deal reweighting probabilities is proposed for the finite state and finite action distributionally robust Markov decision processes (DRMDPs) with good-deal risk measure. Numerical results are presented to show the differences among the performances of risk-neutral Markov decision processes, minimax Markov decision processes and DRMDPs with good-deal risk measure. In the last part of this thesis, distributionally robust Markov decision processes with good-deal risk measure are proposed for the multistage stochastic generation capacity expansion problem. This model takes the decision maker's risk attitude, uncertainties in electricity demand and fuel price, and sequential decision making into account. An adjusted fitted value iteration algorithm and a discretization approach are introduced to solve this model. We also show that how the closed-form solution provided in the third problem substantially decreases computation time in the process of solving problems in practice.

Available for download on Friday, February 05, 2021