Date

2013

Document Type

Thesis

Degree

Master of Science

Department

Computer Science

First Adviser

Muñoz-Avila, Héctor

Abstract

Explaining anomalies in a stochastic environment is a complex task and a very active research field. In this thesis we present our contributions to this fascinating topic by introducing the concept of Stochastic Explanations, which associate to any unexpected event or anomaly a probability distribution over the possible causes of the anomaly. In this thesis, we present the EXP_GEN agent, which uses stochastic explanations to explain anomalies, implementing a synergy of Case-Based Reasoning and Reinforcement Learning mechanisms to, respectively, store and retrieve as cases (event, stochastic explanation) pairs, and to learn the probability distribution of the stochastic explanation for each anomaly. We claim that an agent using stochastic explanations will react faster to unexpected events than an agent that uses a deterministic approach to explain anomalies. We compare the performance of EXP_GEN against an agent which utilizes a greedy heuristic to explain anomalies.Unexpected events should be considered as a huge opportunity for an intelligent agent to learn something new about the environment. Providing explanations to these events is just the first step: we must reutilize the previous experience and explanations to avoid making the same mistakes in the future. For this purpose, we present the GENERATE_GOAL+ algorithm, which replaces the basic goal generation mechanism of the EXP_GEN agent. This algorithm takes into consideration the previous mistakes made by the agent and uses them to generate better goals. In this way, the agent is effectively learning from mistakes, improving its performance over time. We consider the ability to learn from one's own mistakes crucial to the implementation of more complex intelligent agents in the future. In our results, we show how the EXP_GEN agent that uses GENERATE_GOAL+ greatly outperforms the same agent that uses naïve goal generation.To show the effectiveness of learning from mistakes, we also present an enhanced version of the GENERATE_GOAL+ algorithm, called CombatPlan. Here we show how the EXP_GEN agent can handle broader and more complex scenarios and learn effectively by the mistakes it makes. To show this, we train the agent for a certain number of iterations, so that it makes mistakes and explains them. We then show how the performance of the agent on the test scenario is directly related to the amount of training performed, or in other words, to the amount of explanations generated during the training phase.

Share

COinS