Browsing by Author "Edge, Robert"
Now showing 1 - 2 of 2
- Results Per Page
- Sort Options
Item Empowerment as a Task-Agnostic Measure of Domain Competence(2020-12) Edge, RobertA ubiquitous challenge for humans and learning agents is the ability to measure and forecast how competent they can become in a specific domain. The problems created by the inability to forecast and measure achievable competence are manifold. We don't know who will be good at which jobs - we don't know which problems a machine learning architecture can solve, we don't know whether other agents might be better, etc. Performance measures exist in all sorts of domains: e.g. video games, athletics, academics, etc. that traditionally capture task-specific performance heuristics such as points accrued, time remaining, accuracy, etc. Assessment is problematic because we don't have the ideal battery of tests, and the time-cost of extensive testing on all plausible tests is prohibitive. In reinforcement learning we desire that agents are able to learn reasonable behavior on novel tasks in new environments, but it is unclear on how to best design tasks and scheduling to provide the agent with a general understanding of its capabilities. These quantities based on task heuristics may not even be appropriate for measuring an agent’s general ability if they do not accurately reflect the agent’s true goals, especially in environments with multiple available tasks. What does it really mean to be competent in an environment? Rather than domain-specific heuristics, a more fundamental notion of skill is the agent’s ability to understand, predict and control their environment. While we normally impute a player’s capabilities indirectly from their score or rank, in this dissertation we show that it is possible to create a direct measure of a player’s capabilities via the empowerment measure. We then use this measure to show the value of using a better universal objective in the context of reinforcement learning. Navigation is a task where it is advantageous to understand, predict, and control the environment. Methods for localization using information-theoretic quantities focus on where to sample signals to reduce uncertainty, rather than use the agent’s understanding of its own capabilities to understand where it might fail. We demonstrate a proof of concept using empowerment to predict navigation failure, and also how it can be used to produce a safer route. This measure is poised to have broad impact across multiple domains, including RL, education, and entertainment.Item Predicting Player Churn in Multiplayer Games using Goal-Weighted Empowerment(2013-09-09) Edge, RobertThe extent to which players feel compelled to play a game is a primary factor in determining that game's success. Using ideas from self-determination theory, we propose that the drive to play games is related to player’s ability to exercise their mastery drive. In self-determination theory, increasing one’s control is intrinsically rewarding, however it is difficult to instantiate this theory to make concrete predictions of when a player will quit a game early in favor of another activity. The problem of predicting within-game churn events involves modeling and predicting a player's motivational state to remain in a game. We combine new motivation theory ideas with machine learning methods to develop a computational model that postulates that player’s satisfaction is directly related to their perceived locus of control, extrinsic vs. intrinsic, and hypothesize a set of measurable signals that mediate a player's locus of control. An agent will continue goal pursuit within the game based on its "empowerment", which is essentially its ability to predict the future based on its own actions. This indicator of locus of control accounts for both the feasibility of the goal and whether expected progress toward a selected goal is being achieved. A player's behavior can be used to estimate their empowerment by correlating their continuation of goal pursuit with measures of their expectations of progress and goal-achievability. We apply these concepts and build a model to predict player when players will quit a game early for the multiplayer game, Dota2.