Geração procedural de conteúdo utilizando aprendizado por reforço com design de iniciativa mista e entropia como feedback de recompensa

Abstract

Over the years, video games have become one of the main ways of entertainment in the digital environment and are often used in the machine learning research. In machine learning, we can find different branches, among them, reinforcement learning, which is commonly used to train agents to play games. Currently, there are many games that use procedural generation methods to generate some content in order to increase the player’s experience. Recently, academic articles have approach the problem of procedural generation of content in games though reinforcement learning. In this work, we investigate how we can approach procedural content generation with reinforcement learning and mixed-initiative design. A second question discussed here is how we can use metrics to evaluate the diversity of the generated level. Our proposal has as its main hypothesis to use scenario models, provided by an expert human in level design, for the reinforcement learning agents in order to generate new scenarios. The levels provided by the specialist are separated into segments or blocks that are used to compose the new scenario structures. Also, a new reward function based on the use of entropy was proposed to measure the diversity of the generated scenarios. Initially, we trained our model for three different 2D Dungeon crawlers game environments. We analyzed our results through the value of the entropy, and it shows that our approach can generate levels with a wide diversity of segments. A second analysis of the results is through expressive range, to evaluate the expressivity of the levels using linearity and leniency metrics.