Dynamic-object-aware simultaneous localization and mapping for augmented reality applications

Abstract

Augmented Reality (AR) is a technology that allows combining three-dimensional virtual objects with an environment predominantly real in a way to build a new environment where both real and virtual objects can interact with each other in real-time. To do this, it is required to nd the pose of the observer (camera, HMD, smart glasses etc) in relation to a global coordinate system. Commonly, some well known physical object, called marker, is used to de ne the referential for both virtual objects and the observer's position. The Simultaneous Localization and Mapping (SLAM) problem borns from robotics community as a way to build truly autonomous robots by allowing they to localize themselves while they build a map of the observed scene from the input data of their coupled sensors. SLAM-based Augmented Reality is an active and evolving research line. The main contribution of the SLAM to the AR is to allow applications on unprepared environments, i.e., without markers. However, by eliminating the marker object, we lose the referential for virtual object projection and the main source of interaction between real and virtual elements. Although the generated map can be processed in order to nd a known structure, e.g. a predominant plane, to use it as the referential system, this still not solve for interactions. In the recent literature, we can found works that integrate an object recognition system to the SLAM in a way the objects are incorporated into the map. The SLAM map is frequently assumed to be static, due to limitations on techniques involved, so that on these works the object is just used to provide semantic information about the scene. In this work, we propose a new framework that allows estimating simultaneously the camera and object positioning for each camera image in real time. In this way, each object is independent and can move through the map as well as in the marker-based methods but with the SLAM advantages kept. We develop our proposed framework over a stateof- the-art SLAM system in order to evaluate our proposal and demonstrate potentials application in Augmented Reality.