Facebook research open-sources tools to improve AI and AR in virtual spaces

Facebook wants to improve AI capabilities to ensure that virtual assistants can understand and navigate in our physical surroundings better. So, it has open-sourced a simulation called AI Habitat and a dataset called Replica for this very reason. [caption id=“attachment_6820151” align=“alignnone” width=“1280”]  Replica simulation of an apartment by Facebook researchers. Image: Facebook.[/caption] Replica is a research project that has been created by Facebook Reality Labs (FRL). It’s a photo-realistic version of 18 sample spaces including a living room, office conference room and a two-storey home. The idea here is to enable these virtual spaces to train AI bots about the real world without having to go through the complexities of creating physical prototypes and letting them out in the real world. So, if an AI agent can successfully search for missing keys in the virtual environment, it will be able to do the same in the real world someday. Facebook researchers say that these virtual spaces need to get certain aspects of our real world accurately. Tiny details such as mirror reflections and rug textures will be necessary to make these spaces virtually identical to the real objects. This is exactly where Replica comes in. While the virtual environment has been taken care of, the AI bots will still require a platform to learn what to do and how to do it. AI Habitat is an open platform to train AI agents in simulated home and office environments. Using this platform, researchers will be able to place an AI bot in a Replica environment and teach them how to carry out different tasks. Essentially, it can learn how to search for keys that’s located on the desk of your bedroom. This means that the bot has to be capable enough to locate your bedroom in the virtual environment, then search for a desk inside it and finally spot the keys whether they are present or not.

Facebook also mentioned about bringing social presence into the real world using AR and VR. It envisions a future where digital versions of people could be created and placed in the physical world, as if they were right there even though they were physically miles away from that location. FRL Research Director Richard Newcombe said, “Much as the FRL research work on virtual humans captures and enables transmission of the human presence, our reconstruction work captures what it is like to be in a place; at work, at home, or out and about in shops, museums, or coffee shops.” With Replica, AI Habitat and its social presence dream using AR, Facebook researchers are trying to build a future where AI will enable a more effective and engaging world.