Extended Reality Research: Reinforcement Learning with Unreal Game Engine

Motivation —

I was curious about the applications of Machine Learning (ML) in XR. As the Metaverse is increasingly becoming a popular research area. I wanted to creatively identify, formulate and solve problems in this space. This was my first opportunity researching on a XR technology which allowed me to stretch beyond my personal comfort zone.

My Contribution —

I was a member of the ML team with 2 students. I was in charge of our team scheduling and progress reporting. I worked on implementing the MindMaker AI plugin, specifically for the CartPole balancing task to explore the usability of the platform.

Reflection —

I developed new skills in XR, HCI and ML. I learned a lot about how ML techniques are used in the MindMaker: the extent that coding is required or not required from the user of the platform, and how to train agents using RL. Also, this experience helped me understand the fundamental differences between Augmented Reality, Virtual Reality, and Extended Reality.