My favorite technology that my group has been working with so far is easily Unity Hub, which is a desktop app that runs Unity Games that allows for easily editing and testing of code. Changes made locally in the Unity Window can also be included in Pull Requests to make setup and presets much simpler. Once the code has began running, you can pause and inspect different variables and even change them if some minor tweaking is required. It is a very handy tool that helps abstract away some of the challenges that come along with video game development.
The technology that has had the highest learning curve so far has been the ML-Agents training software. It is extremely complicated, the recommended tutorial is around 10+ hours in length for the beginner project. Luckily one of our group members who is more experienced with Machine Learning created a very helpful instructional guide on how to create and train new Machine Learning Models, as well as how to manipulate the reward based on conditionals inside the game’s code. This however, requires a lot of fine tuning and training, the latter of which can take upwards of 30 minutes -> 2 hours to converge (which means the model’s mean reward stops improving and the error rate approaches a constant).
The current best performing models on for the ML-Breakout game are still only as effective as a human player would be on their first few attempts at the game. On an average attempt it will probably clear around a quarter of the available bricks before losing all 3 lives. The models I trained have a tendency to stick to the right wall, often causing them to miss balls that are skewed to the left side and I don’t yet understand why. The machine learning agents are difficult to evaluate, as the resulting “code” from training is not understandable in text form, and each model must be observed manually to accurately assess its competency. We are in the process of making a script to help us evaluate the agents by recording an average score over 10 runs after the training is complete. With the addition of this script and our improving knowledge of machine learning, we should be able to improve the efficacy of the ML-Agents to at least that of the average player by the end of this term.
