This week, I was able to implement the first version of my machine learning model for our Breakout game. We also realized that the singleton instance pattern could potentially interfere with our AI training if we wanted to scale and multiply our training level. So, after some architectural changes to remove the usages of the singleton instance pattern, I attached the machine learning agent to our Breakout pong game with the initial learning model and point reward system below:
Initial model
Base points = 0.01
When the paddle hits the ball = reward base points
If the ball hits a brick = reward base points times (x) brick’s point value
Example: Green brick = 3 points * 0.01 base points = 0.03 reward
If the ball goes out of bounds = lose base points 0.01 * 10 = -0.1 punishment
If the game is over for the Ai (lose all 3 lives) = lose base points 0.01 * 50 = -0.5 punishment
This past week was the midpoint of the term, and it was a hectic one. The positive side is that we got a lot of work done this week.
Lately, we have been focusing on tightening up the quality of our game, fixing bugs, and adding more levels. In particular, we’ve been experimenting with new multiplayer level experiences.
Ultimately, we had so many great ideas we decided to implement all of them. Here is what we came up with:
1 – Player 1 vs. Player 2 Split Screen
The idea behind this level is to have a way for 2 human players in the same room to play competitively against each other. It divides the play area in half, giving each player their own screen real estate. This is to allow the players to clearly see their own play area without getting any distractions from the other player.
2 – Player vs. Computer AI
The motivation behind this level is to let the player enjoy the full game experience, utilize the full screen to play, and have more bricks to break, while adding a competitive multiplayer component. This multiplayer component is controlled by an artificially intelligent computer to challenge the player. The AI player of this level controls balls, bricks, and a paddle that are transparent in the middle, and have a colorful glow on the outer edge. We call these the “Ghost Assets”. This design is to allow both human and AI assets to be visibly layered on top of each other on the game screen.
In this screenshot, some Ghost bricks and some normal bricks have been broken. You can easily distinguish between them. Also in this screenshot, the first player has lost all of their lives, so their paddle and ball have de-spawned from the game, leaving only the AI’s ball and paddle in play.
Conclusion
As of now, the Player vs. AI level has been created and integrated into the project. The only thing missing for the level is the actual AI component.
I do feel that we are slightly behind on the AI component, because it took us longer than expected to come up with a final prototype that we are comfortable with before we start adding the machine learning.
But, now is the time to start, and I feel that we still have enough time to make it happen and train it well.
As for the split level, we decided to remove the “Exit” button on the Main Menu, because we no longer need that button if we will be deploying this game to the Web. In place of the “Exit” button will be a new button to access the Player vs. Player split screen level.
You can check out our first deployment at: https://kenkoolio.github.io/ML-Breakout-Capstone/
In terms of development, this week’s pace calmed down a little bit after the milestone of combining our prototypes. Most of our development efforts went to fixing bugs and improving the aesthetics of our game.
My teammates came up with some pretty creative solutions to the bugs we were facing. For example, whenever the ball gets stuck bouncing back and forth between the left and right walls, after 5 bounces there is an upward force added to the ball to kick it out of the loop.
This week, I also incorporated higher quality sprites that a teammate found for the ball, paddle, and bricks. These sprites have a vibrant color selection, and graphical details like a bold outline, bezels, and shadows.
Current game progress – Upgraded game sprites
I was also able to implement an architectural design that allows the game to progress to the next level, loading the new scene while unloading the old level’s scene. This is necessary because our architecture loads scenes additively.
We were also able to set up the ML-Agents environment for our project. There were setbacks and hurdles due to conflicting operating systems, but we were able to work through those and successfully integrate the libraries.
Our next major milestones will be to implement a machine learning model, build a human vs. AI level, and add more single player levels.
That’s it for this week’s check-in. See you in the next one.
After mostly working individually on our initial game prototypes, this past week was dedicated to combining together all of our work that was done in parallel. Good o’ parallelism. Much efficient.
Since each of us created a replica of the same game, this naturally resulted in our game assets being very similar. This made it easier to combine projects. I decided to use my game as the base and import the best parts of everyone else’s game.
Another group member had already made a main menu and game over screen. We had the parts, but they just weren’t tied together into a coherent game that flows. As a goal of having a minimum viable prototype, this means it was time to design an architecture.
Main MenuPrototype Level 1Game Over Menu
When designing the architecture of our game, I had to keep in mind 2 things: 1) The player’s total score and number of remaining lives must be tracked as the player progresses through different levels. 2) Design an architecture that easily allows for adding new levels.
ML Breakout High Level Architecture Diagram
The diagram above shows a high-level flowchart of the architectural design I decided to implement. The game can only start from the scene titled “Main”. This is the root scene that contains global scripts that will control which scenes get loaded and unloaded. This Main scene holds the “GameController” script, which will be the main driver for the game. The GameController keeps track of the player’s score and lives, and controls when scenes are loaded and unloaded. The scenes are loaded additively, so the Main scene stays loaded the entire time.
In contrast, each separate level must have its own local controller, called the “LevelHandler”. This controller uses the “LevelHandler” script, and contains all of the game’s assets as children in its hierarchy.
The LevelHandler must have a “BrickManager” as one of its children elements. The BrickManager uses the “BrickManager” script, and is responsible for creating the brick layout of the level, as well as destroying the bricks, and re-constructing the level if necessary. Both of these components are required because the root GameController component utilizes them to control the game.
To add a new level, you would add a new method that constructs the level’s brick layout inside the BrickManager script. You would also need to add the new scene to the project’s File > Build Settings, and to the corresponding Enum of Scene names in the SceneLoader (not shown).
That’s about it for this week’s update. Stay out of trouble y’all.
As we approach the end of this week, this wraps up our inaugural dive into Unity. I have now, metaphorically, “dipped my toes” into the tides of game development in Unity, and the water feels … nice 🙂 .
Using Unity feels like a breath of fresh air. After only working in IDEs and code for so long, it was invigorating to use a very high-level, sophisticated, and graphical tool that also gives the developer some control by writing their own code scripts. The way Unity set up their framework to facilitate this is impressive.
This first week of development was a good chance for all of our team members to start experimenting and playing around in Unity. I’m confident that we all learned a lot in such a short amount of time.
In my own personal attempt at developing a prototype for our Breakout game, I have been able to set up all the necessary game pieces. I have bricks that are different colors, and each color has a different point value. I might have worked ahead and implemented a scoring system, but it’s a very rough prototype and needs a lot more mesh rendering and polish.
We have a meeting this week to demo each of our individual prototypes and decide what direction we want to take this project. I would also like to implement the concept of limited number of lives, a menu screen, and a game over screen before we demo our projects.
I also need to research how to change to different scenes so that we can start developing new levels, and how to implement ML-Agents in Unity.
It also doesn’t help that I needed to do some last-minute traveling for some things that came up at my job. Luckily, I reached out to the team about my situation and they were understanding.
A sneak peak of what I’ve been cooking up in the laboratory.
This week, our group met for the first time. When I say “met”, I mean we got on a voice chat together on Discord. The meeting went well, and we were able to discuss a lot. Especially, the fact that all three of us have zero experience with Unity or AI/ML. This means, we all have some studying to do.
So pretty much, for the past two weeks, I have been looking up and studying whatever resources I could find for various, various topics.
At first, before we were assigned to a specific project, I sought out introductory material in Unity and TensorFlow. It felt like taking two more classes this semester in itself.
I had no idea if I was wasting my time or not. I knew most of the projects I requested had to do with machine learning (ML), and some required Unity. So, I was confident in my choices to learn a broad amount of introductory information about these topics.
I did spent 5+ hours watching a course on TensorFlow. And even though we won’t be using TensorFlow directly in our projects, I really enjoyed what I learned in those 5+ hours. I’m not even done with the course (barely halfway). I’m also pretty happy about what I learned about TensorFlow so far, so I definitely want to finish that course later. I feel that what I learned is already useful for me going into this project.
The next step is to learn Unity.
This must be the most exciting, and most difficult technology to learn for this project. My mind is already starting to think about future game ideas.
So far, learn.unity.com has been a huge resource to learning Unity. What better resource than from the source, right?
In addition, there are tons of guides on YouTube that give us some good ideas on how to approach this project.
The process of choosing my project was packed with more emotional ups and downs that I ever would have expected. After browsing through the catalog of projects, I found myself getting excited for so many of them. At first, I picked out my top 10 most preferred projects. This list was already difficult to decide at first, but when I finally saw the requirement was to choose our top 5, this decision became even harder.
My original top 10 projects were chosen based mainly on 2 factors:
It was an Artificial Intelligence/Machine Learning project, or
It was a game developed in Unity, or
It was a mobile project
Even though I have never worked on an AI/ML project before, I have always expressed an interest to do so. After reading the project descriptions, some of them sounded like they were designed to be a great learning opportunity for someone with zero experience. The same goes for a few Unity projects.
I also took an elective at OSU in mobile app development, so I have had the chance to learn Flutter and Dart. The mobile app projects sounded like a great way to get practice and experience creating mobile apps.
To narrow my choices down to only 5, I decided that I would want to get an AI/ML project above other projects because I have always wanted to learn more about it, and perhaps pursue a specialty in this field one day.
I picked 4 of the most accessible-looking AI/ML projects from the standpoint of someone that had zero experience in these technologies. Although, I do have experience working in Python and C#.
The 5th project I chose was a fun looking mobile app idea, that behaves similarly to Uber or Toro to rent out your bicycles, called “The Bike Kollective.” I chose this in case I wasn’t chosen for an AI/ML one due to lack of experience. Essentially, I chose the one mobile project that I would want to work on the most, if I didn’t get an AI/ML project.
This whole process has been very exciting. My top project choice was chosen because it gives the opportunity to learn Unity and AI/ML using TensorFlow. I got very excited at this option. So much so, I realized that I am determined to learn Unity and TensorFlow, even if I do not chosen for one of these projects.
After some research, learning Unity and TensorFlow seemed very approachable and attainable. So, I have started my journey in learning those technologies by exploring online resources. Here’s hoping I learn fast enough to have a successful project this term.
Without further ado, my top 5 project choices for Capstone:
1 – ML Breakout
2 – Algorithmic Stock Market Trading Strateges
3 – Fire Risk Prediction
4 – Top-n Music Genre Classification Neural Network
