{"id":1,"date":"2024-07-30T16:30:05","date_gmt":"2024-07-30T16:30:05","guid":{"rendered":"https:\/\/blogs.oregonstate.edu\/machinelearningbreakout\/?p=1"},"modified":"2024-07-31T14:09:25","modified_gmt":"2024-07-31T14:09:25","slug":"hello-world","status":"publish","type":"post","link":"https:\/\/blogs.oregonstate.edu\/machinelearningbreakout\/2024\/07\/30\/hello-world\/","title":{"rendered":"Training the Paddle"},"content":{"rendered":"\n<h3 class=\"wp-block-heading has-text-align-center\">Our OSU Capstone journey into machine learning.<\/h3>\n\n\n\n<h6 class=\"wp-block-heading alignwide has-text-align-center has-large-font-size\">James Ramsaran | Daniel Fontenot | Trevor Phillips<\/h6>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"766\" height=\"1024\" src=\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-10.45.09-AM-766x1024.png\" alt=\"\" class=\"wp-image-5\" style=\"width:284px;height:auto\" srcset=\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-10.45.09-AM-766x1024.png 766w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-10.45.09-AM-224x300.png 224w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-10.45.09-AM-768x1027.png 768w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-10.45.09-AM-1149x1536.png 1149w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-10.45.09-AM.png 1152w\" sizes=\"auto, (max-width: 766px) 100vw, 766px\" \/><\/figure>\n<\/div>\n\n\n<p>Hello! This term, our team of enthusiastic student developers is venturing into machine learning (ML) for the first time. This blog will serve as a brief summary of the significant moments in our efforts and seek to be a resource for future students.<\/p>\n\n\n\n<p>Let&#8217;s cut to the chase. Here&#8217;s what we had accomplished prior to our training efforts:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Initial prototype our breakout clone built in Unity. We based our prototype (right) off of the 1976 Atari version (left).<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"886\" height=\"1024\" data-id=\"8\" src=\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-11.02.25-AM-886x1024.png\" alt=\"\" class=\"wp-image-8\" srcset=\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-11.02.25-AM-886x1024.png 886w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-11.02.25-AM-260x300.png 260w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-11.02.25-AM-768x887.png 768w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-11.02.25-AM.png 1018w\" sizes=\"auto, (max-width: 886px) 100vw, 886px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"767\" height=\"1024\" data-id=\"9\" src=\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-11.04.52-AM-767x1024.png\" alt=\"\" class=\"wp-image-9\" srcset=\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-11.04.52-AM-767x1024.png 767w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-11.04.52-AM-225x300.png 225w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-11.04.52-AM-768x1025.png 768w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-11.04.52-AM-1151x1536.png 1151w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/8059\/files\/2024\/07\/Screenshot-2024-07-30-at-11.04.52-AM.png 1160w\" sizes=\"auto, (max-width: 767px) 100vw, 767px\" \/><\/figure>\n<\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Installation of the ml-agents toolkit via the GitHub repo (<a href=\"https:\/\/github.com\/Unity-Technologies\/ml-agents\/blob\/develop\/docs\/Installation.md\">https:\/\/github.com\/Unity-Technologies\/ml-agents\/blob\/develop\/docs\/Installation.md<\/a>). Warning: This is currently (as of July, 2024) a major pain still on an ARM-based Mac M1 setup. Lots of troubleshooting required.<\/li>\n<\/ul>\n\n\n\n<p>With these preparations in place, we set about constructing our first agent script for the paddle. To newcomers: while it might seem overwhelming at first the basic functionality of an agent script can be divided into three categories:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Observations<\/li>\n\n\n\n<li>Actions<\/li>\n\n\n\n<li>Rewards\/Penalties<\/li>\n<\/ul>\n\n\n\n<p>Let&#8217;s look at each briefly.<\/p>\n\n\n\n<p><strong>Observations:<\/strong> Observations represent the data that the agent will track during the course of training. For our first training runs we are collecting three observations:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The angle between the paddle and ball.<\/li>\n\n\n\n<li>The position of the ball.<\/li>\n\n\n\n<li>The position of the paddle.<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code has-stratosphere-color has-paddletail-black-background-color has-text-color has-background has-link-color wp-elements-5f6118537f7970b03b87c8956fd82841\" style=\"font-size:12px\"><code>    \/\/ Relevant observations for agent.\n    public override void CollectObservations(VectorSensor sensor)\n     {\n         sensor.AddObservation(PaddleBallAngle());          \n         sensor.AddObservation(ball.transform.position);    \n         sensor.AddObservation(paddle.transform.position);  \n     }<\/code><\/pre>\n\n\n\n<p><strong>Actions: <\/strong>Actions represent what the agent is able to do within the environment. For instance, a car agent might be able to accelerate and decelerate. Our paddle agent has three potential actions. The ML agents trainer of course doesn&#8217;t control these actions directly. It simply produces an integer for each decision:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>0 = Stay still<\/li>\n\n\n\n<li>1 = Move left<\/li>\n\n\n\n<li>2 = Move right<\/li>\n<\/ul>\n\n\n\n<p>We must then connect that integer to a meaningful action via code. In our model this is accomplished by accessing the paddle script.<\/p>\n\n\n\n<pre class=\"wp-block-code has-paddletail-black-color has-paddletail-black-background-color has-text-color has-background has-link-color wp-elements-6fe9b0bb511aa2c5b49b450c41d2c0c7\" style=\"font-size:12px;font-style:normal;font-weight:400\"><code>    \/\/ Control paddle movement\n    public override void OnActionReceived(ActionBuffers actions)\n    {\n        int dir = actions.DiscreteActions&#091;0];\n        if(dir == 0)\n        {\n            \/\/stay still\n        }\n        else if(dir == 1)\n        {\n            paddle.GetComponent&lt;Paddle&gt;().PaddleMoveLeft();\n        }\n        else{\n            paddle.GetComponent&lt;Paddle&gt;().PaddleMoveRight();\n        }\n    }<\/code><\/pre>\n\n\n\n<p><strong>Rewards\/Penalties: <\/strong>Finally, in order to train the results must have meaning to the agent. This is accomplished through rewards and penalties. Our model begins with a very simple system:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Ball is bounced: +5<\/li>\n\n\n\n<li>Ball dies: -100<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code has-paddletail-black-background-color has-background\" style=\"font-size:12px\"><code>    \/\/ Reward bouncing ball\n    private void OnCollisionEnter2D(Collision2D other)\n    {\n        if(other.gameObject.GetComponent&lt;Ball&gt;())\n        {\n            AddReward(5.0f);\n        }\n    }<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code has-star-canvas-color has-paddletail-black-background-color has-text-color has-background has-link-color wp-elements-00524ec7d7d85683bb8475dd8ed15810\" style=\"font-size:12px\"><code>    private void Update()\n    {\n        \/\/ If ball dies give penalty.\n        if (deadBall)\n        {\n            AddReward(-100.0f);\n        }\n    }<\/code><\/pre>\n\n\n\n<p>The goal here is straight forward. Can we train an agent to keep the ball alive by continually bouncing it off the paddle? Let the training begin!<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/miro.medium.com\/v2\/resize:fit:1400\/1*pOGwxbJaiRMgUtFWxp9OIQ.gif\" alt=\"\" \/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>Our OSU Capstone journey into machine learning. James Ramsaran | Daniel Fontenot | Trevor Phillips Hello! This term, our team of enthusiastic student developers is venturing into machine learning (ML) for the first time. This blog will serve as a brief summary of the significant moments in our efforts and seek to be a resource [&hellip;]<\/p>\n","protected":false},"author":14352,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/blogs.oregonstate.edu\/machinelearningbreakout\/wp-json\/wp\/v2\/posts\/1","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.oregonstate.edu\/machinelearningbreakout\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.oregonstate.edu\/machinelearningbreakout\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/machinelearningbreakout\/wp-json\/wp\/v2\/users\/14352"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/machinelearningbreakout\/wp-json\/wp\/v2\/comments?post=1"}],"version-history":[{"count":14,"href":"https:\/\/blogs.oregonstate.edu\/machinelearningbreakout\/wp-json\/wp\/v2\/posts\/1\/revisions"}],"predecessor-version":[{"id":24,"href":"https:\/\/blogs.oregonstate.edu\/machinelearningbreakout\/wp-json\/wp\/v2\/posts\/1\/revisions\/24"}],"wp:attachment":[{"href":"https:\/\/blogs.oregonstate.edu\/machinelearningbreakout\/wp-json\/wp\/v2\/media?parent=1"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/machinelearningbreakout\/wp-json\/wp\/v2\/categories?post=1"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/machinelearningbreakout\/wp-json\/wp\/v2\/tags?post=1"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}