{"id":50,"date":"2023-03-10T05:57:29","date_gmt":"2023-03-10T05:57:29","guid":{"rendered":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/?p=50"},"modified":"2023-03-10T05:57:29","modified_gmt":"2023-03-10T05:57:29","slug":"captsone-retrospective","status":"publish","type":"post","link":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/captsone-retrospective\/","title":{"rendered":"Captsone Retrospective"},"content":{"rendered":"\n<p>There were many interesting projects in the capstone catalog. In general, I gravitated toward those that involved embedded development since that is an area I know well. The <a href=\"https:\/\/www.cymaspace.org\/projects\/audiolux-one\/\">AudioLux<\/a>, an embedded project, caught my attention because it aims to help others. Years ago, I prototyped a toy for children with severe congenital brain malformations using an Arduino, music, and lights. The work was gratifying just from knowing it would bring a smile to a child. The AudioLux is designed to process music and produce light patterns in real-time so that the hard of hearing can see a representation of the audio. It is a great idea, and the device does its job well.<\/p>\n\n\n\n<p>The device is based on <a href=\"https:\/\/learn.adafruit.com\/adafruit-huzzah32-esp32-feather\">Adafruit\u2019s ESP32 Featherboard<\/a>. The <a href=\"https:\/\/en.wikipedia.org\/wiki\/ESP32\">ESP32<\/a> is a powerful system-on-a-chip\u00a0 based on a dual-core RISC processor by <a href=\"https:\/\/www.cadence.com\/en_US\/home\/tools\/ip\/tensilica-ip\/tensilica-xtensa-controllers-and-extensible-processors\/xtensa-lx-processor-platform.html\">Tensilica<\/a>. It includes many devices like Bluetooth and WiFi radios, general purpose I\/O pins, several chip-to-chip communications interfaces (I2S, I2S, SPI, etc.), analog to digital converters (ACD), digital-to-analog converters (DAC), an Ethernet controller, LED drivers and a bunch more stuff.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"419\" src=\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/1920px-Espressif_ESP32_Chip_Function_Block_Diagram.svg_-1024x419.png\" alt=\"\" class=\"wp-image-51\" srcset=\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/1920px-Espressif_ESP32_Chip_Function_Block_Diagram.svg_-1024x419.png 1024w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/1920px-Espressif_ESP32_Chip_Function_Block_Diagram.svg_-300x123.png 300w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/1920px-Espressif_ESP32_Chip_Function_Block_Diagram.svg_-768x314.png 768w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/1920px-Espressif_ESP32_Chip_Function_Block_Diagram.svg_-1536x629.png 1536w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/1920px-Espressif_ESP32_Chip_Function_Block_Diagram.svg_-500x205.png 500w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/1920px-Espressif_ESP32_Chip_Function_Block_Diagram.svg_.png 1920w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">ESP32 architecture. (2023, March 2). In <em>Wikipedia<\/em>. <a href=\"https:\/\/en.wikipedia.org\/wiki\/ESP32\">https:\/\/en.wikipedia.org\/wiki\/ESP32<\/a><\/figcaption><\/figure>\n\n\n\n<p>Using this module gives the AudioLux a lot of power to expand in the future. Currently, the resources used are a couple of ADCs to sample the incoming audio and an SPI driver to illuminate the visualization LEDs. This year we added the WiFi radio.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"703\" src=\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/IMG_5847-1024x703.jpg\" alt=\"\" class=\"wp-image-54\" srcset=\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/IMG_5847-1024x703.jpg 1024w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/IMG_5847-300x206.jpg 300w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/IMG_5847-768x527.jpg 768w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/IMG_5847-1536x1055.jpg 1536w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/IMG_5847-2048x1406.jpg 2048w, https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/6098\/files\/2023\/03\/IMG_5847-437x300.jpg 437w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Internals of the AudioLux. On the bottom right is the ESP32 module.<\/figcaption><\/figure>\n\n\n\n<p>The concept is interesting enough to draw the attention of at least seven more people. I soon learned that the project was offered to both the E-Campus and On-Campus capstone classes. While that may seem like a good idea, I think there are logistical problems that made its management somewhat challenging:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The two classes do not have the same assignments and deadlines. Producing design documentation was difficult because each sub-team had different rubrics and objectives.<\/li>\n\n\n\n<li>The project does not lend itself to being neatly split so that each sub-team can take a few discrete components and work on them. This is an embedded project using the Arduino system. Everything is in one file. We managed to find seams to divide the work, but it was not as natural as it would be with a stack-based or component-based development.<\/li>\n\n\n\n<li>The partner is a professor at OSU and not the main stakeholder, and he is managing many other projects. The main stakeholder did not attend meetings, and the partner is spread so thin that we spent most of our meetings bringing him up to speed.<\/li>\n<\/ul>\n\n\n\n<p>This is not to say that the project was dysfunctional by any means. I think the team, especially the On-Campus one, is driven and responsible. They spent a lot of time working on light patterns and sound processing improvements. But the separation of teams made coordination difficult and, I think, stunted the development of a whole-team culture. Nonetheless, we made it work as best as we could.<\/p>\n\n\n\n<p>My part in the project was improving the AudioLux\u2019s usability by giving it a user interface. Given its size, a screen or complex controls are out of the question. So creating the user interface involved programming the WiFi radio of the embedded microcontroller and running a web server with it. This was very exciting and had several interesting challenges:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The device has severe memory constraints. About 4 MB. All application code, both the firmware and a web application, must fit in that footprint.<\/li>\n\n\n\n<li>Since there is no screen, there is no easy way to know the device\u2019s IP address on boot. This causes a chicken-and-egg problem, we need the device\u2019s IP address to connect to it, but we can\u2019t get that address unless we connect to it first.<\/li>\n\n\n\n<li>Web applications can get big very fast. So, whatever web application is used, its size must stay below 1MB of memory.<\/li>\n\n\n\n<li>There is no operating system and no resources. All the code running in the device is in the form of a single C++-based executable. Anyone that has worked with C\/C++ knows that string and JSON manipulation are not very pleasurable. Mainly because there is no room to bring ancillary libraries like boost or the STL.<\/li>\n<\/ul>\n\n\n\n<p>Some of these challenges have no solution. For example, there is no way to change the amount of available memory. While others required some careful design to overcome.<\/p>\n\n\n\n<p>To solve the problem of needing a connection to find out the device\u2019s IP address, I decided to take advantage of the WiFi radio\u2019s ability to work simultaneously as an Access Point (AP) and a Station (STA). In AP mode, the AudioLux is similar to a router, so it has a hard-coded IP and allows other devices to connect to it. We gave it the address 192.168.4.1. On the other hand, STA mode is similar to how computers connect to WiFi. In STA mode, the AudioLux will connect to a specified nearby wireless network. I wrote a set of routines in the firmware that scan the available networks and can connect to any of them, given the right credentials. A counterpart in the web application presents the networks to the user and then can select which one the device will use. From then on, it will try to automatically connect to that network until the setting is changed.<\/p>\n\n\n\n<p>The web application itself was a challenge. We wanted it to be a modern, reactive Single Page Application (SPA). However, modern SPAs easily measure dozens of megabytes. We needed one that could be less than one megabyte. After some research, the team decided to use <a href=\"https:\/\/preactjs.com\/\">Preact<\/a>, a tiny yet surprisingly complete implementation of React.js. With it, we managed to keep the web application at about 400KB. Of course, using a reduced framework meant I had to create most of the controls that would usually be provided by a fully-fledged one.<\/p>\n\n\n\n<p>Finding small libraries was a common topic during the whole development cycle. I found a very light JSON management library that could be leveraged to implement the WiFi connection API. Combined with Arduino\u2019s small String library, it made coding that functionality in C++ less painful.<\/p>\n\n\n\n<p>All in all, the project has been challenging and rewarding. Knowing it will help others expand their enjoyment of music despite physiological barriers is a great motivator. Since it is open source, I look forward to continuing to work on improving it in the future.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>There were many interesting projects in the capstone catalog. In general, I gravitated toward those that involved embedded development since that is an area I know well. The AudioLux, an embedded project, caught my attention because it aims to help &hellip; <a href=\"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/captsone-retrospective\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":12856,"featured_media":53,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-50","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/wp-json\/wp\/v2\/posts\/50","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/wp-json\/wp\/v2\/users\/12856"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/wp-json\/wp\/v2\/comments?post=50"}],"version-history":[{"count":3,"href":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/wp-json\/wp\/v2\/posts\/50\/revisions"}],"predecessor-version":[{"id":56,"href":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/wp-json\/wp\/v2\/posts\/50\/revisions\/56"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/wp-json\/wp\/v2\/media\/53"}],"wp:attachment":[{"href":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/wp-json\/wp\/v2\/media?parent=50"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/wp-json\/wp\/v2\/categories?post=50"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/eventualconsistency\/wp-json\/wp\/v2\/tags?post=50"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}