There were many interesting projects in the capstone catalog. In general, I gravitated toward those that involved embedded development since that is an area I know well. The AudioLux, an embedded project, caught my attention because it aims to help others. Years ago, I prototyped a toy for children with severe congenital brain malformations using an Arduino, music, and lights. The work was gratifying just from knowing it would bring a smile to a child. The AudioLux is designed to process music and produce light patterns in real-time so that the hard of hearing can see a representation of the audio. It is a great idea, and the device does its job well.

The device is based on Adafruit’s ESP32 Featherboard. The ESP32 is a powerful system-on-a-chip  based on a dual-core RISC processor by Tensilica. It includes many devices like Bluetooth and WiFi radios, general purpose I/O pins, several chip-to-chip communications interfaces (I2S, I2S, SPI, etc.), analog to digital converters (ACD), digital-to-analog converters (DAC), an Ethernet controller, LED drivers and a bunch more stuff.

Using this module gives the AudioLux a lot of power to expand in the future. Currently, the resources used are a couple of ADCs to sample the incoming audio and an SPI driver to illuminate the visualization LEDs. This year we added the WiFi radio.

The concept is interesting enough to draw the attention of at least seven more people. I soon learned that the project was offered to both the E-Campus and On-Campus capstone classes. While that may seem like a good idea, I think there are logistical problems that made its management somewhat challenging:

• The two classes do not have the same assignments and deadlines. Producing design documentation was difficult because each sub-team had different rubrics and objectives.
• The project does not lend itself to being neatly split so that each sub-team can take a few discrete components and work on them. This is an embedded project using the Arduino system. Everything is in one file. We managed to find seams to divide the work, but it was not as natural as it would be with a stack-based or component-based development.
• The partner is a professor at OSU and not the main stakeholder, and he is managing many other projects. The main stakeholder did not attend meetings, and the partner is spread so thin that we spent most of our meetings bringing him up to speed.

This is not to say that the project was dysfunctional by any means. I think the team, especially the On-Campus one, is driven and responsible. They spent a lot of time working on light patterns and sound processing improvements. But the separation of teams made coordination difficult and, I think, stunted the development of a whole-team culture. Nonetheless, we made it work as best as we could.

My part in the project was improving the AudioLux’s usability by giving it a user interface. Given its size, a screen or complex controls are out of the question. So creating the user interface involved programming the WiFi radio of the embedded microcontroller and running a web server with it. This was very exciting and had several interesting challenges:

• The device has severe memory constraints. About 4 MB. All application code, both the firmware and a web application, must fit in that footprint.
• Since there is no screen, there is no easy way to know the device’s IP address on boot. This causes a chicken-and-egg problem, we need the device’s IP address to connect to it, but we can’t get that address unless we connect to it first.
• Web applications can get big very fast. So, whatever web application is used, its size must stay below 1MB of memory.
• There is no operating system and no resources. All the code running in the device is in the form of a single C++-based executable. Anyone that has worked with C/C++ knows that string and JSON manipulation are not very pleasurable. Mainly because there is no room to bring ancillary libraries like boost or the STL.

Some of these challenges have no solution. For example, there is no way to change the amount of available memory. While others required some careful design to overcome.

To solve the problem of needing a connection to find out the device’s IP address, I decided to take advantage of the WiFi radio’s ability to work simultaneously as an Access Point (AP) and a Station (STA). In AP mode, the AudioLux is similar to a router, so it has a hard-coded IP and allows other devices to connect to it. We gave it the address 192.168.4.1. On the other hand, STA mode is similar to how computers connect to WiFi. In STA mode, the AudioLux will connect to a specified nearby wireless network. I wrote a set of routines in the firmware that scan the available networks and can connect to any of them, given the right credentials. A counterpart in the web application presents the networks to the user and then can select which one the device will use. From then on, it will try to automatically connect to that network until the setting is changed.

The web application itself was a challenge. We wanted it to be a modern, reactive Single Page Application (SPA). However, modern SPAs easily measure dozens of megabytes. We needed one that could be less than one megabyte. After some research, the team decided to use Preact, a tiny yet surprisingly complete implementation of React.js. With it, we managed to keep the web application at about 400KB. Of course, using a reduced framework meant I had to create most of the controls that would usually be provided by a fully-fledged one.

Finding small libraries was a common topic during the whole development cycle. I found a very light JSON management library that could be leveraged to implement the WiFi connection API. Combined with Arduino’s small String library, it made coding that functionality in C++ less painful.

All in all, the project has been challenging and rewarding. Knowing it will help others expand their enjoyment of music despite physiological barriers is a great motivator. Since it is open source, I look forward to continuing to work on improving it in the future.