Month: November 2020

This week has been one of the lighter weeks throughout the term, as thanksgiving was during it. My workplace gave me two days off and I took another day off, so I had time to develop some more on the neural networks that our team has thought up.

significantly this week I created a convolutional neural network that handles some gesture recognition. Currently it only handles three separate gesture, however the architecture is all there for using different data sets for more categories. This will be a launching off point for our team as we can use OpenVINO to optimize it, one of the requirements that our client has set up for us. It can also be used as a backup in case we don’t get the video recognition working for the capstone .

Along with this, my teammate and I have been working on a white paper covering the various topics that make up the preproccessor side. Currently the paper is in it’s rough draft stage, so there are quite a grammatical errors within it. However it goes over all the fundamentals topics and ideas that we will be using or attempting to use when designing the preproccessor aspects of the project. The purpose of this white paper is so that when we finish our tasks on the project, or drop it because the deadline was hit, then the next person who picks the project up will have an easier time. Not having white papers to documents what aspects worked and which didn’t work can lead to future designers trying to the same tactics.

This week saw the creation of a timeline that helped our client understand how we will be continuing forward on our project. Creation the timeline was particuarely difficult as it was difficult to project what how well we will be doing on early parts of the project. Along with this, it’s hard to see what needs to be worked on later on. Some issues can crop up that are n’t even considered during the process of the project. Sometimes small functionality issues can cause hard stops in the programming process. So the timeline was very specific early on in the develop process but as the timeline continues, it becomes very vague.

I also worked on an idea for my sections of the team project, which is preproccessing. The idea comes from the general architecture of cross correlation. My sections of the project pertains to detecting when a hand is making a new sign, so that the main neural network doesn’t need to constantly be parsing data. Cross correlation is involved in the fundamentals of a convolutional neural network. In our case it can be used in detecting the difference in two images by simply finding the difference in each pixel and attributing it to a distinct value. Then if we keep track of the difference between this value and the previous threshold, we will know when the movements transfer between a still hand and one that’s moving. I don’t know whether this will actually work or not, but it’s worth it to just test out.

Finally, our team has begun really starting our project. A lot of different aspects of the project, like the video recognition and preprocessing have been discussed thoroughly. This means that we are finally starting to proceed forward and get a feel for the cod base on our project.

This week we have finally gotten around to working on the code base that has been provided by the previous project group. Some things have cropped up while looking through the code, such as what composes the convolutional neural network. There is a severe lack of documentation that clarifies what the Resnet18 is made up of. As such, it is hard to see whether the network has essentially components like batch normalization or dropout. These layers are essential for preventing overfitting of the model when training. Not knowing whether they are included or not can lead to our teams detriment as attempting to include them will lead to underfitting the model if they are already contained within the model.

We are also beginning to develop our timeline. This is a particularly difficult thing to decide as it’s hard to tell how long it will take to tune the hyper parameters. Along with this, the model itself could be flawed, meaning that the model may be scrapped if that cross roads is come to. This culminates into many things that are really beyond our ability to foresee. However, I will be discussing with my teammates Travis what the best option will be when moving forward. It shouldn’t really matter in the long run, as a timeline is just a best estimate for how long it will take, and any roadblocks can be discussed with our sponsor.

Finally, we’ve been discussing what our first steps will be when it comes to implementing changes to the code base. Github will be our primary centralized version control, along with pep8 and black to handle the documentation style. One thing that most of our team agrees upon is the lack of style that the previous code has needs to change. The most common complaint that I personally have is the fact that random semicolons are added throughout the program. This is unnecessary, and was likely caused by someone who was extremely used to C style coding. Albiet, none of this really effects the program capabilities, it makes it harder to read.

This week has seen the actual introduction to the project codebase. The actual GUI setup is apparently auto generated from a application. It’s an interesting API to learn about as the GUIs I have worked on the past have all been manually created. I don’t really know whether to think that this is a better option than the alternative. I can see it being helpful if someone wants an easier time to create a general front end interface that does not provide much functionality. However, for very specific actions can problem cause some road blocks in the future. This is not very important, as much of the front end interface has already been finished.

Setting up the webcam and the actual program started with some faults. The Setup page didn’t describe exactly which folder the virtual environment was supposed to be setup in python. This led to some frustration. Along with this, one of the included libraries in the requirements.txt didn’t work, so I had to go out through some discussions online to find a workaround for it. It’s good to keep this in mind in the future, as the program we create will need to be tested on a clean environment setup wise.

The design document so far has been a pretty confusing document to create. The document example helps a bit but it will take a while for my team to figure out what exactly we will want to put down for each of the different design concerns and concepts. Usually a the group documents consists of a lot of combining the different individual documents. However this one requires a lot more conversation about what the contents will be.