Month: October 2020

This week we started to talk about the actual changes we needed to make in our project. It was confirmed that unfortunately, because of how fundamentally different image recognition is from video recognition, almost the entire neural network will need to be scrapped and created again from scratch. The new network will be much more difficult to create, as the extra dimension to allow for recognition over time means that there will be even more layers of complexity. However, I am somewhat glad we are doing this. Having a fully furnished project handed to us with the most interesting parts already completed would have been a dull task. We get to recreate a network that I have never worked with before, which means that there will be a lot of interesting topics to research.

We also discussed our individual technology review and came to the agreement that it was done incorrectly. We each chose separate sections of the project that are pretty dissimilar to one another, meaning each of us would have to research all aspects of the project. Instead, it was decided that the project should be compartmentalized into separate sections, that way one person would really only need to research and work on their specific section of the project. Pretty obvious in hindsight and I feel like I should have seen this as I have discussed this before with my Mentor at Garmin.

Finally, we got our Intel cameras in the mail. The depth sensing feature using LiDAR is an extremely unique technology. It is undetermined whether it will actually help when it comes to lowering the loss of a neural network. In my opinion, I feel as though the adding of depth may actually make the network perform worse. A hand is usually a singular colored object in a small spectrum of the three primary colors. However, a depth based camera would rapidly shift the colors throughout the whole spectrum. As a 2D CNN would have to keep track of these with all three of the color channels, it would have to be fed more data for this pattern. A harder pattern to distinguish means a more difficult time finding a good minima.

A lot of this week was talking about different aspects of the project and what came with them. The requirements were mostly already gone over in our previous meeting. Unfortunately, the intern who previously worked on this project was out of office, meaning we could not get good in depth requirements for what we were specifically supposed to do. Luckily, a lot of conclusions can be drawn based on the github code provided. We have created our own requirements that we think should be done along with the project. For example, refactoring the code base would a good idea. A lot of the program is developed into individual scripts that are handled independently. We believe it would be a good idea to merge all of these individual pieces so that we could run them from one main file.

We also had to take our time to designate tasks between us. It was especially difficult as attempting to split parts of the project up can lead to carryover between work. Sometimes a project member may have nothing to do because sections of the project he needs to work on are currently being held back by someone else. We managed to get something cohesive between the three of us, but we have not yet managed to contact our other two team members. I am not surprised, as they had very short notice of our meeting to discuss these things. This is something that we as a group work on in the future.

As for the project itself, we may be re-writing sections of it in c++, as the client has requested we port some of it over into an API that is handled in C++. Most of the ideas we have come up with needs to run by the client themselves, but I think our team has a good grasp on where we need to go next.

As opposed to last week, we have finally contacted our client and gotten information on what we are specifically supposed to do on the project. Most of the work on the neural network has already been done. We are instead supposed to optimize how well the network can handle the recognition process. Currently the project can only handle images so well, with an error in recognition which is above what is allowable.

Another part of the project which I didn’t even consider is the inclusion of three dimensional convolutional neural networks to pattern match a video segment as opposed to a single image. Because some letters in American Sign Language require an aspect of movement, like the letter J, the network would need to analyze multiple images that are in sequence. Out of everything that has been gone over so far, things like LiDAR (Very similar to how GPS signals function) and CNN’s, this is what I’m most unfamiliar with. I believe the ramp up to understanding how we will incorporate a system to recognize movement will be fairly fast, but it will be difficult in of itself to implement. Many tensor libraries have a general lack of support for them and not very many guides that cover the subject.

It’s a bit disappointing to pick up directly where someone left off when all the most exciting segment has already been setup. From here on out it will essentially just be changing the size of the neural network, or changing the the hyperparameters. The same goes for the actual LiDAR camera, creating the proccessing for the images for the images would be really cool, but that is not an aspect of the project we are really working on. Although, this is a minor complaint, as there is still a lot of cool things to mess around with.

We are supposed to be working with PyTorch on this project. Getting into the headspace of programming in machine learning is a difficult task to accomplish. However, Neural Nets are given a strange degree of “deceptive complexity”. In reality, high level understanding of neural networks make them seem simple, by most standards. Mind, the calculus that makes up a Neural Network still requires a lot of background knowledge to understand. However, basically anyone with a limited understand of computer science can make a Neural Network with the tools that are currently provided.

Tensorflow, Keras and PyTorch are prime examples of this, with many of the most difficult to create features (back propogation and 2D kernel generation) being boiled down into one or two lines of code for the creator. This is the main reason why projects like these are even possible for students. Building a Densely Connected Neural Network from the ground up, even a very small one, takes a load of time and knowledge. This isn’t even entertaining the idea of training the models after the Network has been created. Instead, a student can simply reach a level of enough understanding to get at least a jist of how Neural Networks work and then let something like PyTorch do the rest of the work.

As for what has gone well and poorly with the project on this week, my teammates are very diligent when it comes to communication and work done already, which is always a good change of pace. Unfortunately, out client has not yet responded to us yet, which is quite worrying. Hopefully, it just a minor setback, as it is the weekend.

What could be done better in the course? I think more transparency on how the selection process works would help. I have heard that it was simply a random selection, which didn’t give me a large amount of confidence in the rest of the course. Of course, this is conjecture, as I don’t actually know whether any of that is true. What I enjoy about this coruse currently is the amount of leeway we get when it comes to oversight on the project. Most things are left to the client and the students, which is nice.