Wild Wacky Adventures in Senior Software Engineering

This week was mostly uneventful project wise, it’s been a lot of discussion about direction, and doing research. Work however got crazy this week, which resulted in me neglecting some of my school work so this weekend was meant to be playing catch up on the things I haven’t been working on.

Friday comes around and I go to check the calendar to see what’s coming due. Two big stats assignments that I usually do one at a time on the same day by midnight… great. First I have to go back and watch the lectures I had to skip for work meetings. After a full day of grinding I make the deadline. Nice.

Saturday morning check the Operating Systems II calendar on it’s own site. Oh my god, lab3 is due by midnight? It’s been out for three weeks and they’re usually huge assignments that I knock out exercise by exercise. It’s going to be a close one. After sitting at my computer panic coding, not eating, not showering, not doing anything but slog through kernel code; from 11:00AM to 11:40PM, I’m close but haven’t finished. I don’t think I’m going to make it. In an act of desperate hope, I check the due date one last time hoping I misread. I did. It’s due Monday, 9 Nov, the course calendar for that class outlines the current day in blue which when I looked at it, said Lab 3 was due. What I didn’t notice until the wee hours of the night, was that the calendar doesn’t include the weekends and just highlights the next available entry. Rough day, but at least I made progress.

Our project has some peculiarities to it that are going to make implementing a neural net a little bit trickier than it might otherwise be. More or less, we are supposed to be using a LiDAR camera interpret someone signing in American sign language and transcribe it. Only problem is training a neural network takes massive amounts of data and there simply aren’t many people recording themselves signing in front of a LiDAR camera. I’m pretty sure that we can’t train the neural network on one type of data and then feed in an entirely different kind of datatype and expect it to accurately classify. Which leaves a couple of options. Create a dataset of signing with LiDAR camera data, or ditch the LiDAR and try to use regular video. I don’t know sign language, and no one else in our group does and this would require a massive amount of data, probably thousands of hours, so that makes the first option unattractive. Even if we did try to learn enough to record a lot of examples, we’d likely end up with a neural network trained to only recognize really bad amateurish sign language and not much else. The second option still requires finding a dataset…

Which I did! It’s called the WLASL, or Word-Level American Sign Language. They basically created a massive json with thousands of labelled of entries to videos of people speaking sign language, a download script, and a preprocessing script. It has a ton of examples, too many in fact. I talked my way into getting access to the pelican cluster for training but after pulling the repo, starting a screen session, running the download script, and going to bed, I woke up to a ton of warnings about overfilling my allotted hard drive space on LDAP, and having a basically unresponsive server account. My solution was to delete everything, fork the repo, modify the download script to download a smaller subset of the training data, and start from scratch. I figured if we could get a neural net working to a reasonable level training with a smaller dataset, we could figure a way to download the rest of the dataset and increase our accuracy later on, that way we can at least be somewhat confident in our proposed architecture before going all in. Well that was going well after tons of refactoring, but then I realized the preprocessing script was massively inefficient, copying videos that didn’t require preprocessing, and leaving the original raw videos in place either way, effectively doubling the amount of space required for the raw and preprocessed videos. In fact it actually converted everything to mp4 in a separate directory, then preprocessed only the frames of interest from those into a new directory, effectively tripling the required storage. Some people… Anyway I’m still in the process of trying to fix that too but opencv is giving me cryptic issues.

On top of all of that, it’s Halloween weekend. A wise professor once told me

“Sleep more than you study,
Study more than you party,
Party as much as possible”

– I can’t remember his name, but he was awesome

If you know me you know that I’m a huge advocate for using Linux, terminal, and Vim. Out of the box, Vim may not be the most approachable piece of software to use. With a little (a lot) of time spent customizing it, it can become one of the most powerful tools in your arsenal. I’m so hopelessly down the rabbit hole of Vim, that I’m currently writing this blog entry in my terminal and when I’m done I’ll copy it into the browser.

I find most IDEs to be bloated and confusing. There are options and settings deep in menu trees and drop down menus. You have to touch the mouse all the time. I’m sure there is a lot of functionality they provide but I don’t know that it’s there, where it is or how to use it. Instead of figuring all that out, I decided to take this lightweight, fast, sleek, portable, ubiquitous text editor and bloat it to hell myself. Piece by piece, one at a time, on my own terms kind of bloat. Every feature I add to my Vim has been put by me, I know how it works, I know what it does, and it’s configured to do work the way I expect. At this point my Vim behaves pretty much like an IDE. There’s code completion, linting, auto pairs, indentation, git diff indications, file tree navigation, debugging instances, snippets, filetype specific settings, and more. Discovering plugins has been a blessing and a curse. The first thing I do when I find out I’ll be coding in a new language is google ‘ vim plugins’, start shopping, and download everything. If I spent less time streamlining my setup trying maximize efficiency, I might actually be able to get something done.

Anyway, last night I decided to burn it all down.

Auto completion is one of the most vital pieces and it’s the hardest to implement correctly. ‘YouCompleteMe’ has been my go to code completion engine for a long time but it’s a pain to maintain. The core server requires compilation so every update to YCM or Vim requires a recompile and the config and completion flags are finicky. Overall it works as advertised once you get it to work, but getting there can be a struggle. The most recent update to YCM changed it’s requirements so it no longer works with my install of Vim. Further, the maintainers of YCM aren’t tagging their commits with versioning so I had to manually go find the breaking commit, check out the hash, and recompile.

My operating system is Debian, which is great except for the fact that packages in the repo are often older. A lot of plugins require a Vim install of a certain version and often patches up to a certain point, which aren’t always easy to find. Which means compiling Vim from source; not too difficult, but enabling certain features requires having the proper libraries or header files installed which are tricky to track down.

All of this started to seem excessively tedious so I decided to look for other options and I stumbled upon ‘Conquer of Completion’. CoC works on Vim8 and Neovim. Neovim is a heavily modified fork of Vim, inspired by the fact that the original author, maintainer, genius, and benevolent dictator for life behind Vim, Bram Moolenaar; Is very picky about what goes into Vim and has ridiculously high coding standards. Anyway I was interested so I installed Neovim, low and behold version too low (thanks Debian). Uninstall, pull source, compile (worked out of the box whaaaat?), install, then stay up until 1 am playing with it.

Ok, cool it works. I can do my homework now… Why am I the way that I am?

This week flew by and there was a lot to do. Our group finally met with our sponsor, and the project sounds really interesting. We’re working on a machine learning algorithm that uses a LiDAR camera, capable of sensing distance, to recognize and interpret sign language. This project seems like it will be a challenge since machine learning is not something I’m very familiar with. Learning how to integrate the LiDAR camera into code is also something I’ve never done but I’m excited to experiment with.

The project is very open ended, which is both a good and bad thing. We are given a lot of freedom around where we’d like to take the project. Last year another capstone group started the project and is passing it on to us. If we chose to, we could redesign the project from scratch, using different tensor libraries or languages, move computation to the cloud, or enable it to interpret video information, rather than just stills. The downside being though that the direction and requirements of the project are not very well defined. As a group we’ll have to discuss what we want to accomplish and what we are able to accomplish in our time frame. Estimating how much time it will take to complete a given project, even of much smaller scale, is always a difficult task, so planning out nine months is going to be a challenge.

I think this project is really cool and a fun challenging problem, but I’m a little confused about the actual usefulness or application of the system. It’s supposed to be assistive for people who have difficulty communicating by speech, but it’s primarily deaf people who communicate through sign language… but deaf people can still see, which means they could watch someone signing and understand it, or read. So I’m imagining a deaf person spending over $300 on a LiDAR camera, setting up our project and running the code, signing into the camera, and then reading what they just signed on their computer monitor. I can understand speech recognition because there’s a lot of audio based media and information that’s inaccessible to people with impaired hearing, but I’m struggling to see the use case for this. Doesn’t matter to me though, since I’m not trying to make any money off of it and also not paying for any of the stuff. If someone wants to hand me a fancy camera and teach myself how to do machine learning, I’m all over it.

I woke up at 8am yesterday, got dressed, sat down at my desk and got to work. Before I knew it, it was pitch black out 10 o’clock at night and I hadn’t eaten. This is fine.

Spent the bulk of the day working on stats homework, writing a probability calculator in R which I have decided is the worst ‘programming’ language ever invented. Arrays are indexed at 1, member fields and functions of their janky class implementations are accessed with a ‘$’, dots; which usually access member fields, actually are just interpreted as an arbitrary name, builtin functions have no naming convention consistency either; mixing camelCase, smashedtogetherfunctionnames, and their confusing dot.access.names(‘why?’).

Work in October dried up due to my primary client running out of budget for the quarter, sidelining several big upgrades I had in the works; and we were between contracts on another big client I was going to be added to. First week of school, old client pops up out of nowhere, an internal project finally gets some movement, new client starts on-boarding, and I get asked to build some packages for a repo and address some issues on their projects. Cool, but where was this energy last month.

Guess I should talk about the project, since I’m here. Got in touch with my group mates and we met. I’m feeling pretty good about my team. I’m a little scarred from my last nightmare group project I wrote 90% of the code for (confirmed by github analytics). These people seem very smart and proactive, so I’m going to do my best to keep up.

I emailed our sponsor as soon as the contact information was sent out, then again the next day, then my group mates emailed as well; only to wake up to an announcement from our professor saying the class isn’t being proactive enough and it’s unprofessional. Probably not talking about us specifically, but still. Since we haven’t been able to get a hold of our sponsor, and the project page is short and vague; my writing assignment may be a brain dump of speculative nonsense but you work with what you’ve got. Try not to get burned out.

Welcome to my blog where I’m completely voluntarily and of my own accord going to be  sharing stuff about myself and computers I guess.

I have always been vaguely computer savvy but nothing too fancy, and never learned to code. I originally was going to school to be a pilot but took a CS class and decided that’s what I was going to do instead. I transferred to OSU and am graduating this year plus a term, if all goes well. I joined Linux Users Group and accidentally became the project  manager my first term, joined the OSU Open Source Lab as soon as a position became available, did an internship at Tag1 Consulting last summer and have decided to stay on with them part time as I finish school. I love coding but most of my experience has taken me down a specialty in infrastructure and configuration management. I’m a big fan of Linux and encourage everyone to learn it. There is definitely a learning curve to some aspects, but if you put in the time to learn and configure your environment, you’ll never be able to look at anything else the same.

I’m looking forward to working on a project and getting to know some of you!