Taco Libre

One of my teammates made a very nice CI/CD pipeline for us in GitHub for our capstone project. During the last few weeks, I ran into an issue where some of the test cases would run for me on my local machine but not in the pipeline. In the end, it turned out to be something to do with the path to an sqlite database being different when imported between two python projects and inside of virtual environments. I still don’t know for sure what the problem was, but the short answer is that we solved the problem by just using an environmental variable to the absolute path.

While trying to debug the pipeline, I ended up pushing at least a dozen commits, each which caused the pipeline to run. My teammate was clever and used some caching to speed up the process; nevertheless, it got me wondering about how much it was costing to use the service. I asked my other teammate how much GitHub actions cost and he informed me that they were free. For some reason I had it in my mind that we only had a limited number of minutes for the pipelines and the free part had to do with being students and having a pro account. This turned out to be false. After some googling I discovered that some time ago GitHub actions were made free for all public repos. I was living in some alternate universe.

I immediately looked through some of my personal GitHub projects and tried to see where I could exploit this new free feature. I set up a pipeline to run unit tests for my incremental backups project (https://github.com/erietz/incremental-backups). The template that GitHub provides for python projects includes the ability to run your tests using multiple different versions of python in parallel. I quickly discovered that my code failed on python 3.8 while it passed on 3.9 and 3.10. On my personal machine I think I had 3.9 so I never would have discovered the issue without the pipeline.

I then got to thinking that I could create a pipeline for my machine setup repo (https://github.com/erietz/machine-setup). In this repo I use Ansible to clone my dotfiles (https://github.com/erietz/.dotfiles), install a bunch of software that I depend on, and run some install scripts to prepare a system how I like it. Since I have dotfiles for Manjaro, Ubuntu, and macos, this turned out to be more work to get everything working correctly. My most used operating system is Manjaro and it is slightly disappointing to see that the only Linux OS available as a runner for the pipeline in GitHub is Ubuntu. For the Manjaro workflow, I run the entire thing in a docker container on the Ubuntu host.

In an ideal world, the code required to configure a Manjaro machine versus an Ubuntu machine would be practically identical. However, many of the packages between the two systems have different names and there are other quirks which make the Ansible roles different enough that I can’t find a way to reuse the code. I favor simplicity over complexity and having to configure three different types of machines leads me to want to abandon Manjaro and switch completely over to Debian based distributions. In this way, I could run my Ansible playbooks directly on the machine in GitHub and keep things simple.

Those System 76 laptops look pretty slick and they come with Pop OS 🙂

I completely forgot to do this before the due date. We will see if I can submit a link a day late and still get the 10 points.

If you are reading this, please, please, pretty please 🙂

Python was the first language that I learned. I started dabbling with it long before I ever started at OSU. During my time here in the post-bacc program, I would say that the majority of my course work was completed in python. I was very comfortable in the language and I would go so far as to say that I used to think in python. However, I started working full time as a software developer several months ago and now code entirely in C# at work. Recently I have been learning go in my free time and even started a project at work in golang for a hackathon.

For our capstone project, we have agreed to implemenent all of our micro-services related to our machine learning project in python. This week I put together some of the scaffolding for the API that we will use to connect our neural network predictions to a web interface. I have to say that when I fired up neovim and opened up a new python file, I felt completely lost. At my place of employment, it is not uncommon that before implementing a large feature, or when starting on a new project, we are first required to open a PR for a detailed design. In C# this could entail writing interfaces which describe the methods that will be later implemented. Those methods will have both parameters and returns values with types that are clearly specified. This may also involve creating classes with typed data members which establish a data contract between services. These ideas do not exist in python since there is no concept of an interface and types are optional.

I was stuck when it came to creating a detailed design for a relatively simple API in python. After reading and writing so much in strongly types languages, the lack of types in python makes the code unintelligible. How do you indicate to another programmer reading your code what the function is suppose to take as parameters and what you expect it to return? Do you write comments? I know that optional typing is becoming more common in more recent versions of python3, but even so you have to rely on linters to see if you made a mistake. Typescript solves some of these problems by having to compile your code into javascript so you can see at ”compile” time type related errors.

I will include one example here from the go project that I am currently working on which shows a concept that does not exist in python.

In go (and in C#) you can decode json into defined types. I have a struct like this which holds options related to an azure service bus.

type Options struct {
	Topic          string    `json:"topic"`
	Subscription   string    `json:"subscription"`
	Count          int       `json:"count"`
	Start          int64     `json:"start"`
	Unzip          bool      `json:"unzip"`
	EnqueuedAfter  time.Time `json:"enqueuedAfter"`
	EnqueuedBefore time.Time `json:"enqueuedBefore"`
	Save           bool      `json:"save"`
	Queue          string    `json:"queue"`
	MetaData       bool      `json:"metaData"`
	Query          string    `json:"query"`
}

I also have this method called ParseFromJson which takes a pointer to an Options struct, and sets the values by parsing json into the Options struct type.

func (options *Options) ParseFromJson(filename string) {
	file, err := os.Open(filename)
	if err != nil {
		exitError(fmt.Sprintf("Could not open config file %s\n", filename))
	}
	defer file.Close()

	bytes, err := io.ReadAll(file)
	json.Unmarshal(bytes, options)
	options.Validate()
}

After the ParseFromJson method executes on the Options struct, the options struct is modified in place. In python, when you read content from a json file using the json package from the standard library, you get back a dictionary. There is no way to specify that shape of that dictionary. By that I mean you cannot say ahead of time that you are expecting to decode your json into an Options class with a property of EnqueuedAfter which is of type time. You just get back a dictionary and the keys and values may be completely random.

By the end of this term, I will write another blog post where I will reflect again on my feelings towards python after finishing the project. Maybe I will become comfortable again with simple dynamic duck typing.

For my first blog post, I think it is only appropriate that I am to blog about vim, the greatest text editor ever created and the reason that I am in this program today. I first started learning vim in about 2017 when I saw a guy on YouTube editing $\LaTeX$ very fast. He was showing off his set up and I knew that I had to have that.

It has been five years and I can say that I am a professional vim user, but I still learn new things all the time, even some features that are quite basic. Over the course of the last five years, I have developed muscle memory for accomplishing routine text editing tasks. It was not until watching another vim user that I realized that there are other, possibly more efficient ways, to accomplish the same task in vim. While watching The Primeagen on YouTube in this video https://www.youtube.com/watch?v=uL9oOZStezw, I caught on to two different ways in that he does the same task as I do.

I will share with you here the first task as it is easy to explain in text. Given a few lines of code like the following:

| Header 1 | Header 2
| ---      | ---
| thing 1  | thing 2
| thing 3  | thing 3

one may want to add a | character after each line to complete the markdown table. Normally, I would perform this series of steps

1. Move the cursor to the end of the first line (where the “2” is)
2. Hit “ctrl-v” to go into visual block mode
3. Navigate the cursor to the “3” on the last line by using a series of movements
4. Then use the command '<,'>norm A | to append a | character to the end of each line
5. Use a vim plugin to align all text at the | character so my code looks good

I have found that The Primeagen does something different which uses less key strokes.

1. Hit vip to visual select the paragraph.
2. Use the command '<,'>s/$/ |/ to substitute the end of each line in the visually selected area with a space and a | character.
3. Use a vim plugin to align all text at the | character

Now that I have broken down the steps, I realize that in my first approach I could have also avoided visual block mode and used vip and the norm command works just the same. Now, spending the time to improve this tiny text editing habit that I have to perform at least once a day, I will shave off many seconds in my life!