Month: November 2023

Course SWOT Analysis

Note: This is the final blog post required for this course. At the time of this writing, the blog is hosted on Oregon State University’s WordPress site. Since this capstone course is my final degree requirement, my login credentials will expire permanently in the near future. That said, I do have further thoughts I’d like to post if I have some breathing room before that happens.

SWOT

A SWOT analysis is an assessment of Strengths, Weaknesses, Opportunities, and Threats in a plan or project. It’s generally presented as a 2×2 matrix that facilities identification of areas in which further inquiry and action may be most fruitful.

A SWOT matrix with row and column labels providing context for the areas of analysis. Image licensed under CC BY-SA 2.5 DEED.

CS 467 — Online Capstone Project

A refresher: I’m on a 6-student team in the online, 3-month version of the course. Our project is SaaS Application for Fire Department 911 Risk Analysis for industry sponsor Levrum Data Technologies. We are charged with continuing work on an existing application that provides response time data visualization for fire departments.

With this in mind, I will analyze the course within the context of my experience with an industry sponsor as part of a larger-than-average team.

The definitions of each category are not necessarily in line with those in the above graphic and are provided as part of the headings when this is the case.

Strengths

  • Sponsor: In our case, the sponsor has been very communicative and helpful. At no point have we been concerned about lack of responsiveness. The instructional team told us they have confidence in the sponsor and it’s well placed.
  • Coordination: Especially with so many team members, this course requires a lot of planning, communication, and feedback within the team. It’s great way to get some realistic project experience.
  • Existing code: While some courses require parsing small amounts of code, the existing code base we had to get familiar with forced us to come to grips with just learning enough to get to work. It was an important lesson.

Weaknesses

Time: Our project is meant to become a commercial product at some point. Making significant contributions in three months of part-time work (including project selection and onboarding), almost invariably results in tempering expectations. It certainly did in our case.

Opportunities (for improvement)

The Microsoft .NET logo. Image licensed under CC0 1.0 DEED.

Stricter Selection: Our team took on a project written chiefly in .NET/C# having stated that only one out of the six of us had any experience with the framework and language. While we did learn enough to get to work, it took us several weeks. The result is that we spent almost half the course in research rather than production.

This is certainly a learning experience on its own and I appreciate it. However, we may have been able to write a lot more working code if we had been forced to take something other than our first choice that was more in line with our strengths.

Threats (significant weaknesses)

Documentation: In this course, the main method of documentation is a video of a narrated slide presentation submitted once a week. Each team member produces a minimum of four slides with speaker notes. A single team member compiles and records the presentation with the addition of in introduction, overview, conclusion, and video clips of the project working.

For a team of six, this results in a 25-30 minute video. When editing time is taken into account, it’s a significant undertaking. For 4 credit course, this consumes a huge amount of available time that could otherwise be used on project development.

A less time-intensive method of documentation would be a great improvement.

Conclusions

On the whole, I found the course to be an effective learning experience. Even the items listed under weaknesses and opportunities are a kind of lesson for both students and course designers.

I certainly won’t be emphasizing languages and frameworks I’m only vaguely familiar with on my resume! Thanks to this course, I am now comfortable with .NET, C# and the Entity Framework for SQL databases in C#. It’s not quite the way I would have preferred to learn it, but I’m glad I did.

Thanks to the instructional team for some great lessons, both practical and personal.

Experimenting with GitHub Copilot

Since this is my first post since before projects were selected, it would be good to get readers up to speed.

I did get placed with the two team members I coordinated with before class (in addition to three others) and we got our first choice project: SaaS Application for Fire Department 911 Risk Analysis for our sponsor, Levrum Data Technologies.

The application calculates and visualizes response times for networks of fire stations.

A detail of the existing UI showing the fire station response time map visualization for Corvallis, Oregon; credit: Levrum Data Technologies

This is a continuation of an existing OSU project. A prototype of the server and browser-based front end already exist. Our team is continuing that work by implementing a database and a control-plane. There are a few issues with that:

  • Only one member of our team has prior experience with .NET & C#
  • The existing code is very well modularized but that means it’s very interconnected
  • The existing code is minimally commented
  • There is very little external documentation

In order to understand and document the code, we’ve employed traditional tools like auto-generated code maps. I’ve also experimented with GitHub Copilot to attempt to speed up the process.

Here are my findings:

People who have spent much time interacting with Large Language Models (LLMs) and generative AI will point out that it’s sometimes necessary to change the wording of requests to get what you want (especially if you’re trying to bypass security features with a prompt injection).

This remains true with GitHub Copilot. But, for certain functions, the precise details of the prompts are invisible to the user. You have to experiment with canned choices by selecting varying lengths and sections of code.

When highlighting a class method and selecting “Explain the selected code,” I’ve found the generated text is generally accurate but not necessarily easier to understand than the code itself. It doesn’t reach outside the highlighted code for context. It will tell you about an argument and its type but it won’t explain what else in the code base might call the method and pass the argument.

In fact, when I tried to use the chat to prompt Copilot to do so, I was informed it does not have access to the file structure. Of course that makes sense; the LLM isn’t on my workstation.

However, highlighting a larger block, like a class with multiple methods, is much more useful. Copilot will provide a formatted breakdown of the class.

A partial example of an explanation of a C# class generated by GitHub Copilot; credit Microsoft

It even provides a helpful prompt suggestion:

Copilot suggestion to define the Strategy design pattern; credit Microsoft

There’s also a plugin for Visual Studio Code only called GitHub Copilot Labs. It’s for experimental Copilot features and that’s apparent when you use it.

The options look promising and helpful:

Code modification options provided by the GitHub Copilot Labs plugin ; credit Microsoft

In practice, you can get different, usually broken results every time you try one. Below is an example of the automatic documentation. Not only did it repeat the same two lines 10 times, it provided an incorrect definition of coverage and broke the code by duplicating the class definition line.

Copilot Labs automatic documentation may create good comments or it may do this; credit Levrum

So, Labs is experimental, as advertised. It may be interesting to play with but it’s not a tool to use for development.

Conclusions

I haven’t had a chance to try the main attraction: pair coding with Copilot. From what I’ve read, it’s useful except when it’s not.

The basic functions of GitHub Copilot are quite useful once you understand how they are best employed. There’s just a very slight learning curve. The Labs functions may be useful in the future if they ever work.

It’s important to realize that LLMs don’t understand anything. They’re statistical inference engines. They can help speed things up a lot but you always have to check their work. To be fair, that’s also true of people. Hopefully not the understanding part, in most cases.