Agile vs. Waterfall

I’ve work in product management for a decade now. Over that course of time, I’ve watched the company change software development philosophies several times. Not all of these changes resulted in positive improvements, but the impact of these changes on the software development lifecycle and engineering culture is fascinating. For this blog, I discuss the four different approaches taken during my time with the company and the results of those approaches. Note that not everything in here is typical of these processes. In many cases, we implemented the approach in an ineffective way, which lead to some process failure. These are my experiences, and your mileage may vary.

Stage 1: Kanban

When I first joined the product team, many moons ago, the young company had only a single team of engineers. Back then, software development used Kanban, including a Kanban board that listed the various tasks needed. Under this form of Kanban, sprints do not exist. Instead engineers accept new tasks immediately after finishing a prior task. Product managers were required to constantly groom new tasks to keep up with the engineers.

One of Kanban’s focuses is on constant deployment. This frequent deployment and change in tasks made software evolve at lightning speed. Customer requested features were often implemented within days or weeks of getting the requests. Everyone delighted in the constant smorgasbord of new features. Of course, no one was very happy when a new feature broke something. However, we could repair any outage within hours or rollback with minimal effect. The product was fun to work on, even if keeping up with the constant change was taxing. UX and marketing experienced the most hardships. The fast timelines left UX little room for testing designs and the impact of new features. Marketing also struggled to keep up with engineering and complained that the constant deployment meant they lacked a big press announcement about new features. Note that Kanban can accommodate bigger changes with proper notice to marketing and UX of what projects are coming up. However, in this case, planning that far in advance never happened.

Kanban was super fun and, for me, the upsides of speed and agility outweighed the negative aspects. The teams were forced into a tight-knit group in order to meet the need for speed and constant change. This made going to work very pleasant and rewarding.

Stage 2: Classic Scrum

As the company grew, Kanban became increasingly difficult to manage. With additional engineers and new teams, conflicts arose both in work and in code. In turn, the conflicts increased the number of outages and rollbacks. I believe Kanban can work with large teams and comprehensive unit tests, but the discipline required from engineering proved too much. We switched to classic scrum.

Using classic scrum, the teams switched to two-week sprints, daily standups, and a dedicated scrum master who held the members of the team accountable for their performance. Each product manager was assigned as specific area of the project, allowing the engineers on that team to focus their skills .In addition, we added true cross-functional stakeholder meetings and sprint demos. These meetings were a relief to both UX and marketing as the heads-up gave those teams room to plan.

The primary downside of scrum was the noticeable loss of speed compared to Kanban. With sprint planning and team coordination, fixes that used to take a day could take up to three weeks. One week of planning plus two weeks of waiting for the change to reach the top of the priority list. The other perceived downside was the leadership team felt a lack of control over the planning process. Roadmaps in scrum are ordered lists of priorities. This mean that the executive team was never sure when a project would complete. Combined with the lack of immediate changes, the use of scrum led to frustrated business leaders.

Of the various approaches, I thought classic scrum worked the best. Although we operated slightly slower, the total amount of product produced over all teams was significantly higher. I also liked having slightly more time to plan and the focus of product on future changes rather than the immediate tasks.

Stage 3: Large Scale Scrum (LeSS)

The engineering teams expanded with additional company growth. Eventually, we reached a critical mass where, once again, the engineering teams were stepping on each other. With six teams working on one project, each with their own product manager, conflicts became common. The PM releasing first usually ended up owning that particular part of the product.

To fix the conflicts, the company pivoted to large scale scrum (LeSS). LeSS is similar to traditional scrum but is designed to deal with multiple engineering teams working on the same system. LeSS features a main product manager that coordinates between the product owners, each of which has responsibility for certain areas of the product. The product manager decides which stories are a priority based on the the customer impact and completeness of the specification.

The main issue with LeSS is political. With one person deciding priorities, the different product owners felt more like advocates for particular agendas than dedicated to a single team. The lack of embedment with engineering corroded the relationship between product and engineering, leading to less trust in the product plans. On top of that, the UX team could never be sure which product owner had the most important task. Sometimes UX would end up working on a part of the product that never received prioritization. With LeSS, if the product manager is constantly diligent, the whole process breaks down. LeSS also didn’t solve the management oversight problem.

I liked LeSS slightly less than Kanban. LeSS ended up feeling too political. The product manager simply didn’t have the time to review all of the proposals and sort out the presentations effectively. I think LeSS would have been more successful if the engineering teams aligned more closely with the product owners rather than the product manager at the top.

Stage 4: Waterfall

The final, and current, evolution of software development approaches was a retreat to waterfall. Despite the CTO trying to pivot towards a more agile framework, the CEO and current SVP of product really like the theoretical oversight waterfall provides. The waterfall approach features a beautiful project plan. Managers and sales can share this project plan, pointing to what the future holds. When the project plan came out, everyone was excited about the end-state product. Customers especially appreciated the long notice of what we planned to release and the end-state specifications.

Unfortunately, we ended up experiencing every classic waterfall pitfall. The project has significantly deviated rom the project plan. All parts of the project are late. That big fancy document that the SVP touts around? No one reads it. Everyone still likes the end-state, but the actual plan is more of a marketing and sales tool rather than a product specification.  This causes friction between the business and engineering.

Although the waterfall process appears super organized on paper, the reality is chaos. Despite being happy about the documentation, the management team is more frustrated than ever because of the missed timelines. I personally have found the decline to waterfall fascinating. It’s a textbook case about the pitfalls listed in every article on agile.

Summary

My recommendation, based on experience, is to stick with agile. There are lots of agile methods available, and every single one worked better than waterfall. I loved Kanban, but I definitely experienced issues in scaling Kanban to large organizations. If I could change one thing, I’d go back to classic scrum.

File Explorer and Unity

Card Creation

One of the most interesting parts of this capstone project is allowing users to create cards that are then saved to a database and used to build custom playing decks. Naturally, I volunteered for this assignment. Most of the card creation system implementation is straight forward. I set up text information for card statistics and flavor text information. I added images that change based on a card’s characteristics. This information is saved via an HTTP request to the server (which part is still under construction).

However, there was one very difficult requirement to get working. The project specification allowed a user to upload an image and then use that image in the card design. Initially, I thought this would be something readily built into Unity. I was wrong. Very wrong.

There is a lot of complexity surrounding this one little command:

Uploading Images

Uploading local images in Unity during a game’s run time is surprisingly complex considering this is a common Window’s task. The complexity must be why so many games allow users to pick skins from a predefined list. Accessing predefined information in Unity is pretty straight forward. Simply access the “Resources” folder and load the asset into the project, assigning the loaded asset to the desired UI element. You cannot use the “Resources” folder approach if the information needed is not known before compiling the program.

Unity has logical reasons for making loading local assets a difficult task. First, Unity is designed to support multiple platforms. The different platforms do not share a uniform System.IO object, meaning that the windows-type file I/O is not a universal solution (c# – Attempting to upload .png file to web directory – Stack Overflow). Second, there are concerns about copyright and needing to monitor what images are uploaded. I can see why this a concern for commercial developers. The concern is almost zero for a class project.

Non-Solutions

I was surprised that the Unity community forums contained only a few good ideas on how to solve this issue. Most of the answers I found are examples of how to work with files when you already know the file location. If you know the location, you can use the UnityWebRequest object to load the file and use it in the project. Despite the name, the UnityWebRequest object can read local files across platforms.

Other suggestions used the deprecated www object in combination with the Unity EditorUtility. For example: Unity – Manual: Uploading raw data to an HTTP server (PUT) (unity3d.com). These solutions used Unity’s EditorUtility to open a file and read the information using www tools. This works great…in theory. Unfortunately, EditorUtility only works in editing mode and will not work with deployed code, making the solution impractical for the final product. The real answer is: “There is no built in file browser for the player, so you would need to roll your own using the .NET frameworks classes for file system access” How do I let the user select a local file for my application to use? – Unity Answers

The Real Answer

Frustrating, right? At this point, I became very worried about the project’s scope. Writing a new .NET application seemed like a capstone project by itself. I didn’t think the team could create both a new application and develop a game. Frustrated, I was about to move on to a different part of the project. Luckily, I remembered the unity asset store. Eureka!

Searching the store finds a few different option. The free one I used is:

https://assetstore.unity.com/packages/tools/gui/runtime-file-browser-113006

This plugin allows you to load multiple files, save files, and specify search criteria. Likely overkill for what I need, but the price sold me.

Getting It Working

When I first imported the plugin, I could not get the tool to work. This was my mistake, and I failed to import the C# script into the project library after importing the package from the asset store. This failure caused Unity to replicate a similar script in the resources location and crash when run, stating that two assets were detected with the same name. To get the plugin working correctly, I needed to import the plugin into my C# code before testing. I also had to delete the created file. After fixing my installation issues, the plugin worked great and included very comprehensive user documentation. You can see it working below.

Working File Explorer

Yay! The code to make this work is a small modification on the example implementation.

	private void GetPath()
	{
		FileBrowser.SetFilters(false, new FileBrowser.Filter("Images", ".jpg", ".png"));
		FileBrowser.SetDefaultFilter(".jpg");
		FileBrowser.SetExcludedExtensions(".lnk", ".tmp", ".zip", ".rar", ".exe");
		FileBrowser.AddQuickLink("Users", "C:\\Users", null);


		FileBrowser.ShowLoadDialog( ( paths ) => { path = paths[0]; },
								   () => { Debug.Log( "Canceled" ); },
								   FileBrowser.PickMode.Folders, false, null, null, "Select Folder", "Select" );

		// Coroutine example
		StartCoroutine(ShowLoadDialogCoroutine());
	}

Conclusion

In retrospect, the first place I should have looked for an answer to my problem is the asset store. Being new to unity, the idea didn’t occur to me. I thought of the asset store as a place to buy sprites and images, not useful tools. Turns out the asset store is a real boon to the community and filled with useful items. The biggest takeaway from this headache is that when you get stuck, look at the asset store. There’s a good chance someone already solved your problem.

LucidCharts: A Love Letter

In this blog, I wanted to share one of my favorite pieces of software, LucidCharts. I use LucidCarts quite a bit for work, and I find that it’s one of the best ways to coordinate across large teams. LucidCharts helps me organize my projects and trains of thought.

Apologies to Elizabeth Barrett Browning on this parody of her sonnet. To break up the poem, I included images of a few projects I’ve done in LucidCarts over the course of my tenure at OSU.  

The Poem

LucidCharts, how do I love thee? Let me count the ways.

I love thee because of thy ERDs and UML diagrams, which are so easy to create.

ERD for the gaming project

My soul can reach (to LucidCharts), when feeling like I need to start mapping out a project.

Laying out a personal project

For the end of hand-drawn images and the ideal packages that contain all the icons I need.

Example of how many icons exist just for users

Most quiet need, whether I’m working on OSU homework by sun or candle-light (mostly candle-light).

I love thee freely, as I strive to get my project plan just right.

I love thee purely, as you turn diagrams into creative layouts.

Design of Card Creator

I love thee with the passion because it’s so easy to use,

Compared to my old griefs with my childhood’s faith in spreadsheets.

I love thee with a love I seemed to lose using Microsoft Visio.

Especially with my lost files because of failures to save, I love thee with your cloud-based saving,

And the lack of tears when doing all my homework; and if my instructor choose,

I shall but love thee better after this class ends.

The End

I really like LucidCarts.

You can find the real poem by Elizabeth Barrett Browning here: How Do I Love Thee, Let Me Count The Ways: Sonnet Analysis (nosweatshakespeare.com)

Ready…Set…Go!

For me, the hardest part of any project is getting started. New projects always feel like you’re lost in a big world with every direction leading to new options. The best way to get somewhere interesting is just start moving.

One of the most interesting elements of this class is the lack of a set directions on how our team accomplishes our goal. Previous classes set forth the required language, the desired outcome, and the specifics on how you implemented that outcome. With CS492, everything changes. Suddenly, you have a wide selection of tools and a project where “complete” is left to the imagination. This is both exciting and terrifying!

Just Start Already!

The best way I have found to overcome the anxiety that always seems to accompany new experiences is to just start moving. Type out notes. Sketch a framework. Do something or anything to just start moving! Starting early often means moving in the wrong direction. However, the act of starting starts the gears in motion, allowing me break through the nervousness and focus on the problem.

For my capstone project, I applied this practice by writing out what I wanted to accomplish, consolidating the project outline provided online with what I thought sounded interesting. started by writing out what I hoped to accomplish. I also watched several videos on how to code in unity and set up the repository in Github. These small acts made me feel more confident in the undertaking, got things organized, and was fun to do.

The Second Step

Despite my initial nerves calmed by the act of starting, I still found the project very overwhelming. The project just seems so big! To further “just get started”, I diagrammed a possible flow/architecture for the project. This is the diagram I came up with:

A possible project flowchart

Sketching out diagrams like this help give you direction and help solidify what you want to accomplish.

Rough diagrams like this help me visualize the various steps along the way, breaking the project into more manageable chunks. They also give me direction. Even though the team hasn’t scoped the size or efforts of these chunks, diagrams can help split up an huge undertaking into smaller, and doable, steps.

Added Benefits

One added benefit of starting on a project is the process of starting might find holes in your thought planning process. For example, putting together the flow diagram made me realize that the team never discussed a very important part of the project in planning – how do you actually play the game! The project’s main goal is to allow users to create customizable card games, but we need a simple set of modifiable rules that a computer can follow when becoming an opponent. We are working on creating these now, but thank goodness for staring into the project! 

Conclusion

For me, jumping straight into the work is the best way to start a project , feet first. Even if the project initially feels overwhelming, the act of starting helps calm my nerves, organizes the process, and allows me to better see the project’s bigger picture. Just doing something towards the end-goal makes big tasks seem smaller.