Planning for Program Intersections

“The wise speak only of what they know.”

They say hindsight is 20/20, you always know what the right approach could have been in retrospect.  That is where the value of experience comes in.  In our capstone project we are learning a lot of valuable lessons, ones that have probably been learned many times over by professionals already in the industry.  The most important lesson learned recently is planning for program intersections.  Or, figuring out how everything talks to each other.

We are young professionals.  We have an academic understanding of how to implement frontend frameworks and RESTful APIs.  So we were excited to work with them.  How the restful API will work was a good portion of conversation, coupled with what technology we were going to use on the front end.  What we didn’t pay as much attention to was how these two pieces were going to talk, where they intersected.  Which is arguably, the most important part of it all.

There are endless ways to correctly program a RESTful server (endless wrong ones too).  There are a million plus ways to design a front end application correctly.  But if attention isn’t being paid to how they communicate with each other, or the structure of the program intersections.  Then you can end up with two programs that are supposed to be part of a whole, but are not working together.  

When I talk about the program intersections, I mean the structure of the GET/POST requests.  Are you using JSON, if so what is the structure?  What work is being done in what part of the program?  The way the applications communicate in this space directly impacts how they are both built.  If it is not talked about ahead of time, then chances are someone is going to have to redo their work.  Different assumptions made by different team members can be a dangerous thing!

Luckily in our group we caught this early on.  It became very obvious once we saw it, so it didn’t pose a major problem.  Still, it gave us all something to think about for our future projects.  We chatted for a bit about what would have happened had we not caught this early on.  What is the worst case scenario when two halves of a project show up for deployment and they aren’t even speaking the same language. 

Even writing about this blog post, this feels like it should have been more obvious than it was.  But that is the nature of things.  You spend so much time thinking about how you are going to spend your time building the part you are responsible for, that you don’t think about how it will correlate with the parts of your team members.  Software development in a team is a collaborative process, and if you lock yourself away and focus on your responsibilities, chances are you are missing part of the full picture you are working on.

Job Hunting

“All’s well that ends better.”

I have begun the Job Hunt. Well more appropriately, I have been on the job hunt for awhile now. This blog post won’t answer any questions, in fact it might posit more questions than anything else. But sometimes it is important to step back and process everything you have gone through.

I feel like I am a strong candidate for employment (who doesn’t feel like that though). I graduate in December, I have a 4.0 in the OSU program, I am continuing my education with an online Graduate Program in January (Georgia Tech, designed for working professionals), and I have successful job experience from my previous career. I have spent time building an online portfolio, cleaning up and formatting my resume, and practicing l33tcod3 problems when I should be cleaning the kitchen.

Still, in an industry that is suffering a labor shortage, I am finding it really tough to even get interviews! I picked my brain about this for awhile, and I have started realizing a few things.

  1. As entry level Software Engineers/Developer/Code Monkeys we are an investment by any company that picks us up. We learn theory and get a broad exposure to a lot of different technologies in undergrad. But what we see in this program is a puddle in comparison to the ocean of the world of Computer Science. But there is no other way to be, if we were to become experts in all aspects of the field, we would never graduate!
  2. So far those people that do the technical interviews (as opposed to the behavioral) are in more technical focused roles. That means they are usually taking time out of their day to meet with you. That is a big ask for a lot of people, and it can show sometimes.
  3. I am use to interviews coming within a week of submitting an application (from my previous career), that it is not the case in the Tech world (for me). I have had recruiters and companies reach out a month plus after the application was submitted. Things seem to move a bit slower in this industry.
  4. Career Fairs have given me more success than anything else to get past initial interviews.

As I analyze these trends, I am going to try to levy them in to more success in the job hunt. I recently made it to the final round of interviews for a company I had interest in. But I bombed the technical interview. It was a painful learning experience as I knew all the questions that were asked, but I found myself nervous and thus stumbling over my answers. Learning to manage your own expectations and ego (for both bruised and big egos) is a lifelong learning challenge for all of us.

But. You learn and you move on to the next interview. I am working hard to keep the 5 P’s in mind, as they have served me well up until this point. Proper Planning Prevents Poor Performance.

QuantConnect 101

“You have no authority here! Your orders mean nothing!”

QuantConnect is a platform for designing, testing, back testing and trading your algorithmic trading strategy. With the rise of algorithmic trading and retail interest in the stock market, I thought I would highlight some of the outstanding parts of this platform. Welcome to QuantConnect 101!

I Go, You Go, We Algorithmic Trading

So what is algorithmic trading and why do we need a platform for it? Well first and foremost, lets look at what an algorithm is. An algorithm (in the most simplest of senses) is a finite set of steps, that are computer-implementable, used to solve a specific (or group of) problem(s). The are the implemental expression of the logic of how to solve a problem.

Most algorithms can be generalized to solve a set of similar problems. So can we design algorithms, that can be programable, to make a profit in the stock market? Well… that it is the whole base of what algorithmic trading is! QuantConnect is a platform that works to give you all the tools to create those successful trading algorithms.

But what tools are there? Firstly, they provide packages and support for two different languages for you to program you algorithm in. You may choose between C# and Python. These are two of the most popular, powerful, and widely known languages currently, and they both have a lot of extensive libraries to help you succeed.

Baby’s Got Back … Test

So you are a Pythonista or C Sherpa, and you have leveraged QuantConnect library to write the one algorithm to rule them all (or so you think). What comes next? Well, you are going to need to test that algorithm before you throw some money at it. You know, unless you want to live on the wild side.

Thanks to QuantConnect, we are in luck! They have a giant library of historical financial data to back test your algorithm with. You can build it right in to your algorithm! This is great if you want to train your algorithm, or just want to see how it holds up over certain periods of time.

QuantConnect also computes a lot of those pesky success (or failure I guess…) metrics for you. The QuantConnect web app will tell you your Sharpe ratio, as well as your Probabilistic Sharpe Ratio (how likely your algorithm has >1 Sharpe Ratio). It also calculates win and loss ratios, rate of return, and plenty of other metrics which I am still understanding!

Stocks, Crypto, and Options … Oh My!

“But Calvin!” you say as we swirl our $15 ‘craft’ Lemon Drop martinis at the local Farm-to-Table restaurant, “my father trades stocks, it is so passé… We all know the future is in crypto… or actual Futures.” Well. I got good news for you! QuantConnect has support and back test data for ALL of it: Stocks, Options, Cryptocurrency, Futures, and even Tulip bulbs (this last one might not be true).

By supporting so many different financial assets, QuantConnect gives you flexibility to really create strong and powerful algorithms. Being able to trade across indexes and even asset types, allows you to create very powerful Risk Management and profit driven algorithms.

Tu-Tu-Tu-TuTORial

But what if you don’t know where to start building an algorithm? Well Quant Connect also has many different tutorials. From Market Basics, Trading Strategies, Algorithm Design, all the way to actual QuantConnect algorithm implementation. Though, my favorite learning tool are the user submitted trading algorithms. They read like small academic papers. They share the theory behind the algorithm, and the source code to implement it. It is a great starting point as you not only get insight in to market strategies, but how to leverage the power of QuantConnect as well.

This embodies the whole QuantConnect approach though. They work to give you as many tools as possible for you to be successful, for you to optimize your portfolio. This is important in the current economic climate. If your money isn’t working for you, it is working against you. Honestly, this is why I have become a fan of QuantConnect. Take charge of your Finances.

P.S. Did I mention the majority of its functionality is FREE?

Financial Signal Processing and Multiresolution Analysis Or:

Woof.  A Primer.

“You may learn something, and whether what you see be fair or evil, that may be profitable, and yet it may not. Seeing is both good and perilous.

I was very excited to be chosen for my top choice capstone project this quarter.  The project focuses on short term algorithmic trading strategies.  Definitely an interest for me.  I run a small personal portfolio and have been fairly active in trading options for the last few years.  I am all self taught when it comes to the financial world, but I have done modestly well!  So I am very excited to get to grow my knowledge and skills related to my personal interests!

The first meeting with the sponsor was definitely eye-opening.  There were a lot of terms used in the context of trading that I was not familiar with.  So I made notes with the determination of looking up as much as I could up.  Two of the big ones were Signal Processing and Multiresolution Analysis.

Woof.

So… I present to you a primer on that subject matter.  A high level overview that even an ex-liberal-arts major like me can understand. 

I. Sending You Those Signals

Our first task before we get into how all this applies to financial markets is to garner a basic understanding of what Signal Processing itself is.  For that we need to hop fields from Finance and Computer Science to Electrical Engineering.  

Electrical Engineers spend a lot of time working with, you guessed it, electricity.  More than just the juice that powers our cell phones, it is a very complex field.  There are lots of varying measurements to keep track of.  Some of note include current, voltage, and my blood pressure.  And much like my blood pressure, it is important to be able to work with these measurements.  You need to be able to process, understand, and utilize the information they are giving you.

But how would you process something that is constantly fluctuating?  Enter our new friend, signals.  A signal in its simplest form is a very easy concept to understand.  It is quantifiable numeric data, like a measurement of voltage or current, that changes throughout another completely independent variable, think something like time.

So a very basic signal would be the data points produced by measuring the electrical current of your cell phone every second.  Boom, you have a signal!  So what do you do with the signal?  Well that is WAY out of the scope of this blog post.  But know that there is an entire subsection to Electrical Engineering dedicated to figuring and processing these signals to get the most useful information possible out of them.

II. A Sea of Signals

The really exciting thing that came out of all this Signal Processing they were doing in Electrical Engineering is how applicable it is to so many other fields!  Everything from image processing to audio processing uses this technique.  The camera on your cell phone uses Signal Processing to interpret moving pictures even!

But we aren’t interested in those fields for this post.  No, we are interested in the world of Finance.  This seems like an odd place to find Signal Processing as we have defined it so far, until you abstract what a signal is.  A measurement over an independent variable.  Say something like, the closing price of a stock recorded every day.  You have a measurement, the closing price of the stock, defined over an independent variable, which is time.  

This was the birth of the field of Financial Signal Processing.  As a field it is attempting to leverage all the work done with Signal Processing so far, but with respect to the financial markets.  Super cool right, albeit a little nerdy.  But hey, nerds rule the world.  

Financial Signal Processing has led to all new approaches in market strategies.  It has led to the creation of many new indicators in the market, and new ways to use them.  Most of these strategies are guarded jealously by their Hedge Fund and Market Maker owners.  But as amateur investors and traders, we can see how a basic understanding of the Signal Processing field can help us translate strategies to our own portfolio management.

III. Multiresol-whoseit-what-now

Multiresolution Analysis is the last new term and idea we are going to look at in this post.  The best way to get a grasp on this (at least personally) intimidating word is to break it down into its parts.  Multi is pretty clear as a prefix, it means more than one.  Analysis is clearly referring to the act of analyzing a specific set of data.  Great!  We have the majority of this word figured out already.

But what is a resolution?  The word is (hopefully) not referring to my New Year’s promises I never keep.  I would hate anyone to analyze that!  No, a resolution in this context is more appropriately thought of as the change in the independent variable between a measurement of a signal.

Woof.

So Multiresolution Analysis is just analyzing a signal over multiple resolutions.  I think maybe an example would be in order…

Let’s say I am a person who really really really likes dogs.  In fact, I usually pet multiple dogs a day.  For science purposes I decide to start tracking my average dogs petted for certain periods of time.  I keep a running average of the dogs I have petted over the last 3 days, the last 10 days, and the last 30 days.  That is to ensure that I am not slacking on how many dogs I am petting.

My grumpy boy Rocko

Now I recalculate these running averages at the end of every day.  My signals will be the measurement of those averages. At the end of each day.  So for the last 3 days, I have averaged petting 4 dogs a day, the last 10 days I have averaged petting 2 dogs a day, and for the last 30 days I have averaged petting 2.7 dogs a day (gross, .7 of a dog…).

So the running averages of petted dogs are our measurement part of the signal.  But everytime we measure these signals, we are measuring them against our independent variable (time), for different lengths (2, 10, 30 days), or… resolutions!  BOOM.  Multiresolution Analysis of petted dogs in my life!

Me and my 16 year old Kimmy (she gets carried everywhere, she deserves it)

Back on topic, we can see how if we were to utilize this technique with stock prices it could provide some valuable insight into trends and data to analyze!

IV. Conclusion and Resources

Let me start by saying that I am no Electrical Engineer, and until a week or so ago I thought Signal Processing was something my Engineer brother held over my head to make himself feel better for losing at Smash.  But it is really interesting and I know I am looking forward to learning more about it.

I hope this very very very high level overview helped you as much as it helped me writing it!  I will link some of the resources I used for research at the bottom of the post so you can read from some real professionals if you are interested in learning more!

Resources:

  1. https://core.ac.uk/download/pdf/4408024.pdf
  2. https://www.sciencedirect.com/topics/physics-and-astronomy/signal-processing
  3. https://www.youtube.com/watch?v=y7KLbd7n75g
  4. https://www.mccdaq.com/TechTips/TechTip-1.aspx
  5. https://sites.tufts.edu/eeseniordesignhandbook/2015/signal-processing-in-finance/

Project Planning: The Pool Party Approach

“It’s the job that’s never started as takes longest to finish.”

Project Planning is to successful projects as Pool Parties are to summertime: a staple.  With pool parties you find a bunch of friends and family hanging around the grill, enjoying their cold beverage of choice, and sweating in the palpable summer heat.  All the guests were told this was a pool party.  Everyone side-eyes the refreshing pool, scratching at the swimsuit they wore under their shorts.  Everyone knows they will feel so much better once they take just a quick dip in the pool.  The host has even told everyone to feel free to enjoy the pool at their leisure!  There is nothing holding anyone back.  Except you know, even though everyone is prepared for the pool party, no one wants to be the first one in the pool.

Between my career in Restaurant Management and my time in the Oregon State University’s Computer Science program, I have been a part of a lot of different types of projects with many different types of teams.  Overall, I have been extremely lucky with my teammates.  I have had lots of teammates who are very talented, and I was able to grow from working alongside them.  Other teammates have been gifted leaders whose abilities left me in awe.  Others still, who were so creative in arriving at their solutions, I had to ask them how they got from point A to point B.

But.  One thing I have found to be more common than anything else in these teams, was trouble getting the project planning started.  Usually everyone on the team has had some time to prepare and think up solutions and approaches to whatever the problem we were all brought together to solve was.  But rarely does anyone want to be the first one to propose anything that is not immediately offered by the project leader or client.  

I don’t know if it is a culturally instilled sense of politeness that keeps us from speaking up.  Or if most everyone is naturally shier than I thought we were.  But I am empathetic to it.  For a long time I sat in the back of the classroom, never raised my hand, and never volunteered to go first in group presentations.  I always waited for someone else to start to share my ideas.  No one likes going first.

When I started working on more complex projects, and project planning became more and more important with less and less time to do it, I realized that this hold up was hindering our work.  We would eventually get to where we needed to be.  But there was wasted time in the beginning, waiting for everyone to wait for someone else to start the conversation.  So I started a more direct approach in my groups.  I was the first one in the pool.  

And it wasn’t even that I was just the first one in the pool.  As soon as the host greeted me at the door, I ripped off my tear away pants exposing my swimsuit underneath and ran hootin’ and hollerin’ through the party and did a straight cannonball into the pool (trying to get as many of the partygoers as wet as possible).  Now that the empty pool mystique was broken, it wasn’t long until everyone was comfortable jumping in the pool themselves!

“Except you know, even though everyone is prepared for the pool party, no one wants to be the first one in the pool.”

But what does that 10/10 first cannonball look like in a more project planning setting?  A lot of the time it can come off as enthusiasm.  I will show up to the first meeting for the group with an outline or rough draft of a project plan.  Before we even have a chance for that awkward silence to set in after we all introduce ourselves I have that plan in front of everyone, asking for feedback.  I KNOW that this plan is going to get reworked, even likely completely re-written.  But it gets everyone talking from the same starting point.  It allows me to rope in people with specialties as well.  “Michelle!  I know you are great with Machine Learning, do you have any thoughts on the algorithm section?”.  It also lets people get to their ideas faster, without waiting around for a more traditional slow build planning meeting.  “Does anyone know a better way we can store the data for the login server?”.  All in all, the goal is to break the ice, or in this case the surface tension of the pool.

After my initial cannonball entrance, as the pool becomes more filled with revelry, I myself settle into a nice back float.  I observe, contributing where I have something to add, but mostly letting the those in the pool dictate the pace of the party now.  Comfortably I get out of the water for awhile and grab a bite from the grill.  Because I know, now that the party has moved into the pool, it will be going strong all night.

Part I: A Career Change

Not All Those Who Wander Are Lost

If you were to ask a dozen professionals why they decided to make a career change, chances are that you would receive a dozen different answers. Many have changed because they are not happy in their current profession, some change to make the world a better place, and many more change to increase their salary. My career change to the Tech and Computer Science industry, oddly enough, came about in thanks to a bottle of 1999 Tenuta San Guido “Sassicaia” Super Tuscan.

I love wine. Ever since I was old enough to appreciate it, I have always had a fascination with the history of viticulture. It seems that every wine region in the world has a storied history, each more romantic than the next. The many farmers and vintners across the world gush about how their fruit tell a story. The story of a struggling season, or a plentiful harvest, or the history of a family who have farmed this specific small patch of land for generations.

And I have always loved stories. My first degree was in English with a focus on Creative Writing (I wanted to be the next Kerouac, but it turns out I don’t like jazz enough). When I graduated the first time, there were not many jobs for an aspiring writer. So naturally I fell in to what I knew, what had just paid my way through school. Bartending.

While behind the bar you are a part-time therapist, part-time baby-sitter, full-time entertainer, and of course the Artful Dodger when it comes to sales. Thus, to keep up I began my studies about my product, specifically wine. This lead me around the world (figuratively not literally). I fell in love with the rustic reds of Rhone, dreamt of Riesling from the Rhein, and chased carafes of Champagne. Ultimately I moved in to a role as a Sommelier. It was my second full-cellar program where I met the bottle that would motivate me to change my career.

“…watch them gaze through this looking glass that was laser focused on 20th century Tuscany.”

Tuesday nights are the graveyard shifts in fine dining. Rarely do people go out for big experiences on a Tuesday. Some restaurants even choose not to open during these early week shifts, that way they do not waste the labor. So if you are open on the slow days, you are staffed to reflect it. A skeleton crew for a graveyard shift.

Thus it is not uncommon to see managers serving tables, chefs doing dishes, hostesses manning the fryer (ok I have never actually seen this, but it has to have happened somewhere). This Tuesday night was particularly quite. I encouraged my server to break down the back and get out early. I would watch the floor for the last few minutes of the shift.

If you have known anyone in the Service Industry, or watched any of the movies on the subject matter, you have undoubtedly heard it’s favorite trope: “The table that comes in five minutes before closing”. Those of us that have been in the Service Industry for awhile don’t mind the late tables. We are open after all, and it is our job to take care of our guests. So when the inevitable last minute table arrived to my half-shut down restaurant that Tuesday, I rolled out the red carpet.

If you do not know the story of the Super Tuscan red wine Sassicaia, I recommend reading in to it. I believe it truly is one of the great wine stories out there. Consequently, that Tuesday night, I just happened to have a bottle of the 1999 vintage sitting in my (well, the restaurant’s) cellar. The cool damp cellar was my favorite place to spend time, and situated in a prominent spot was the bottle of Sassicaia. It always elicited a lingering glance from me, coupled with the math of how many weeks I would have to go without eating to afford it.

But I could not help myself! This was a truly exciting bottle, it had such a great story to share! So I was elated when our late night Tuesday table ordered the bottle. What a pleasant surprise to end the night on, and of course, the sales helped out too.

I retrieved, opened, and decanted the bottle for them. Two parents and their young son. They weren’t particularly talkative, and that was fine. Not every table wanted to hear a man in a suit talk about wine. Besides, if any wine could tell its own story it was this Sassicaia.

It was tasted, approved and poured. I gracefully retreated to my corner of the dining room to let them enjoy their experience in peace. But really, and more selfishly, to secretly observe their journey. I wanted to watch them enjoy this beautiful piece of work, watch them gaze through this looking glass that was laser focused on 20th century Tuscany.

But alas, I was treated to something else entirely. I witnessed an eager story waiting to be told wrapped up without being so much as read. I watched in muted abject despair as the lady poured her diet coke into her wine glass, completely topping it off. She had created something I never thought I would, or wanted to, see: a Sassicaia highball. Silently, I grieved for the grapes whose stories were now lost to the aspartame void .

“But that night the first domino in a short line that would lead me from the Service Industry to Tech fell.”

Let me clear something up. I have no space in my life for people who are snobs about wine. Wine needs to be presented as approachable. It deserves to be shared with all who have interest in it. The story of the evolution of wine is irrevocably intertwined with the story of humanity. Everyone has a right to taste the fruits (pun intended) of our labor, our history, our inheritance. But, that night, money changed hands and that bottle of 1999 Sassicaia was theirs to do with as they pleased. It is not, nor ever has been, my place or inclination, to judge people for how they enjoy their food or drink.

Still there was a certain level of heartbreak over this wine. It was akin to an estranged lover cutting of all communication with you and disappearing from your life like the marine layer retreating in the shadow of the sunrise. Hard to prove that they were ever there, though you want to believe they were. An ache. A desire. A longing you do not know quite how to process, and not even sure you would, if you did. I searched for the cork after the table had left. A reminder to keep close, a memorial of a friend. But it was on the child’s plate. Covered in polpette sauce.

Sometimes the universe sends you an unmistakable sign. A nudge that it is time to move on. If, or how, we listen to these universal breadcrumbs is up to us. I won’t embellish and say that this one incident alone was all the motivation I needed to make my career change. But that night the first domino in a short line that would lead me from the Service Industry to Tech fell. My story in restaurants, like the bottle of Sassicaia’s, had wrapped up.

This is where the story leaves the romance of a restaurant. I traded my wine key for a keyboard, my suit and tie for textbooks and blue-blockers. Tech and programming had always been a passionate hobby for me. I had built countless computers, taught myself a few different programming languages, designed websites, built blogs, and written scripts for everything from automating my morning routine to causing mischief with my roommates computer over the LAN. But I wanted more, so I strapped on my bartender shoes for one last curtain call, and went back to school to learn what Computer Science really was all about.

There is not big lesson here. No plot point or greater narrative. The decision to move on from my Service Industry career started because an over priced bottle of wine got some diet coke in it.

But I am finding that the romance that gives wine and fine dining its mystique also exists in the world of Tech. The design of the terraced vineyards of the Mosel has haunting echoes to the relationship between D.F.S. and the graph data structure. Old code doesn’t age as well as wine, but it lets us see and learn from those that came before. Much in the way that old wine tells the story of the generations of farmers who sweated the land it was grown upon. Tech, and code, will take us to Mars and beyond, and wine will continue it’s epic and endless chronicle of humankind and the Earth.

My previous career taught me a lot. I will always be grateful for the leadership experience, business acumen, and communication skills I honed during my time in the Service Industry. But I am incalculably more excited for my career in Tech and Computer Science. I am excited to continue learning. I am excited to explore and surpass the limits modern technology. I am excited to share my insights, my experiences, my perspective, and by far most importantly of all: I am excited to share what not to mix with a nice bottle of wine.

(It’s diet coke. Please don’t do it.)