Hypothesis Testing For Entrepreneurs

Hypothesis testing appears to be a simple task. Just write down a question, devise a methodology to test it, elicit a response and analyze the results. Some entrepreneurial experts suggest that these tests must be pass or fail. In other words, either the hypothesis is true or it is not. In my experience pass/fail questions created without consideration of other factors is not effective.

For example, Team A reports: “Well, we thought we would get a 50% hit rate, but only got as high as 38%. That is good enough. We pass the test.” Did Team A pass the test?

The first two rules of entrepreneurship are (1) to be honest with yourself and (2) learn from your mistakes. Team A just violated both rules. First, they justified their projected hit rate and were not honest with themselves about what that really meant to their company. Secondly, they didn’t learn from the exercise. They never found out WHY they only had a 38% hit rate, rather than their predicted 50%. This is a terrible, missed opportunity. Why did they originally believe that they could get 50%, and why didn’t that occur? What needs to be changed? Can it be changed? Is it the test or the product? There are too many important questions in this scenario that will never be answered.

One interesting model for creating a more quantifiable hypothesis testing is the HOPE model. This model looks at four factors:

Hypothesis: What is your theory? Is it both “falsifiable” and quantifiable?

Objective: Are your tests objective rather than subjective?

Prediction: What do you think you will find?

Execution: How are you going to test?

The most important element of creating a hypothesis is that it must be “falsifiable.” That means your guess can be rejected after an initial experiment of the hypothesis. If your plan is to see what happens, then your hypothesis will always be true.

Second, all hypotheses should be quantifiable. In other words, you must be able to predict, account, and analyze your results. A good hypothesis includes both a question and good methodology to uncover the results. After determining the question and developing your methodology, you should then run a test to analyze the information obtained.

Additionally, your tests must have a good source of data, as well as represent your demographic population as accurately as possible. Your results should be objective rather than subjective.

Conducting good tests is a subject unto itself, and requires a more lengthy discussion than this blog entry addresses. I will save that for another day.

In my work with both scientists and entrepreneurs, the predictive element is often missing in hypothesis testing. This is even true of scientists and economists who use hypothesis testing on a regular basis. Included within a good hypothesis test must be a predictive indicator of the results. A predictive indicator might include how fast an event might occur and whether there are any stress points in the experiment and where the stress might be located. I believe that failure to quantify your results may mean that the hypothesis is not completely tested, and the result is incomplete. However, if you place a value or a number in the hypothesis, you can learn more about how close you came to hitting the mark.

Without quantifying hypotheses there is a tendency to justify the data to fit the results. In analyzing the results, teams need to be careful to differentiate between causation and correlation. For example, more ice cream is sold in the summer. More people drown in the summer. Therefore, they must be related. Of course, they are not.

Scientists and statisticians also discuss null hypothesis—a hypothesis that is assumed to be true, (e.g. in a courtroom, the defendant is presumed innocent until proved guilty) as opposed to alternative hypothesis—a statement that contradicts the null hypothesis (e.g., the courts would rather the guilty go free than send innocents to jail). What I am advocating in statistical terms is a criterion of judgment based on probability in quantifiable statements. For example, in the courtroom jurors would be asked to determine “beyond a reasonable doubt” whether the defendant is guilty.

So, in your hypothesis testing, will your test confirm beyond a reasonable doubt that your hypothesis is true? If you tested correctly, then you know the honest answer and just reduced the uncertainty of moving forward with your enterprise.

A Meditation on Entrepreneurial Strategy

Most of contemporary strategy literature is based on big company strategy. Larger companies focus their strategies on a cost based model because costs are within their realm of control. They almost never think about the revenue side of the equation. After all, one can control costs, but the customer controls your revenue. This is where entrepreneurial training differs, and provides a winning strategy.

If you think of strategy in terms of costs you can win only half the battle. A recent Harvard Business Review Article by Roger Martin, calls this The Big Lie of Strategic Planning. Martin clearly sees the entrepreneurial view, to focus on the sources of revenue, i.e., customers as the key element of strategy.

Henry Mintzberg called this differential—intended strategies versus emergent strategies. Entrepreneurs work in the emergent section, because they are very opportunistic about revenues. Good entrepreneurs learn quickly that you cannot control the future, but you can try to reduce the uncertainty in getting there. Strategists would call this the resource-based view of strategy.

Resource based strategy states that an organization should use the strongest competencies of a firm to determine a strategy. Entrepreneurs think about what they could be doing with the resources in hand in order to find the opportunities. The planning school holds the thought about what the organization “should be doing” corner of the spectrum rather than the “could be doing” corner. Other strategists might view this as the Blue Ocean strategy. Swim to where no one else is playing; find a niche where there is no competition. Entrepreneurial strategy might also fall into the Michael Porter School of Positioning strategy, which is very analytical.

For information of the various schools of strategy read Henry Mintzberg’s book Strategy Safari. Unfortunately, the entrepreneurial school has changed dramatically since the book was published and it shows less relevance for entrepreneurs. However, this book is recommended as a great summary on the various strategic schools of thought and it is still relevant today as a great primer on the major thought patterns in the strategy discipline.

A good strategy (or whatever term is used – mission statement, mantra, culture) communicates behavior to employees. This strategy communicates what decisions should be made and the boundary limits for what should be the focus of the organization.

Another way to determine and validate a strategy, entrepreneurs may prefer the VRIO framework as popularized by Jay Barney. Are you building something Valuable? Is it Rare? Can it be easily Imitated? And can your Organization implement on the concept?

Possibly, the most important considerations for a startup concern (1) whether a strategy is necessary, and (2) at what point does a strategy become necessary for an organization. Should every company act like entrepreneurs and be opportunistic? Early stage startups do not necessarily engage in long-term strategies. For them, it comes down to tactics and execution. Execution trumps all organizational strengths every time.

The bottom line is that entrepreneurs should talk to their customers. Entrepreneurs have a venture. A business is created when the product or service of the venture can reach at least twenty customers who will make a purchase at a price that provides sufficient margins. If the entrepreneur doesn’t have a product and a price then they don’t have a business…yet.

The Entrepreneur as Magician

We want our startups to act convincingly with passion and a compelling vision for their invention or process. We want them to suspend our disbelief and stunningly convey the magic of their innovation. A magical story is the single best way a startup can acquire the talent required to build a successful company.

Passionate people join startups when magic is invoked. Good people want to work in great environments with great leaders who bring their magic to that business. This type of magic is an honest and creative vision of the innovation with a vision about how it can change the world.

Magicians tell stories during their performances. For both the magician and entrepreneur, story telling is about engagement. Magicians need a story to set up the trick; entrepreneurs use the story to bring relationships and talent onboard.

Sometimes, a magician undersells and overstates the difficulty of the trick. Similarly, entrepreneurs may under promise and over deliver. Entrepreneurs are very good storytellers. They tell stories about products that define a path to the future, a problem solved, and a job accomplished, or increased efficiency. These stories paint an exciting picture of the future and these stories are what it takes to make early sales possible.

Entrepreneurs use good story telling setups. They map the benefits of the product back to customer needs and mesmerize future customer while doing so. If you think about it, stories are much more memorable than statistics. Everyone remembers a good story or compelling image. Do any of you remember a statistic from a commercial for a product or service? I don’t.

Magicians use small props to divert attention away from the real magic. Similarly, startup entrepreneurs effectively use limited resources to magically pull a rabbit out of a hat. They know that if they can’t pull off that trick, there is always a plan B.

As Elmer Fudd will tell you, wabbits can be very, very rascally and… tricky. The startup environment is tricky as well. Entrepreneurs need to learn to navigate its puzzling waters, pivot when necessary, and be ready to take advantage of new opportunities. Ready to execute on plan B.

Magicians use their talented assistants wisely. Magicians’ assistants make their boss look good. Startup entrepreneurs must also employ their assistants in order to efficiently build the business. Warning to both magicians and entrepreneurs: Never cut your assistant in half. Make the work fun and don’t overwork your assistants.

Magicians engage the audience during each of their tricks. They often invite an audience member onstage to participate. Good entrepreneurs engage their target audience – customers—early in the product development cycle as well as the sales and marketing process. Engaging customers’ interest is the single best goal for startups. Early involvement in the process helps prospects listen and buy.

Last of all, magicians and entrepreneurs both develop a special rapport with their audience or customers and receive appreciation. They also show their appreciation as well. When entrepreneurs show customers how special they are, the customers will return for more.

Remember that as a startup entrepreneur, your job is to show the passion and convey the magic of your innovation.

Out-Of-The-Building Blocks

I had the opportunity to chat with Steve Blank about the Business Model Canvas and his Stanford class. I suggested that there is much work to be accomplished even before embarking on the Lean Launchpad Class. When I taught at the University of Southern California (USC), the cornerstone class before the Business Plan class was the Feasibility study. In fact, there were two tracks for entrepreneurs: one for technology and another for other businesses, and each had separate feasibility and business plan classes. (Disclaimer: I am not a believer in the traditional business plan, but rather focus on the operational and business-building sides of startups.)

I adopted many of the topics in both courses as a precursor to the Lean Launchpad class for mechanical engineers at Cal State Los Angeles, which I developed. When I taught at USC, the feasibility class was part of the MBA program. Therefore, many of the entrepreneurial and business concepts were already familiar to many of the students. However, the engineers at Cal State had no experience with business and entrepreneurial terminology and concepts.

I found the key to starting a new feasibility class for engineers was to introduce a few creativity concepts and exercises as a way of enriching their way of thinking. Engineers tend to think in a very linear fashion, and are sometimes uncomfortable with ambiguity in business. They weren’t as interested in variations in valuations or marketing—they just wanted the formula.

At some point in any business process, the research stops and the marketing begins. I am also a big subscriber to effectuation – the concept that entrepreneurs must think differently than ongoing businesses; using evolving means to reach new goals. Startups that are resource poor can’t throw money at a problem in order to make it sellable. A large business may be able to afford such a play, but not a startup. Entrepreneurs must use a different toolkit and a different way of thinking in order to succeed. Both effectuation and the business model canvas provide those tools.

In my entrepreneurship class at USC, students were required to meet 25 strangers. Steve Blank’s classic line is “to get entrepreneurs out of the building.” Entrepreneurial knowledge can only be gained through meeting people in their industry, advisors, and anyone else who can help budding entrepreneurs think through the parts of their business, how they compete, what niche does their competitor target, and just about everything required to be known before thinking about spending a dime.

So what kind of research could an entrepreneur do before starting on their entrepreneurial journey? This is secondary research to be completed before getting out the door. This needs to be done before to make the primary research (getting out the door) more valuable and shorter in duration. After all, having better hypotheses will yield more confirmations:

  1. Know thyself. What are you good at, who do you know that can advance the concept, and why is this an important undertaking? Can you articulate the value and benefits? Is the opportunity clear?
  2. Know the industry. Every aspect of the industry, every competitor and their niche, how would you find the pattern of change in the industry to become the leader of the pack, who are the major suppliers, can you successfully target the underserved market?
  3. Know the value chain – who gets paid and how, can you gain access to critical supplies on good terms? Do you know every step of your business process from order taking to fulfillment and post sale customer service? Where is the most value added?
  4. What is the best way to reach the customer? What effects their buying decisions?
  5. Who are your target customers and their demographic data? Are they big enough to constitute a market? What color underwear do they wear? Yes, you need to know everything about the client down to their undergarments.
  6. Financials – Don’t even think of continuing if you can’t do a spreadsheet forecast called a pro-forma. We all know the spreadsheet is a work of fiction, but how can you guestimate the entrepreneur’s bet? What are the premises underlying the data on your targeted financials? What is the delivered cost of the product? What are the multiple streams of revenue? Get the premises right and be convincing to yourself.

Possibly the most important skill to teach engineers and principle investigators is a little marketing and a whole lot of sales. Let’s face it; with few exceptions scientists are not famous for being extroverts. We need to impart to them that sales is not a shady profession, but rather an exercise in building relationships and helping potential clients solve problems. Good sales and marketing skills help build relationships allowing engineers to fix the world and its problems. Sales and engineering make for a good match.

However, relationship building is sometimes difficult for engineers and principle investigators. Social engagement must be built into all classes. In my class, everyone had to do a one-minute sales pitch that you could give someone in an elevator—a classic elevator pitch. After the first three or four intrepid students made their pitches, the whole class understood the key elements. Not everyone ended up being comfortable with selling, but then again not everyone will become a CEO.

So remember that before jumping into the Business Model Canvas, there is still much thought and research to put into your plan. If you are able to do the homework before embarking on validation tests in the canvas, my hypothesis is that you may have better information through good secondary research techniques before embarking on validation testing in each of the nine building blocks.