Tuesday, February 19, 2008

Who Needs Market Research?

This is a question that was brought to mind by the current episode of the NextNYers video series, where every week they interview entrepreneurs who are building companies in the New York area. This episode is about a company with an online video speed dating web service called Camlink.

Indeed it wasn't a bad interview, but the host, Courtney Nichols, asked one question that troubled me. She asked what sort of market research the company had done to determine if there was a market for this kind of product.

This, it seemed to me, was an unfair question, because it implied that market research is a needed and/or valuable way of determining whether a web service will be successful.

I don't generally believe this is the case.

In fact, I would go as far as to say that without actually building a consumer service, it is rarely possible to determine whether it will be successful. This is because with these kinds of products, the devil is in the experience. Who would have thought that Facebook would have been able to come into the market so strongly and to catch up to or beat MySpace? On paper, it would probably not have been compelling.

The only thing you could have perhaps determined from market research is whether people want to connect with each other. But the truth is it's like asking someone if they would prefer Coke with lemon or lime when they have never tried either. You can ask people if they like lemon. You can ask if they like lime. You can even ask if they get thirsty. But the only way to know what they really are going to like is to observe their behavior after trying both.

In this case, I don't think there is much question that there is a demand or need for tools to help people find dates, partners, or spouses. Will their particular approach work? I don't know, but I don't think any market research would tell me.

But this does bring to mind the question of how you determine if there is a market for a web product. I think the best way is by solving a problem that you personally relate to. Of course its never that simple, particularly if you do not have sensibilities that match those of the mainstream. No matter what, some combination of insight, luck, and execution will always be required, and indeed I think finding that sweet spot is more art and iteration than science.

For developers doing consumer web products, doing pre-launch market research surveys, or anything other than testing product usage is a silly exercise. Post launch, real market reaction is the only research you can really trust.

8 comments:

  1. While I agree that a great deal of market research is wasteful, I disagree with everything else you've written.

    There's nothing wrong with a reporter, or an investor, asking an entrepreneur about their market research. If they have any research, they should say so. If they do not, let them say so. Either way, the decision to use or not use market research is a real decision that other entrepreneurs and strategists will want to know about.

    If a business owner feels bad about not using research or fears criticism for that decision, frankly they need to toughen up -- either learn to deal with unjustified critiques or learn to make a good case for your own strategy and opinions about market research. Ms. Nichols shouldn't be afraid of hurting her guest's feelings.

    As to your other generalizations about market research, I contend that you should be criticizing *bad* research, or more specifically badly interpreted research, not the collection of research itself. To contend that gathering information about your market and analyzing it is always a bad idea is, frankly, pretty dumb, or at least it's an excuse for a startup to be lazy, to exclusively rely on luck and hope and instinct when they could be focusing their strategy with wisdom instead.

    Good research can be conducted on *any* topic or product idea. You can find similar products that have succeeded or failed, and you can examine how or why they did. You can find niches within your intended general market where potential customers are underserved (it's easier to compete where competition doesn't exist than in a crowded space). You can learn where to focus and where to relax your development efforts to more precisely match user needs.

    Market research isn't just about "do you want X?", either. Research can reveal how to market a product, too, how (or if) to advertise it. Or how to design it. Or how much people are willing to pay for it. Or when and where to introduce it. The list goes on and on.

    Investors are perfectly justified in asking about market research, and startups damn well be able to answer it, either by citing their research or by making a good case that it's impossible to do any valid research on their breakthrough product.

    Believe me, I am a HUGE doubter of using market research to predict success or failure of a product, but I wouldn't dream of developing anything for sale -- or supporting someone else who is -- without collecting as much information as we can about the market we are entering.

    ReplyDelete
  2. D said:Feb. 19, 3:25 PMGreat advice Hank.
    I'm glad I subscribed :0
    --------------------------------------------------------------------------------
    Courtney Nichols said:Feb. 19, 4:05 PMAs the host of the interview you refer to, a few thoughts: I agree that the devil is in the experience (and a couple of other places as well) but to say that market research is irrelevant is ignoring the many examples that point to its significance or at the bare minimum to its helpfullness and relevance to investors evaluating a business and a decision by founders on where to put their limited dollars.

    In this particular case, market research could have told you the following: average market price for dating services, average turn over in dating services, market size, crossover between match.com, chemistry.com (their EHarmony)etc. and therefore whether you are stealing customers or just attracting an untapped market, a very different proposition indeed. Look at the advertisers those sites have or don't have, contact them, are there dollars available? i can think of several critical questions that market research would have given me at least some indicators for.

    Of course, we can all find examples where it was the feedback from the market after the fact that mattered ( I mean, that's not really rocket science, is it?) or even contradicted what the research may have told you but why would someone who is going into a particular business actively refuse to research? I never suggested you swallow what you find during the course of that research as truth from the mount, only that you take the time to do it so you know you have not tried to hide from news that you may not like but that your potential investors are likely to ask you about.

    Almost any investor (angel or VC) would ask this question. The answer itself may not even be relevant, but I would certainly want to know that the founders have done their homework, regardless of which insight they decide to heed.

    You are failing to acknowledge the thousands of companies that are not brand new services (therefore may have less relevant research) but companies that are based on pre-existing markets. In addition, market research does not only include focus groups but even that, if conducted properly (i have seen both) i.e. knowing how to ask questions/moderate to get the information they need, can be helpful and avoid expensive errors. Even better, the online world today allows for informal focus groups, which even for allowing for a large margin of error, can help businesses see which way the wind is blowing for little to no money. Market research includes looking at a lot of different information related to the particular field you are interested in (even papers published by your potential or pre-existing competitors!).

    How exactly do you think companies come up with market size opportunities? they research related industries of comprables, create a ven diagram of those market values and then estimate their slice in the middle. They wont be exact but they give potential investors an order of magnitude to think about when evaluating a business. And, in my experience they tend to get it right most of the time.

    No one said that market research is the salve for the world's problems but to dismiss it as silly in every case just sounds absurd to anyone, myself included, who has started and funded companies. Outliers do not a theory make.....

    ReplyDelete
  3. From the context, I think Hank was referring to consumer-focused market research and not to competitive market research that analyzes what the existing players are up to. At least from this post, I don't think he was saying the latter is irrelevant.

    ReplyDelete
  4. Chris,

    We will just have to agree to disagree. I have never heard an investor thinking of investing in a web startup ask for the market research. I'm not saying it doesn't ever happen, but I promise you its rare. The truth is angels generally go by gut, intuition, experience, and faith in the entrepreneur, and VCs generally want to see a working product with real users.

    ReplyDelete
  5. Courtney,

    "In this particular case, market research could have told you the following: average market price for dating services, average turn over in dating services, market size, crossover between match.com, chemistry.com (their EHarmony)etc. and therefore whether you are stealing customers or just attracting an untapped market"

    Who cares whether you are stealing users. Everyone steals everyone's users. The only relevant question is whether the offering is highly compelling and will attract users, regardless of where they come from.


    i can think of several critical questions that market research would have given me at least some indicators for.

    But would these critical questions have made *any* difference to the product design at all?

    "How exactly do you think companies come up with market size opportunities? they research related industries of comprables, create a ven diagram of those market values and then estimate their slice in the middle. They wont be exact but they give potential investors an order of magnitude to think about when evaluating a business. And, in my experience they tend to get it right most of the time."

    Wow, if you know people that tend to get product/market fit via research right most of the time you are in the wrong business. Thats huge. The VCs I know tell me they have no idea how to evaluate consumer web services other than to watch them for a bit in the marketplace.

    ReplyDelete
  6. Gabe,

    You are exactly right. I was reacting to the question Courtney asked which was if there was any market research to validate that there is demand for the product. I don't think you can do that with market research.

    ReplyDelete
  7. Market research is valuable in certain situations. Chris states that good market research can be done on any topic or product idea. I beg to differ. Good market research cannot be done on any topic or product idea and that is the problem. Market research has value but its value is circumscribed.

    If you were doing market research on home computing in say 1980, you would have found out that most people had no interest in them and saw no need for them.

    Market research may not be of much value when a need has not yet been established. On the other hand, it may be of value in assessing existing markets, products and current "felt" needs as well as strategies that service them.

    The point of market research is to manage risk. Market research can help manage risk in numerous ways from investment decisions to deployment of resources to marketing strategies. It's value to high risk investors, i.e. venture investors will be more with respect to existing markets and competition than to products that break paradigms or are aimed at undeveloped markets. Conventional wisdom has thought it to be of great value to investors in liquid assets - hedge funds, private equity, financial institutions, etc. Yet, given what is occurring the markets even that is up for debate these days.

    Getting back to my main point, there are times when market research may increase rather than decrease risk. The risk incurred here is often the loss of an opportunity that will be a big winner. In many games this is a good trade. In others - certain kinds of investing as well as our society in general it often is not.

    Certain products, strategies, services etc, need time to develop, to reach what you might say is their "tipping point" which is when they really start to catch on. Before that they may be viewed by the masses as uninteresting, unnecessary or unworthy. If they were researched, the research would present a snapshot in time before the phase transition would have occurred.

    For example, would Seinfeld ever have been able to take its place in the pantheon of American TV if NBC was operating in an entirely "quant" kind of way. The answer is a resounding no.

    Today, Seinfeld probably could not happen basically because of the increasing fetishization of market research. Yet, still the execs seem content with this equation. It is better to forgo a big winner than to be the exec responsible for a loser. While the exec may benefit, we as a society do not. We lose the benefit of the creativity, the innovation and the "surprise" of its success.

    In terms of usage, market research has been on an upward swing for decades now. It has ventured into territories inconceivable at one point in time. What would market research have told us about James Joyce's Ulysses or the paintings of Van Gogh.

    All of this is not to ultimately disparage market research. It can be of great value when its limitations are understood. It has its downsides though and these days, more often than not, they are not articulated.

    ReplyDelete
  8. Don't agree
    Pre-startups market research is important come on. It depends what you do but a pre-launch can cost a lot, and then what? Another second pre-launch, and again and again.
    If you can avoid making mistakes before starting your company, launching a new product/service, then do it!
    It is for sure less expensive to do some prevention than to repare mistakes! Pre-start market research can be a real market testing, sending the product to respondents to have their feedback...

    Starting a business? Launching a new product/service?
    Test the market first!!!

    Marketest

    ReplyDelete