Thanks for sharing your thoughts!

First off, I'd wouldn't characterize the process as product/customer research but rather product/customer validation. Sometimes the only way to test a hypothesis is to go a little further which can mean building out an app. The trick is balancing how far you go. The process of customer discovery is to help you pair down the feature list to a minimum viable product which you then use to validate again if you've build something people want.

Even user driven apps like twitter/facebook started out solving a problem the founder's first envisioned in their heads. No matter what type of app, I believe the initial success metric needs to be around user retention. Can you build something that people come back to… Beyond that, user driven apps typically track their success metrics around user acquisition while SaaS apps track their success metrics around revenue.

What you are describing though is the “marketplace” model which has mutli-sided complexity. Having never built a marketplace app, I don't consider myself qualified to answer but I do think user retention is where you start then balance with user acquisition and revenue (from your agencies).

Hope this helps…

I have a question though. While I can see this method working for applications where there is a customer need, what about web apps which are user driven. For e.g. Twitter or Foursquare. The only way these apps become meaningful is if there is mass user adoption. How would one go about doing product/customer research for such a problem ? Is it even applicable in such cases ?

Reason I ask is because I am building a web app which is only going to be meaningful to the paying clients (i.e. Ad agencies) if there is a user base. And I have polled quite a few “target” users who like the idea when I showed it to them on paper drawings. But ultimately the only validation is going to be user adoption… which means I am right back where I started… I need to build the app in order to get customer feedback.

Any thoughts ?

While both are used to demonstrate feasibility of some principle, they differ in other ways. First, a POC is usually a prototype on it's way to becoming a fully functioning product while an MVP is the minimum set of features that make up the first working product. Second, the objective of a POC could be to validate technical, business, or market feasibility, while the MVP is always about validating customer feasibility and prescribes specific techniques for how you validate that learning.

While both are used to demonstrate feasibility of some principle, they differ in other ways. First, a POC is usually a prototype on it's way to becoming a fully functioning product while an MVP is the minimum set of features that make up the first working product. Second, the objective of a POC could be to validate technical, business, or market feasibility, while the MVP is always about validating customer feasibility and prescribes specific techniques for how you validate that learning.

I was about to ask how the product is going but I see you shared some metrics in a follow-up post.

pete

I recently was pushing for the ersatz sign-up page method for the conversion of our TogetherTag.com service to a freemium model and my partners convinced us to build the minimum product on the premise you make that if you do the fake signup you are only testing the adsense ads and the fake signup, not the product itself, which means you mostly have non-actionable data. They convinced me if we took the extra 4 weeks to build it, we’d be testing real adsense against a real product offer (albeit highly minimized.) Took more time but now we’re getting real data about the product conversion and foresee getting highly actionable data when we start driving traffic.

Great to read it’s worked so well for you!

As I’ll outline in my next post, we had little trouble getting people to see and understand our differentiated value after a face to face demo but had a completely different outcome if they only saw the first version of our landing page. Most immediately jumped to a premature conclusion about how our service worked and did not identify the differentiated value. It took a lot of tweaking to make it better… I think there’s still room for improvement.

To me the landing page tests, helped create a more crisp message which in turn helped feed more users into the conversion funnel. We haven’t and don’t intend to turn the knob all way by spending on advertising, but we are interested in testing the engagement level of the promise. As Eric Ries pointed out in one his talks, what’s the point of building a downloadable product, if no one makes it to the download page?

That said, I do intend to follow Sean Ellis’ advice and close the loop with retention stats with our existing user base.