A framework to think about experiments for your startup

Experimenting is at the core of building and tying new ideas. A opposed to having a clear problem to solve, experiments are designed to “try” out ideas that you have and yet know if they will work or not.

At most startups, I notice two primary “ends of the pendulum” issues. Most (over 90% of startups) dont run enough experiments. The rest (< 10%) run too many experiments in parallel.

If you fall into the first category, then my only suggestion is to think about experimenting and commit to doing one. The AirBnB blog is a great place to start, in terms of understanding the user experiments they run. They actually have a experiment reporting framework, which shows how evolved their thinking is in terms of this facet of work.

I also am a fan of the “test and learn” mindset. Since I believe that working on your startup is more a journey in self realization than of markets, problems and customers, it is important to keep learning. I do also believe the best way to learn is to try and do small experiments which you can scale.

I have been a fan of disciplined experimentation.

This post is about the < 10% who run too many experiments in parallel. That’s the biggest challenge I see with startups that hire amazingly entrepreneurial talent for their first few hires.

Since each of the first 5-10 employees are entrepreneurs themselves, they all tend to run multiple experiments, either with product, marketing, customer acquisition, sales, etc.

The framework I have for thinking involves 3 “sets of steps”. I call it “Trail, Nail, Scale”.

The “Trail” comprises of 5 steps, the “Nail” comprises of 3 steps and the “Scale” comprises of 2 steps.

Here is a visual to think about it.

Trail Nail Scale Disciplined Experimentation

Trail Nail Scale Disciplined Experimentation

Obviously this is very early thinking, but I’d love your feedback.

The way to think about experiments is you to pass through gates and assign the appropriate resources at each stage and have a “rough sense” of what you are trying to achieve. If you know exactly what you want to get out of your experiments, you are not “experimenting”.

What I have noticed is that the 3 stages end up being a funnel. There are many experiments you run, a few of them you will nail and a fewer of them you will scale.

If you have 100% of your “experiments” when you start, (on the left of the graphic), then 20% (or less) will be nailed and 10% you will scale.

In terms of allocating time and resources (if you dont have a large team as a startup, allocate your time), 10% is spent on “Trailing”, then twice that time or 20% on nailing and 70% on scaling.

There are many questions that this throws up, which I want to address over the next few days.

1. How many experiments should you run at the same time?

2. How do you define the success of an experiment?

3. How do you internalize and document the learning from your experiment?

4. How much “money” should I spend on trailing? How about in nailing?

5. How do I leverage lean principles into this thinking of Disciplined Experimentation?

Anyway, I’d love your feedback on this framework. As I share more of my work, which I am interviewing people in larger (Unicorn) startups at, I will also give you some case studies to see what they learned.

Why it is so difficult to spot unicorns when they are hatched – I need your opinion

Yesterday I was a judge at the TechCrunch event in Seattle. The meetup had 500+ folks from the Seattle area, with 9 companies being selected to pitch at the event. Over 100 startup applied – some that were idea stage, others with prototype and still others with a few customers and early traction.

TechCrunch Meetup

TechCrunch Meetup

Here are the 9 startups in a nutshell.

1. A bar robot that helps mix cocktails (home, personal use). It comes with 4 ingredients standard and you can add more ingredients (think of vodka as an ingredient and same for tomato juice as an example). It is meant to be counter-top friendly and starts at $99. They are building a prototype.

2. An Outdoor ad projection system for personal messages. According to the founder, as an individual, when you are trying to get your message out, there are few options other than those online (the rest are prohibitively expensive). So, he designed a projection system, which beams the messages at key spots outdoors. They are in 2-3 locations in Seattle right now.

3. Cashless POS and payment system for bar owners. The big problem is that busy bars lose money since it takes 2-5 min to get payment from the users – either with cash or credit card. Many bar owners lose money since patrons lose patience and instead opt to give up standing in line for a drink. This company produces a band (wearble) which you tap for contactless payment.

4. A lock screen app that replaces your lock screen (which most people stare at 50-100 times daily. This will replace it with images and photos from your collection on the cloud or the phone. The app has 100K users already.

5. A Car on demand service, replacing owning a car. Long term leasing or renting of cars. You can “rent” a car when you want for however long you want and keep changing. Need a small car for the weekday, rent it. If you however need a boat-towing Ford F150 during the weekend, no problem, replace your compact car with that for the weekend. In beta at Austin.

6. A curated collection of interactive STEM content (videos right now) for kids between the ages 7 and 15. This product provides way for children to get interested in STEM content watching engaging and interactive videos online. Has many users already.

7. A soccer ball sized Mic for student participation in class. Most teachers have trouble getting kids to listen to their voice, so they use a mic and speakers in the classroom. If they have to get the kids to talk, then they pass another mic in the classroom. This product has a mic built into a soccer-sized ball so kids can “engage” with the class and have fund doing it. Being tested by a few teachers in their classroom.

8. A learning system with games for your dog. They provide hardware that allows your dog to “learn”. These toys come with hardware that can be controlled and upgraded for new toys. When you dog plays with these games, you get notified via your smartphone. Currently looking for beta testers.

9. Uber for beauty. There are some days you want to look your best – party, interview, etc. For those days you can get a beautician “on demand” locally, who will come in and help you get your hair, clothes, etc. right and ensure you look the best. They use local beauticians and are a marketplace. Have a few users trying the service.

So, there you have it. 9 ideas and startups.

If you were the jury and had 1 min to listen to each of the founders, which ones would you pick to “have a deeper” conversation. I am not sure which of these companies will become a unicorn, or if any of them will be venture scale, but they are all promising ideas, and some have traction as well.

We declared one winner – the car on demand company and one runner up – the Uber for beauty. The audience choice was the learning system for your dog.

  1. If you were an early stage investor with $50K to put in one of these companies, which one would you invest in?
  2. Do you think any of them will become large in a short period of time?
  3. Which ones are cash-flow businesses – generating enough cash to be profitable, but not large.

Some of these ideas are really interesting, others I am not sure about at all – in terms of scale.

I’d love your opinion.

One way to explain the “What is your strategy” question to an investor

Inevitably when you get into second discussion with an investor you will get the question “What is your strategy for X”? The X could be mobile, pricing, customer acquisition, fund raising, etc.

What they are really asking is, have you thought through the various elements, documented your assumptions and have a point of view on the market that’s unique and different than others.

For larger companies with strategy teams and dollars to spend on McKinsey, the strategy question is usually delivered by means of a presentation with data to back up basic assumptions, historical trends and a prediction (again based on data), on 2-3 scenarios on what might happen in the future.

Given that most institutional investors have the rigor and background either in operational expertise (having been an entrepreneur before) or the experience with strategy in a far more detailed level than most entrepreneurs might, the question might intimidate you.

At the basic level, any discussion of strategy should be based on 3 things: Your idea of the landscape, your assumptions on what are the fundamentals and your vision for the future.

Strategy is not a pedantic or meticulous exercise in creating more PowerPoint slides though. Even if you explain it in a logical fashion with the 3 basic pillars of landscape, assumptions and vision, you should be good to go.

Lets take an example. I read the SouthWest Airlines strategy chart in a single “Activity System”, which is similar to what I have seen in many presentations.

SouthWest Airlines Strategy

SouthWest Airlines Strategy

So how did they come up with this chart?

First, as always, start with your customer. A day in the life audit will help you understand their “must haves” versus their “nice to haves”.

The most important thing I have learned so far is that it matters which customers you talk to. Talk to the late adopters and you will get the “Give me a faster horse” response.

Picking the right customers and documenting the vision for understanding the future is your first step.

Then you want to start to document what the world will look like in a few years (5, 10 usually suffice) but that “vision” is usually not that obvious. Most entrepreneurs tell me “I dont know what’s going to happen in 6 months, how can I tell you what’s going to happen in 5 years”?

That’s a fair statement, but a cop out. You have to come up with a view of the world and if you cant come up with one yourself, find a way to get to original thinkers in your domain and ask them to help.

The second step is to document the current state.Those will be the assumptions that are the key to revisit when questions come up. They also help you put the workable “plan” as opposed to a great set of strategy charts and graphs with no actionable information.

Finally the core tenets of your offering / product / solution (see the smaller orange balls in the SouthWest example above) explain your strategy in simple enough terms.

To recap, showcase the vision, then document the assumptions and finally put together the tenets of your offering and how it is different.

To get a more detailed view on how this is done in theory, read the HBR post by Michael Porter.

The rise of the Micro VC fund and the future of seed stage funding – 5 questions for @SamirKaji

Samir Kaji at First Republic has been following seed stage investing for a while now. I have written before about the rise in Micro VC funds, which are typically <$100 Million funds run by a solo General partner or two partners. Many angel funds are in the sub $25 Million range, as well.

Samir Kaji First Republic

Samir Kaji First Republic

Over the last 10 years (since Jeff started Soft Tech Venture Capital) there are over 250 Micro VC firms that have been started. I had a chance to talk to Samir yesterday and asked him 5 questions to explain and understand this phenomena. Here is an edited version of our conference call.

1. What are Micro VC funds and why are they a thing now?

As the cost of the software startup drops, the amount of money required at the early stage has reduced as well. At the same time the number of startups has increased and so has the number of angel investors. Micro VC funds are deploying smaller capital at the earlier stages than traditional Venture firms. Typically 10 years ago, the VC fund would be $100 – $200 Million and invest in 20-30 companies, between $2 Million to $20 Million per company. Now the larger funds are deploying $5 Million to $50 Million as their fund sizes have increased to $500M to $1 Billion.

2. How many of these Micro VC funds are there are where are they located?

Close to 40% of the 250+ Micro VC funds are in the Silicon Valley said Samir. 25% of them, or about 60 are in Asia and Latin America. About 30 are in Europe (largely London and Germany) and the rest in Chicago, Boston and New York.

3. What value does a Micro VC provide besides cash?

Depending on the Micro VC, expertise and connections are the most important things they provide besides cash. A good Micro VC is already thinking about your next round and has built a good network of connections to help you with follow on funding. Since most Micro-VC funds are small by nature, many often cannot exercise their full pro-rata in follow on financing rounds, and almost certainly do not have the ability to serve as a lead in the next round of capital. Instead, they have a well curated list of investors who they have worked with before to help introduce startups to their network. Expertise varies by investor, but most will be able to help you hire good people and also help you with Go To Market, Pitch preparation and some early business development.

4. When and how should entrepreneurs approach Micro VC?

Typically after you have a little traction is when most Micro VC’s will be interested is what I learned. Most want to see some validation of your startup. Some of them do work with Accelerator programs, but I have seen many of them working their network of entrepreneurs to get introductions to other potential entrepreneurs.

5. Will there be more or fewer Micro VC’s in a few years? Where is this headed?

Micro VC’s are here to stay, said Samir. In fact, he sees more of them will emerge in the next few years. The main reasons are that there are enough entrepreneurs who have had some modicum of success and dont want to do a startup again, but want to use their expertise, connections and network to help diversify their risk and give back by helping other entrepreneurs. So, they will in fact continue to exist. He did say that the Micro VC’s with more tenure will likely move “up market” and start to raise larger funds and become the next generation VC funds, while newer, smaller GP’s will start to become Micro VC funds.

Being clear about the type and kind of help you need makes it easier to get support

I get about 10 emails or LinkedIn messages a day from entrepreneurs and potential entrepreneurs who are seeking help. About 2 of them will be referrals, but most are prospecting cold, either having read somewhere that I am an investor or looking at my profile on LinkedIn and hoping to connect.

I tend to read most (not all) messages) and am still not sure on which ones I do read and which ones I am unable to. I know that the ones I get referred to from known or trusted connections will get a response from me, but a good % of those that are cold also get a response.

Over time I started to notice when I do respondthe easy no is the fastest response, the easy action gets a response as well. The ones that require me to do a lot of thinking, work to connect, look up another piece of information or a lot of cut and paste get ignored.

I have an entrepreneur who is very tactical and is a friend who only reached out to me when he need to connect with one of my contacts. Initially I would do it since I’d want to help, but over time it has waned – it has become a chore to connect him to my connections – not because I dont want to, but because it involves my going to two different email boxes (Outlook for work, vs. Gmail for personal) which involves more than 30 seconds and I lose interest.

Over time I have figured out the things that will get you a response (quick yes or no) and those that will get you a favorable response (mostly this means the potential target is willing to give you time – for a meeting, for a call, for advice, etc.).

When the bar for the response is high, and especially when I dont know the person, expect a quick “no” response. For e.g. even if you are referred to me and your first request is for either an investment or an mentor opportunity, 99% of the time I would say no. That seems strange given that I might miss a “great” opportunity, but I am okay with that. Lack of time forces me to miss great opportunities, and I have found that my own sourced opportunities to invest generate a bigger return (so far) than the ones referred to me.

There is no shortage of warm introductions that most of us get from others, so the bar is now getting even higher on what requires a response.

A quick yes response is usually to a specific question that I may have written about before, which requires me to refer the person to a blog post.

A no response is when the person is not clear about what they want. Unfortunately I get 40% of emails of this type.

Do you want to have me invest? Do you want me to refer you to someone else? Do you want an introduction to an investor? Would you like to apply to the accelerator in India and need my referral? Or do you want me to mentor you? Or would you like feedback on you product?

When it is not clear after reading 3 sentences what your ask of me is, I will likely not respond at all.

“I’d like 30 min of your time to tell you about my new idea” is hard to justify time for. When I know 2 sentences will explain your product or idea, why would I be willing to spare 30 min? If you cannot explain it in a short sentence or two, then there’s more refinement needed on your part, which means there’s more work to be done.

I like napkin stage companies, but clarity of problem is what I expect at napkin stage.

Customer Validation as a Service (CVAS) – an agency for small startups

Customer discovery and validation is a pretty challenging area for most startup entrepreneurs. While most can build a product and maybe even hire people to help it grow, validating with customers and cold calling people to get feedback is hard for most technology entrepreneurs.

Customer Discovery

Customer Discovery

An entrepreneur suggested to me an idea to start an agency that does Customer Validation as a Service (CVAS) or Customer Discovery as a Service (CDAS).

The key part of this service is setting up the problem statement for the entrepreneur, distilling the list of potential customers (Both B2B and B2C) and finding – emailing, talking to and interviewing potential customers to find the top 3 pain points for which they’d pay money for.

Imagine if you were a technical developer entrepreneur and you can build a good product, but your customers were in another location, or they were people you were not able to get to easily.

If a service built a website landing page, setup Google adwords, Facebook campaigns, Twitter profile and ran a campaign for a week or so to give you feedback on what’s the interest, what are potential customers interested in and what would they pay for?

I think this type of service would be very valuable for entrepreneurs. I can easily see various offerings being add-ons to this service.

1. Pricing validation

2. Content (what to write, what medium – email newsletter or Youtube Video, etc.) validation

3. Budget validation

4. Technology landscape validation (what other products should we integrate with)

5. Go to market validation (Where should we advertise, how should we market, etc.)

In the Steve Blank Model for customer discovery above, this set is most useful in the Test problem phase, followed by the Verify phase.

What do you think?

Would you buy this service for $500 – Consumer startups and $1000 for Enterprise startups?

How we podcast; Our setup, flow and agenda: India Startup Chat

Over the last 4 months, I have been blogging every day to understand the discipline of writing and work myself into thinking and learning about new mediums of communication. I published a post on What’s working in B2B blogs and what’s not working a few weeks ago to capture my experiences.

One of the things that’s clear to me is that the number of people reading text-based blog posts is dropping and video is growing in popularity worldwide.

Combined with Mobile usage growth, the trends are pointing to a huge number of people watching short form video on the go.

Particular to the US, an increasing number of people are listening to podcasts during the commute or heading to the gym.

Noticing these trends, I reached out to Lakshmi Rebecca and Ravi Gururaj, two of my good friends from India, who are celebrities in the Indian startup ecosystem. I wanted to do a podcast every week on the Indian startup ecosystem.

The audience for this podcast was largely folks in the US (investors, entrepreneurs) who were keen to learn about the Indian startup scene. We also noticed a lot of people from India (entrepreneurs in particular) would be interested in listening to this podcast.

Lakshmi took the lead (as expected, since is the most creative and a media personality in her own right) and gave us detailed instructions on what we should prepare.

There were 3 steps we followed:

3 Step Podcasting Process

3 Step Podcasting Process

First was the preparation. We decided on the format (3 sections of 10 minutes each), time for each section and the topical items that mattered.

Since I was in the US, Ravi at his home in Bangalore, and Lakshmi at her studio and home office, we felt the best way to execute this was using Skype. After 3 false starts, poor recording quality and endless frustration with Internet connection speeds, we gave up.

Lakshmi then did some research to figure out that “double ender” format was the best way for each of us to record our podcasts on each end and then combine and produce it together later.

We also needed a logo and a name. Descriptive name won, although we were tempted to go with a more generic name like the Views podcast, we decided that we’d start by naming it “what it was” – a podcast about the Indian Startup ecosystem. The “Chat” was to indicate the informal nature of our riffing. So, India Startup Chat was born.

Lakshmi also put our logo together. We needed a 1400px X 1400 px sized logo for iTunes, BTW.

The next step was to do the actual recording. After a start with our own microphones on our machines, we found that  we needed a professional quality mic, since there was too much static white noise when we recorded.

I got a Blue Snowball Ice USB mic, which I have been pretty happy with, Lakshmi has a more professional setup.

Blue Snowball Mic

Blue Snowball Mic

Each of us also downloaded Audactity for recording.

We then got on a conference call ( so we could each listen to one another), and recorded at the same time on our own machines.

Lakshmi Mic

Lakshmi Mic

The final step was to upload our individual recording files to Google drive and Lakshmi then went to the Production studio to mix the audio tracks together. She mentioned it took her a few hours, but will be less as we get more familiar with the setup.

Production Lakshmi

Production Lakshmi

I then setup a SoundCloud account and posted the images and file to the site.

Please do give us a listen and let us know what you think on Facebook!