What matters more at the earliest of early stages? Team? Or Idea?

I have attended over 45 hackathons in the last year. Some as a mentor, some as a participant and most as a judge. There are many biases I have, including a “developer bias” – I prefer developer heavy teams to MBA teams or sales teams etc. I know some of these biases and many I am not aware of.

So, it is always fascinating when I get a chance to meet the individuals first before they form teams and then see them selling their ideas to get quality hackers, hustlers (presenters, more than sales guys are needed for a hackathon) and hipsters (designers) on board.

I have been doing an experiment to help me understand how to evaluate teams and ideas better.

The experiment is fairly simple. On a white sheet of paper, I put the names of 5 people I think are the likely to win at the beginning of the hackathon. This may be based on just a 2-3 minute conversation with them. I dont know what goes in my mind, but I want to document it. I think I judge how they introduce themselves, what the first few words are, the background and education, etc. I think I have the same judgment biases as most investors and entrepreneurs have, so it is very likely that the top 10 on my list will be the same as any of the other folks who are judging or mentoring. I have shared these notes and the top 10 list with other mentors and asked them to share theirs as well.

Then when the ideas are being pitched I write down the top 5 ideas. I am sure many biases are being played in my mind again. I generally dislike the education and ideas aimed at solving problems that college students have, for example looking for mentors or finding internships. I am very partial to solutions aimed at developers or marketing audiences.

Usually at the end of this process I have my top 5 or 6. The reason is that I have found that in 4 out of 10 cases the “good person” on the list does not choose a “good idea” or the “good idea” has some “average teams”.

I have tried to then map my initial “picks” against those that win the hackathon eventually. In the 11 hackathons that I have consistently done this over the last 2 months, there has been only 1 case when a team from the “other list” has won the hackathon. In 6 of the remaining 10, my first pick has gone on to win and in the remaining 4, my first pick has made it either to 2nd or 3rd place.

I thought I was doing pretty good. Over the last 5 hacakthons I have let other judges and mentors in on this experiment and asked them to rate before and after so they can get their before and after picks.

Turns out most of the judges were in the same ballpark. So, my entire basis for being able to judge good teams was as good or as bad as other judges. There is no proprietary information to leverage since the meetings and information exchanges are so short.

The second part of the question is what mattered more – a good team or a good idea – this is only to win the hackathon.

Turns out good ideas win more hackathons than good teams in 5 of my 11 cases. I have insufficient information to find out how many of the good ideas will eventually turn out to be good startups, given how nascent some of the ideas and teams were.

What I also think is that even if I took someone reasonably tech savvy regardless of experience, they would have a similar “wining picks” ratio.

This also calls into question the “experience” judges and mentors have gathered over the years and the exposure they have to all other competing ideas and companies. Turns out all that experience is possibly useful at certain other things, but at the earliest of early stages, it does not require a rocket scientist to figure out who wins the hackathon, within a low margin of error.

What do you think? Or am I taking limited data and generalizing to a trend?

6 thoughts on “What matters more at the earliest of early stages? Team? Or Idea?”

  1. Thanks for your honesty. There are enough experiments to prove that i.e students asked to rate prof. after interacting for 2 mins or 2 hours or 2 days will hardly deviate, the same applies in medical field with diagnosis. Sadly, what actually happens with the extra time is increase in confidence on the early assumption which can be counter productive. Our lizard brain makes the determination quickly, and we post rationalize as though it was a scientific and logical process. The same happens with investors looking at entrepreneurs. With all the BigData bruhaha one can only imagine the the disastrous outcomes from over confidence or people cogitating over data instead of acting.

  2. I loved this article! I had to read it a couple of times to arrive at a conclusion.

    Is it possible to generalize from the results that
    great team + great idea = best chance of winning
    good team + great idea = better chance of winning
    great team + good idea = good chance of winning

    An idea has a slight edge over a team?

  3. “what mattered more – a good team or a good idea – this is only to win the hackathon.”

    What about in real? What matters the most in real-life practice?

Comments are closed.