Read the Evaluation Criteria Carefully Before a Contest

I try not to sit on the judging side very often, but presumptuous as it may be, I have been doing so more and more¹.

The one thing I would really like to tell those who are about to take the stage at subsidy screenings or pitch contests, speaking as a judge, is this: "If the evaluation criteria are published in advance, read them carefully before you go in."

No matter how good the idea or how good the business, if the evaluation criteria differ even slightly, the evaluation will differ, and the scores will come out completely different. Even for a company whose business I personally love, its score can end up relatively low depending on how the criteria are set.

For example, suppose a contest that evaluates a company's technical initiatives prepared the following criteria, and judges were asked to give each a score out of 5.

• Size of the problem
• Quality of the solution
• Feasibility of the business
• How the organization is run

Now suppose a different contest used the following criteria, again scored out of 5.

• Size of the problem
• Quality of the solution
• Potential of the business
• Leadership of the management team

The criteria are similar, but subtly different. If you took the same presentation to both contests, the scores would change, and so would the rankings.

In other words, a presenter's "goodness" or "value" in a given contest changes depending on the "yardstick" that is the evaluation criteria.

Conversely, you should properly change the content of your presentation depending on which criteria have been set.

Something similar happens in fundraising. For example,

• The materials to show a VC (equity financing)
• The materials to show a bank (debt financing)

are different from each other. VCs evaluate growth potential, while banks evaluate certainty. Because the criteria differ, if you showed a bank the materials meant for a VC, the bank would rarely lend you money.

For participants, reading the contest's or screening's evaluation criteria carefully and aligning with them is like preparing for an exam. Just as you find out what the exam covers and what kinds of questions will appear and prepare accordingly, if you take the tactical approach of tailoring yourself to score highly on the specific criteria the contest has prepared, your business should be properly appreciated within that contest.

This is, in a sense, hacking the contest. But if you want to win, you are better off understanding the criteria thoroughly. People do exactly that for school exams, yet in business contests like these, I find that surprisingly few people do.

That said, winning a contest is different from winning in business. A contest has criteria that exist only for that contest, whereas a business is evaluated by a different set of criteria: sales and profit. And even if you lose a contest, it is no problem at all if you win in business, I think.

On the other hand, what I want to say to those who design contests and screenings is: "Evaluation criteria really are important, so please choose them carefully." If a screening for startups has several criteria related to feasibility but only one that evaluates potential, the scores will end up relatively weighted toward feasibility.

To avoid that, you first need to think carefully about "what you value" in this contest or screening. Accordingly, you will also need to be resourceful: prepare the criteria properly, and place greater weight on particular criteria.

Or, as I wrote in the article below, devising a better way of selecting is another option.

Innovation in How We Choose and EvaluateI sometimes serve as a judge or evaluation committee member for grants, awards, and the like. What I have come to feel is that the way we choose may also need innovation. We pay little attention to "choosing." In particular, when it comes to the "process of choosing" and the "method of choosing…takaumada.com

I have also summarized a particular kind of document screening in the following article, so please take a look.

Tips for Applying to and Preparing for J-Startup Impact, from a Recommendation Committee Member (2023 Edition)(Note: I wrote this article during and immediately after the recommendation review. I am publishing it now because the J-Startup Impact selected companies were announced on October 6, 2023.) I have read through about 500 applications and finished my evaluation. First, including both self-nominations and nominations by others, ...takaumada.com

A good evaluation system guides those being evaluated in a good direction. Conversely, if you build a bad evaluation system, those being evaluated will be optimized further and further in a bad direction. This is true of evaluation systems in education as well, but evaluation is very sensitive. I hope organizers will take great care when running contests, too.

The top image is from the following slides.

Footnotes

1. That said, I turn down judge and commentator requests more often than not, so please don't ask me too often... ↩