The Power of Randomness: Democracy, Science, and Social Implementation

• Allocating science grants at random (investing in uncertainty)
  • The problem
  • The solution
  • Feasibility
• Allocating social implementation grants at random (investing in uncertainty)
• Using randomness in business
• Other discussions: Having randomly selected citizens participate in democracy (fairness)
• Summary

Suppose you have become interested in art and decide to buy something. Art includes paintings, sculptures, and music. At first you don't even know what you like. Everything looks good, and everything looks bad. Even if you ask friends who know art for comments and reviews, they evaluate by different criteria, so the evaluations are inconsistent.

Each piece costs a fair amount and takes up space, so two or three a year is the limit. So you try to choose carefully, comparing various options. But gradually you tire of choosing, think "oh, forget it," and may give up on buying at all.

In a moment like that, "just pick one at random for now" may be one optimal approach. That is what this article is about.

The introduction of random selection is beginning to be considered and put into practice as one such method.

Allocating science grants at random (investing in uncertainty)

Randomness in research is well known through the RCT (randomized controlled trial), but this time I will talk about other areas, particularly the allocation of grants, where an allocation method called "random funding" or "lottery funding" has begun on a trial basis.

The problem

Research needs money. To obtain it, researchers propose the research they want to do to various grants. The government has to award grants to the ones that look promising. Usually peer review among scholars decides where grants go, but especially in basic science, even experts tend not to be able to predict which research will hit.

For example, the 2005 research of Dr. Katalin Karikó, which became the basis of mRNA vaccines, reportedly did not attract much attention at first. In fact, the University of Pennsylvania, where Dr. Karikó worked, sold the related patent in 2010, and she subsequently could no longer even afford to rent a lab at the university (so she moved to BioNTech). At the time, it probably could not have been foreseen that her research would lead to the major achievement of the mRNA vaccine.

This is not to say the University of Pennsylvania was at fault. In highly uncertain areas, you cannot know which option is truly promising. Even the great Newton apparently judged alchemy and theology to be as promising as physics, or more so ¹.

In particular, highly novel ideas tend to be rejected, and it has been pointed out that they fail to receive grants. Evaluators also tend to give lower scores to proposals that are more novel in areas close to their own expertise. Because of such biases, new ideas in a field can be unjustly undervalued. For example, I have heard that when someone tried to bring quantitative research into an academic field centered on speculative argument, the older generation pushed back.

Max Planck is said to have stated that "a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it." In other words, generational change. If we really have no choice but to wait for generations to turn over, then now that longer lifespans are lengthening the time until that turnover, it may be that it is becoming systematically harder for new ideas to emerge.

The funders who award the grants also lack the courage to put money behind things that are likely to fail. As a result, investment goes only to research that is an extension of the status quo, the base of the research field does not broaden, and truly promising ideas can end up buried.

Furthermore, the selection process also admits biases related to gender and minorities. Female researchers and researchers from minority groups tend to have difficulty receiving funding, and in the UK, roughly 90% of engineering and physics grants reportedly went to male-led projects.

The solution

Random funding offers an alternative, "try choosing at random," as a way to address such systemic biases.

In New Zealand, since 2013, a program called the "Explorer Grant" has been providing grants at random. Each grant is a small amount, NZ$150,000 (12 million yen), and the cumulative total amounts to about 2% of the funding overall.

Year	Applications	Judged ineligible by the panel	Not funded in the random draw	Funded in the random draw
2013	116	99	14	3
2014	24	18	2	4
2015	45	38	3	4
2016	38	29	0	9
2017	34	21	2	11
2018	60	50	0	10
2019	77	53	9	15
Total	394	308	30	56

The table is from The acceptability of using a lottery to allocate research funding: a survey of applicants | Research Integrity and Peer Review

However, this is not simply a matter of handing out many small sums at random. The condition for applying is that the proposal be "innovative and transformative of existing understanding." In other words, funding is given on the condition that the proposal might offer a large return with a small probability. It does not fund incremental improvements to existing research; it provides money so that things that "might take off" are not crushed.

This is like seed-round venture capital investing in startups. It can also be called a practice of the Convex Tinkering that Taleb proposed in The Black Swan and Antifragile. By betting on the condition "unlikely, but big if it hits," you can fund ideas that might become future Nobel Prize candidates and let them run experiments.

It has further been suggested that such random funding leads to gender equality and diversity. If selection is random, grants are awarded in a way that directly reflects the diversity of the applicants. Women will no longer be unjustly rated low for being women.

Separately from the matter of small-scale random funding, there is a survey finding that "awarding many small grants seems to be more cost-effective." According to a study of past science grants in Canada, the result was that "research impact is positively related to the amount of funding, but only weakly." In other words, the impact per dollar is lower the larger the grant. Conversely, it can also be said that giving smaller grants to many yields better results overall than giving large grants to a few.

However, trying to give small grants to many people entails correspondingly higher costs of evaluation and screening. This is where random selection can be used.

Random selection makes the time spent on evaluation more efficient. The greatest efficiency gain comes in deciding whether to accept mid-ranked proposals. In general, in a review, the top and bottom tiers are quickly rated, but the proposals in the middle tier are hard to tell apart, and because the number of awards is limited, they still have to be ranked carefully. If you instead cut off the bottom tier and choose the middle ones at random, the time spent on evaluation can be kept down.

Furthermore, if selection is random, an application only needs to be of reasonable quality, and it is expected that the effort of raising the quality of applications would not be so great ². It has been pointed out in many places that Japanese scientists are swamped with writing applications, and the time and cost of applying could be reduced as well. Incidentally, in Australia, it is said that researchers as a whole spend time equivalent to 180 years in a single year preparing documents for grant applications.

A paper called Talent vs Luck simulates random allocation. Its result was that distributing grants equally to all researchers gave the best performance, followed by allocating 10% at random, and then 20% at random (commentary article).

To sum up, it saves time, removes bias to secure diversity, and gives new ideas a chance: these are the benefits of random funding.

Feasibility

The effects of actually awarding grants at random have also been measured. Nesta tried changing its internal grants from selection to random allocation. As a result, more projects by women were accepted, and diversity increased.

It was not just that women were selected; initiatives that had previously been only reports also began to be tried in new ways, including videos and prototypes.

So how do the scientists who actually receive grants at random feel about this approach? In a survey of how New Zealand researchers feel about random funding in this Explorer Grant, 63% were in favor and 25% opposed. For random funding of other grants, 40% were in favor and 37% opposed; support fell, but the two sides are close. In other words, opposition to random funding is not that strong.

Following this trend, Switzerland is also about to start random funding initiatives.

Of course, purely random selection could end up funding proposals that are scientifically impossible or unscientific. To guarantee a minimum level of quality, a cutoff would probably be necessary. I also do not think all funding should be allocated at random. For large-scale projects and fairly well-understood areas, the existing peer-review format may be better.

However, investing small amounts in early-phase attempts that might take off is a cost-effective way to invest, and introducing randomness may make it possible to shed light on good ideas that might otherwise have been overlooked. And broadening the base of research can ultimately benefit science as a whole.

Allocating social implementation grants at random (investing in uncertainty)

Something similar can be said when thinking about social implementation.

In particular, the early phase of social implementation carries the large uncertainty of customer needs. For that reason, I think funding in a form close to random funding could be effective, as in science. No one can tell which one will hit.

VC investment also has a diversity problem. For example, under the current pitch process, women are not valued. Investment also likely tends to flow only to trending themes.

I also feel that the tendency seen in science, to give lower scores to highly novel proposals in areas close to one's own expertise, is seen among VCs too: ideas that have competing products are rated low, whereas areas VCs are not very familiar with, such as services for young people, are rated relatively high precisely because they don't know them well (especially in contests for students and the like).

VCs have to return funds to their LPs, so it may be unavoidable that they choose carefully. In that case, one option would be for the government to take on the investment risk in the very early phase (on the order of a few million yen) and mix randomness into the investment method. As a result, the absolute number of successful startups would increase, and if the startups and VCs pay their taxes properly, the returns should be commensurate with the investment.

Of course, a qualitative cutoff is important. For that reason, random funding can also be combined with training. For example, by combining it with customer development training offered as I-Corps in the US and ICUR-e in the UK ³, and awarding grants after that training, a minimum quality guarantee might be possible. Research on the effects of I-Corps is advancing, and I believe training that focuses in particular on the number of hypotheses and the number of customer interviews is effective.

Furthermore, attaching such training also leads people to give up on bad ideas early. Giving up on bad ideas quickly raises the failure rate, but by increasing the total number of attempts, it can also improve the performance of the entire ecosystem.

Using randomness in business

There are also results showing that randomness is effective in business.

One is in the context of investing. In a study that compared, through simulation, four representative investment strategies against entirely random investing, the result was that, except at certain points in time, choosing at random basically gave lower volatility and better performance. In other words, for an individual, choosing at random is easier and lower in risk than paying high consulting fees to have someone choose for you.

It is said that randomness might also be used in organizations. The Peter principle is well known: "a person rises to the limit of their competence within an organization; people keep being promoted until they become incompetent." To get past that limit, a simulation compared (1) a strategy of promoting the best members, (2) a strategy of promoting the worst members, and (3) a strategy of promoting at random, and the result was that "choosing those to promote at random makes the organization perform better as a whole." This research won an Ig Nobel Prize in 2010.

In optimization problems in computer science as well, simulated annealing is an example that incorporates randomness. It is an algorithm that starts by sampling at random and then gradually narrows down, which lets it arrive at the best choice (while allowing, at times, changes for the worse). It costs extra for that, but it is less likely to fall into local optima.

Looking at it overall, we might say that the more uncertain things are and the less the future can be foreseen, the more choosing at random and entrusting ourselves to chance may steer us in a good direction.

Other discussions: Having randomly selected citizens participate in democracy (fairness)

I said that funding at random can also address bias against gender and minorities.

In fact, it is known that in ancient Greece and ancient Rome, lotteries were used to select rulers. The famous example is the archon of Athens (2, 3). Many members of councils and administrative officials were also chosen by lot from among the citizens.

One idea behind the lottery was that holding a particular office for a long time was thought to cause injustice and corruption. It was thought that selecting at random could avoid such wrongdoing.

Afterward, the lottery system receded, and in the modern era representative democracy (parliamentary democracy) was introduced in countries around the world. But there is also a theory that this was not because representative democracy was better, but because the aristocracy disliked democratization and wanted to justify their own rule. If aristocrats were "democratically" elected as representatives, their position could be justified.

Times have changed, so let us turn to the present. Currently, forms of deliberation incorporating randomness are being practiced, mainly in the EU. Consensus conferences and citizens' assemblies, in which citizens selected by open call or at random deliberate, are examples. In a consensus conference, experts provide information to citizens selected by open call or at random, and then the citizens deliberate. By doing so, it is thought that, though not perfectly, public opinion can be better reflected in politics.

Behind this seems to be frustration that citizens' voices are not well reflected under representative systems. In fact, in modern society there are more and more issues that need deliberation, and they are becoming more complex. There are so many and they are so complex that we cannot think through every issue properly. And there is also the aspect that "because they have no time to deliberate, some people choose leaders by whether they like the candidate's hairstyle."

Meanwhile, the representatives chosen that way push their own opinions believing they have been "entrusted by the citizens," but in fact they were not chosen after careful deliberation, so they are not acting in a way that reflects citizens' voices, and a gap in perception grows between representatives and citizens.

Representative democracy cannot work like this. It also preserves the existing power structure. In Japan, it becomes something like a Diet for middle-aged and older men. Then a situation arises in which "our representatives do not represent us." Voices become harder to reach, disillusionment with politics spreads, citizens participate even less in politics and deliberation, the distance between politics and citizens widens further, and representative democracy stops functioning.

So, as a way of selecting the people involved in deliberation, choosing people at random in order to gather voices that reflect the population is now being widely tried. This is also called lottocracy (democracy by lot) rather than democracy.

What matters is not simply gathering people at random and deciding by majority vote, but having them deliberate and reach a conclusion. In Japan too, the city of Sapporo holds the "Sapporo Climate Citizens' Assembly," in which citizens chosen by lot deliberate on what a decarbonized society should look like.

It is as if the lay judge system in the judiciary, which began in Japan in 2009, is now being applied to legislation and administration as well.

Summary

The usefulness of randomness in investing is also described by Taleb, famous for The Black Swan, in Antifragile. We fear randomness and fluctuation so much that we prefer stability. But stability is stability under certain assumptions. When those assumptions collapse, fragility is exposed all at once. Conversely, one of the things Antifragile argues is that by having a fragility that tolerates change, a system becomes stable.

I think part of the reason Japan has lost strength in science and technology over the past few decades is that, in order to avoid failure, it put too much money into "projects likely to succeed", that is, it avoided risk, and so could not invest in things that were likely to fail but might be tremendously successful. By not taking risks, we become fragile as a whole.

Of course, random selection does not solve everything. Careful design is needed. There is also a theory that the archons of Athens gradually lost real power after becoming selected by lot, and generals chosen by direct election began to gain power instead. The lay judge system that began in Japan in 2009 has also been recognized as having some effect, but many problems remain. A system is not perfect from the start, nor does it produce effects as soon as it is introduced. It always needs improvement.

Still, if choosing at random gives opportunities to more people and raises the chance of finding ideas that might truly take off, it may be worth considering as one method.

Incidentally, the simplest way to give opportunities to more people and "fund many small projects" is to "increase the total budget." In fact, the EU set a budget of €60 billion for Horizon 2020 in 2014–2020 and €100 billion for Horizon Europe in 2021–2027, which by simple calculation is an increase of 67%. On top of that, it is trying to find new seeds through things like random funding, which I find an interesting development.

Footnotes

1. From the description that "of the 1,624 books left in Newton's library after his death, 259 books, or 16 percent, related to mathematics, natural philosophy, and astronomy, while 518 books, or 32 percent, related to theology and philosophy" ↩

2. That said, there is also a survey result indicating that the time spent on application materials does not change much even when selection becomes random ↩

3. FY2018 Industrial Technology Survey Project: Survey of Trends in Industrial Technology in Japan and Abroad and similar materials give details ↩