Not Sure? Ask Everyone

Crowdsourcing is becoming an increasingly common tool to solve scientific challenges both big and small. It is even being put to the test in AIDS vaccine research

By Kristen Jill Kresge

Science is all about discovery. And new discoveries can come about in many different ways. In biomedical research, several companies and organizations are now exploring different approaches to encourage new discoveries or stimulate innovation, some of which rely on collective wisdom.

Crowdsourcing is one of these approaches. This principle, dubbed by Jeff Howe, a contributing editor at Wired, a popular technology magazine, describes a phenomenon by which an undefined, generally large group of people or crowd takes on tasks in response to an open call. The open call is often issued via the Internet. This approach is used to solve all kinds of simple tasks, such as digitizing books and periodicals published before the Internet, as well as for more complicated scientific problems. Crowdsourcing is now even being used to address some of the challenges confronting AIDS vaccine researchers.

Reaching the crowd

There are several different ways to get the public or crowd involved in solving scientific challenges. One way is to use online games. Last year, researchers at the University of Washington introduced the online game Foldit, which aims to find the lowest possible energy structure of different proteins. Foldit players use their computer mouse to move around parts of proteins, which are displayed on the screen. They score points by getting the protein in a conformation closer to its lowest energy state.

Recently, Foldit announced a new component of the game that allows players to manipulate HIV’s Envelope protein (Env), which covers the exterior of the virus, to expose areas of that protein that would be potentially vulnerable to neutralizing antibodies (Y-shaped proteins that bind to viruses and disable them).

Originally, a team led by David Baker, a University of Washington professor of biochemistry, developed a program called rosetta@home. This program, which could be downloaded by anyone, used the downtime of multiple computers to sort through protein structures. The results of the calculations were then displayed as a screensaver. Foldit was created because users of rosetta@home wanted to participate, not just watch, Baker says. “They thought they could do better,” he adds. And it seems that they can. People see which particular options to try in a more efficient way than computers would, says Zoran Popović, a computer scientist at the University of Washington who developed Foldit with Baker and others. “They can find solutions that the computers have not found,” Popović says.

New companies have also sprung up to facilitate crowdsourcing of scientific or engineering challenges. InnoCentive and NineSigma are two of these companies. They run websites, where, for a fee, organizations (referred to as seekers) can post specific challenges they want solved. Anyone can view the challenge or have it sent to them by email and then propose a solution. The seeker can then review the submitted solutions and determine if any of them meet their requirements.

At InnoCentive, some challenges only require a written proposal of ideas about how to solve the problem, while others require additional evidence showing that the solution actually works, such as original data from experiments or even a physical sample. The seeker then pays a cash award to the solver who provides the solution that they find suitable.

NineSigma was founded in 2000 by Mehran Mehregany, a professor of electrical engineering and computer science at Case Western Reserve University. Mehregany says he founded the company once he realized that the elaborate system the government uses to issue open calls to academic researchers wasn’t available to industry. “Industry does not have a similar systematic infrastructure to broadcast its science and technology needs,” he says.

The solutions to challenges posted on InnoCentive or NineSigma can come from anyone, anywhere, and they often do—the success rate for the challenges posted through these sites is surprisingly high. InnoCentive says that about a third of its challenges get solved. Karim Lakhani, an assistant professor in the technology and operations management unit at Harvard Business School, says it’s hard to know how this compares with the in-house success rate of companies, since most don’t keep track of that or share it publicly. But, he says, in his conversations with research and development chiefs at various organizations, they seem “very surprised” by the high success rate of InnoCentive, especially considering that the challenges that get posted on InnoCentive’s website are likely there because the companies couldn’t solve them in-house.

Ed Melcarek, a 60-year-old Canadian engineer and scientist, says he has made over US$115,000 for solving seven challenges on InnoCentive since 2003. InnoCentive declared him one of the most successful solvers of 2007.

On average, it takes two weeks, or 80 hours, for solvers to come up with a solution to an InnoCentive challenge, according to a study of 166 challenges solved through the company’s website between 2001 and 2004. The study also found that the further removed the background of the solver was from the area the challenge pertained to, the more likely it was that the problem got solved, says Lakhani, who helped conduct the study. “In our analysis the problem solvers said that the problem that they tried to create a solution to was typically outside their own field of expertise,” he says.

For example, John Davis solved a challenge to help with oil spill recovery. The challenge, from the non-profit Oil Spill Recovery Institute, was to find a way to liquefy the oil/water slush collected on barges from arctic waters in the case of an oil spill so that it could be pumped from the barges to larger storage tanks on land. Davis says he remembered that construction workers used a vibrating device to keep the concrete from solidifying at construction sites. He thought the same approach might work on the oil/water slush. After a day of work, and a call to the company asking if they could modify the vibrating device for this purpose, he filed the solution. A few months later, he received $20,000.

InnoCentive most often has companies as clients but it also tries to attract non-profits to post challenges, says Dwayne Spradlin, president and CEO of InnoCentive. “We try to make it very appealing for non-profits because we think [they] have not had access to the same innovation channels that commercial interests have,” Spradlin says.

From 2006 until 2008, the Rockefeller Foundation collaborated with InnoCentive to encourage non-profits to participate. The foundation would typically pay the fee required to post a challenge as well as half of the award money on behalf of the non-profit, according to Amanda Sevareid, a research associate at the Rockefeller Foundation. Once a problem was solved, the foundation paid the rest of the award money if there was evidence that the solution was successfully implemented. Six non-profits have taken part in the program, and most of their challenges have been solved. In late 2008, the TB Alliance announced two awards of $20,000 each for improving the synthesis of a tuberculosis drug candidate.

In 2008, IAVI posted a challenge on the InnoCentive website as part of the Rockefeller Foundation program. The challenge issued was to create a stable version of HIV Env. In its natural state, the Env protein is unstable and breaks down easily when entering the body, according to Kalpana Gupta, director for new alliances and initiatives at IAVI, who was involved in developing the challenge. As a result, it has been difficult to trigger antibody responses against this protein. Having a stable form of HIV Env—the primary target for neutralizing antibodiesÑwhich researchers could experiment with in the laboratory, could help in the development of AIDS vaccine candidates.

This article was adapted from an article written by Andreas von Bubnoff in the May-June 2009 issue of IAVI Report.