Crowd Testing

What is Crowd testing?

Crowd testing, or crowdsourced testing, is when a virtual group of testers troughout the world (a crowd) is involved in testing  rather than a traditionally managed test team on a single location. It is a form of test organization.

Crowd testing takes its roots from crowd sourcing. The key to the success of crowd sourcing is that within a larger group of people a lot more ideas can be generated and those ideas can influence and support each other. Current cloud en web 2.0 technologies enable the sharing of ideas within large groups.  This same advantage goes for crowdsourced testing: a lot of testers can find a lot of places to look for bugs and the bugs that one tester finds can influence other testers to look for similar bugs.  

Crowd testing has a second advantage.  It is not just the ideas and experience of the different members that are more diverse, but also the different hardware and software configurations that the software under test is tested against. Crowdtesting enables a test where software can be tested on a large amount of devices, browsers and operating systems where all tests are executed in parallel by different testers, minimizing the throughput time of the tests.

A cloud test is best applicable when the software under test is meant for public use. Websites and apps especially. If the software under test is for private use – for instance within a company – then a internal test team that has the relevant knowledge of the application is more obvious.

People Involved

Generally, there are three parties involved. The client, the organizer of the test and the cloud of testers.


The artifacts of a cloud test can be the same as of any other test. In most cases, the client is mostly interested in finding bugs, so a bug report is the most obvouis artifact. But in most cases, test plans, test cases and all other test artifacts can be made as well.  

Success Factors

The success factors of crowd testing depend on the goals of the client

The size, skills and diversity of the crowd.
The goals of the client. Generally, a cloud test gives very valuable results in terms of finding bugs and device coverage. If the client is looking for guarantees and testcase coverage, the results can be less clear.