Testing different versions of components on different hardware for different types of functional usage brings up the challenge of combinatorial explosion. For example, a typical SpikeSource release integration involves optimizing 272 parameters distributed across 189 configuration files across 63 components yielding 7 preconfigured stacks for developing applications in Java, PHP, Python, Perl, C, and C++.

We need a new approach in testing to address these challenges. Our recommendation is to apply the very nature and strength of the "architecture of participation" to open source testing. To quote Tim O'Reilly, "we need a system that allows for a real free market of ideas, in which anyone can put forward a proposed solution to a problem; it becomes adopted, if at all, by acclamation and the organic spread of its usefulness."

Hence the requirement to build a Participatory Testing System that allows developers to participate by testing their applications as the payload.

Author: Murugan Pal
Published: O'Reilly Network, April 7, 2005