OpenSTA Summit 2005 - Hebden Bridge, UK

[textile] Saturday 28th May - Sunday 29th May 2005 "OpenSTA":http://opensta.org, possibly the best Open Source, Free Performance/Load Testing tool available has had an interesting past. Based on the outcome of last weekend, I would say it has an even more interesting and exciting future.

The best OpenSource tool for general web based load generation is OpenSTA "-Scott Barber":http://www.testingreflections.com/node/view/908

Well, this event was probably too informal to call it a summit, but it was a coming together of some of the key players in the project where we discussed a great deal. The agenda for OpenSTA's road-map was firmed up and I am sure that I wasn't the only one who felt inspired by the experience. Project goals, opportunities, migration of legacy code and some exciting concepts for OpenSTA's short, medium and long-term future were all discussed. Read on to find out more... h2. People Attendees included some of the original OpenSTA team, particluarly Dan Sutcliffe and Geoff Hall. For anyone that doesn't know, Dan is one of the original developers of OpenSTA, responsible for designing a good part of it too. Geoff was the project manager and played a major role in designing OpenSTA, developing some parts of it too. And, of course, there is "yours truely":http://www.testingreflections.com/user/view/2 - a key player in the OpenSTA community for the last four or five years. Meeting Dan Sutcliffe was like catching up with a long lost friend, even though this was actually the first time we had ever met face to face in the four or more years I have been involved in OpenSTA. Day one entailed talking a lot about history and learning why some aspects of OpenSTA aren't necessarily ideal, whilst others are probably still better than many commercial alternatives. We discussed both its strengths and weaknesses. Many ideas were bounced around for its future which Dan and I focussed on during the second day. Day two involved just Dan Sutcliffe and me. We both had some specific ideas about how OpenSTA should move forward. After a bit of a hiatus, Dan is now dedicating his time 100% to OpenSTA. His vision for OpenSTA includes making it XML driven, enable call outs to externally written code (e.g. using SOAP), making it cross-platform and updating the technologies that it is built on. This opens up numerous possibilities. h2. Goals Behind the ideas we both have for OpenSTA, there are some core goals. Particularly: 1. Increase the user-base. The more people that use OpenSTA, the more people are likely to hear about it through word of mouth. Making it easier to use will ensure that potential users of any level will not be discouraged from using it. Although performance testing is a technical activity and OpenSTA is pretty straight forward to a seasoned performance tester, making it even easier to use may encourage novices to start learning the craft. Dan highlighted that one of these individuals may discover a previously unrealised aptitude and grow to become a great performance tester/engineer. Another idea is to set up a not-for-profit organisation to raise funds and help to steer OpenSTA's future, under the influence of the user-community. In addition to funding development, the OpenSTA NFP would also fund marketing campaigns. Such marketing campaigns may include sponsoring speakers at major testing conferences or just direct/indirect advertising. Regular releases and high visibility of the user and developer mailing lists would also raise confidence that the tool not only has a past but also has a future. With an increased user-base, there will also be more opportunities to generate commercial revenue that, ultimately, will benefit the ongoing development of OpenSTA through re-investment in the tool. 2. Increase Investment and Fund Raising Development of OpenSTA may be accelerated if more companies invest in it, either with cash or by donating resources. Red Hat and Suse generate revenue by selling commercial support packages and enhanced features with their free, open source Linux distributions. They re-invest in the core of linux, which is freely available, advancing it and growing it as time progresses. We would like to create opportunities for this model to work for OpenSTA. Investment in OpenSTA is essential for it to grow and improve at the pace we hope it to. It was assumed that commercial users of OpenSTA would be encounraged to invest if they benefit from using the tool, however, this seems to rarely be the case. Many companies seem to prefer to use it without giving anything back. This is despite it being in their interests for OpenSTA to grow and keep pace with technology so that they can continue to benefit from OpenSTA's existence. If there was a commercial advantage or even profit to be made by investing in OpenSTA, companies might be much more inclined to invest. By making it easier to build enhanced functionality as commercial add-ins, or be able to wrap the core in a more sophisitacted user interface and sell it as a commercial product, OpenSTA may become increasingly attractive for investment. It will also increase the opportunities for consultants to write their own proprietary helpers to increase their own efficiency, giving them an edge when bidding for work. Although commercial opportunities are important, unrestricted community development is very much a key value of the project. 3. Make it more attractive to community developers OpenSTA is built on C++ and MFCs using Visual Studio 6. This is not particularly attractive for developers that wish to practice their skills in a community development project. By moving OpenSTA to more open, cross platform technologies using the latest protocols and platforms, it will become much more attractive to community developers. This presents another attractive prospect for developers. Moving from legacy code to new, employing some of the latest patterns and metaphors (such as Fowler's Strangler Metaphor) in practice will be both challenging and exciting. This is public-domain experience that could enhance the CV of participating developers. h2. Brain Storming Dan and I covered numerous directions that OpenSTA could go in. This included his ideas on wrapping, and eventually consuming the existing Corba layer with SOAP. One of OpenSTA's strengths is its clever threading model. Unlike some other load testing tools, each VU doesn't get its own CPU thread. Efficiency of resource utilisation is maximised through a task brokering mechanism, passing tasks for a given user out to a thread pool. These are features that we want to keep - especially with OpenSTA's impressive concurrent VU capacity per load-injector. An early stage will be to have an XML schema that supports all the capabilities of the TOF files (compiled scripts). This will exist in parallel with SCL (scripts). Eventually, SCL itself will, instead of being compiled into a proprietary TOF, will be transformed into XML. It will be possible to script natively with XML or SCL. Call-outs from XML will be enabled. Call-outs will enable calls to external code, perhaps initally using SOAP. We discussed COM but one key principal is to make OpenSTA truely cross platform - over time. The use of other standard languages, such as Ruby, Python and more, was discussed, however, these all present challenges in enabling them to take advantage of the threading model. The Script IDE and user interface was also covered, where we discussed numerous possibilities including Eclipse or a QT based rewrite and redesign of the GUI as two of the many possibilities. h1. So, now what? With Dan dedicating his time to OpenSTA, those that care, benefit or stand to benefit, need to support the ongoing development of the tool with time, money or by spreading the word! OpenSTA is such a capable tool already but has never realised its full potential. Moving to a cross-platform build environment and modernising the OpenSTA core is essential to making this possible. Once OpenSTA has a more accessible and open API, the sky will be the limit! Viva la "OpenSTA!!!":http://opensta.org ----------------- Links: * "OpenSTA Home Page":http://opensta.org * "OpenSTA Community Portal":http://portal.opensta.org * "Martin Fowler's Strangler Metaphor":http://www.martinfowler.com/bliki/StranglerApplication.html * "Brian Marick's Suggestions on Legacy Code":http://www.testing.com/cgi-bin/blog/2005/05/11#fixing-legacy-code