Skip navigation.

Archives

Python - Web Load Tester - Multiple Processes and Threads

Here is my latest HTTP load generator in Python (web performance test). You just give it a URL and some runtime parameters and it will hammer a resource with HTTP requests. You could easily adapt this to run against a web service or set of links. It is useful for quickly loading a web resource with synthetic transactions for performance testing or tuning purposes.

I have built lots of different load testing tool in Python in the past (see: Pylot), but they all suffered a similar problem. Their concurrency model was based on Threads. Because of this threaded design, combined with Python's GIL implemenatation, my tools were unable to fully utilize multiple cores or processors.

Here be dragons

I like maps. I wasn’t sure why maps appealed to me until it dawned on me one day that maps are a form of data visualization. The phrase “here be dragons” was used by mapmakers many years ago in reference to uncharted or unexplored areas. I like the phrase. Maps were marked with dragons and in some cases, other animals were drawn to signify danger or unknown, uncharted territories. I find the concept and the phrase kind of cool in a geeky sort of way. You can read more about this old mapmaker’s phrase here.

I also like SQL. I like being able to access a database and spend time with data. I teach a couple of different SQL classes and one thing I’ve found repeatedly has been people’s intimidation by having to write a join in SQL. It’s amazing how far people will go to avoid the dreaded join – it’s a bit like “here be dragons” - a territory marked with fear and uncertainty.