We should learn from 60-80ties, shouldn't we? Server resource usage

Scott Barber begins his recent white paper about performance testing with a short paragraph I would call just another "we have made no progress for years". I do believe that one of the reasons could be found in a history. During the era of "big-shared-machines" both developers and architect were really concerned about hardware resource usage. However the natural progress in this area halt during Personal Computer (PC) era – building countless (relatively) cheap PCs and making servers stateless (with HTML/cgi/jsp). The new era is coming: hardware costs are low, maintenance cost - high. Online applications, SOA, thin clients become more and more popular bringing back the need for big-shared-resources. Are we going to reinvent a wheel or could we learn something from the practices developed years ago? [textile] As a QA person (not developer or architect) I'm not going to research what to learn - I hope to give an inspiration for someone to study the topic, learn from past and share what learned.

Prescribed budget of resources: do developers care less today ?

I'm reading The Mythical Man-Month - a book first published in 1975 (I've got second edition dated 1995). As a mathematician I love old books that contains universal truth rather than today's hot news. I could get the news from blogs and publications. What paid my attention in that project management and no way a performance book? The following statement:

The program must also be designed so that it uses only a prescribed budget of resources-memory space, input-output devices, computer time.

And even more later in text:

A discipline that will open an architect's eyes is to assign each little function a value: capability x is worth not more than m bytes of memory and n microseconds per invocation

As far as I understand that comes from legacy mainframe systems where each process have been required to prescribe (forecast) the resources it is going to use, the system in a turn ensure (guarantee) that the required resources are available for that process. In nowadays operation systems resource are not forecasted by processes - it's usage is extended on demand (e.g. demand more and more memory until OS runs out of it and hang and tools like garbage collectors make the management job for us, while great debugging tools help to identify memory leaks). So developer doesn't need to think about resources during coding any more - only about the logic, do he/she?

Well, that's bad engineering practice, isn't it? I've seen numerous projects canceled (from small to multi-million ones) because the developers did not think/forecast upfront either of resource demand: network (as nowadays input-output devices) or processor time (the computer time). Memory seems to be of less concern due to it's scalability, low cost, OS restrictions and tools I mentioned above. I hope developers think of the right algorithms and correct architecture patterns when writing their code. But do they have the right scale in mind (e.g. don't use bubble sort algorithm for large data sets, but use it if you know the size is going to be only a few items). However even as fastest as possible may appear not to be fast enough because of 3rd party or legacy system limitations. Forecast resource usage and develop code to mitigate risks of abuse (e.g. do not allow flooding) - that's what should be done, shouldn't it?

A little bit of history (of mainframes) in my improvisation

I don't like history until it is on discovery and with a bottle of beer in my hand. However I still remember and agree with my history teacher that there are so much if not everything we could learn from history. That's why more than a half of my favorite IT related books are more than 10 years old. So what I know about the history? As far as I remember the initial reason for constructing and using the mainframes with a big processing power was the cost of the hardware and it's low performance (not only processing but especially data storage and input data access speed) compared to what we have nowadays. The idea was to create a single processing unit that will be used by a great many people, thus literally sharing hardware the cost among all of them.

In those days, when hardware cost was the main reasons it was quite obvious that each individual process (brought in by individual user) had limited quota of processor and memory usage. More over the processor time should be reserved up-front (as far as I remember for each program you have to forecast how long it will run or something). When I was in university the mainframe era was over and we only learned the main concepts of the big computers, so if anyone with real experience could correct me it would be great. For those who hear about it the first time I would like to draw the following picture. Imagine that the only way to cut grass in front of your house is using a highly automated haymaker that costs 20 000$. But you only need to use it once per few weeks for a few minutes due to it's high speed. More over it's highly reliable due to security reasons. So you get all your neighbors or even a whole town to buy a single instance to share it later. You create a schedule of it's usage. And you don't allow using it anyone too much (for example in countryside to cut his 100 arks).

The PC-era changed this. Initially PCs was somehow cheap. Today full fuel tank in my car cost more than used Pentium. However the same fuel tank cost almost approximately the same as Internet connection per year, also reducing my phone bills so that I literary got it for free. And the speed of connection allows me to download in a second the size of all the free memory that first PCs had. PC-era is going to an end soon, but not the way I expected it 5 years ago. PCs are so cheap that they are not going to die. The software is changing, not the hardware. And it is changing already. Do you know what wikipedia today says about mainframes?

a single mainframe can replace dozens or even hundreds of smaller servers, reducing management and administrative costs while providing greatly improved scalability and reliability.

You see - even for mainframes the recognized idea is reducing management and administrative costs, no more hardware costs.

So what I wanted to say - the history is repeating itself. Although the reasons have changed from hardware to maintenance costs, the idea of single big computing unit is back in form of web servers, SOA, etc. Take a look at popularity gaining online-games that don't require you to install and configure the game software on our computer: even if they are simple to trivial I've seen non-IT people to play them a lot. Not to mention mobile movement (I'm not big fan of using mobile phones for web browsing, I still think that phone is for calling, computer - for browsing, but there is a great money investing to change this attitude, and you know - at least the idea of JAVA is not as dead as someone expected it to be as a single language for any device...).

Last but not least the security is also a concern: the less data processing happens on your computer, the less access your computer and processes running on your computer has to the central data storage, the easier it to secure the data . Security is not my field however so I will not extend this topic although I believe this may soon become the key reason for going from thick to thin clients, not the installation/maintenances costs.

Read more soon ... Once again - my conclusion is that we go back to idea of a big computer and a “terminals” - PCs connected to it doing only visual rendering. And it is time to consider how to limit each individual PC using the shared resources. That's not true that systems nowadays don't do that. They just don’t do it systematically. Prescribe/estimate/forecast resource usage – you could find evidence of that if you search. I once described my rule of thumb – no single user request should use more that 2 seconds of server CPU time. Well there could be exceptions however (report generation, complex search, etc.). What I’ve seen – that business applications allow users to use such features but restrict to abuse them. I’m going to blog about it soon. Appendix: An illustration

Let me take an abstract example. In this example all request response times for one user are equal, for example 0.5 seconds. All the times goes to utilize the CPU, which is single processor. It means that maximal throughput theoretically is 2 requests per second. Let's now assume that we've got 60 users for that site. If each of them will submit on an average a request per 30 seconds the server will not be overloaded and the response times should be close to those 0.5 seconds (well less than 3 seconds for sure). If on the other hand they will TRY to submit requests more frequently (let's assume they will do it within 5 seconds) - system will not be able to process them faster, so the response times will be the other (30-5=) 25 seconds. Right? So they will win nothing, only frustration.

That's theory, let's become a little bit more realistic. What I do on this site as a user: scenario enter blog: log in, press create content, press blog entry, type in the blog and pres preview, fix the bugs press preview again, save. Now for some steps I don't want to wait another 30 seconds, no way. For others - for example preview - I could do the preview quite a long time. If I have the blog text copied into local file (which I do always) I could fix all the problems in it while looking at preview, I don't need to fix one bug and press preview again. Well, not always I do that because I'm not interested to optimize server resource usage. If there would be a statistics kept for me that analyze that and show something like my rating and the current server load (so that I know if I need to optimize my activities) and the rating changes based on server load and frequency of my interactions - that would make me care. Maybe another rating such as how good a contributor to the site am I - how much a good blogs I create, do I do denote, etc. My idea is too fresh yet, and you know - I'm not the architect, I'm a QA only. I am to provide feedback not to design.