Re: Overwhelming a Web Application Server

[Thorbjoern Ravn Andersen <ravn@xxxxxxxxxx>]

Nathan Andelin skrev:
> > From: Thorbjoern Ravn Andersen
> > If there is one thing experience has taught me, it is that you must 
> > measure instead of guessing when analyzing performance problems.
> >     
>
> Maybe we spent too much time talking about session limits and expiration.  Hmmm, who came up with that idea in the first place ;-)
>   
I did.  It is the lowest hanging fruit :-)

It doesn't fix anything but it may buy you time to figure out what the 
next fruit is.

> Your point about measurement is well taken.  Walden pointed it out too.  Unfortunately, my information in this case is from a user's perspective, and a developer's perspective - not that of a site administrator.  I don't have profiling data.
>   
First of all, what is the actual application doing stuff written in?   
Apache itself is so fast to serve static pages that it can easily 
saturate a 100 Mbps network connection.

Then, if profiling data is not easily available then you can always add 
logging yourself to the application.  Just having the start time and end 
time of each invocation (this probably requires a job number or so?) 
allows you to derive the processing time of each request, the number of 
concurrent requests at any time plus more I cannot remember right now.

It might be something as simple as a lock forcing some processing step 
to be strictly sequential, causing all jobs to stand in line waiting.
> But if you're willing to indulge in a somewhat hypothetical discussion, what specific measurements would you look at? Under IBM i, I think most operators would check CPU utilization & page faults, first - just because that's easy. Suppose the Tomcat server is consuming 95%.  Would that mean much?  What would you check after that?
>   
Generally if a modern PC (I'm not familiar enough with the AS/400 to say 
if it applies here too) has so much computing power so if Tomcat uses 
95% there is something doing unneccesary work (busy looping e.g).   Page 
faults are interesting, as the Classic JVM is basically unbounded, so 
the heap may grow very large which doesn't go well with small pools.

For the Classic JVM, I would look at ANZJVM and DMPJVM over some time to 
get an idea what the system is doing.   Also enabling verbose garbage 
collection can get you an idea if the system does excessive garbage 
collections implying the process needs more memory.

Then I would ask again :-)