Re: Thinking out loud about a new RPG web framework

[Ralph Daugherty <rdjfc@xxxxxxxxxxxxx>]

Re: DIM'ing data structures for 10,000 sessions for state web serving -

Aaaron wrote:
<i>It seems this would immediately put a limitation on the framework 
being used for public purposes because it would be limited by the RPG 
language vs. being limited by processing power (you can buy more 
processing power, but you can't buy more RPG DIM element entries).  One 
could argue that 10,000 users is a lot, but in my mind that number could 
be eaten up pretty fast based on current RPG customers using this 
framework to take online orders and such.</i>


 I agree, this is a fundamental question, and subsequent decision that 
needs to be made, Aaron. Do we take full advantage of i5/OS and ILE to 
go beyond the same ol' same ol', or instead limit ourselves to the same 
ol' that everyone else can do?

 If we use any aspect of the iseries capability, then we should go for 
it and use it all to go beyond. There might be a philosophy of just use 
as little as possible, such as calling RPG programs from Java or PHP, 
with the idea that the RPG could be rewritten into another language if 
moved to another system, but we're not going to produce something 
superior by staying lowest common denominator as much as possible.

 My goal is to exceed the norm with iseries i5/OS and ILE capability. 
Yes, that's iseries only, but everyone else has their "alleged" generic 
solutions, and if that's the best then the iseries will run it, if it's 
even close to actually being generic.

 But a ton of solutions are being developed in .NET/.ASP for IIS under 
Windows, they utilize every aspect of their Windows environment to 
produce the optimal solution, and you don't see them sweating over 
whether it works under i5/OS or AIX or Linux. Because it doesn't, it's a 
proprietary solution.

 But more importantly, to the point of session users and the data 
structre dimensioning scheme, the following:

1. Dimensiong data structures is the easy way to do this and explain it, 
but in a large and complex environment I would provide in the code an 
alloc of memory for each of the data structures for a session and access 
them with basing pointers. I wouldn't even deallocate the memory, just 
build up the pool as needed for concurrent sessions. This would be 
equivalent to dynamic dimensioning.

2. Data structure indexes can go to 32K. I set 10,000 as an example of 
an ambitious goal of running a major web app system from one iseries, 
which given optimal use of RPG and dataq servers as I describe with 
state maintained I think is possible. I would give something like a 20% 
overhead factor for web page processing versus 5250 processing and say 
that the concurrent users capable of being served would be 20% less than 
interactive session capability for a given box. That's just off the top 
of my head, no benchmarks. It could be more of an overhead than that.

3. In a robust environment where we're talking about of 10,000 or more 
logged in users of a business system, e-commerce, and the like, you 
start getting into a server farm of iseries. But the if the session 
maintains state, then session web pages have to be directed to the 
iseries where that user is logged in. Now here's where it gets interesting.

 The standard wisdom is that stateless web serving allows any web page 
to go to any farm web server in any mixture from different sessions. But 
then the state has to be built up and restored, and that's the red 
herring. A database server has to have the state stored in it, and you 
have to get to the right database server, unless you can keep multiple 
database servers synched in real time. Which we need to do anyway for 
high availability, but there's a differnce between high availability 
synching for failover and keeping multiple production systems in synch, 
such as two or n way synching between the databases to start with 
instead of one way to the high availability backup server.

 Less than acceptable ways of doing this store state in a cookie and/or 
hidden fields in the web page, and with encrypted data will get a lot of 
people by, but I think if anyone imagined our 5250 sessions and programs 
through the years storing all the program variables, authorities, and 
everything else we have in an interactive session in the 5250 binary 
stream which carries it around so that it could "go to any AS/400" in a 
server farm, we would not be in any way, shape, or form the reliable, 
secure, and nearly bulletproof system that we are, I think most anyone 
in the iseries community would agree with that. And that's what using 
cookies and hidden fields is doing.

 So rather than store state in a central database that all the farm 
servers access to restore state for a given web page, the session 
communications in a very large complex environment in my scheme would be 
 directed to the farm iseries the session is logged into, so the 
routing software has to have a level more intelligence than round 
robining to next farm server. But then state is there and appropriate 
database servers are just accessed to retrieve business data as we do in 
normal programming.

 In reality, a complex environment would have more than one business 
system, and part of that smart routing would be routing to appropriate 
iseries running the system rather than all iseries running every 
business system and users logging in on random ones, etc.

 That's just from one of your comments, Aaron. But these are good 
questions for us to come up with an optimal solution, as you suggest.

 rd