Re: QRCVDTAQ

[Simon Coulter <shc@xxxxxxxxxxxxxxxxx>]

On 10/07/2006, at 2:20 PM, Steve Richter wrote:

> come on.  networking is different story. I have not worked on MQ
> applications but I have written a number of dtaq applications. I wrote
> a fax system that used a lot of data queues between the PC that had a
> fax board and the as400 that had users that wanted to fax stuff. It
> started as something that seemed doable, but it got very complicated.
>
> The fundamental problem with data queues is your client code does not
> get a definitive return code from the server.

Again, that has nothing to do with queues per se. The queue APIs inform 
you of successful send or receive and provide error messages or 
exceptions in the case of failure. Anything over that is an application 
issue. If your sender needs to know the data was successfully received 
then your design should include that. For example, the receiver could 
send a status entry to another data queue that is periodically checked 
by the sender. The sender could maintain a list of sent entries and 
check them off as a completion status is received. If you think about 
it you'll see that the confirmation doesn't have to synchronous so you 
still get the asynchronous benefits of queues but don't lose track of 
what's happening.


> > 3. What you have described repeatedly and well is what I understand
> > to be the contract model of development - a "user" and a "service"
> > contract to behave certain ways, such as that the user will validate
> > data and deliver it to the service in the form the service expects to
> > see, then the service will process it and return certain data as
> > agreed upon. I think we do this implicitly, but there is a framework
> > in which this is more defined, IIRC. Even a development environment
> > called Eiffel, I think.
>
> that seems fundamentally flawed to me. The server has to validate the
> transaction before it is applied to the database. If the client does
> validation also that is a duplication of work.

Your thinking is flawed again. The web is a perfect example of where 
both ends probably should do validation. Consider a browser client that 
relies on Javascript to perform field validation. Given that it is 
really easy to disable Javascript in browsers, and that the server has 
no real way of knowing whether the data it received came from it's 
expected client or anything capable of communicating over HTTP then the 
server application has a responsibility to validate the data.

Web applications would typically validate at both ends because 
validating at the client gives a quicker response to the user, and 
validating at the host ensures accurate data. At least with a web 
application you have a synchronous operation during the POST/RESPONSE 
cycle so it is easy to have the server return error messages to the 
client.

Queues are by their nature asynchronous and thus impose a greater 
responsibility on the sender to provide valid data. However, even here 
it is possible to build in asynchronous confirmation if the application 
requires it.


> > 4. RPC - remote program calls, for some reason, creep into my
> > thinking here. Or call it asynchronous processing or think about MQ.
>
> Can RPC throw exceptions back to the remote caller? If it can it is
> superior to MQ.

Certainly can in the Java implementation. In Unix implementations it's 
usually done by return codes.

> > 5. As to robust production code that depends on this kind of thing,
> > it is well known that much of the iSeries code is using user indexes
> > and user queues (an even lower-level variant of a data queue). I
> > guess IBM has made a mistake depending on these "unreliable" things.  
> >  <vbg>
>
> Hah! i5/OS had to run on a 1 CPW B10. Queues and batch processing are
> efficient. That is their advantage. The downside is you dare not mess
> with the system once you have it working. IBM for example cannot get
> the interactive subsystem to support multi threading.

This has nothing to do with queuing. It has everything to do with 
storage assumptions. The device management code may use static buffers 
for 5250 processing, it may save state between calls, or any one of a 
dozen other things that make retrofitting threads difficult. Not 
impossible only difficult. Whether IBM add multi-threading support to 
interactive jobs is a question of how much money and time would it take 
and who would make use of it. I can count on the fingers of one hand 
the OS/400 programmers capable of making good use of threads in an 
interactive job. Add to that the block mode interface of interactive 
terminals and there is no business case to support threading in 
interactive jobs.

Of course, if IBM develop a native replacement for 5250 then it would 
make sense to design that for threads.


> The spawn and SBMJOB APIs are also very slow.  Is that the nature of 
> the beast or is
> it that all the queueing and dequeing cannot be circumvented at risk
> of breaking the entire mechanism?

It's the nature of the beast as you'd work out if you thought about it. 
SBMJOB and spawn start JOBS. Jobs are heavy-weight processes. It is the 
creation and destruction of a job that makes these slower than starting 
a thread. Processes on other systems are usually much lighter in system 
requirements than OS/400 jobs but then they don't provide as much 
feedback either. Want to see the search path for a Unix process? Want 
to see the call stack for a WinDOS process? Want to see open files? 
Want to see a running log of the job? Hah! You get what you pay for.

Threads are not a straightforward replacement for spawn--even on other 
systems. Using spawn (or even fork and exec) to start a separate 
process is a trivial task. Using threads requires careful thought and 
even more careful implementation.

Regards,
Simon Coulter.
--------------------------------------------------------------------
   FlyByNight Software         AS/400 Technical Specialists

   http://www.flybynight.com.au/
   Phone: +61 3 9419 0175   Mobile: +61 0411 091 400        /"\
   Fax:   +61 3 9419 0175                                   \ /
                                                             X
                 ASCII Ribbon campaign against HTML E-Mail  / \
--------------------------------------------------------------------