Re: Normal CPU? -- DOMINO400

See, that's another weird thing.  The originating server is an network 
device sending automatic notifications to some of our network admins.  The 
device is outside our firewall, sending to an address at our ISP.  That 
address is then configured to forward anything it receives to our three 
admins' work accounts.  The ISP is the same organization that hosts our 
antispam and antivirus filters, so everything from outside our network 
(including, one would presume, this message) comes through them.  They 
don't see any sign of this ghost message after the original send.  The 
only thing between their server and mine is network routers and firewalls. 
 I guess that could be something to investigate next, but it doesn't seem 
likely that either of them would cache a message, not just a packet, but 
the entire assembled message, for two days.

The entire thing might be moot, now.  It's been over eighteen hours since 
the last of these duplicate messages was received and, to my knowledge, 
the SMTP process hasn't spiked in the interim, either.  Maybe whatever was 
caching the message (if that's what happened) finally decided that it has 
been delivered.

Thanks for all your assistance,
Patrick




Walter Scanlan <wscanlan@xxxxxxxxxx> 
Sent by: domino400-bounces+ptrapp=nex-tech.com@xxxxxxxxxxxx
12/29/2005 06:08 PM
Please respond to
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>


To
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>
cc

Subject
Re: Normal CPU?






Well if it isn't in mail.box I would guess a remote system is sending 
(repeating) the message.

If you add smtpdebug=1 to your notes.ini and restart SMTP the console will 

report all remote connections.

will tell you where this message is comming from.

 
Walter Scanlan 
Senior Software Engineer
Domino & Workplace for iSeries
507-286-6088
wscanlan@xxxxxxxxxx



"Patrick Trapp" <ptrapp@xxxxxxxxxxxx> 
Sent by: domino400-bounces+wscanlan=us.ibm.com@xxxxxxxxxxxx
12/29/2005 04:58 PM
Please respond to
Lotus Domino on the iSeries / AS400


To
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>
cc

Subject
Re: Normal CPU?






I think I need to rethink my question for you.

The original symptom that I was seeing was a SMTP process running at 
outrageous levels (80%+).  It was at the point that none of my users could 


connect (we are a one-system shop), so I took SMTP down and left it down 
for a while so they could catch up.  I left it down for quite a while, and 


Server started to go through the roof.  I turned SMTP back on and Server 
dropped down to what seems more reasonable levels, but SMTP ratcheted up 
to 95% after a while (sorry, not sure how long).  My WRKACTJOB just showed 


me 100.2% utilization -- now that's efficiency!  If only my users could 
connect...

A user contacted me to let me know that he's getting repeat copies of a 
message, and it seems to be each time I have to kill SMTP, he gets another 


copy.  There's nothing in mail.box for this message, but it has been 
delivered to his mailbox over twenty times.  The server has restarted (for 


backup) since this all started yesterday.  Any suggestions for where I can 


be looking to find and kill this ghost message?  Everything else seems to 
be related to this one message.  The secondary question, of course, would 
be what might make a message start doing this?

Patrick

PS  It looks like I was wrong.  I see PM400 on the system, but it doesn't 
appear to be started.




"Patrick Trapp" <ptrapp@xxxxxxxxxxxx> 
Sent by: domino400-bounces+ptrapp=nex-tech.com@xxxxxxxxxxxx
12/29/2005 04:15 PM
Please respond to
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>


To
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>
cc

Subject
Re: Normal CPU?






I think so, but I will have to ask.  Am I correct in assuming I would have 



to order the software trial to proceed?  (As opposed to downloading an 
image from somewhere.)

Patrick




Walter Scanlan <wscanlan@xxxxxxxxxx> 
Sent by: domino400-bounces+ptrapp=nex-tech.com@xxxxxxxxxxxx
12/29/2005 04:07 PM
Please respond to
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>


To
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>
cc

Subject
Re: Normal CPU?






The 25ba is the i810-2467  The 2467 is a 1470 CPW system. 

The 522A is the i520 -8953 is the 2400 CPW system. 

(now you know how much growth you have)

So I would say if CPU popped after the upgrade we do have a problem.

Can you install PM400 and collect the data as I suggested?

 
Walter Scanlan 
Senior Software Engineer
Domino & Workplace for iSeries
507-286-6088
wscanlan@xxxxxxxxxx



"Patrick Trapp" <ptrapp@xxxxxxxxxxxx> 
Sent by: domino400-bounces+wscanlan=us.ibm.com@xxxxxxxxxxxx
12/29/2005 03:44 PM
Please respond to
Lotus Domino on the iSeries / AS400


To
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>
cc

Subject
Re: Normal CPU?






The 520 shows 9406 for the Main Card Enclosure (that's the number that 
looks familiar to me) and 522A for the System Processor Card

The 810 shows 9406 for the Main Card Enclosure, also, and 25BA-000 for the 





System Processor Card

Patrick




Walter Scanlan <wscanlan@xxxxxxxxxx> 
Sent by: domino400-bounces+ptrapp=nex-tech.com@xxxxxxxxxxxx
12/29/2005 03:36 PM
Please respond to
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>


To
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>
cc
domino400-bounces+wscanlan=us.ibm.com@xxxxxxxxxxxx, Lotus Domino on the 
iSeries / AS400 <domino400@xxxxxxxxxxxx>
Subject
Re: Normal CPU?






If you issue a go hardware option 4 
you will find something like this
 MP01             26F2       Operational           System Processor Card 

you can't view performance at a thread level without PM400, so a 
multithread job like server would be hard to debug.

PM400 ofters 30 day trials so you can use it to debug this without paying.

 
Walter Scanlan 
Senior Software Engineer
Domino & Workplace for iSeries
507-286-6088
wscanlan@xxxxxxxxxx



"Patrick Trapp" <ptrapp@xxxxxxxxxxxx> 
Sent by: domino400-bounces+wscanlan=us.ibm.com@xxxxxxxxxxxx
12/29/2005 03:26 PM
Please respond to
Lotus Domino on the iSeries / AS400


To
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>
cc

Subject
Re: Normal CPU?






Both systems are in another town.  Short of looking at the label on the 
front, how can I tell which models they are?  I'm relatively sure that it 
is a 520-9406, but haven't needed to know the 810's model for a while.

I don't believe I have PM400 on this system.  Should I?  Is there another 
way to see that information?

Thanks,
Patrick
 



Walter Scanlan <wscanlan@xxxxxxxxxx> 
Sent by: domino400-bounces+ptrapp=nex-tech.com@xxxxxxxxxxxx
12/29/2005 03:17 PM
Please respond to
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>


To
Lotus Domino on the iSeries / AS400 <domino400@xxxxxxxxxxxx>
cc
domino400-bounces+wscanlan=us.ibm.com@xxxxxxxxxxxx, domino400@xxxxxxxxxxxx
Subject
Re: Normal CPU?






You don't say which 810 and which 520 but.

start with wrksysact  (you need pm400 for this).

It will provide a task list of processes consuming CPU.

Then issue set config debug_threadid=1 on the domino console.

match the thread from wrksysact using cpu to the console message with the 
same thread,


You now know WHAT is consuming CPU. 

Next, WHY?

that will depend on the what..


 
Walter Scanlan 
Senior Software Engineer
Domino & Workplace for iSeries
507-286-6088
wscanlan@xxxxxxxxxx



"Patrick Trapp" <ptrapp@xxxxxxxxxxxx> 
Sent by: domino400-bounces+wscanlan=us.ibm.com@xxxxxxxxxxxx
12/29/2005 03:03 PM
Please respond to
Lotus Domino on the iSeries / AS400


To
domino400@xxxxxxxxxxxx
cc

Subject
Normal CPU?






I have a new 520 (V5R3) -- online less than a week -- that is running the 
exact same load as the 810 (V5R2) that it replaced.  Same configuration, 
just saved and restored from server to server.  Something is bringing the 
new server completely to its knees (97%+ CPU utilization) and I'm not 
seeing what it is, so I'm wondering if anyone can provide ideas where I 
should be looking.  Originally, it appeared to be an SMTP issue and 
bouncing the SMTP task temporarily would resolve it, but I tried that a 
short time ago and never saw the CPU dip below 60% (the vast majority of 
it being the Server task).  We are a small shop and a third of my users 
aren't even on the server, so I can't imagine that we have legitimate 
traffic hitting me this hard when the old box would handle everything we 
threw at it without a problem.

Thanks in advance,
Patrick
_______________________________________________
This is the Lotus Domino on the iSeries / AS400 (Domino400) mailing list
To post a message email: Domino400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/domino400
or email: Domino400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/domino400.


_______________________________________________
This is the Lotus Domino on the iSeries / AS400 (Domino400) mailing list
To post a message email: Domino400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/domino400
or email: Domino400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/domino400.


_______________________________________________
This is the Lotus Domino on the iSeries / AS400 (Domino400) mailing list
To post a message email: Domino400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/domino400
or email: Domino400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/domino400.


_______________________________________________
This is the Lotus Domino on the iSeries / AS400 (Domino400) mailing list
To post a message email: Domino400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/domino400
or email: Domino400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/domino400.


_______________________________________________
This is the Lotus Domino on the iSeries / AS400 (Domino400) mailing list
To post a message email: Domino400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/domino400
or email: Domino400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/domino400.


_______________________________________________
This is the Lotus Domino on the iSeries / AS400 (Domino400) mailing list
To post a message email: Domino400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/domino400
or email: Domino400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/domino400.


_______________________________________________
This is the Lotus Domino on the iSeries / AS400 (Domino400) mailing list
To post a message email: Domino400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/domino400
or email: Domino400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/domino400.


_______________________________________________
This is the Lotus Domino on the iSeries / AS400 (Domino400) mailing list
To post a message email: Domino400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/domino400
or email: Domino400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/domino400.


_______________________________________________
This is the Lotus Domino on the iSeries / AS400 (Domino400) mailing list
To post a message email: Domino400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/domino400
or email: Domino400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/domino400.