The mpi API on beowulf at Hanford

Legend:
green ball Normal status or debugging message
yellow ball Notable condition which may be a non-fatal error
orange ball Error condition not fatal to job
red ball Error condition fatal to job
blue ball Notable condition which is not an error
purple ball Currently undefined
email Condition requires email notification of the responsible administrator of this API
telephone Condition requires phone notification of the responsible administrator of this API

Link: API Status Page for Hanford

02/28/07-12:47:10 PST 02/28/07-20:47:10 GMT 856730844 STARTUP archiveLog file "/ldas_outgoing/logs/LDASmpi.log.html" already closed. (archived as /ldas_outgoing/logs/archive/mpiAPI/LDASmpi.856629908)
02/28/07-12:47:10 PST 02/28/07-20:47:10 GMT 856730844 STARTUP closeListenSock no cid registered for service 'data'
02/28/07-12:47:10 PST 02/28/07-20:47:10 GMT 856730844 STARTUP mpi::init unused data port 10018 closed
02/28/07-12:47:10 PST 02/28/07-20:47:10 GMT 856730844 STARTUP mpi::init port 10018 (jobstate) opened on beowulf as sock7
02/28/07-12:47:10 PST 02/28/07-20:47:10 GMT 856730844 STARTUP bgLoop Looping process watchlogs started
02/28/07-12:47:10 PST 02/28/07-20:47:10 GMT 856730844 STARTUP openListenSock port 10016 (operator) opened on beowulf as sock8
02/28/07-12:47:10 PST 02/28/07-20:47:10 GMT 856730844 STARTUP openListenSock port 10017 (emergency) opened on beowulf as sock9
02/28/07-12:47:10 PST 02/28/07-20:47:10 GMT 856730844 STARTUP leakLogger inital size of mpi API: 21032 kB
02/28/07-12:47:10 PST 02/28/07-20:47:10 GMT 856730844 STARTUP bgLoop Looping process etchosts started
02/28/07-12:47:10 PST 02/28/07-20:47:10 GMT 856730844 IDLE bgLoop Looping process statpagefile started
02/28/07-12:47:10 PST 02/28/07-20:47:10 GMT 856730844 IDLE bgLoop Looping process killedjobreaper started
02/28/07-12:47:10 PST 02/28/07-20:47:10 GMT 856730844 IDLE bgLoop Looping process logrotate started
02/28/07-12:47:11 PST 02/28/07-20:47:11 GMT 856730845 IDLE setFTPandHTTPinfo (::FTPURL 'ftp://198.129.208.245') (::FTPDIR '') (::HTTPURL 'http://198.129.208.245/ldas_outgoing/jobs') (::HTTPDIR '/ldas_outgoing/jobs') (::GRIDFTPURL 'gridftp:/export/grid/ldas') (::GRIDFTPDIR '/export/grid/ldas') (::LDAS_GATEWAY 'ldas 198.129.208.245') (::LDAS_SYSTEM 'ldas-wa') (::RUNCODE 'LDAS-WA')
02/28/07-12:47:13 PST 02/28/07-20:47:13 GMT 856730847 IDLE setFTPandHTTPinfo (::FTPURL 'ftp://198.129.208.245') (::FTPDIR '') (::HTTPURL 'http://198.129.208.245/ldas_outgoing/jobs') (::HTTPDIR '/ldas_outgoing/jobs') (::GRIDFTPURL 'gridftp:/export/grid/ldas') (::GRIDFTPDIR '/export/grid/ldas') (::LDAS_GATEWAY 'ldas 198.129.208.245') (::LDAS_SYSTEM 'ldas-wa') (::RUNCODE 'LDAS-WA')
02/28/07-12:47:15 PST 02/28/07-20:47:15 GMT 856730849 STARTUP mpi::killAllMpirun cleaning up for user ldas
02/28/07-12:47:16 PST 02/28/07-20:47:16 GMT 856730850 STARTUP mpi::killAllMpirun ran kill 10 times in 0.939 seconds
02/28/07-12:47:16 PST 02/28/07-20:47:16 GMT 856730850 STARTUP mpi::prestartLamds running lamboot for user search01
02/28/07-12:47:17 PST 02/28/07-20:47:17 GMT 856730851 STARTUP mpi::prestartLamds running lamboot for user search02
pehrens@ligo.caltech.edu, gmendell@ligo-wa.caltech.edu, bjohnson@ligo-wa.caltech.edu 856738051 STARTUP mpi::prestartLamds Subject: LDAS-WA mpiAPI lamboot error!; Body: lam::boot: lam::pushSchema: search02@beowulf: Permission denied (publickey,gssapi-with-mic,password). lost connection
02/28/07-14:47:17 PST 02/28/07-22:47:17 GMT 856738051 STARTUP mpi::killAllMpirun {ldas@beowulf:mpirun: child process exited abnormally} {ldas@beowulf:wrapperAPI: child process exited abnormally} {ldas@beowulf:lamd: child process exited abnormally}
02/28/07-14:47:17 PST 02/28/07-22:47:17 GMT 856738051 IDLE mpi::updateCmonNodelist updated ::beowulfNodes in cntlmonAPI to 'beowulf beowulf beowulf beowulf beowulf beowulf beowulf beowulf beowulf'