Deugging, log locations - dmwm/WMCore GitHub Wiki

Once test is set up you can monitor through wmstats for the progress of jobs. If something goes long, check the logs for each component or application

Log files:

  • for reqmgr2, workqueue, reqmon (wmstats) : log files are located under "/data/srv/logs/", Each thread has its own log file and gets rotated everyday, Old logs are zipped.

    bash-4.1$ ls /data/srv/logs/reqmgr2/
    auxCacheUpdateTasks-20181210.log  heartbeatMonitor-20181211.log  reqmgr2-20181210.log
    auxCacheUpdateTasks-20181211.log  heartbeatMonitor-20181212.log  reqmgr2-20181211.log
    auxCacheUpdateTasks-20181212.log  old-logs-20181202-0007.zip	 reqmgr2-20181212.log
    couchCleanup-20181205.log	  parentageFixTask-20181210.log  statusChangeTasks-20181210.log
    couchCleanup-20181212.log	  parentageFixTask-20181211.log  statusChangeTasks-20181211.log
    heartbeatMonitor-20181210.log	  parentageFixTask-20181212.log  statusChangeTasks-20181212.log
    
  • For WMAgent: Each component has "ComponentLog" log file under /data/srv/wmagent/current/install/wmagent

    > /data/srv/wmagent/current/install/wmagent/JobAccountant $ ls
      ComponentLog  Daemon.xml.BAK.10-39-2018  Daemon.xml.BAK.29-39-2018
    

    Logging level can be controlled by changing log level for each component in config file: /data/srv/wmagent/current/config/wmagent/config.py - component has to be restarted ( to apply the new log level. (INFO, DEBUG)

    config.WorkQueueManager.logLevel = 'INFO'