Global Status Monitor

It is the centralized monitor about the statuses of various online components in E1039. It shows the monitor summary at https://e906-gat1.fnal.gov/SpinQuestDAQStatus/.

For Shift Crew

• No shift-crew task is defined yet, but just in general...
• A false error or warning sometimes appear in a spill or two. Such can be ignored.
• All red messages (errors) must be resolved immediately.
• All yellow messages (warning) must be recorded to elog.

System Structure

• The main script is /data2/e1039/daq/slowcontrols/status_monitor/status_monitor.py.
  • It must be executed by e1039daq on e1039scrun.
  • It will be auto-launched by /data2/e1039/daq/slowcontrols/scripts/slowControlMasterLoop.sh.
  • For now it is sometimes manually launched by nohup /data2/e1039/daq/slowcontrols/status_monitor/status_monitor.py &>/dev/null &.
• The monitoring results (= HTML files) are outputted to /data2/e1039_data/status_monitor/html.
• e906-gat1:/var/www/html/SpinQuestDAQStatus is a symbolic link to the output directory.

Components

• Inherited from E906
  • Status monitor script
  • Disk space
  • Main DAQ
  • Spill counter script
• Will be inherited from E906
  • FMag, KMag
  • Beam DAQ
  • Scaler DAQ
  • Scaler-DAQ decoder
• Will not be inherited from E906
  • Slowcontrol script ... it should be able to be monitored by its own viewer
• Will be newly created
  • Target
  • Others??

To-Do Items

• Decide which components should be added further.
• Kenichi doesn't prefer using /data2 because the monitoring output is temporal and frequent, which causes extra I/O. A web server on e1039scrun should be better.