Home - dmwm/WMCore GitHub Wiki
$${\color{red}\huge These \space pages \space are \space obsolete.\space We\space have\space migrated\space WMCore\space documentation\space to\space CERN\space Gitlab.}$$
New documentation URL: https://cms-wmcore.docs.cern.ch/
Introduction
This wiki is meant to provide a description as complete as possible of the different applications of the Workflow Management system (ReqMgr2, WMStats, GlobalWorkQueue, WMAgent) on what they do and how they do it. (WMCore library is also used for other applications, DBS,CRABServer CRABClient and WmAgentScript
If you are a developer, this should help you get on track to start making changes to the code but you must also checkout the developers tutorials for DMWM. Finally, there is no better documentation than the code itself if you want to work with the project, you'll likely spend many hours just going through it even after reading this.
If you are a non-developer, this will give you basic overview on WMAgent System.
General information for both users (data operators) and developers
Workflow Management System Overview
The following pages contain detailed information about the different projects and how they work.
MicroServices documentation
Microservices are also key elements in the Workload Management system. Documentation for the existent microservices follows.
- MicroService Transferor
- MicroService Monitor
- MicroService Output
- MicroService Unmerged
- MicroService RuleCleaner
- MicroService Pileup
Some important logics and concept on the system.
This section contains special pages which describe a particular phenomenon in the system, usually the information here can be inferred after reading the project pages completely.
Installation (Deployment):
Deploy agent (assume condor is already installed and connected to global pool. For Oracle backend, oracle account should already set up.
Usage and APIs
- Injecting, assigning and validating test requests
- Store result requests
- ReqMgr2 APIs
- WMStats APIs
- ReqMgr2 MicroService APIs
Operations-related documents
Monitoring documentation
- WMAgent monitoring documentation
- Global WorkQueue monitoring documentation
- ReqMgr2 monitoring documentation
- Prometheus AlertManager wrapper APIs
- Monitoring of CMSWEB services with Prometheus AlertManager
Data Management related documentation
Trouble shooting
Information for Developers
Development Tutorial
- New WMCore member setup
- WMCore developer responsibilities
- Setting up WMCore developing environment using github
- Useful git commands
Test setup
- How to set up unittest env (linux)
- All about Jenkins and our automated tests
- How to patch production/test machine (apply patches to WMAgent)
- Injecting, assigning and validating test requests
- Debugging, log locations
- Give access to other users to your vm
Issue tracking and release procedure
CMS Development Tools and Support
Some Implementation details
- WMBS (db backend for WMAgent)
- CouchDB Diagram
- Workqueue replication procedure
- JobSplitting Argorithms
On going developments
DMWM deployment transition (packaging with PyPi and using docker)
- WMCore in Kubernetes
- Improving WMAgent deployment (temporary transitional plan)
- DWWM packaging with PyPi
- WMCore with Docker
Python3 Transition
PhEDEx to Rucio Transition
WMAgent Refactoring (dividing jobs in work units)
Unified Porting
Go transition
Miscellaneous
Deprecated and/or outdated documents
Just keeping them for the information and reference purposed or need to be updated and put in a proper category.