All in one test - dmwm/WMCore GitHub Wiki
Brief Summary (for Linux) : this set up is not complete so submitting request might not work. Need to improve
- This is the set up for quick integration test. This set up is NOT providing the same environment as real system.
- It doesn't contain apatch frontend, no authentication.
- reqmgr and workqueue is set as component rather than the cherrypy daemon.
- To make wmstats working there will be manual changes required (not added in this tutorial)
- don't use this set up if you want to test cmsweb application (reqmgr, workqueue, reqmon), use this only for agent side integration test.
- This tutorial assumes condor is installed in the testing machine and proper cert is available to submit the jobs
Setting up all in one agent (wmagent, reqmgr, reqmon, workqueue)
1. setting up the directory structure.
(follows wmagent deployment directory structure - change to your convenient location )
$ export ADMIN_DIR=/data/admin/wmagent
$ export DEPLOY_DIR=/data/srv/wmagent
$ mkdir -p $ADMIN_DIR
# for env script.
2. get deployment script
$ cd /data
$ git clone https://github.com/dmwm/deployment.git
$ git checkout branch/tag
(optional if need specific tag or branch)
3. Deploy wmagent rpm (example shows 1.0.0.patch4 version)
$ cd /data/deployment
$ ./Deploy -r comp=comp.pre -s prep -A slc6_amd64_gcc481 -t v1.0.0.patch4 $DEPLOY_DIR [email protected]
$ ./Deploy -r comp=comp.pre -s sw -A slc6_amd64_gcc481 -t v1.0.0.patch4 $DEPLOY_DIR [email protected]
$ ./Deploy -r comp=comp.pre -s post -A slc6_amd64_gcc481 -t v1.0.0.patch4 $DEPLOY_DIR [email protected]
Setting up environment
1. create secrete file (set db password and port)
$ vi $ADMIN_DIR/WMAgent.secrets
type following information (change user/password and port - don't use port 9999,
it is used by WMBSService component - need to be changed to getting in from secrete file)
MYSQL_USER=devrecoagent
MYSQL_PASS=passwd
COUCH_USER=devrecoagent
COUCH_PASS=passwd
COUCH_PORT=5984
COUCH_HOST=0.0.0.0 (WARNING: this has to be ip address not hostname)
GLOBAL_WORKQUEUE_URL=http://devrecoagent:[email protected]:5984/globalwq
GLOBAL_WORKQUEUE_DBNAME=globalwq
LOCAL_WORKQUEUE_DBNAME=localwq
WORKLOAD_SUMMARY_URL=http://devrecoagent:[email protected]:5984/workloadsummary
REQMGR_HOSTNAME=131.225.206.104
REQMGR_PORT=8687
WMSTATS_URL=http://devrecoagent:[email protected]:5984/wmstats
COUCH_CERT_FILE=/data/certs/servicecert.pem
COUCH_KEY_FILE=/data/certs/servicekey.pem
REQMGR_URL=http://cmssrv95.fnal.gov:8687/reqmgr/rest
REQUESTCOUCH_URL=http://cmssrv95.fnal.gov:5984/reqmgr_workload_cache
ACDC_URL=http://devrecoagent:[email protected]:5984/acdcserver
DBS3_URL=https://cmsweb-testbed.cern.ch/dbs/int/global/DBSWriter
PHEDEX_URL=https://cmsweb-testbed.cern.ch/phedex/datasvc/json/prod/
DQM_URL=https://cmsweb-testbed.cern.ch/dqm/offline/
DASHBOARD_URL==http://dashb-ssb.cern.ch/dashboard
REQMGR2_URL=http://cmssrv95.fnal.gov:8688/reqmgr2
CENTRAL_LOGDB_URL=https://cmsweb-testbed.cern.ch/couchdb/wmstats_logdb
for Oracle version of unittest, env.sh file need to contain oracle variables instead of mysql
ORACLE_USER=(oracle user name)
ORACLE_PASS=(oracle password)
ORACLE_TNS=(tns name)
COUCH_USER=devrecoagent
COUCH_PASS=passwd
COUCH_PORT=5984
COUCH_HOST=0.0.0.0 (WARNING: this has to be ip address not hostname)
GLOBAL_WORKQUEUE_URL=http://devrecoagent:[email protected]:5984/globalwq
GLOBAL_WORKQUEUE_DBNAME=globalwq
LOCAL_WORKQUEUE_DBNAME=localwq
WORKLOAD_SUMMARY_URL=http://devrecoagent:[email protected]:5984/workloadsummary
REQMGR_HOSTNAME=131.225.206.104
REQMGR_PORT=8687
WMSTATS_URL=http://devrecoagent:[email protected]:5984/wmstats
COUCH_CERT_FILE=/data/certs/servicecert.pem
COUCH_KEY_FILE=/data/certs/servicekey.pem
REQMGR_URL=http://cmssrv95.fnal.gov:8687/reqmgr/rest
REQUESTCOUCH_URL=http://cmssrv95.fnal.gov:5984/reqmgr_workload_cache
ACDC_URL=http://devrecoagent:[email protected]:5984/acdcserver
DBS3_URL=https://cmsweb-testbed.cern.ch/dbs/int/global/DBSWriter
PHEDEX_URL=https://cmsweb-testbed.cern.ch/phedex/datasvc/json/prod/
DQM_URL=https://cmsweb-testbed.cern.ch/dqm/offline/
DASHBOARD_URL==http://dashb-ssb.cern.ch/dashboard
REQMGR2_URL=http://cmssrv95.fnal.gov:8688/reqmgr2
CENTRAL_LOGDB_URL=https://cmsweb-testbed.cern.ch/couchdb/wmstats_logdb
2. set the environment script
servicecert and servicekey can be your own cert. but make sure key is not encrypted.
$ vi $ADMIN_DIR/env.sh
#change this block according to your directory setting
export DEPLOY_DIR=/data/srv/wmagent
# Necessary variables
export WMAGENT_SECRETS_LOCATION=/data/admin/wmagent/WMAgent.secrets
export X509_HOST_CERT=/data/certs/servicecert.pem
export X509_HOST_KEY=/data/certs/servicekey.pem
export X509_USER_CERT=/data/certs/servicecert.pem
export X509_USER_KEY=/data/certs/servicekey.pem
export X509_USER_PROXY=/data/certs/myproxy.pem #(need to set proxy for condor)
#Very convenient variables
export install=$DEPLOY_DIR/current/install/wmagent
export config=$DEPLOY_DIR/current/config/wmagent
export manage=$config/manage
Start components and setting up other thing for request submission.
1. start server and create the db
$ source $ADMIN_DIR/env.sh
$ $manage activate-agent
$ $manage activate-reqmgr
$ $manage activate-workqueue
$ $manage start-services
$ $manage init-agent
if you change config file manually need to remove init files and start agent again
i.e rm $install/.init; rm $install/../workqueue/.init; rm $install/../reqmgr/.init
Setting up threshold so you can submit the job
$ $manage execute-agent wmagent-resource-control --add-all-sites --plugin=PyCondorPlugin --pending-slots=10 --running-slots=10
Or set up one T1 or T2s
$ $manage execute-agent wmagent-resource-control --add-T1s --plugin=PyCondorPlugin --pending-slots=50 --running-slots=50
$ $manage execute-agent wmagent-resource-control --add-T2s --plugin=PyCondorPlugin --pending-slots=50 --running-slots=50
1.1 step to modify agent
To be able to abort workflow
in reqmgr db reqmgr_assigned_prodmgr table workqueue url should include the passwd.
update reqmgr_assigned_prodmgr set prodmgr_id='http://devrecoagent:[email protected]:5984';
TODO: fix this in secure way.
2. start agent
$ $manage start-agent
3. check the status (url and port is from secrete file)
agent and db
$ $manage status
request manager
http://cmssrv95.fnal.gov:8687/reqmgr/
couchdb
http://cmssrv95.fnal.gov:5984/_utils/
wmstats (There need some manual changes to make inner link to work)
http://cmssrv95.fnal.gov:5984/wmstats/_design/WMStats/index.html
3. add user, group team (this should be exactly like this unless you want to change config.py)
add 'fbloggs' as user (this is the default user for wmagent test)
http://cmssrv95.fnal.gov:8687/reqmgr/admin/users/
add DATAOPS as group (this should be matched with your request)
http://cmssrv95.fnal.gov:8687/reqmgr/admin/groups/
add DATAOPS group to the user
http://cmssrv95.fnal.gov:8687/reqmgr/admin/user/fbloggs
add cmsdataops as team
http://cmssrv95.fnal.gov:8687/reqmgr/admin/teams/
add cmssw versions (click update from tc button)
http://cmssrv95.fnal.gov:8687/reqmgr/admin/versions/
3. Create the Request and submit to ReqMgr
This doesn't have to be in the same machine (unless the test request manager is under firewall)
Just assume python is installed.
in any directory
$ git clone [email protected]:dmwm/WMCore.git
$ cd WMCore
change parameters on the test. (i.e. MonteCarlo test)
make config cache exist in the config cache db (default config cache db is pointed to the
cmsweb-testbed which should work)
$ vi test/data/ReqMgr/requests/MonteCarlo.json
"Team": "Team--OVERRIDE-ME", should be "Team": "cmsdataops"
"Requestor": fbloggs
other OVERRIDE-ME needs to be over written whatever string you want
"Campaign": "Test_sryu"
"RequestString": "sryu_test"
"AcquisitionEra": "AcquisitionEra-test",
"ProcessingString" : "TestSryu",
"SiteWhitelist": ["T1_US_FNAL"],
"dashboard": "mc"
get right python version
$ source /data/srv/wmagent/current/apps/wmagent/etc/profile.d/init.sh
inject the request
$ python test/data/ReqMgr/reqmgr.py --reqMgrUrl http://cmssrv101.fnal.gov:8687 -f test/data/ReqMgr/requests/MonteCarlo.json -j '{"createRequest" : {"RequestString" : "test_MC_Files","Campaign": "Test_Campaign"}}' --createRequest --assignRequests