JobAccountant - dmwm/WMCore GitHub Wiki

JobAccountant is responsible for the post-processing of a job and file-parentage association.

• It fetches all jobs in "complete" state in WMBS
• and process the job report file (Report.pkl), cross-checking and registering output modules, output files and their metadata information into WMBS (and DBSBuffer).
• unmerged output files are not registered into DBSBuffer, since those are never injected into DBS. Only the WMBS job/output file relationship is stored.
• on the other hand, merged output files have the WMBS job/output relationship and they are also stored in the DBSBuffer tables, recursively looking up for the parent files.

Expanding on the parentage association. Input files are associated to wmbs jobs. AccountantWorker finds all output files produced by a job (child) and it maps all those files to the jobid. When inserting these output files into WMBS, the DAO fetches the job input files (from wmbs_job_assoc) and creates a parent-child bind that’s inserted into wmbs_file_parent table. If the algorithm does not find any merged parents, then the file is inserted into DBSBuffer parentless.