PID created and release on each call to rocotostat - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki

TerrenceMcGuinness-NOAA: In the rocotostat.py code an Executable object is created called rocotostat and a deep copy is made each time rocoto_statcount is called and rocotostat() is called multiple times with attemp_multiple_times. Is there a new PID instatated for the process and release each time that call is made to the rocotostat executable?

GitHub Copilot: Yes, each time the rocotostat executable is called within the attempt_multiple_times function, a new process is instantiated and a new PID (Process ID) is assigned to that process. When the process completes, the PID is released.

Here's a breakdown of what happens in the rocoto_statcount function:

  1. Deep Copy: A deep copy of the rocotostat object is created to ensure that the original object is not modified.
  2. Add Default Argument: The --all argument is added to the rocotostat command.
  3. Attempt Multiple Times: The rocotostat command is executed up to 4 times with a 120-second interval between attempts if a ProcessError occurs.
  4. Process Output: The output of the rocotostat command is processed to count the occurrences of each status case.

Each call to rocotostat(output=str) within the attempt_multiple_times function results in the creation of a new process. This means that a new PID is assigned for each execution of the rocotostat command.

Here is a simplified example to illustrate this:

import subprocess

def run_command(command):
    result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
    return result.stdout

def attempt_multiple_times(func, attempts, delay, exception):
    for _ in range(attempts):
        try:
            return func()
        except exception:
            time.sleep(delay)
    raise exception

def rocoto_statcount(rocotostat):
    rocotostat = copy.deepcopy(rocotostat)
    rocotostat.add_default_arg('--all')

    rocotostat_output = attempt_multiple_times(lambda: run_command(rocotostat), 4, 120, subprocess.CalledProcessError)
    rocotostat_output = rocotostat_output.splitlines()[1:]
    rocotostat_output = [line.split()[0:4] for line in rocotostat_output]
    rocotostat_output = [line for line in rocotostat_output if len(line) != 1]

    status_cases = ['SUCCEEDED', 'FAIL', 'DEAD', 'RUNNING', 'SUBMITTING', 'QUEUED']

    rocoto_status = {}
    status_counts = Counter(case for sublist in rocotostat_output for case in sublist)
    for case in status_cases:
        rocoto_status[case] = status_counts[case]

    return rocoto_status

In this example:

  • The run_command function uses subprocess.run to execute the command, which creates a new process each time it is called.
  • The attempt_multiple_times function attempts to run the command multiple times if an exception occurs.

Each call to run_command(rocotostat) results in a new process with a new PID. When the process completes, the PID is released.