Registration: Fully compliant (required) - openpmix/openpmix GitHub Wiki

The following list of information required for full compliance is taken from v4.2 of the PMIx Standard - please note that the list is subject to change (primarily in the form of additions) as the Standard continues to evolve.

Session-realm information

  • PMIX_UNIV_SIZE: Maximum number of processes that can be simultaneously executing in a session. Typically, this is a constraint imposed by a scheduler or by user settings in a hostfile or other resource description.

Job-realm Information

  • PMIX_SERVER_NSPACE: Name of the namespace to use for this PMIx server. Identifies the namespace of the PMIx server itself

  • PMIX_SERVER_RANK: Rank of this PMIx server. Identifies the rank of the PMIx server itself

  • PMIX_NSPACE: Namespace of the job - may be a numerical value expressed as a string, but is often an alphanumeric string carrying information solely of use to the system. Required to be unique within the scope of the host environment. Identifies the namespace of the job being registered.

  • PMIX_JOBID: Job identifier assigned by the scheduler to the specified job - may be identical to the namespace, but is often a numerical value expressed as a string (e.g., "12345.3").

  • PMIX_JOB_SIZE: Total number of processes in the specified job across all contained applications. Note that this value can be different from PMIX_MAX_PROCS. For example, users may choose to subdivide an allocation (running several jobs in parallel within it), and dynamic programming models may support adding and removing processes from a running job on-the-fly. In the latter case, PMIx events may be used to notify processes within the job that the job size has changed.

  • PMIX_MAX_PROCS: Maximum number of processes that can be executed by this job. Typically, this is a constraint imposed by a scheduler or by user settings in a hostfile or other resource description.

  • PMIX_NODE_MAP: Regular expression of nodes currently hosting processes from this job

  • PMIX_PROC_MAP: Regular expression describing processes from this job on each node

  • PMIX_CREDENTIAL: Blob containing security key for communications across fabrics requiring secure messaging.

Application-realm Information

If more than one application is included in the namespace, then the host environment is also required to supply data consisting of the following items for each application in the job:

  • PMIX_APPNUM: The application number within the job

  • PMIX_APP_SIZE: Number of processes in the specified application, regardless of their execution state - i.e., this number may include processes that either failed to start or have already terminated

  • PMIX_MAX_PROCS: Maximum number of processes that can be executed for this application.Typically, this is a constraint imposed by a scheduler or by user settings in a hostfile or other resource description.

  • PMIX_APPLDR: Lowest rank in the specified application.

  • PMIX_WDIR: Working directory for spawned processes. This attribute is required for all registrations, but may be provided as an individual pmix_info_t entry if only one application is included in the namespace.

  • PMIX_APP_ARGV: Consolidated argv passed to the spawn command for the given application (e.g., "./myapp arg1 arg2 arg3"). This attribute is required for all registrations, but may be provided as an individual pmix_info_t entry if only one application is included in the namespace.

Node-realm Information

Node-realm information may be passed as individual pmix_info_t entries if only one node will host processes from the job being registered, or as part of a pmix_data_array_t using the PMIX_NODE_INFO_ARRAY attribute when multiple nodes are involved in the job. The list of data referenced in this way shall include:

  • PMIX_NODEID: Node identifier expressed as the node’s index (beginning at zero) in an array of nodes within the active session. The value must be unique and directly correlate to the PMIX_HOSTNAME of the node

  • PMIX_HOSTNAME: Name of the host, as returned by the gethostname utility or its equivalent. As this information is not related to the namespace, it can be passed using the PMIx_server_register_resources API.

  • PMIX_HOSTNAME_ALIASES: Comma-delimited list of names by which the target node is known. As this information is not related to the namespace, it is best passed using the PMIx_server_register_resources API.

  • PMIX_LOCAL_SIZE: Number of processes in the specified job or application realm on the node.

  • PMIX_NODE_SIZE: Number of processes across all jobs that are executing upon the node

  • PMIX_LOCALLDR: Lowest rank within the specified job on the node

  • PMIX_LOCAL_PEERS: Comma-delimited list of ranks that are executing on the node within the specified namespace

  • PMIX_NODE_OVERSUBSCRIBED: True if the number of processes from this job on the node exceeds the number of slots allocated to it

In addition, the server must provide the following information for its own node:

  • PMIX_TMPDIR: Full path to the top-level temporary directory assigned to the session

  • PMIX_NSDIR: Full path to the temporary directory assigned to the specified job, under PMIX_TMPDIR.

  • PMIX_LOCAL_PROCS: Array of pmix_proc_t of all processes executing on the local node

  • PMIX_TOPOLOGY_SIGNATURE: Topology signature string.

Proc-realm Information

Process-realm information shall include the following data for each process in the job, passed as a pmix_data_array_t using the PMIX_PROC_INFO_ARRAY attribute, with the process rank being the first element of that array:

  • PMIX_RANK: Process rank within the job, starting from zero

  • PMIX_APPNUM: The application number within the job in which the specified process is a member. This attribute may be omitted if only one application is present in the namespace.

  • PMIX_APP_RANK: Rank of the specified process within its application. This attribute may be omitted if only one application is present in the namespace.

  • PMIX_GLOBAL_RANK: Rank of the specified process spanning across all jobs in this session, starting with zero. Note that no ordering of the jobs is implied when computing this value. As jobs can start and end at random times, this is defined as a continually growing number - i.e., it is not dynamically adjusted as individual jobs and processes are started or terminated.

  • PMIX_LOCAL_RANK: Rank of the specified process on its node - refers to the numerical location (starting from zero) of the process on its node when counting only those processes from the same job that share the node, ordered by their overall rank within that job.

  • PMIX_NODE_RANK: Rank of the specified process on its node spanning all jobs- refers to the numerical location (starting from zero) of the process on its node when counting all processes (regardless of job) that share the node, ordered by their overall rank within the job. The value represents a snapshot in time when the specified process was started on its node and is not dynamically adjusted as processes from other jobs are started or terminated on the node.

  • PMIX_NODEID: Node identifier indicating where the proc is executing, expressed as the node’s index (beginning at zero) in an array of nodes within the active session.

  • PMIX_REINCARNATION: Number of times this process has been re-instantiated - i.e, a value of zero indicates that the process has never been restarted.

  • PMIX_SPAWNED: true if this process resulted from a call to PMIx_Spawn. Lack of inclusion (i.e., a return status of PMIX_ERR_NOT_FOUND) corresponds to a value of false for this attribute.

In addition, the following information must be provided for processes that are local to this server:

  • PMIX_LOCALITY_STRING: String describing the process’ bound location. The string is prefixed by the implementation that created it (e.g., "hwloc") followed by a colon. The remainder of the string represents the corresponding locality as expressed by the underlying implementation. The PMIx_server_generate_locality_string API can be used to create the string in the correct format.

  • PMIX_PROCDIR: Full path to the subdirectory under PMIX_NSDIR assigned to the specified process.

  • PMIX_PACKAGE_RANK: Rank of the specified process on the package where this process resides - refers to the numerical location (starting from zero) of the process on its package when counting only those processes from the same job that share the package, ordered by their overall rank within that job. Note that processes that are not bound to PUs within a single specific package cannot have a package rank.

Return..............................................................................Optional values