torch.onnx Namespacing - pytorch/pytorch GitHub Wiki

This document lists conventions on namespacing in the torch.onnx module.

Make sure new functions and classes are private to the module by default. New modules should be private to torch.onnx by default.

Only "promote" them to public names when needed, by removing the prefix _

1. We don’t want users to depend on interfaces meant for internal use. This makes it hard for the code and architecture to evolve.
2. We want to quickly understand the visibility and origin of a name when reading the code, both to reduce mental load and to make refactoring easier.
3. Happy type checkers -> happy engineers

1. Import only modules (in most cases).
   • Import only the module instead of classes and functions to (1) keep the namespace clean and provide readers with more context on where its members come from, and (2) prevent circular import errors.
   1. Prevent circular import errors: Programming FAQ — Python 3.10.4 documentation

      Circular imports are fine where both modules use the “import ” form of import. They fail when the 2nd module wants to grab a name out of the first (“from module import name”) and the import is at the top level. That’s because names in the 1st are not yet available, because the first module is busy importing the 2nd.

   2. Clean namespace: For example, readers don’t need to backtrack to see sleep is a function from time, as opposed to a function defined in the file. https://google.github.io/styleguide/pyguide.html#22-imports

      • As another example,

        

        Having the module name “opset9” clearly signals to the reader where the op comes from. This helps reduce errors caused by confusion when different versions of a function are defined in different modules and used together.

2. _module is private to torch.onnx
   • Not for external consumption
   • If a module is inside a folder which has a _ name. It is ok for the module to not be prefixed by _ for simplicity.
     • E.g. torch.onnx._symbolic_opsets.symbolic_opset9 is good.
3. _function is private to its module, this means

   • We should not use it in other modules (in most cases)

   • Example:

     from torch.onnx.symbolic_helper import _unimplemented

     should become something like

     from torch.onnx._symbolic_helper import unimplemented

     for it to be consumed internally.

     Explanation: _symbolic_helper (as an example name; doesn’t mean we should hide what is already exposed) is a private module to torch.onnx so we can import it in other modules. unimplemented is not private to its own module, so we can use it in another module.

   • This provides clarity to readers and consistency in code structure, reinforcing the convention that private functions should not be used outside of their modules.

   • Additionally, a function not prefixed by _ signals that we should create a unit test for it. A function prefixed with _ may be implementation details not worthy of unit tests.

   • It keeps type checkers happy by only using non-private functions in a module.

4. When a name is aliased in torch.onnx.__init__, import them from where they are defined. Do not import the alias for internal use.

   • Why? It can reduce the possibility of loading a partially initialized torch.onnx (which causes errors). It also makes it clear to readers where the names come from.

   • Example:

     In __init__.py we alias

     TensorProtoDataType = _C._onnx.TensorProtoDataType

     In symbolic_helper.py, instead of using torch.onnx.TensorProtoDataType.UINT8 internally, we should use _C_onnx.TensorProtoDataType.UINT8.

   • Alias defined in __init__.py should only be used outside the module (by users).

5. Similar to 4, do not define classes and functions in __init__.py meant for internal use. Define them in internal modules and expose them in __init__.py by aliasing instead. This prevents internal modules from depending on a partially initialized torch.onnx, causing import errors.
6. Expose names in torch.onnx to users by declaring them in __all__ within __init__.py as specified in Public API definition and documentation
   • As opposed to not declaring the __all__ variable in __init__.py
   • This helps tighten the set of public APIs and serves as a safeguard against accidental declaration of public functions.