07. Managing Embedded Python Dependencies - intersystems/ipm GitHub Wiki
To include Python dependencies in an IPM package, a developer can list all dependencies in a requirements.txt
file at the root level of the package (in the same directory as module.xml
). The format of requirements.txt
should follow PEP 508.
When installing an IPM package, IPM will automatically check for the existence of requirements.txt
at the package root and, if found, install all Python packages listed in it using pip
.
In IRIS 2024.2 and later, certain platforms (such as Windows and Linux) support flexible Python runtime, or, as we call it, bring-your-own-python (BYOP). Users can dynamically load a Python dynamic library (.dll, .so) of their own choosing and make use of various syntax features and optimizations in different Python versions.
If you are using this feature on Windows, irispip.exe
is no longer shipped with IRIS. This causes IPM 0.7.2 and earlier to be unable to install Python dependencies. It is recommended that you upgrade to IPM 0.7.3 or higher, then manually specify the path to either 1) a Python executable or 2) a pip executable.
- If you want to specify the path of a Python executable, you need to run
zpm "config set UseStandalonePip 0"
andzpm "config set PipCaller </absolute/path/to/python/executable>"
. - If you want to specify the path of a pip executable, you need to run
zpm "config set UseStandalonePip 1"
andzpm "config set PipCaller </absolute/path/to/pip/executable>"
. - After doing either of the above, you can verify the configuration by running
zpm "config list"
.
Up until 0.7.2, IPM uses irispip.exe
on Windows and whichever pip
is found first in $PATH on Unix systems.
Starting from 0.7.3, the process is more customizable.
- If
PipCaller
is set, it will be used for callingpip install ...
orpython -m pip install ...
, depending on the value ofUseStandalonePip
. - If
PipCaller
is missing (by default), IPM detects the proper pip caller in the following order:- Where available (typically 2024.2+) - Use PythonRuntimeLibrary in iris.cpf to find the directory containing the Python executable.
- On Windows only, try to find /bin/irispip.exe.
- Unless
UseStandalonePip
is explicitly set to 1, try to find whichever python3/python is first available in $PATH. - Unless
UseStandalonePip
is explicitly set to 0, try to find whichever pip3/pip is first available in $PATH.
Starting from 0.7.3, IPM automatically detects the version of your Python runtime when installing Python dependencies. This means you can specify any pip/Python path using config set PipCaller <pip/python>
, which doesn't have to be the same one that's specified as dynamic Python at the IRIS level.
Certain dependencies, such as numpy, require compiling Python ABI-dependent C code and are not reusable across different Python versions. Additionally, both irispip.exe
and IPM
always install Python packages to the same directory regardless of Python versions. Therefore, if you install numpy
while using Python 3.10 and then switch to Python 3.12, import numpy
will fail due to incompatible Python ABI.
This inherently cannot be addressed by IPM itself without some heavy hacking around sys.path
. The recommended and surefire way to fix it is to uninstall the Python package completely and reinstall it in the new environment.