Python_Packages - RicoJia/notes GitHub Wiki

========================================================================

========================================================================

1. package structure

   • Files included automatically: setup.py, README.md
     • setup.py contains metadata needed during installation time.
     • It can even read stuff from README.md and changelog, but they need to be included. That's where MANIFEST.in comes into play.
   • sdist (source distribution) and wheel: wheel is faster cuz it's prebuilt distribution. If both are present, wheel is preferred. doc
   • python interpreter:
     • .pyc: python compiles .py into byte code, then store them in .pyc. Platform dependent
     • python_runtime takes in byte code, then python_virtual_machine (PVM) will run the runtime binaries
     • because of this process, Python is slower than C++. But pyc files are saved, so subsequent runs will be faster.

2. virtual environment: you can install pkgs in a separate space. https://packaging.python.org/tutorials/installing-packages/

3. makefile in python: avoid a bunch of bash scripts to get stuff working on different machines. source

   • creating a virtual env is kinda hassle / make packages. run make.
   • makefile uses dependency graph (otherwise bash scripts may have complex dependency). Also it uses timestamp.

     target: dependency 1    #target is a file to be built, dependency are files needed
       shell_command_1_for_building
       shell_command_2_for_building

   • example: copy a file:

     dist/config.yaml: src/config.yaml
         cp -f src/config.yaml dist/config.yaml

4. ROS - setup.py: export a python module to other packages in the same workspace. see code

   • example for structure: https://github.com/sea-bass/turtlebot3_behavior_demos/tree/main/tb3_autonomy
   • Need to run catkin_make
   • User need to use rosrun

5. **Powerful Use: ** import a class using its name all classes included in init will be visible.

   module = importlib.import_module('diligent.plannable_steps')
   getattr(module, classname)
   

========================================================================

========================================================================

1. pip

   • --no-dependencies means to force install and ignore all dependencies
   • -e edible mode, to install locally, linking to the package.
   • nice read
   • uninstall -y -y is all yes
   • pip install -U will be installed outside of virtual env

2. local install will create a link in /usr/... dist package.

3. upload - python setup.py sdist bdist_wheel creates wheel and source dist both. - python3 setup.py sdist upload -r arpypi uploads to arpypi

4. Avoid putting in passwords, usernames in private repos: ~/.netrc

   pip3 uninstall pkg
   echo "machine pypi.XXX.com
       login name
       password SOMEPASS_WORD
   pip3 install -i https://pypi.build.accelrobotics.com/simple/ ocean-projector3==2.0.0

5. Name of file: do not name your file the same name as module, else python will do circular import!

========================================================================

========================================================================

1. entry-point can be used to generate a script that invokes a python function, defined in __main__.py

   from setuptools import setup, find_namespace_packages
   setup(name="production_tracker",
         version="2.2.0",
         description="Runtime tracker node for AR stores",
         author="AR",
         packages=find_namespace_packages(),
         install_requires=[
             "production-time-sampling",
             "production-video"
         ],
         entry_points={
             "console_scripts": [
                 "tracker=production.tracker.__main__:execute",
                 "tracker-playback=production.tracker.playback:execute",
             ],
         },
         zip_safe=False
         )

2. Python impport has absolute and relative imports

   A.py
   sub_dir/: 
       B.py
       C.py
   

   • By default, python uses absolute import. i.e., If in A: import B, B has import C, there will be an error, because absolute import will think C is in the same directory as A.
   • However, You can make B from . import C, or import .C, because this is relative import
   • What if A has from .sub_dir import C? it's fine but needs to run as python3 -m A.py. Because relative import doesn't work with script being run as __main__. '' ========================================================================

========================================================================

1. upgrade Python 3.5 to python 3.7:

   wget https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tgz
   tar -xf Python-3.7.0.tgz
   apt-get update
   apt-get install -y build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline6-dev libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev libffi-dev
   cd Python-3.7.0
   ./configure --enable-optimizations
   make
   sudo make install
   
   # creates a soft link, points to another file, like a shortcut in windows
   sudo ln -sf /usr/local/bin/python3.7 /usr/local/bin/python3
   
   sudo reboot
   python3 --version

   • https://samx18.io/blog/2018/09/05/python3_raspberrypi.html
   • https://www.raspberrypi.org/forums/viewtopic.php?t=291310

========================================================================

========================================================================

1. if you have a file that directly import a file without relative import

   import SOME_MODULE
   

   • and you can't change it (e.g., Read-only, or auto-generated code), add the root directory into sys path: sys.path.append(DIR)