Python Package Management - UMKCgeg/Wiki GitHub Wiki

Python package managment is one area where Python is not very good. There is a hodgepodge of ways to accomplish the same thing, and many of the solutions aren't all that great. This page won't cover everything, but some of the best ways to set up a Python installation.

Simplest solution

pip is a tool included with Python that allows you to easily install other packages. Simple call pip install the_package to install packages. This might throw an error, so call sudo pip install the_package. This is not recommended, since this installs all packages globally. Different projects may require different packages that can conflict in their dependencies. Installing everything globally, while good enough for simple use, may cause some problems down the road. Also, some packages like SciPy can be hard to install otherwise.

Note that pip can also install packages from GitHub, no matter which solution here you end up using. To do that, follow this recipe:

pip install git+git://github.com/user/repo.git@branch

For example, if you want to install EzGal, call pip install git+git://github.com/dpgettings/ezgal.git@master . You'll almost always want the master branch.

Virtual Environments

Virtual environments solve the problem above of conflicting Python installations. They allow you to have multiple sandboxed Python environments. Things can still be difficult to install, but having virtual enviroments is the best practice to ensure that you don't have any conflicts.

Virtual environments are made with the Python package virtualenv. To install it, simply type pip install virtualenv, with sudo if that throws an error.

To create a virtual environment, call virtualenv name_of_new_environment from the command line. This will install a few things in the folder ./name_of_new_environment. That path is relative to wherever you were when you created the environment. We then need to activate the environment. To do this, call source name_of_new_environment/bin/activate.csh (assuming you're on a C shell, like the GEG machines are). You'll then see the name of the environment to the left of the command line prompt. You can then install packages with pip, and they will be local to your environment, not global.

Conda

Conda is the easiest and probably best way to install everything. It makes things very easy. The main idea behind Conda is having environments, which are like independent Python installations that are easy to modify. You can have multiple environments, with different packages installed in each or even different versions of Python.

One major downside: This isn't fully compatible with CSH.

If you're going to use Conda, the simplest way to do things is just to download the Anaconda Package. It contains everything you need to get started. It does have some extra stuff like a bunch of R packages that you probably don't need.

The preferred way to get going is to install Miniconda, which just contains the package manager portion.

No matter how you install it, you are now ready to get going. This page has some commands that will get you started, but you can also follow the 30 minute test drive, which is fantastic and really gets you off on the right foot.

The first thing we need to do is create an enviroment. This can be done by calling the following command from the command line

conda create --name test

If you aren't using CSH or TCSH, tehn you can type

source activate test

to enter the environment. You'll see the name of your environment in parenthesis to the left of your command line prompt. You can then install packages into it. This is done by

conda install scipy

Most packages you will want are available via a conda install, but others will not. This is because Conda takes the complicated packages (like SciPy), and puts them into a binary that is easy to install. Installing these packages otherwise can require various things such as a working Fortran compiler or various other nonsense that can be a total pain to get working. Conda takes care of all of this. This is the primary advantage of Conda. Things are so much easier to install than they would be otherwise.

For an astronomy specific installation, you'll definitely want to install the following things (all with Conda):

  • scipy
  • astropy
  • matplotlib

Some others are also great:

  • ipython
  • jupyter

For packages that aren't availble via conda install, you can simply pip install them.

Using Conda on CSH

If you are using a C Shell variant, you won't be able to enter different environments. You should, however, still be able to install everything into the root and have it all work. You won't have different environments, but you will have the ease of installing things with Conda.

Your own packages

If you develop your own packages that you would like to import, or have downloaded someone else's code (without using pip or Conda) that you want to import, you'll need to put them in a directory that Python knows about. To do this, you'll need to edit your shell configuration file. You'll need to add lines like the ones below (for tcsh).

setenv PYTHONPATH /path/to/your/code
setenv PYTHONPATH ${PYTHONPATH}:/other/paths/code/is

The first line sets the variable, the second appends to it. Wherever your code that you want to import is, add those files to thePYTHONPATH variable. After restarting your shell, Python will be able to find the code located there.