Notes on using py2exe and py2app

py2exe and py2app are used to create application bundles or installers. This page is used to keep track of any issues that have to be dealt with in order to successfully use these two distutils extensions.

Problems dealing with plugins

The plugin architecture of peppy causes problems with py2exe and py2app because neither of those programs seems to deal with python eggs. This means that the eggs must be unpacked into some directory so that py2exe/py2app can locate the sources before making the native executables.

There is a lot of makefile and bash scripting to do this, and it's a hack to say the least. I need to find a better solution, but the only un-hackish solution will be when py2exe supports eggs.

Missing dependencies

If it would have worked, the best solution would have been to leave the plugins packaged as eggs and let the native application pick them up by scanning a directory. Unfortunately, there are two problems:

1. When creating eggs, setuptools doesn't include any dependencies within the egg. The only modules within the egg are the modules that make up the plugin, nothing else. The setuptools code wasn't designed for native bundles, and relies on all the dependencies being resolvable through the use of easy_install. This works fine in a normal python distribution, but not when you're trying to deliver a standalone application.
2. py2exe scans the application source code to find dependencies and includes only the necessary dependencies in library.zip. py2exe doesn't know how to scan eggs, so the egg dependencies are missing from the application.

For example, a plugin can depend on parts of the python system library that the main application doesn't:

            +--> wx
            |
peppy ------+--> wx.stc
            |
            +--> wx.grid


            +--> wx
            |
egg #1  ----+--> wx.gizmos
            |
            +--> xml.dom.minidom

If the egg is distributed separately from the application, the wx.gizmos and xml.dom.minidom dependencies won't be available because py2exe didn't include them when generating the library.zip that the exe uses.

But, it's not really py2exe's fault that it only sees wx, wx.stc, and wx.grid as dependencies, because how is it supposed to know what plugins will be added by the user at some later time? That's sort of the point of plugins, to be able to extend the functionality of the main program.

Workaround

The only way I've been able to figure a way around this is to unpack the eggs before py2exe is run. This means that plugins really aren't available in the py2exe bundled app -- everything that was a plugin is distributed as part of the main source.

In the peppy source code, the bash script {{{make-dist.sh}}} generates a distribution directory that can be used to run py2exe. It does the following:

1. make a normal source distribution using the Makefile target {{{make distdir}}} to create the directory {{{peppy-0.x.x}}}
2. creates eggs from the plugins in the toplevel {{{plugins}}} directory ''(note: not the {{{peppy/plugins}}} directory -- those are left over from the old plugin system and will be moved into the new plugin system at some point)''
3. unpacks the eggs into {{{peppy-0.x.x/eggs}}}
4. runs the python script {{{make-py2exe-plugin-list.py}}} to create a special file {{{peppy-0.x.x/peppy/py2exe_plugins.py}}} which contains a bunch of import statements that forces py2exe to include the source code for the plugins (and therefore, the dependencies of the plugins)

In ur egg, findin ur dependencies

What I'd like is some way to including any missing dependencies of the egg into the egg itself. This would require that setuptools be able to reference the py2exe library.zip to see what's in there, and also for setuptools to include the appropriate dependencies that it currently doesn't.

I wonder if it could be faked by creating a py2exe bundle out of the egg, using that library.zip to compare with the peppy library.zip, and adding the difference between the two sets to the egg?

Would the exe pick up the new files? What would happen inside the wx package in the wx.gizmos example above? No way to find out other than trying it, I suppose.