Table of Contents Virtual Filesystem Implementations itools wxWidgets PycURL fish WebDAV ftp sftp/scp URL Nesting Schemes Apache Commons Conclusion

The new itools vfs uses GIO, a heavyweight compiled library that requires daemon support on the host computer. I didn't want to force that dependency when using peppy for virtual filesystems, so I chose to stay with the pure python implementation of the virtual filesystem that was present in the older version of itools.

Note that wxWidgets/Python also has a virtual filesystem implementation, but it is limited to read only -- there are no API calls to open a file for writing.

I'm attempting to provide support for network filesystems by using pycurl, a python binding for the cURL library. However, it looks like it is very low level and additional code will be needed to handle the responses from the individual protocols.

• cURL options
• C examples
• Save libcurl equivalent to a curl command: curl --libcurl example_libcurl.c http://www.google.com
• Some interesting mailing-list threads:
  • Only way to get SFTP file size is to start downloading with a callback that aborts

PyCurl doesn't support fish, but perhaps the ssh: protocol is equivalent?

• RFC 2518: http://www.webdav.org/specs/rfc2518.html
• client parser: http://chandlerproject.org/Projects/Davclient
• example: https://lists.sdsc.edu/pipermail/rbnb-dev/2008/000422.html

Example of an unlocked file:

{'/davtest/locked.txt': {'creationdate': '2009-11-06T02:10:17Z',
                       'getcontentlength': '22',
                       'getcontenttype': 'text/plain',
                       'getetag': '"466c549-16-477aa5855b440"',
                       'getlastmodified': 'Fri, 06 Nov 2009 02:10:17 GMT',
                       'lockdiscovery': None,
                       'resourcetype': None,
                       'supportedlock': {'lockentry': {'lockscope': {'shared': None},
                                                     'locktype': {'write': None}}},
                       '{http://apache.org/dav/props/}executable': 'F'}}

Example of a locked file:

{'/davtest/locked.txt': {'creationdate': '2009-11-06T02:10:17Z',
                       'getcontentlength': '22',
                       'getcontenttype': 'text/plain',
                       'getetag': '"466c549-16-477aa5855b440"',
                       'getlastmodified': 'Fri, 06 Nov 2009 02:10:17 GMT',
                       'lockdiscovery': {'activelock': {'depth': 'infinity',
                                                      'lockscope': {'exclusive': None},
                                                      'locktoken': {'href': 'opaquelocktoken:b7d79899-58e5-45eb-8e7b-f11fb83fdf1b'},
                                                      'locktype': {'write': None},
                                                      'owner': {'href': 'http://www.flipturn.org'},
                                                      'timeout': 'Infinite'}},
                       'resourcetype': None,
                       'supportedlock': {'lockentry': {'lockscope': {'shared': None},
                                                     'locktype': {'write': None}}},
                       '{http://apache.org/dav/props/}executable': 'F'}}

FTP doesn't have a standard directory listing protocol, so C library to parse LIST responses.

UPDATE: found a high-level FTP library.

For testing, here's how to set up a chroot environment for sshd.

The interesting idea that wxWidgets provides is the idea of a left location and a right location. For example, the url file:myzipfile.zip#zip:index.htm is decomposed into:

        GetLeftLocation("file:myzipfile.zip#zip:index.htm") == "file:myzipfile.zip"
        GetProtocol("file:myzipfile.zip#zip:index.htm") == "zip"
        GetRightLocation("file:myzipfile.zip#zip:index.htm") == "index.htm"

Arbitrarily nested protocols are possible, e.g. file:archive.tar.gz#gzip:#tar:filename

Although implemented in Java and not directly applicable, Apache Commons VFS also has a method to chain handlers in URLs.

    * jar:../lib/classes.jar!/META-INF/manifest.mf
    * zip:http://somehost/downloads/somefile.zip
    * jar:zip:outer.zip!/nested.jar!/somedir
    * jar:zip:outer.zip!/nested.jar!/some%21dir
    * tar:gz:http://anyhost/dir/mytar.tar.gz!/mytar.tar!/path/in/tar/README.txt
    * tgz:file://anyhost/dir/mytar.tgz!/somepath/somefile

I like the wxWidgets scheme better, because it provides a way for filename completion to work on a nested file, e.g. file:myzipfile.zip#zip:in<TAB> could complete to file:myzipfile.zip#zip:index.html