File transfers - aaronwmorris/indi-allsky GitHub Wiki

Overview

indi-allsky supports several methods of file transfers. All are implemented using native python features and modules.

Protocol Class Name Port Description
ftp pycurl_ftp 21 FTP via pycurl
python_ftp 21 FTP via ftplib
ftpes pycurl_ftpes 21 FTPS (explicit) via pycurl
python_ftpes 21 FTPS (explicit) via ftplib
ftps pycurl_ftps 990 FTPS (implicit) via pycurl
sftp pycurl_sftp 22 SFTP via pycurl
paramiko_sftp 22 SFTP via paramiko
webdav (https) pycurl_webdav_https 443 HTTPS PUT via pycurl

PycURL Options

File transfer variables

Name Type Info
timestamp datetime https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes
ts datetime Shortcut for timestamp
day_date date Day reference for exposure. Referencing the date when the timelapse set starts.

SFTP

PycURL SFTP

SFTP protocol implemented via libcurl.

  • Does not support relative remote paths. You may use tilde ~ for the remote home directory.
  • You may have to manually accept the remote hostkey using ssh on the command line before file transfers can occur.
  • Use Private and Public key inputs or PycURL Options for key based authentication
    • Both private and public keys must be specified
    • Only supports rsa keys in PEM format when libcurl compiled with libgcrypt (Debian, Raspbian, etc)
      • Does not support dsa, ecdsa, or ed25519 keys
    • Convert keys using ssh-keygen -p -m PEM -f /home/pi/.ssh/id_rsa
      • Correct key: -----BEGIN RSA PRIVATE KEY-----
      • If ssh key starts with -----BEGIN OPENSSH PRIVATE KEY----- IT WILL NOT WORK

Unencrypted key

{
    "SSH_PRIVATE_KEYFILE": "/home/pi/.ssh/id_rsa",
    "SSH_PUBLIC_KEYFILE": "/home/pi/.ssh/id_rsa.pub",
    "SSH_AUTH_TYPES": 1,
    "VERBOSE": 0
}

Encrypted key

{
    "SSH_PRIVATE_KEYFILE": "/home/pi/.ssh/id_rsa",
    "SSH_PUBLIC_KEYFILE": "/home/pi/.ssh/id_rsa.pub",
    "KEYPASSWD": "foobar",
    "SSH_AUTH_TYPES": 1,
    "VERBOSE": 0
}

Paramiko SFTP

SFTP protocol implemented via the paramiko module.

  • Use Private Key input for key based authentication (public key is not used)
  • Supports relative and full remote paths.
    • Does not support tilde ~ shortcut
  • Automatically detects and uses ssh keys (rsa, dsa, ecdsa, ed25519) if located at default paths
  • Host key validation is disabled by default

FTP

PycURL FTP

FTP protocol implemented via libcurl.

Python FTP

FTP protocol using Python's native ftp support.

PycURL FTPES

FTPES protocol via libcurl.

  • ftpes sessions start like a regular FTP session on port 21, but negotiate TLS encryption for authentication and data transfers. Similar to START_TLS with SMTP or LDAP.
  • Certificate validation is disabled by default

Python FTPES

FTPES protocol via Python's native ftp support.

  • ftpes sessions start like a regular FTP session on port 21, but negotiate TLS encryption for authentication and data transfers. Similar to START_TLS with SMTP or LDAP.
  • Certificate validation is disabled by default

PycURL FTPS

FTPS protocol via libcurl.

  • FTPS is implicitly encrypted for the full session.
  • Certificate validation is disabled by default

HTTP

PycURL WebDAV

HTTP(S) protocol via libcurl.

  • Remote server must support webdav/web disk.
  • Encrypted when endpoint is https://
  • Certificate validation is disabled by default

Notes