Service: Retransfer - EyevinnOSC/community GitHub Wiki

Retransfer

Retransfer (serviceId eyevinn-docker-retransfer, catalog page) copies a file from one location to another as a one-shot job. The source is an HTTP/HTTPS URL or an S3 object; the destination must be an S3-compatible bucket. It is a job (runs the transfer and exits), not a long-running service. License: MIT.

What it can do

  • Pull a file from a web server into S3-compatible object storage.
  • Copy objects between S3 buckets or providers.
  • Write to S3-compatible storage not on AWS, such as OSC MinIO, by setting a custom endpoint URL.

Creating an instance

If you need a destination bucket on OSC, create a MinIO instance (serviceId minio-minio) first and confirm it is running. Store the S3 credentials as service secrets (create-service-secret, serviceId: "eyevinn-docker-retransfer"), for example accesskeyid and secretaccesskey. Then call create-service-instance with serviceId: "eyevinn-docker-retransfer":

{
  "name": "mycopy",
  "awsAccessKeyId": "{{secrets.accesskeyid}}",
  "awsSecretAccessKey": "{{secrets.secretaccesskey}}",
  "s3EndpointUrl": "https://<minio-instance-url>",
  "cmdLineArgs": "https://example.com/video.mp4 s3://mybucket/incoming/video.mp4"
}

The job runs the transfer and exits. Check with get-instance-status, and get-logs-for-instance for output.

Options

Option Required Type Meaning
name Yes string Name of the job. Alphanumeric, at most 20 characters.
cmdLineArgs Yes string Command-line arguments for the transfer, in the form <source> <destination>. Source supports http://, https://, s3://; destination supports s3:// only. The destination path must include the full object key.
awsAccessKeyId No string, sensitive Access key id for the S3 destination (AWS_ACCESS_KEY_ID). Pass as {{secrets.NAME}}.
awsSecretAccessKey No string, sensitive Secret access key for the S3 destination (AWS_SECRET_ACCESS_KEY). Pass as {{secrets.NAME}}.
s3EndpointUrl No string Endpoint URL for S3-compatible storage not on AWS (S3_ENDPOINT_URL). Omit for AWS S3.

{{secrets.NAME}} must be the entire field value. Overwriting a secret does not restart a running job.

Using the instance

Download from a web server to S3:

npx -y @osaas/cli create eyevinn-docker-retransfer mycopy \
  -o awsAccessKeyId="{{secrets.accesskeyid}}" \
  -o awsSecretAccessKey="{{secrets.secretaccesskey}}" \
  -o s3EndpointUrl="https://<minio-instance-url>" \
  -o cmdLineArgs="https://example.com/video.mp4 s3://mybucket/incoming/video.mp4"

Copy between S3 buckets by giving an S3 source as the first argument:

npx -y @osaas/cli create eyevinn-docker-retransfer s3copy \
  -o awsAccessKeyId="{{secrets.accesskeyid}}" \
  -o awsSecretAccessKey="{{secrets.secretaccesskey}}" \
  -o s3EndpointUrl="https://<minio-instance-url>" \
  -o cmdLineArgs="s3://sourcebucket/archive/clip.mp4 s3://destbucket/publish/clip.mp4"

Limitations

  • Destination must be S3; the destination must be a full object key.
  • The same S3 credentials and endpoint apply to the transfer; the options provide one set of S3 credentials.

Resources

⚠️ **GitHub.com Fallback** ⚠️