Service: Subtitle Generator - EyevinnOSC/community GitHub Wiki

Subtitle Generator

Subtitle Generator (serviceId eyevinn-auto-subtitles) generates subtitles for a video or audio file using OpenAI Whisper (whisper-1 model, via OpenAI Speech to Text). In Eyevinn Open Source Cloud (OSC) it is a long-running service instance with a REST API. License: Apache-2.0.

What it can do

  • Transcribe, and translate to English, the speech in a video or audio file; see the OpenAI documentation for supported languages.
  • Create a compressed audio file split into smaller chunks with time codes preserved, and combine all segments into one subtitle file with correct timing, aiming at readable, broadcast-standard subtitles.
  • Read the media from S3 and write the subtitle file (for example SRT or VTT) to an S3 bucket through /transcribe/s3.
  • Notify your application with an HTTP POST callback when processing has started and completed.

Creating an instance

Prerequisites:

  • An OpenAI API key.
  • S3-compatible storage with an input bucket (for example input) and an output bucket (for example subs), such as a MinIO instance on OSC (create it first and confirm it is running), and its access key and secret.

Store the OpenAI key and S3 credentials with create-service-secret (serviceId eyevinn-auto-subtitles), for example openaikey, accesskeyid and secretaccesskey, then create the instance with create-service-instance using serviceId: "eyevinn-auto-subtitles". The name must be alphanumeric and at most 20 characters.

{
  "name": "mygenerator",
  "openaikey": "{{secrets.openaikey}}",
  "awsAccessKeyId": "{{secrets.accesskeyid}}",
  "awsSecretAccessKey": "{{secrets.secretaccesskey}}",
  "awsRegion": "osc",
  "s3Endpoint": "https://<minio-instance-url>"
}

Wait with wait-for-service-ready, then use describe-service-instance for the instance URL.

Options

Option Required Type Meaning
name Yes string Name of the instance. Letters, digits and underscore only, at most 20 characters.
openaikey Yes string, sensitive OpenAI API key used to call OpenAI Whisper. Without it transcription cannot work.
awsAccessKeyId No string, sensitive Access key id for the S3 buckets. Needed when using the /transcribe/s3 endpoint; use together with awsSecretAccessKey.
awsSecretAccessKey No string, sensitive Secret access key for the S3 buckets. Needed when using /transcribe/s3.
awsRegion No string Region of the S3 buckets. For MinIO instances on OSC use osc. Can also be given in the API request payload.
s3Endpoint No string Endpoint URL for non-AWS S3-compatible storage such as a MinIO instance (can be associated with a MinIO instance on OSC); omit for AWS S3.

openaikey, awsAccessKeyId and awsSecretAccessKey are sensitive: store them as service secrets and reference them as {{secrets.NAME}}, which only works as the entire field value.

Using the instance

Generate subtitles by POSTing to /transcribe/s3 with a body such as:

{
  "url": "s3://input/osc-reel.mp4",
  "language": "en",
  "format": "srt",
  "bucket": "subs",
  "key": "osc-reel_en1"
}

This transcribes (and translates to English if the spoken language is not English) the file osc-reel.mp4 from bucket input and writes osc-reel_en1.srt to bucket subs. Allow a few minutes. Check the result with the S3 CLI, for example aws s3 --endpoint-url=<minio-endpoint-url> ls s3://subs/.

Calling the API

To call the API from your own application or from the command line (for example with curl), you need to provide a Service Access Token (SAT).

Generate a Service Access Token

Use an OSC personal access token (see Finding Your API Token). Store it in your terminal:

% export OSC_ACCESS_TOKEN=<personal-access-token>

Generate a Service Access Token using the OSC CLI:

% npx @osaas/cli service-access-token eyevinn-auto-subtitles
<service-token>

Call the API with curl

Use the service token in the Authorization header:

% curl -X POST \
  -H 'Authorization: Bearer <service-token>' \
  -H 'Content-Type: application/json' \
  -d '{"url": "s3://input/osc-reel.mp4", "language": "en", "format": "srt", "bucket": "subs", "key": "osc-reel_en1"}' \
  "https://<your-instance>.eyevinn-auto-subtitles.auto.prod-se.osaas.io/transcribe/s3"

All API endpoints are protected by an authentication gate. Without a valid Bearer token, requests will be rejected with an authentication error.

Include subtitle in VOD package

We can now take this generated subtitle file and include it when we create a VOD package. For example using Shaka Packager in Open Source Cloud and the Open Source Cloud CLI. For Shaka Packager we need to have the subtitles in the VTT format. Let us run a subtitle generation job again but with vtt as subtitle format.

{
  "url": "s3://input/osc-reel.mp4",
  "language": "en",
  "format": "vtt",
  "bucket": "subs",
  "key": "osc-reel_en1"
}

We should now have a file in VTT format in addition to SubRip.

% aws s3 --endpoint-url=<minio-endpoint-url> ls s3://subs/
2025-06-07 22:59:15       1332 osc-reel_en1.srt
2025-06-07 23:13:15       1307 osc-reel_en1.vtt

Copy this VTT file to the input bucket.

% aws s3 --endpoint-url=<minio-endpoint-url> cp s3://subs/osc-reel_en1.vtt s3://input/

Now we can create a VOD package by creating a Shaka Packager job in Open Source Cloud. Before running the command below make sure you have created the secrets accesskeyid and secretkey for the Shaka Packager service. These are not the same as you created for the Subtitle Generator service.

% npx -y @osaas/cli create eyevinn-shaka-packager-s3 subs \
  -o cmdLineArgs="-s s3://input -d s3://subs/vod/ -i a:audio=osc-reel.mp4 -i v:2069=osc-reel.mp4 -i t:en=osc-reel_en1.vtt" \
  -o awsAccessKeyId="{{secrets.accesskeyid}}" \
  -o awsSecretAccessKey="{{secrets.secretkey}}" \
  -o s3EndpointUrl=<minio-endpoint-url>

Provide callback endpoint for status reporting

In the subtitle job request you can provide an URL to a callback endpoint. This endpoint will receive an HTTP POST when subtitle process has started and completed. This way your application can get a notification when the submitted job has finished. Extend the JSON payload with this.

{
  "url": "s3://input/osc-reel.mp4",
  "callbackUrl": "<your-callback-listener-endpoint-url>",
  "language": "en",
  "format": "vtt",
  "bucket": "subs",
  "key": "osc-reel_en1"
}

Resources