Solution: TAMS storage - EyevinnOSC/community GitHub Wiki
TAMS Gateway is a POC of the Time-addressable Media Store (TAMS) that stores segmented media flows by combining a media store that holds the individual media flow segments with a service that provides a database index for these segments.
- If you have not already done so, sign up for an Eyevinn OSC account
- Requires 3 available services in your plan. If you have no available services in your plan you can purchase each service individually or upgrade your plan.
Create a NoSQL database for the TAMS Gateway system by navigating to the CouchDB open web service. Click on the button "Create couchdb".
Give the database instance a name and enter a secret admin password.
When the database is up and running click on the instance card. Take a note of the URL and AdminPassword to the database instance.
Login with the username "admin" and enter the AdmindPassword that you set.
Click on the button Create Database in the top right corner and create 3 databases with the names "flows", "sources" and "segments.
Your should now see:
Create a minio S3 Storage by navigating to the Minio open web service. Click on the button "Create objstorage".
Give the minio instance a name, a RootUSer and a RootPassword. Take a note of the URL, RootUser and RootPassword of the minio instance. If you have forgotten these, you can click on the three dots at the top right of the card and then click on "Instance parameters".
Now go back to the Eyevinn Open Source Cloud web console and navigate to the TAMS Gateway web service. Create a new gateway by clicking on the button "Create gateway".
Entering the values:
- Name:
<name of your TAMS Gateway>
- DbUrl:
<URL to your couchDB instance>
- DbUserName:
admin
- DbPassword:
<AdminPassword to your couchDB instance>
- AwsAccessKeyId:
<RootUser to your minio instance>
- AwsSecretAccessKey:
<RootPassword to your minio instance>
- S3EndpointUrl:
<URL to your minio instance>
When the instance is up and running click on the card.
You should now see "TAMS-Gateway - OK". This means TAMS Gateway is up and running.
To access the API of your TAMS Gateway you will need an SAT (Service Access Token). To create an SAT Token first go back to the Eyevinn Open Source Cloud web console and navigate to "Settings" at the bottom left and then go to the "API" tab. Copy your "Personal Access Token".
Now, using npx (you need npm installed), you can create an SAT Token. In a console enter:
export OSC_ACCESS_TOKEN=<PAT TOKEN>
then enter:
npx -y @osaas/cli service-access-token eyevinn-tams-gateway
Take note of your SAT Token.
Our goal is to create a flow (which is stored in our couchDB) and store some segments to our minio S3 Storage which can then be used to play some media. To interact with our TAMS Gateway API we can use an API client like ThunderClient or Postman or simply CURL to the API. In the following examples we will be using ThunderClient.
To view the Swagger documentation of your TAMS Gateway API you can navigate in your browser to:
<URL to your TAMS Gateway instance>/docs
All requests to the API need to be authenticated with our SAT Token. Use the "Authorization" header with the value
Bearer <SAT Token>
In this example we will use the values:
- flowId: "FlowID"
- sourceId: "SourceID".
To create a flow we will be making a PUT request to:
<URL to your TAMS Gateway instance>/flows/:id
Where :id is our flowId. We will also include the JSON body:
Click me to view body
{
"id": "FlowID",
"source_id": "SourceID",
"label": "string",
"description": "string",
"created_by": "string",
"updated_by": "string",
"tags": {
"additionalProp1": "string",
"additionalProp2": "string",
"additionalProp3": "string"
},
"metadata_version": "string",
"generation": 0,
"created": "string",
"metadata_updated": "string",
"segments_updated": "string",
"read_only": true,
"codec": "string",
"container": "string",
"avg_bit_rate": 0,
"max_bit_rate": 0,
"segment_duration": {
"numerator": 0,
"denominator": 0
},
"timerange": "string",
"flow_collection": [
{
"id": "string",
"role": "string",
"container_mapping": {
"track_index": "string",
"format_track_index": "string",
"audio_track": {
"channel_numbers": [
0
],
"channel_range": "string"
},
"mp2ts_container": {
"pid": 0
},
"mxf_container": {
"package_uid": "string",
"track_id": "string"
},
"isobmmf_container": {
"track_id": 0
}
}
}
],
"collected_by": "string",
"container_mapping": {
"track_index": "string",
"format_track_index": "string",
"audio_track": {
"channel_numbers": [
0
],
"channel_range": "string"
},
"mp2ts_container": {
"pid": 0
},
"mxf_container": {
"package_uid": "string",
"track_id": "string"
},
"isobmmf_container": {
"track_id": 0
}
},
"format": "string",
"essence_parameters": {
"frame_rate": {
"numerator": 0,
"denominator": 0
},
"frame_width": 0,
"frame_height": 0,
"bit_depth": 0,
"interlace_mode": "string",
"colorspace": "string",
"transfer_characteristics": "string",
"aspect_ratio": {
"numerator": 0,
"denominator": 0
},
"pixel_aspect_ratio": {
"numerator": 0,
"denominator": 0
},
"component_type": "string",
"horiz_chroma_subs": 0,
"vert_chroma_subs": 0,
"unc_parameters": {
"unc_type": "string"
},
"avc_parameters": {
"profile": 0,
"level": 0,
"flags": 0
}
}
}
If the request responds with a 401 then something went wrong in step 5. You should see something like this:
The TAMS Gateway automatically creates a source with the sourceID provided. If you check your couchDB "flows" and "sources" databases you should see the newly created flow and source.
To create storage for our segments we must first make a POST request with an empty body to:
<URL to your TAMS Gateway instance>/flows/:id/storage
In the response object you should see 2 arrays:
- pre: This is for actions the client should do before uploading segments. In this case we want to create buckets in our S3 Minio Storage using the "put_url" and the XML body under "body".
- media_objects: Once the buckets have been created we can now store our media segments in the S3 Minio Storage using the "put_url". You should store one segment per object.
Once the media segments have been stored you can now make a POST request for each segment to:
<URL to your TAMS Gateway instance>/flows/:id/segments
Include the JSON body:
Click me to view body
{
"object_id": <object_id>,
"timerange": <timrange>,
"sample_count": 0,
"sample_offset": 0
}
Where:
- object_id: the bucket object id
- timerange: the timerange of the segment using the following notation [minutes:seconds_minutes:seconds) e.g. [0:0_0:5)
The segments are now stored in the S3 Minio Storage and we have saved their object_id and timerange in the couchDB.
To fetch our media segments from our S3 Minio Storage we can make a GET request to:
<URL to your TAMS Gateway instance>/flows/:id/segments?timerange=<timerange>
Where we can specify the query parameters:
- timerange: the segments timerange boundaries. e.g. ?timerange=[0:00_0:05) will fetch the segment(s) for the first 5 seconds of media.
Using the get_urls you can now fetch your media segments from the S3 Minio Storage.