CB‐Spider Object Storage API Guide (SigV4, XML Format) - cloud-barista/cb-spider GitHub Wiki

Related Links

CB-Spider Object Storage API Guide (SigV4 / XML Format)

CB-Spider provides an S3 API-compatible Object Storage interface that enables unified management of Object Storage across various Cloud Service Providers (CSPs).

This guide covers authentication using AWS Signature Version 4 (SigV4)-supported tools — including AWS CLI and awscurl. Examples in this guide use awscurl. For the CB-Spider standard method using curl with HTTP Basic Auth, see Object Storage API Guide (Basic Auth / XML-Format).

Prerequisite: Install awscurl before proceeding: pip install awscurl

Key Features

  • Multi-Cloud Support: Unified management of Object Storage across AWS, IBM Cloud, GCP, and other CSPs
  • S3-Compatible API: S3-compatible API structure, endpoints, and XML response format; SigV4 authentication enables use of AWS CLI, awscurl, and other SigV4-supported tools
  • PreSigned URL REST API: Provides PreSigned URL generation REST API
  • SigV4 Authentication: AWS Signature Version 4 authentication; credentials embed {username}@{ConnectionName} for CB-Spider routing

Authentication

CB-Spider SigV4 authentication encodes the spider username and connection name as credentials:

Field Value
Access Key {username}@{ConnectionName}
Secret Key {password}

Note: CB-Spider uses the SigV4 signing algorithm compatible with AWS S3,
but credentials encode {username}@{ConnectionName} instead of a standard AWS Access Key ID.
Standard AWS CLI/SDK can connect by passing these values as environment variables or credential parameters.

AWS CLI

AWS_ACCESS_KEY_ID="admin@aws-config01" \
AWS_SECRET_ACCESS_KEY="your-secure-password" \
aws s3 ls \
  --endpoint-url http://localhost:1024/spider/s3

awscurl

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3"

Note: Examples in this guide use awscurl. The same credential format ({username}@{ConnectionName}) applies to AWS CLI as well.

Note: The ConnectionName is embedded in --access_key, so URLs do not need ?ConnectionName=... query parameter.

Note on form uploads: awscurl does not support multipart/form-data (-F flag). The Upload Object (Form Data) API is not available with awscurl.

Note on CORS preflight: The Test CORS with OPTIONS example uses plain curl (no auth), as browsers send CORS preflight requests without credentials.

API List

1. Bucket Management

Function HTTP Method Endpoint Example
List Buckets GET /spider/s3 Example
Create Bucket PUT /spider/s3/{bucket-name} Example
Get Bucket Info GET /spider/s3/{bucket-name} Example
Check Bucket Exists HEAD /spider/s3/{bucket-name} Example
Get Bucket Location GET /spider/s3/{bucket-name}?location Example
Delete Bucket DELETE /spider/s3/{bucket-name} Example

2. Object Management

Function HTTP Method Endpoint Example
Upload Object (File) PUT /spider/s3/{bucket-name}/{object-key} Example
Upload Object (Form) POST /spider/s3/{bucket-name} Example
Download Object GET /spider/s3/{bucket-name}/{object-key} Example
Get Object Info HEAD /spider/s3/{bucket-name}/{object-key} Example
Delete Object DELETE /spider/s3/{bucket-name}/{object-key} Example
Delete Multiple Objects POST /spider/s3/{bucket-name}?delete Example

3. Multipart Upload

Function HTTP Method Endpoint Example
Initiate Multipart Upload POST /spider/s3/{bucket-name}/{object-key}?uploads Example
Upload Part PUT /spider/s3/{bucket-name}/{object-key}?uploadId={id}&partNumber={num} Example
Complete Multipart Upload POST /spider/s3/{bucket-name}/{object-key}?uploadId={id} Example
Abort Multipart Upload DELETE /spider/s3/{bucket-name}/{object-key}?uploadId={id} Example
List Parts GET /spider/s3/{bucket-name}/{object-key}?uploadId={id}&list-type=parts Example
List Multipart Uploads GET /spider/s3/{bucket-name}?uploads Example

4. Versioning Management

Function HTTP Method Endpoint Example
Get Bucket Versioning GET /spider/s3/{bucket-name}?versioning Example
Set Bucket Versioning PUT /spider/s3/{bucket-name}?versioning Example
List Object Versions GET /spider/s3/{bucket-name}?versions Example
Delete Versioned Object DELETE /spider/s3/{bucket-name}/{object-key}[?versionId=version-id] Example

5. CORS Management

Function HTTP Method Endpoint Example
Get Bucket CORS GET /spider/s3/{bucket-name}?cors Example
Set Bucket CORS PUT /spider/s3/{bucket-name}?cors Example
Delete Bucket CORS DELETE /spider/s3/{bucket-name}?cors Example

6. CB-Spider Special Features

Function HTTP Method Endpoint Example
Generate PreSigned URL (Download) GET /spider/s3/presigned/download/{bucket-name}/{object-key} Example
Generate PreSigned URL (Upload) GET /spider/s3/presigned/upload/{bucket-name}/{object-key} Example
Force Empty Bucket DELETE /spider/s3/{bucket-name}?empty Example
Force Delete Bucket DELETE /spider/s3/{bucket-name}?force Example

Note: CB-Spider special features are unique to CB-Spider and not part of AWS S3 standard.

API Usage Examples

1. Bucket Management

List Buckets

Prerequisites: Create a bucket and upload a sample object so the response shows bucket contents.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Upload a sample object
echo "Hello CB-Spider SigV4 Guide Test!" > /tmp/sample.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/sample.txt" \
  --data-binary -d @/tmp/sample.txt -H "Content-Type: text/plain"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Owner>
    <ID>aws-config01</ID>
    <DisplayName>aws-config01</DisplayName>
  </Owner>
  <Buckets>
    <Bucket>
      <Name>spider-test-bucket</Name>
      <CreationDate>2026-03-25T07:47:16Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/sample.txt"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Create Bucket

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"

Response:

Get Bucket Info

Prerequisites: Create a bucket and upload a sample object so the Contents field is populated.

# Upload a sample object
echo "Hello CB-Spider SigV4 Guide Test!" > /tmp/sample.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/sample.txt" \
  --data-binary -d @/tmp/sample.txt -H "Content-Type: text/plain"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/spider-test-bucket"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>spider-test-bucket</Name>
  <Prefix></Prefix>
  <Marker></Marker>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
  <Contents>
    <Key>sample.txt</Key>
    <LastModified>2026-03-25T07:51:34Z</LastModified>
    <ETag>e6855ab957a1674b0d2b62ed691044df</ETag>
    <Size>34</Size>
    <StorageClass>STANDARD</StorageClass>
  </Contents>
</ListBucketResult>

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/sample.txt"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Check Bucket Exists (HEAD)

Prerequisites: Create a bucket to test the existing-bucket response.

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -v -X HEAD "http://localhost:1024/spider/s3/spider-test-bucket"

Response for existing bucket:

...
'Response code: 200\n'
{'Vary': 'Origin', 'X-Amz-Id-2': '1774426905', 'X-Amz-Request-Id': '1774426905', 'Date': 'Wed, 25 Mar 2026 08:21:45 GMT'}

Response for non-existing bucket:

...
'Response code: 403\n'
{'Vary': 'Origin', 'Date': 'Wed, 25 Mar 2026 08:25:16 GMT'}
''

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Get Bucket Location

Prerequisites: Create a bucket first.

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/spider-test-bucket?location"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<LocationConstraint xmlns="http://s3.amazonaws.com/doc/2006-03-01/">ap-southeast-2</LocationConstraint>

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Delete Bucket

Prerequisites: Create a bucket first.

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Response:

2. Object Management

Upload Object (From File)

Prerequisites: Create a bucket and prepare a local file.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Prepare local file
echo "Hello CB-Spider S3 SigV4 Test!" > /tmp/local-file.txt

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt" \
  --data-binary -d @/tmp/local-file.txt \
  -H "Content-Type: text/plain"

Response:

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Upload Object (Form Data)

Not supported: awscurl does not support multipart/form-data (-F flag). This API is not available with awscurl.

Download Object

Prerequisites: Create a bucket and upload a sample object.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Upload object
echo "Hello CB-Spider S3 SigV4 Test!" > /tmp/local-file.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt" \
  --data-binary -d @/tmp/local-file.txt -H "Content-Type: text/plain"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt" \
  -o /tmp/downloaded-file.txt

Response:

Hello CB-Spider S3 SigV4 Test!

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Get Object Info (HEAD)

Prerequisites: Create a bucket and upload a sample object.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Upload object
echo "Hello CB-Spider S3 SigV4 Test!" > /tmp/local-file.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt" \
  --data-binary -d @/tmp/local-file.txt -H "Content-Type: text/plain"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -v -X HEAD "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt"

Response:

...
'Response code: 200\n'
{'Content-Length': '31', 'Content-Type': 'text/plain; charset=utf-8', 'Etag': 'ae3d5aee551ae166d418bc55af3865b5', 'Last-Modified': 'Wed, 25 Mar 2026 09:12:38 GMT', 'Vary': 'Origin', 'X-Amz-Id-2': '1774430010', 'X-Amz-Request-Id': '1774430010', 'Date': 'Wed, 25 Mar 2026 09:13:30 GMT'}

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Delete Object

Prerequisites: Create a bucket and upload an object.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Upload object
echo "Hello CB-Spider S3 SigV4 Test!" > /tmp/local-file.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt" \
  --data-binary -d @/tmp/local-file.txt -H "Content-Type: text/plain"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt"

Response:

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Delete Multiple Objects

Prerequisites: Create a bucket and upload multiple objects.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Upload three objects
for f in file1.txt file2.txt file3.txt; do
  echo "Content of $f" > /tmp/$f
  awscurl --service s3 \
    --access_key "admin@aws-config01" \
    --secret_key "your-secure-password" \
    -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/$f" \
    --data-binary -d @/tmp/$f -H "Content-Type: text/plain"
done

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X POST "http://localhost:1024/spider/s3/spider-test-bucket?delete" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<Delete xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Object>
    <Key>file1.txt</Key>
  </Object>
  <Object>
    <Key>file2.txt</Key>
  </Object>
  <Object>
    <Key>file3.txt</Key>
  </Object>
</Delete>'

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<DeleteResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Deleted>
    <Key>file1.txt</Key>
  </Deleted>
  <Deleted>
    <Key>file2.txt</Key>
  </Deleted>
  <Deleted>
    <Key>file3.txt</Key>
  </Deleted>
</DeleteResult>

Notes:

  • The delete query parameter is required
  • Request body must be XML with proper namespace declaration
  • Supports deleting multiple objects in a single request
  • Returns detailed results for each deleted object

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

3. Multipart Upload

Important Notes:

  • Part Size Limit: Each part (except the last one) must be at least 5MB in size for AWS S3 compatibility
  • ETag Format: Multipart files have ETag format as {hash}-{part_count} (e.g., 50f9c71a2e1d2cd7706f6dfd0b12c9fd-3)
  • Upload ID Lifecycle: CB-Spider automatically invalidates Upload IDs after a period of inactivity (unlike AWS S3 standard where Upload IDs never expire). It's recommended to upload parts immediately after initiation

Initiate Multipart Upload

Prerequisites: Create a bucket first.

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X POST "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploads" \
  -H "Content-Type: application/octet-stream"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Bucket>spider-test-bucket</Bucket>
  <Key>large-multipart-file.bin</Key>
  <UploadId>I0IgW6DHitIuc.iMWqoTBTTxAMB2hBczsbndV78_4.Gcul49Y5y.ADqwuhazXUUUJ76R0a8wBrsyeLwp4YU6_G3oUBePrhKvjlgfJRvlrw2zkin8_qv1Aum45.LYMdFAE93wfDzTo.Vh_XNOXmWGdg--</UploadId>
</InitiateMultipartUploadResult>

Upload Part

Prerequisites: Initiate a multipart upload and note the UploadId. Prepare a large part file (at least 5MB except the last part).

# Create 6MB test file
dd if=/dev/urandom of=/tmp/large-part1.bin bs=1M count=6

Request:

Note: Replace the uploadId value below with the UploadId returned from the Initiate Multipart Upload response above.

# Upload Part 1 (6MB)
# Replace {UploadId} with the UploadId from Initiate Multipart Upload response
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -i -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId from Initiate response}&partNumber=1" \
  --data-binary -d @/tmp/large-part1.bin \
  -H "Content-Type: application/octet-stream"

Response:

{'Etag': '11be3e8737b74ecddc59633f47db4800', 'Vary': 'Origin', 'X-Amz-Id-2': '1774435648', 'X-Amz-Request-Id': '1774435648', 'Date': 'Wed, 25 Mar 2026 10:47:28 GMT', 'Content-Length': '0'}

Complete Multipart Upload

Prerequisites: Upload all required parts and have their ETags ready.

Note: Replace {UploadId} with the UploadId from the Initiate Multipart Upload response, and replace {ETag of Part 1} with the ETag value from the Upload Part response (e.g., "11be3e8737b74ecddc59633f47db4800").

Request:

# Replace {UploadId} with UploadId from Initiate Multipart Upload response
# Replace {ETag of Part 1} with ETag from Upload Part response (e.g. "c853c31bc491fa878d4129c6089613ea")
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X POST "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId from Initiate response}" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<CompleteMultipartUpload>
  <Part>
    <PartNumber>1</PartNumber>
    <ETag>{ETag of Part 1}</ETag>
  </Part>
</CompleteMultipartUpload>'

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Location>/spider-test-bucket/large-multipart-file.bin</Location>
  <Bucket>spider-test-bucket</Bucket>
  <Key>large-multipart-file.bin</Key>
  <ETag>a39aa9b45765b6abb4dee2443bf897a1-1</ETag>
</CompleteMultipartUploadResult>

Note: The ETag ending with -1 indicates this file was assembled from 1 part.

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Abort Multipart Upload

Prerequisites: Create a bucket and initiate a multipart upload to get an UploadId.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Initiate upload
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X POST "http://localhost:1024/spider/s3/spider-test-bucket/abort-test-file.bin?uploads" \
  -H "Content-Type: application/octet-stream"

Request:

Note: Replace {UploadId} with the UploadId returned from the Initiate Multipart Upload response above.

# Replace {UploadId} with the UploadId from Initiate Multipart Upload response
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/abort-test-file.bin?uploadId={UploadId from Initiate response}"

Response:

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

List Parts

Prerequisites: Initiate a multipart upload and upload at least one part.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Initiate upload and note the UploadId
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X POST "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploads" \
  -H "Content-Type: application/octet-stream"
# Upload Part 1 (6MB) using the UploadId from above
dd if=/dev/urandom of=/tmp/large-part1.bin bs=1M count=6
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId}&partNumber=1" \
  --data-binary -d @/tmp/large-part1.bin -H "Content-Type: application/octet-stream"

Request:

Note: Replace {UploadId} with the UploadId returned from the Initiate Multipart Upload response in the Prerequisites above.

# Replace {UploadId} with the UploadId from Initiate Multipart Upload response
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId from Initiate response}&list-type=parts"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<ListPartsResult>
  <Bucket>spider-test-bucket</Bucket>
  <Key>large-multipart-file.bin</Key>
  <UploadId>u0n.71gq6slvttUu4JJXNGbRC9HCbj3FmSjpzObpka4btqdAJEVYrT8mksk.UV1bvZ4dJQIJpA_chMTyFbAO8_uEcH29P13B.25k58Cn5JXNHk6SwL3T7IlnZuoYT4ny9hQunw5QuxpPrOvbFXrNDg--</UploadId>
  <PartNumberMarker>0</PartNumberMarker>
  <NextPartNumberMarker>1</NextPartNumberMarker>
  <MaxParts>1000</MaxParts>
  <IsTruncated>false</IsTruncated>
  <Part>
    <PartNumber>1</PartNumber>
    <LastModified>2026-03-25T11:08:20Z</LastModified>
    <ETag>&#34;f161f9fc81fd87cdf365cb4d78b5e70e&#34;</ETag>
    <Size>6291456</Size>
  </Part>
  <Initiator>
    <ID>arn:aws:iam::123456789123:user/spider</ID>
    <DisplayName>spider</DisplayName>
  </Initiator>
  <Owner>
    <ID>7d9f77b078e963b2aca023ffa7dc39a3cac3b138397f2214d2d66e33bee3384d</ID>
    <DisplayName></DisplayName>
  </Owner>
  <StorageClass>STANDARD</StorageClass>
</ListPartsResult>

Optional Parameters:

  • part-number-marker: Start listing after this part number
  • max-parts: Maximum number of parts to return (default: 1000)

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId}"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

List Multipart Uploads

Prerequisites: Initiate at least one multipart upload and leave it pending (do not complete or abort it).

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Initiate upload (leave pending — do not complete or abort)
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X POST "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploads" \
  -H "Content-Type: application/octet-stream"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/spider-test-bucket?uploads"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<ListMultipartUploadsResult>
  <Bucket>spider-test-bucket</Bucket>
  <KeyMarker></KeyMarker>
  <UploadIdMarker></UploadIdMarker>
  <NextKeyMarker>large-multipart-file.bin</NextKeyMarker>
  <NextUploadIdMarker>79aWWDbC.OjlK27.CTW02tn39BUDPdsHn.nQ5cUgu7hZO6RIaGsiKgHNtbJJlShOlYvbx2Y8Uqe62Ih7lkr2KqE32yrYcw17ggk8h.QJGyDgzdEodrNjtPIuJqhocioP0j9V8pe8a.FvzITB1a6nHQ--</NextUploadIdMarker>
  <MaxUploads>1000</MaxUploads>
  <IsTruncated>false</IsTruncated>
  <Upload>
    <Key>large-multipart-file.bin</Key>
    <UploadId>79aWWDbC.OjlK27.CTW02tn39BUDPdsHn.nQ5cUgu7hZO6RIaGsiKgHNtbJJlShOlYvbx2Y8Uqe62Ih7lkr2KqE32yrYcw17ggk8h.QJGyDgzdEodrNjtPIuJqhocioP0j9V8pe8a.FvzITB1a6nHQ--</UploadId>
    <Initiated>2026-03-25T11:14:30Z</Initiated>
    <StorageClass>STANDARD</StorageClass>
    <Initiator>
      <ID>arn:aws:iam::123456789123:user/spider</ID>
      <DisplayName>spider</DisplayName>
    </Initiator>
    <Owner>
      <ID>7d9f77b078e963b2aca023ffa7dc39a3cac3b138397f2214d2d66e33bee3384d</ID>
      <DisplayName></DisplayName>
    </Owner>
  </Upload>
  <Prefix></Prefix>
  <Delimiter></Delimiter>
</ListMultipartUploadsResult>

Optional Parameters:

  • prefix: Only list uploads with keys beginning with this prefix
  • key-marker: Start listing after this key name
  • upload-id-marker: Start listing after this upload ID
  • delimiter: Character used to group keys
  • max-uploads: Maximum number of uploads to return (default: 1000)

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId}"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Multipart Upload Technical Notes

Upload ID Lifecycle:

  • Upload IDs must be used immediately after initiation
  • CB-Spider automatically invalidates unused Upload IDs (differs from AWS S3 standard)
  • Always initiate a new multipart upload for each file upload session

Part Size Requirements:

  • Each part (except the last) must be at least 5MB for AWS S3 compatibility

ETag Format:

  • Single uploads: Standard MD5 hash (e.g., "da6a0d097e307ac52ed9b4ad551801fc")
  • Multipart uploads: Hash with part count suffix (e.g., "50f9c71a2e1d2cd7706f6dfd0b12c9fd-3")

4. Versioning Management

Get Bucket Versioning

Prerequisites: Create a bucket and enable versioning.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Enable versioning
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?><VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Status>Enabled</Status></VersioningConfiguration>'

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/spider-test-bucket?versioning"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<VersioningConfiguration>
  <Status>Enabled</Status>
</VersioningConfiguration>

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force"

Set Bucket Versioning

Prerequisites: Create a bucket first.

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Status>Enabled</Status>
</VersioningConfiguration>'

Response:

To suspend versioning:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Status>Suspended</Status>
</VersioningConfiguration>'

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force"

List Object Versions

Prerequisites: Create a bucket, enable versioning, and upload the same object multiple times to create multiple versions.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Enable versioning
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?><VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Status>Enabled</Status></VersioningConfiguration>'
# Upload version 1
echo "Version 1 content" > /tmp/file-v1.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt" \
  --data-binary -d @/tmp/file-v1.txt -H "Content-Type: text/plain"
# Upload version 2 (same key)
echo "Version 2 content" > /tmp/file-v2.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt" \
  --data-binary -d @/tmp/file-v2.txt -H "Content-Type: text/plain"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/spider-test-bucket?versions"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<ListVersionsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>spider-test-bucket</Name>
  <Prefix></Prefix>
  <KeyMarker></KeyMarker>
  <VersionIdMarker></VersionIdMarker>
  <NextKeyMarker></NextKeyMarker>
  <NextVersionIdMarker></NextVersionIdMarker>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
  <Version>
    <Key>same-file.txt</Key>
    <VersionId>EYBBHeItneF43m54Rs8GpTHfRSjGtwBQ</VersionId>
    <IsLatest>true</IsLatest>
    <LastModified>2026-03-25T11:40:33Z</LastModified>
    <ETag>e0ef76395fbb2af0b3213158344bd881</ETag>
    <Size>18</Size>
    <StorageClass>STANDARD</StorageClass>
    <Owner>
      <ID>aws-config01</ID>
      <DisplayName>aws-config01</DisplayName>
    </Owner>
  </Version>
  <Version>
    <Key>same-file.txt</Key>
    <VersionId>sQH.byqZNVGHtwvCK_zs5gDEjIU3eCiV</VersionId>
    <IsLatest>false</IsLatest>
    <LastModified>2026-03-25T11:40:32Z</LastModified>
    <ETag>0e8e102783f3fd35b701b8de312415e5</ETag>
    <Size>18</Size>
    <StorageClass>STANDARD</StorageClass>
    <Owner>
      <ID>aws-config01</ID>
      <DisplayName>aws-config01</DisplayName>
    </Owner>
  </Version>
</ListVersionsResult>

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force"

Delete Versioned Object

Prerequisites: Create a bucket with versioning enabled, upload an object twice (same key), then list versions to get the versionId.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Enable versioning
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?><VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Status>Enabled</Status></VersioningConfiguration>'
# Upload version 1
echo "Version 1 content" > /tmp/file-v1.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt" \
  --data-binary -d @/tmp/file-v1.txt -H "Content-Type: text/plain"
# Upload version 2 (same key)
echo "Version 2 content" > /tmp/file-v2.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt" \
  --data-binary -d @/tmp/file-v2.txt -H "Content-Type: text/plain"

List Versions Response Example (note the VersionId values — use one in the Request below):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/spider-test-bucket?versions"

<?xml version="1.0" encoding="UTF-8"?>
<ListVersionsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>spider-test-bucket</Name>
  <Prefix></Prefix>
  <KeyMarker></KeyMarker>
  <VersionIdMarker></VersionIdMarker>
  <NextKeyMarker></NextKeyMarker>
  <NextVersionIdMarker></NextVersionIdMarker>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
  <Version>
    <Key>same-file.txt</Key>
    <VersionId>XksdMz6s7r_146VV__vHM9t9GXj1h4N4</VersionId>
    <IsLatest>true</IsLatest>
    <LastModified>2026-03-25T11:42:11Z</LastModified>
    <ETag>e0ef76395fbb2af0b3213158344bd881</ETag>
    <Size>18</Size>
    <StorageClass>STANDARD</StorageClass>
    <Owner>
      <ID>aws-config01</ID>
      <DisplayName>aws-config01</DisplayName>
    </Owner>
  </Version>
  <Version>
    <Key>same-file.txt</Key>
    <VersionId>MZ5dYcduobyVVQ_nSHfJWMtnT43gnYaN</VersionId>
    <IsLatest>false</IsLatest>
    <LastModified>2026-03-25T11:42:11Z</LastModified>
    <ETag>0e8e102783f3fd35b701b8de312415e5</ETag>
    <Size>18</Size>
    <StorageClass>STANDARD</StorageClass>
    <Owner>
      <ID>aws-config01</ID>
      <DisplayName>aws-config01</DisplayName>
    </Owner>
  </Version>
</ListVersionsResult>

Request:

Note: Replace {versionId} with a VersionId from the List Versions response above.

# Replace {versionId} with a VersionId from the list versions response above
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt?versionId={versionId}"

Response:

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force"

Versioning Important Notes

Creating Multiple Versions: When versioning is enabled, uploading an object with the same key creates a new version:

# Upload version 1
echo "Version 1 content" > /tmp/file-v1.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt" \
  --data-binary -d @/tmp/file-v1.txt \
  -H "Content-Type: text/plain"

# Upload version 2 (same key)
echo "Version 2 content" > /tmp/file-v2.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt" \
  --data-binary -d @/tmp/file-v2.txt \
  -H "Content-Type: text/plain"

Versioning States:

  • Enabled: New versions are created for each upload
  • Suspended: New uploads don't create versions (but existing versions remain)
  • Unversioned: Default state (no versioning)

5. CORS Management

Get CORS Configuration

Prerequisites: Create a bucket and configure CORS so the response shows the active CORS config.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Set CORS configuration
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?><CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod><AllowedMethod>PUT</AllowedMethod><AllowedHeader>*</AllowedHeader><MaxAgeSeconds>3000</MaxAgeSeconds></CORSRule></CORSConfiguration>'

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/spider-test-bucket?cors"

Response Example (when CORS is configured):

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
    <ExposeHeader>ETag</ExposeHeader>
    <ExposeHeader>x-amz-server-side-encryption</ExposeHeader>
    <ExposeHeader>x-amz-request-id</ExposeHeader>
    <ExposeHeader>x-amz-id-2</ExposeHeader>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
  </CORSRule>
</CORSConfiguration>

Error Response (when CORS is not configured):

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>NoSuchCORSConfiguration</Code>
  <Message>The CORS configuration for bucket 'spider-test-bucket' does not exist</Message>
  <Resource>/spider-test-bucket</Resource>
  <RequestId>1774351430</RequestId>
</Error>

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?cors"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Set CORS Configuration

Prerequisites: Create a bucket first.

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"

Request:

Basic CORS Configuration:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration>
  <CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedMethod>DELETE</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
  </CORSRule>
</CORSConfiguration>'

Advanced CORS Configuration (Multiple Rules):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration>
  <CORSRule>
    <AllowedOrigin>https://example.com</AllowedOrigin>
    <AllowedOrigin>https://app.example.com</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedHeader>Content-Type</AllowedHeader>
    <AllowedHeader>Authorization</AllowedHeader>
    <ExposeHeader>ETag</ExposeHeader>
    <MaxAgeSeconds>1800</MaxAgeSeconds>
  </CORSRule>
  <CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <MaxAgeSeconds>300</MaxAgeSeconds>
  </CORSRule>
</CORSConfiguration>'

Response:

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?cors"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Test CORS with OPTIONS Request

Note: CORS preflight requests are sent by browsers without credentials. Use plain curl (no auth).

Prerequisites: Create a bucket, upload an object, and configure CORS.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Upload test object
echo "Hello CB-Spider S3 SigV4 Test!" > /tmp/local-file.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt" \
  --data-binary -d @/tmp/local-file.txt -H "Content-Type: text/plain"
# Set CORS configuration
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?><CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod><AllowedMethod>PUT</AllowedMethod><AllowedHeader>*</AllowedHeader><MaxAgeSeconds>3000</MaxAgeSeconds></CORSRule></CORSConfiguration>'

Request:

curl -X OPTIONS "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt" \
  -H "Origin: https://example.com" \
  -H "Access-Control-Request-Method: PUT" \
  -H "Access-Control-Request-Headers: Content-Type" \
  -v

Response:

*   Trying 127.0.0.1:1024...
* Connected to localhost (127.0.0.1) port 1024 (#0)
> OPTIONS /spider/s3/spider-test-bucket/test-file.txt HTTP/1.1
> Host: localhost:1024
> User-Agent: curl/7.81.0
> Accept: */*
> Origin: https://example.com
> Access-Control-Request-Method: PUT
> Access-Control-Request-Headers: Content-Type
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 204 No Content
< Access-Control-Allow-Headers: Content-Type
< Access-Control-Allow-Methods: GET,HEAD,PUT,PATCH,POST,DELETE
< Access-Control-Allow-Origin: *
< Allow: OPTIONS, DELETE, GET, HEAD, POST, PUT
< Vary: Origin
< Vary: Access-Control-Request-Method
< Vary: Access-Control-Request-Headers
< Date: Wed, 25 Mar 2026 12:07:00 GMT
<
* Connection #0 to host localhost left intact

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?cors"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Delete CORS Configuration

Prerequisites: Create a bucket and configure CORS first.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Set CORS
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?><CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod><AllowedHeader>*</AllowedHeader></CORSRule></CORSConfiguration>'

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?cors"

Response:

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

CORS Configuration Elements

Element Description Required Example
AllowedOrigin Specifies the origins allowed to access the bucket Yes * or https://example.com
AllowedMethod Specifies the HTTP methods allowed Yes GET, PUT, POST, DELETE
AllowedHeader Specifies the headers allowed in the request No *, Content-Type, Authorization
ExposeHeader Specifies which headers are exposed to the browser No ETag, x-amz-request-id
MaxAgeSeconds Specifies how long the browser can cache the preflight response No 3600 (1 hour)

CORS Configuration Examples

Allow All Origins (Development):

<CORSRule>
  <AllowedOrigin>*</AllowedOrigin>
  <AllowedMethod>GET</AllowedMethod>
  <AllowedMethod>PUT</AllowedMethod>
  <AllowedMethod>POST</AllowedMethod>
  <AllowedMethod>DELETE</AllowedMethod>
  <AllowedHeader>*</AllowedHeader>
  <MaxAgeSeconds>3000</MaxAgeSeconds>
</CORSRule>

Production Configuration (Specific Domains):

<CORSRule>
  <AllowedOrigin>https://example.com</AllowedOrigin>
  <AllowedOrigin>https://app.example.com</AllowedOrigin>
  <AllowedMethod>GET</AllowedMethod>
  <AllowedMethod>PUT</AllowedMethod>
  <AllowedHeader>Content-Type</AllowedHeader>
  <AllowedHeader>Authorization</AllowedHeader>
  <ExposeHeader>ETag</ExposeHeader>
  <MaxAgeSeconds>1800</MaxAgeSeconds>
</CORSRule>

Read-Only Access:

<CORSRule>
  <AllowedOrigin>*</AllowedOrigin>
  <AllowedMethod>GET</AllowedMethod>
  <AllowedHeader>Range</AllowedHeader>
  <ExposeHeader>Content-Length</ExposeHeader>
  <ExposeHeader>Content-Range</ExposeHeader>
  <MaxAgeSeconds>300</MaxAgeSeconds>
</CORSRule>

CORS Best Practices

  1. Security: Use specific origins instead of * in production
  2. Performance: Set appropriate MaxAgeSeconds to reduce preflight requests
  3. Headers: Only allow necessary headers to minimize security risks
  4. Methods: Only specify required HTTP methods
  5. Testing: Use browser developer tools to verify CORS behavior

6. CB-Spider Special Features

Generate PreSigned URL (Download)

Prerequisites: Create a bucket and upload a sample object.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Upload object
echo "Hello CB-Spider S3 SigV4 Test!" > /tmp/local-file.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt" \
  --data-binary -d @/tmp/local-file.txt -H "Content-Type: text/plain"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/presigned/download/spider-test-bucket/test-file.txt?expires=3600"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<PresignedURLResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <PresignedURL>https://spider-test-bucket.s3.dualstack.ap-southeast-2.amazonaws.com/test-file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=AKIA***EXAMPLE%2F20260324%2Fap-southeast-2%2Fs3%2Faws4_request&amp;X-Amz-Date=20260324T141006Z&amp;X-Amz-Expires=3600&amp;X-Amz-SignedHeaders=host&amp;X-Amz-Signature=***-signature</PresignedURL>
  <Expires>3600</Expires>
  <Method>GET</Method>
</PresignedURLResult>

Optional Parameters:

  • expires: URL expiration time in seconds (default: 3600)
  • method: HTTP method for the URL (default: GET, also supports PUT)
  • response-content-disposition: Content-Disposition header for downloads

Usage Example:

# Download a file using the generated PreSigned URL
curl -X GET "{PresignedURL}" -o downloaded-file.txt

Note: This feature is unique to CB-Spider and not part of AWS S3 standard.

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Generate PreSigned URL (Upload)

Prerequisites: Create a bucket first.

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/presigned/upload/spider-test-bucket/test-file.txt?expires=3600"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<PresignedURLResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <PresignedURL>https://spider-test-bucket.s3.dualstack.ap-southeast-2.amazonaws.com/upload-test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=AKIA***EXAMPLE%2F20260324%2Fap-southeast-2%2Fs3%2Faws4_request&amp;X-Amz-Date=20260324T141006Z&amp;X-Amz-Expires=3600&amp;X-Amz-SignedHeaders=host&amp;X-Amz-Signature=***-signature</PresignedURL>
  <Expires>3600</Expires>
  <Method>PUT</Method>
</PresignedURLResult>

Optional Parameters:

  • expires: URL expiration time in seconds (default: 3600)

Usage Example:

# Create a file to upload
echo "Hello CB-Spider S3 SigV4 Test!" > /tmp/local-file.txt
# Upload a file using the generated PreSigned URL
curl -X PUT "{PresignedURL}" --data-binary "@/tmp/local-file.txt" -H "Content-Type: text/plain"

Note: This feature is unique to CB-Spider and not part of AWS S3 standard.

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt"
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Force Empty Bucket

Prerequisites: Create a bucket and upload some objects.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Upload objects
echo "file1 content" > /tmp/force-empty-file1.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/force-empty-file1.txt" \
  --data-binary -d @/tmp/force-empty-file1.txt -H "Content-Type: text/plain"
echo "file2 content" > /tmp/force-empty-file2.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/force-empty-file2.txt" \
  --data-binary -d @/tmp/force-empty-file2.txt -H "Content-Type: text/plain"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?empty"

Response:

JSON Error Response (when the ?empty parameter is not used):

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>BucketNotEmpty</Code>
  <Message>The bucket you tried to delete is not empty. It contains 2 objects. Use force=true parameter to force delete.</Message>
  <Resource>/spider-test-bucket</Resource>
  <RequestId>1774442507</RequestId>
</Error>

Verification:

# Check that bucket is now empty
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/spider-test-bucket"

Empty Bucket Response:

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>spider-test-bucket</Name>
  <Prefix></Prefix>
  <Marker></Marker>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
</ListBucketResult>

Note: This feature is unique to CB-Spider and not part of AWS S3 standard. The ?empty query parameter is required as a safety mechanism.

Cleanup (optional):

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket"

Force Delete Bucket

Prerequisites: Create a bucket and upload some objects.

# Create bucket
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket"
# Upload objects
echo "file1 content" > /tmp/force-del-file1.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/force-del-file1.txt" \
  --data-binary -d @/tmp/force-del-file1.txt -H "Content-Type: text/plain"
echo "file2 content" > /tmp/force-del-file2.txt
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/force-del-file2.txt" \
  --data-binary -d @/tmp/force-del-file2.txt -H "Content-Type: text/plain"

Request:

awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force"

Response:

Error Response (bucket not empty, without force):

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>BucketNotEmpty</Code>
  <Message>The bucket you tried to delete is not empty. It contains 2 objects. Use force=true parameter to force delete.</Message>
  <Resource>/spider-test-bucket</Resource>
  <RequestId>1774443332</RequestId>
</Error>

Verification:

# Check that bucket no longer exists
awscurl --service s3 \
  --access_key "admin@aws-config01" \
  --secret_key "your-secure-password" \
  "http://localhost:1024/spider/s3/spider-test-bucket"

Deleted Bucket Response:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>NoSuchBucket</Code>
  <Message>Resource 'spider-test-bucket' does not exist in connection 'aws-config01'</Message>
  <Resource>/spider-test-bucket</Resource>
  <RequestId>1774361413</RequestId>
</Error>

Note: This feature forcefully deletes the bucket and all its contents. This feature is unique to CB-Spider and not part of AWS S3 standard. The ?force query parameter is required as a safety mechanism.

Related Links

Add a custom footer
⚠️ **GitHub.com Fallback** ⚠️