CB-Spider Object Storage API Guide (Basic Auth / XML Format)

CB-Spider provides an S3 API-compatible Object Storage interface that enables unified management of Object Storage across various Cloud Service Providers (CSPs).

This guide uses curl with HTTP Basic Authentication. For authentication using AWS Signature Version 4 (SigV4)-supported tools such as AWS CLI and awscurl, see Object Storage API Guide (SigV4 / XML-Format).

Key Features

Multi-Cloud Support: Unified management of Object Storage across AWS, IBM Cloud, GCP, and other CSPs
S3-Compatible API: S3-compatible API structure, endpoints, and XML response format
PreSigned URL REST API: Provides PreSigned URL generation REST API
Simple Authentication: CB-Spider HTTP Basic Auth with ConnectionName query parameter

Authentication

This guide uses HTTP Basic Authentication via curl -u {username}:{password}.

Field	Value
Username	CB-Spider username (e.g., `admin`)
Password	CB-Spider password

The ConnectionName specifying which CSP to use is passed as a query parameter in every request URL:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3?ConnectionName=aws-config01"

Note: This is CB-Spider's own authentication method and is not compatible with standard AWS S3 tools (AWS CLI, awscurl, boto3, etc.).
For AWS SigV4-based authentication compatible with those tools, see Object Storage API Guide (SigV4).

API List

1. Bucket Management

Function	HTTP Method	Endpoint	Example
List Buckets	GET	`/spider/s3`	Example
Create Bucket	PUT	`/spider/s3/{bucket-name}`	Example
Get Bucket Info	GET	`/spider/s3/{bucket-name}`	Example
Check Bucket Exists	HEAD	`/spider/s3/{bucket-name}`	Example
Get Bucket Location	GET	`/spider/s3/{bucket-name}?location`	Example
Delete Bucket	DELETE	`/spider/s3/{bucket-name}`	Example

2. Object Management

Function	HTTP Method	Endpoint	Example
Upload Object (File)	PUT	`/spider/s3/{bucket-name}/{object-key}`	Example
Upload Object (Form)	POST	`/spider/s3/{bucket-name}`	Example
Download Object	GET	`/spider/s3/{bucket-name}/{object-key}`	Example
Get Object Info	HEAD	`/spider/s3/{bucket-name}/{object-key}`	Example
Delete Object	DELETE	`/spider/s3/{bucket-name}/{object-key}`	Example
Delete Multiple Objects	POST	`/spider/s3/{bucket-name}?delete`	Example

3. Multipart Upload

Function	HTTP Method	Endpoint	Example
Initiate Multipart Upload	POST	`/spider/s3/{bucket-name}/{object-key}?uploads`	Example
Upload Part	PUT	`/spider/s3/{bucket-name}/{object-key}?uploadId={id}&partNumber={num}`	Example
Complete Multipart Upload	POST	`/spider/s3/{bucket-name}/{object-key}?uploadId={id}`	Example
Abort Multipart Upload	DELETE	`/spider/s3/{bucket-name}/{object-key}?uploadId={id}`	Example
List Parts	GET	`/spider/s3/{bucket-name}/{object-key}?uploadId={id}&list-type=parts`	Example
List Multipart Uploads	GET	`/spider/s3/{bucket-name}?uploads`	Example

4. Versioning Management

Function	HTTP Method	Endpoint	Example
Get Bucket Versioning	GET	`/spider/s3/{bucket-name}?versioning`	Example
Set Bucket Versioning	PUT	`/spider/s3/{bucket-name}?versioning`	Example
List Object Versions	GET	`/spider/s3/{bucket-name}?versions`	Example
Delete Versioned Object	DELETE	`/spider/s3/{bucket-name}/{object-key}[?versionId=version-id]`	Example

5. CORS Management

Function	HTTP Method	Endpoint	Example
Get Bucket CORS	GET	`/spider/s3/{bucket-name}?cors`	Example
Set Bucket CORS	PUT	`/spider/s3/{bucket-name}?cors`	Example
Delete Bucket CORS	DELETE	`/spider/s3/{bucket-name}?cors`	Example

6. CB-Spider Special Features

Function	HTTP Method	Endpoint	Example
Generate PreSigned URL (Download)	GET	`/spider/s3/presigned/download/{bucket-name}/{object-key}`	Example
Generate PreSigned URL (Upload)	GET	`/spider/s3/presigned/upload/{bucket-name}/{object-key}`	Example
Force Empty Bucket	DELETE	`/spider/s3/{bucket-name}?empty`	Example
Force Delete Bucket	DELETE	`/spider/s3/{bucket-name}?force`	Example

Note: CB-Spider special features are unique to CB-Spider and not part of AWS S3 standard.

API Usage Examples

1. Bucket Management

List Buckets

Prerequisites: Create a bucket and upload a sample object so the response contains meaningful data.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload a sample object
echo "Hello CB-Spider S3 Test!" > /tmp/sample.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/sample.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/sample.txt -H "Content-Type: text/plain"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3?ConnectionName=aws-config01"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Owner>
    <ID>aws-config01</ID>
    <DisplayName>aws-config01</DisplayName>
  </Owner>
  <Buckets>
    <Bucket>
      <Name>spider-test-bucket</Name>
      <CreationDate>2026-03-31T01:05:32Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>

Cleanup (optional): Remove the created resources if no longer needed.

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/sample.txt?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Create Bucket

Request:

curl -u admin:your-secure-password -i -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Response:

HTTP/1.1 200 OK
Location: /spider-test-bucket
Vary: Origin
X-Amz-Id-2: 1774919313
X-Amz-Request-Id: 1774919313
Date: Tue, 31 Mar

Get Bucket Info

Prerequisites: Create a bucket and upload a sample object so the Contents field is populated.

# Upload a sample object
echo "Hello CB-Spider S3 Test!" > /tmp/sample.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/sample.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/sample.txt -H "Content-Type: text/plain"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>spider-test-bucket</Name>
  <Prefix></Prefix>
  <Marker></Marker>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
  <Contents>
    <Key>sample.txt</Key>
    <LastModified>2026-03-31T01:09:11Z</LastModified>
    <ETag>d90b98bda557e77b5401019d4da1c383</ETag>
    <Size>25</Size>
    <StorageClass>STANDARD</StorageClass>
  </Contents>
</ListBucketResult>

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/sample.txt?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Check Bucket Exists (HEAD)

Prerequisites: Create a bucket to test the existing-bucket response.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Request:

curl -u admin:your-secure-password -I "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Response for existing bucket:

HTTP/1.1 200 OK
Vary: Origin
X-Amz-Id-2: 1774919429
X-Amz-Request-Id: 1774919429
Date: Tue, 31 Mar 2026 01:10:29 GMT

Response for non-existing bucket:

HTTP/1.1 403 Forbidden
Vary: Origin
Date: Tue, 31 Mar 2026 01:10:51 GMT

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Get Bucket Location

Prerequisites: Create a bucket first.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?location&ConnectionName=aws-config01"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<LocationConstraint xmlns="http://s3.amazonaws.com/doc/2006-03-01/">ap-southeast-2</LocationConstraint>

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Delete Bucket

Prerequisites: Create a bucket first.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Request:

curl -u admin:your-secure-password -i -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1774919561
X-Amz-Request-Id: 1774919561
Date: Tue, 31 Mar 2026 01:12:41 GMT

2. Object Management

Upload Object (From File)

Prerequisites: Create a bucket and prepare a local file.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Prepare local file
echo "Hello CB-Spider S3 Test!" > /tmp/local-file.txt

Request:

curl -u admin:your-secure-password -i -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/local-file.txt \
  -H "Content-Type: text/plain"

Response:

HTTP/1.1 200 OK
Etag: d90b98bda557e77b5401019d4da1c383
Vary: Origin
X-Amz-Id-2: 1774919621
X-Amz-Request-Id: 1774919621
Date: Tue, 31 Mar 2026 01:13:41 GMT
Content-Length: 0

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Upload Object (Form Data)

Prerequisites: Create a bucket and prepare a local file.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Prepare local file
echo "Hello CB-Spider S3 Test!" > /tmp/local-file.txt

Request:

curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" \
  -F "key=test-file.txt" \
  -F "file=@/tmp/local-file.txt"

Response:

<?xml version="1.0" encoding="UTF-8"?>
<UploadResult>
  <Bucket>spider-test-bucket</Bucket>
  <Key>test-file.txt</Key>
</UploadResult>

Notes:

The key form field specifies the object name/key
The file form field contains the file to upload
Optional Content-Type form field can be specified
Supports browser-based HTML form uploads

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Download Object

Prerequisites: Create a bucket and upload a sample object.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload object
echo "Hello CB-Spider S3 Test!" > /tmp/local-file.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/local-file.txt -H "Content-Type: text/plain"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  -o /tmp/downloaded-file.txt

Response:

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Get Object Info (HEAD)

Prerequisites: Create a bucket and upload a sample object.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload object
echo "Hello CB-Spider S3 Test!" > /tmp/local-file.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/local-file.txt -H "Content-Type: text/plain"

Request:

curl -u admin:your-secure-password -I "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01"

Response:

HTTP/1.1 200 OK
Content-Length: 25
Content-Type: text/plain; charset=utf-8
Etag: d90b98bda557e77b5401019d4da1c383
Last-Modified: Tue, 31 Mar 2026 01:19:27 GMT
Vary: Origin
X-Amz-Id-2: 1774919968
X-Amz-Request-Id: 1774919968
Date: Tue, 31 Mar 2026 01:19:28 GMT

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Delete Object

Prerequisites: Create a bucket and upload an object.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload object
echo "Hello CB-Spider S3 Test!" > /tmp/local-file.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/local-file.txt -H "Content-Type: text/plain"

Request:

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1774920021
X-Amz-Request-Id: 1774920021
Date: Tue, 31 Mar 2026 01:20:21 GMT

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Delete Multiple Objects

Prerequisites: Create a bucket and upload multiple objects.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload objects
for f in file1.txt file2.txt file3.txt; do
  echo "Content of $f" > /tmp/$f
  curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/$f?ConnectionName=aws-config01" \
    --data-binary @/tmp/$f -H "Content-Type: text/plain"
done

Request:

curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket?delete&ConnectionName=aws-config01" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<Delete xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Object>
    <Key>file1.txt</Key>
  </Object>
  <Object>
    <Key>file2.txt</Key>
  </Object>
  <Object>
    <Key>file3.txt</Key>
  </Object>
</Delete>'

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<DeleteResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Deleted>
    <Key>file1.txt</Key>
  </Deleted>
  <Deleted>
    <Key>file2.txt</Key>
  </Deleted>
  <Deleted>
    <Key>file3.txt</Key>
  </Deleted>
</DeleteResult>

Notes:

The delete query parameter is required
Request body must be XML with proper namespace declaration
Supports deleting multiple objects in a single request
Returns detailed results for each deleted object

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

3. Multipart Upload

Important Notes:

Part Size Limit: Each part (except the last one) must be at least 5MB in size for AWS S3 compatibility
ETag Format: Multipart files have ETag format as {hash}-{part_count} (e.g., 50f9c71a2e1d2cd7706f6dfd0b12c9fd-3)
Upload ID Lifecycle: CB-Spider automatically invalidates Upload IDs after a period of inactivity (unlike AWS S3 standard where Upload IDs never expire). It's recommended to upload parts immediately after initiation

Initiate Multipart Upload

Prerequisites: Create a bucket first.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Request:

curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploads&ConnectionName=aws-config01" \
  -H "Content-Type: application/octet-stream"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Bucket>spider-test-bucket</Bucket>
  <Key>large-multipart-file.bin</Key>
  <UploadId>343OwmR5colrDYyajh4V4VK5zgOjtVzZYOR_I.EDaJ_JSQuYjo9uj12zFd3VwXhDXeg7gaysMQk6iuHmXSPWXIzBqBBVmvM6BvVJWvTXI.noOIWth5zydwoYAhKSGhhXIQLg6TxjO3eybjPzApfkoA--</UploadId>
</InitiateMultipartUploadResult>

Upload Part

Prerequisites: Initiate a multipart upload and note the UploadId from the response. Prepare a large part file (at least 5MB, except the last part).

# Create a 6MB test file
dd if=/dev/urandom of=/tmp/large-part1.bin bs=1M count=6

Request:

Note: Replace the uploadId value below with the UploadId returned from the Initiate Multipart Upload response above.

# Upload Part 1 (6MB)
curl -u admin:your-secure-password -i -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId from Initiate response}&partNumber=1&ConnectionName=aws-config01" \
  --data-binary @/tmp/large-part1.bin \
  -H "Content-Type: application/octet-stream"

Response:

HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Etag: 2b669f2a67dc844c135b3cf21a5d030e
Vary: Origin
X-Amz-Id-2: 1774920809
X-Amz-Request-Id: 1774920809
Date: Tue, 31 Mar 2026 01:33:29 GMT
Content-Length: 0

Complete Multipart Upload

Prerequisites: Upload all required parts and have their ETags ready.

Note: Replace {UploadId} with the UploadId from the Initiate Multipart Upload response, and replace {ETag of Part 1} with the ETag value from the Upload Part response (e.g., "2b669f2a67dc844c135b3cf21a5d030e").

Request:

# Replace {UploadId} with UploadId from Initiate Multipart Upload response
# Replace {ETag of Part 1} with ETag from Upload Part response (e.g. "2b669f2a67dc844c135b3cf21a5d030e")
curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId from Initiate response}&ConnectionName=aws-config01" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<CompleteMultipartUpload>
  <Part>
    <PartNumber>1</PartNumber>
    <ETag>{ETag of Part 1}</ETag>
  </Part>
</CompleteMultipartUpload>'

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Location>/spider-test-bucket/large-multipart-file.bin</Location>
  <Bucket>spider-test-bucket</Bucket>
  <Key>large-multipart-file.bin</Key>
  <ETag>1cdb33b6f6a46305cdb2500ab69d1c5b-1</ETag>
</CompleteMultipartUploadResult>

Note: The ETag ending with -1 indicates this file was assembled from 1 part.

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Abort Multipart Upload

Prerequisites: Initiate a multipart upload and note the UploadId.

# Create bucket if needed
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Initiate upload
curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket/abort-test-file.bin?uploads&ConnectionName=aws-config01" \
  -H "Content-Type: application/octet-stream"

Request:

Note: Replace {UploadId} with the UploadId returned from the Initiate Multipart Upload response above.

curl -u admin:your-secure-password -i -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/abort-test-file.bin?uploadId={UploadId from Initiate response}&ConnectionName=aws-config01"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1775198576
X-Amz-Request-Id: 1775198576
Date: Fri, 03 Apr 2026 06:42:56 GMT

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

List Parts

Prerequisites: Initiate a multipart upload, upload at least one part, and note the UploadId.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Initiate upload and capture UploadId
curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploads&ConnectionName=aws-config01" \
  -H "Content-Type: application/octet-stream"

# Upload Part 1 (6MB) using the UploadId from above
dd if=/dev/urandom of=/tmp/large-part1.bin bs=1M count=6
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId}&partNumber=1&ConnectionName=aws-config01" \
  --data-binary @/tmp/large-part1.bin -H "Content-Type: application/octet-stream"

Request:

Note: Replace {UploadId} with the UploadId returned from the Initiate Multipart Upload response in the Prerequisites above.

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId from Initiate response}&list-type=parts&ConnectionName=aws-config01"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<ListPartsResult>
  <Bucket>spider-test-bucket</Bucket>
  <Key>large-multipart-file.bin</Key>
  <UploadId>ZnkEEh._l9oTW2PiMRffyqak75ujkL_oX9gObXos1mIiAKJxCWii3fM8CsCEOXm_51caZ1yQ2fJYinYZNmqa5sIZszPMMlyDZP56v4I5efwJZ4Nf987uAMwsf0nsgmjgHEEMKqJ60YPs4ZfYooJ9TA--</UploadId>
  <PartNumberMarker>0</PartNumberMarker>
  <NextPartNumberMarker>1</NextPartNumberMarker>
  <MaxParts>1000</MaxParts>
  <IsTruncated>false</IsTruncated>
  <Part>
    <PartNumber>1</PartNumber>
    <LastModified>2026-04-03T06:53:22Z</LastModified>
    <ETag>&#34;6232919ebd4d085e3e635766b3f3e21a&#34;</ETag>
    <Size>6291456</Size>
  </Part>
  <Initiator>
    <ID>arn:aws:iam::123456789123:user/spider</ID>
    <DisplayName>spider</DisplayName>
  </Initiator>
  <Owner>
    <ID>7d9f77b078e963b2aca023ffa7dc39a3cac3b138397f2214d2d66e33bee3384d</ID>
    <DisplayName></DisplayName>
  </Owner>
  <StorageClass>STANDARD</StorageClass>
</ListPartsResult>

Optional Parameters:

part-number-marker: Start listing after this part number
max-parts: Maximum number of parts to return (default: 1000)

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId}&ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

List Multipart Uploads

Prerequisites: Initiate at least one multipart upload (do not complete or abort it).

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Initiate upload (leave it pending)
curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploads&ConnectionName=aws-config01" \
  -H "Content-Type: application/octet-stream"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?uploads&ConnectionName=aws-config01"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<ListMultipartUploadsResult>
  <Bucket>spider-test-bucket</Bucket>
  <KeyMarker></KeyMarker>
  <UploadIdMarker></UploadIdMarker>
  <NextKeyMarker>large-multipart-file.bin</NextKeyMarker>
  <NextUploadIdMarker>cR3KvhAF697kiQK1J.pKKusbjueVTwePYSUKDQW9NElQwuxUEm0dObjaDSDv4q9ZP.M2MQ_XyTE2f9DZQ1li3imz78Kmo5AJ9eATxoA88jMU_CsWqSTEpVUqWbjSqw3ig3g0Bzh_8S.EF84wZ0lugg--</NextUploadIdMarker>
  <MaxUploads>1000</MaxUploads>
  <IsTruncated>false</IsTruncated>
  <Upload>
    <Key>large-multipart-file.bin</Key>
    <UploadId>cR3KvhAF697kiQK1J.pKKusbjueVTwePYSUKDQW9NElQwuxUEm0dObjaDSDv4q9ZP.M2MQ_XyTE2f9DZQ1li3imz78Kmo5AJ9eATxoA88jMU_CsWqSTEpVUqWbjSqw3ig3g0Bzh_8S.EF84wZ0lugg--</UploadId>
    <Initiated>2026-04-03T07:01:10Z</Initiated>
    <StorageClass>STANDARD</StorageClass>
    <Initiator>
      <ID>arn:aws:iam::123456789123:user/spider</ID>
      <DisplayName>spider</DisplayName>
    </Initiator>
    <Owner>
      <ID>7d9f77b078e963b2aca023ffa7dc39a3cac3b138397f2214d2d66e33bee3384d</ID>
      <DisplayName></DisplayName>
    </Owner>
  </Upload>
  <Prefix></Prefix>
  <Delimiter></Delimiter>
</ListMultipartUploadsResult>

Optional Parameters:

prefix: Only list uploads with keys beginning with this prefix
key-marker: Start listing after this key name
upload-id-marker: Start listing after this upload ID
delimiter: Character used to group keys
max-uploads: Maximum number of uploads to return (default: 1000)

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId}&ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Multipart Upload Technical Notes

Upload ID Lifecycle:

Upload IDs must be used immediately after initiation
CB-Spider automatically invalidates unused Upload IDs (differs from AWS S3 standard)
Always initiate a new multipart upload for each file upload session

Part Size Requirements:

Each part (except the last) must be at least 5MB for AWS S3 compatibility

ETag Format:

Single uploads: Standard MD5 hash (e.g., "da6a0d097e307ac52ed9b4ad551801fc")
Multipart uploads: Hash with part count suffix (e.g., "50f9c71a2e1d2cd7706f6dfd0b12c9fd-3")

4. Versioning Management

Get Bucket Versioning

Prerequisites: Create a bucket and enable versioning.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Enable versioning
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning&ConnectionName=aws-config01" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?><VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Status>Enabled</Status></VersioningConfiguration>'

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?versioning&ConnectionName=aws-config01"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<VersioningConfiguration>
  <Status>Enabled</Status>
</VersioningConfiguration>

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force&ConnectionName=aws-config01"

Set Bucket Versioning

Prerequisites: Create a bucket first.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Request:

curl -u admin:your-secure-password -i -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning&ConnectionName=aws-config01" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Status>Enabled</Status>
</VersioningConfiguration>'

Response:

HTTP/1.1 200 OK
Vary: Origin
X-Amz-Id-2: 1775200756
X-Amz-Request-Id: 1775200756
Date: Fri, 03 Apr 2026 07:19:16 GMT
Content-Length: 0

To suspend versioning:

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning&ConnectionName=aws-config01" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Status>Suspended</Status>
</VersioningConfiguration>'

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force&ConnectionName=aws-config01"

List Object Versions

Prerequisites: Create a bucket, enable versioning, and upload the same object multiple times to create multiple versions.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Enable versioning
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning&ConnectionName=aws-config01" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?><VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Status>Enabled</Status></VersioningConfiguration>'
# Upload version 1
echo "Version 1 content" > /tmp/file-v1.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/file-v1.txt -H "Content-Type: text/plain"
# Upload version 2 (same key)
echo "Version 2 content" > /tmp/file-v2.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/file-v2.txt -H "Content-Type: text/plain"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?versions&ConnectionName=aws-config01"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<ListVersionsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>spider-test-bucket</Name>
  <Prefix></Prefix>
  <KeyMarker></KeyMarker>
  <VersionIdMarker></VersionIdMarker>
  <NextKeyMarker></NextKeyMarker>
  <NextVersionIdMarker></NextVersionIdMarker>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
  <Version>
    <Key>same-file.txt</Key>
    <VersionId>QA4Y6zq1j2TOkjxkFWDpFxpbOh5JtTCJ</VersionId>
    <IsLatest>true</IsLatest>
    <LastModified>2026-04-03T07:20:54Z</LastModified>
    <ETag>e0ef76395fbb2af0b3213158344bd881</ETag>
    <Size>18</Size>
    <StorageClass>STANDARD</StorageClass>
    <Owner>
      <ID>aws-config01</ID>
      <DisplayName>aws-config01</DisplayName>
    </Owner>
  </Version>
  <Version>
    <Key>same-file.txt</Key>
    <VersionId>5SuasC.nPRYm75A5QowJ5oEpwA3UoCWB</VersionId>
    <IsLatest>false</IsLatest>
    <LastModified>2026-04-03T07:20:53Z</LastModified>
    <ETag>0e8e102783f3fd35b701b8de312415e5</ETag>
    <Size>18</Size>
    <StorageClass>STANDARD</StorageClass>
    <Owner>
      <ID>aws-config01</ID>
      <DisplayName>aws-config01</DisplayName>
    </Owner>
  </Version>
</ListVersionsResult>

Cleanup (optional):

# Force delete bucket with all versions
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force&ConnectionName=aws-config01"

Delete Versioned Object

Prerequisites: Create a bucket with versioning enabled, upload an object twice (same key), then list versions to get the versionId.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Enable versioning
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning&ConnectionName=aws-config01" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?><VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Status>Enabled</Status></VersioningConfiguration>'
# Upload version 1
echo "Version 1 content" > /tmp/file-v1.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/file-v1.txt -H "Content-Type: text/plain"
# Upload version 2 (same key)
echo "Version 2 content" > /tmp/file-v2.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/file-v2.txt -H "Content-Type: text/plain"

List Versions Response Example (note the VersionId values — use one in the Request below): curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?versions&ConnectionName=aws-config01"

```xml
<?xml version="1.0" encoding="UTF-8"?>
<ListVersionsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>spider-test-bucket</Name>
  <Prefix></Prefix>
  <KeyMarker></KeyMarker>
  <VersionIdMarker></VersionIdMarker>
  <NextKeyMarker></NextKeyMarker>
  <NextVersionIdMarker></NextVersionIdMarker>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
  <Version>
    <Key>test-file.txt</Key>
    <VersionId>uvB9NuGdLl.t2qgkGh1La.3CqwFqwHiA</VersionId>
    <IsLatest>true</IsLatest>
    <LastModified>2026-04-03T07:59:36Z</LastModified>
    <ETag>e0ef76395fbb2af0b3213158344bd881</ETag>
    <Size>18</Size>
    <StorageClass>STANDARD</StorageClass>
    <Owner>
      <ID>aws-config01</ID>
      <DisplayName>aws-config01</DisplayName>
    </Owner>
  </Version>
  <Version>
    <Key>test-file.txt</Key>
    <VersionId>rJYYT2RyYfaGDvxQNtPGOhtum._7M6CA</VersionId>
    <IsLatest>false</IsLatest>
    <LastModified>2026-04-03T07:59:36Z</LastModified>
    <ETag>0e8e102783f3fd35b701b8de312415e5</ETag>
    <Size>18</Size>
    <StorageClass>STANDARD</StorageClass>
    <Owner>
      <ID>aws-config01</ID>
      <DisplayName>aws-config01</DisplayName>
    </Owner>
  </Version>
</ListVersionsResult>

Request:

Note: Replace {versionId} with a VersionId from the List Versions response above.

curl -u admin:your-secure-password -i -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt?versionId={versionId}&ConnectionName=aws-config01"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1775203397
X-Amz-Request-Id: 1775203397
Date: Fri, 03 Apr 2026 08:03:17 GMT

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force&ConnectionName=aws-config01"

Versioning Important Notes

Creating Multiple Versions: When versioning is enabled, uploading an object with the same key creates a new version:

# Upload version 1
echo "Version 1 content" > /tmp/file-v1.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/file-v1.txt -H "Content-Type: text/plain"

# Upload version 2 (same key)
echo "Version 2 content" > /tmp/file-v2.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/file-v2.txt -H "Content-Type: text/plain"

Versioning States:

Enabled: New versions are created for each upload
Suspended: New uploads don't create versions (but existing versions remain)
Unversioned: Default state (no versioning)

5. CORS Management

Get CORS Configuration

Prerequisites: Create a bucket and configure CORS so the response shows the active CORS config.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Set CORS configuration
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?><CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod><AllowedMethod>PUT</AllowedMethod><AllowedHeader>*</AllowedHeader><MaxAgeSeconds>3000</MaxAgeSeconds></CORSRule></CORSConfiguration>'

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01"

Response Example (when CORS is configured):

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
    <ExposeHeader>ETag</ExposeHeader>
    <ExposeHeader>x-amz-server-side-encryption</ExposeHeader>
    <ExposeHeader>x-amz-request-id</ExposeHeader>
    <ExposeHeader>x-amz-id-2</ExposeHeader>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
  </CORSRule>
</CORSConfiguration>

Error Response (when CORS is not configured):

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>NoSuchCORSConfiguration</Code>
  <Message>The CORS configuration for bucket &#39;spider-test-bucket&#39; does not exist</Message>
  <Resource>/spider-test-bucket</Resource>
  <RequestId>1775203711</RequestId>
</Error>

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Set CORS Configuration

Prerequisites: Create a bucket first.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Request:

Basic CORS Configuration:

curl -u admin:your-secure-password -i -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration>
  <CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedMethod>DELETE</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
  </CORSRule>
</CORSConfiguration>'

Advanced CORS Configuration (Multiple Rules):

curl -u admin:your-secure-password -i -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration>
  <CORSRule>
    <AllowedOrigin>https://example.com</AllowedOrigin>
    <AllowedOrigin>https://app.example.com</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedHeader>Content-Type</AllowedHeader>
    <AllowedHeader>Authorization</AllowedHeader>
    <ExposeHeader>ETag</ExposeHeader>
    <MaxAgeSeconds>1800</MaxAgeSeconds>
  </CORSRule>
  <CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <MaxAgeSeconds>300</MaxAgeSeconds>
  </CORSRule>
</CORSConfiguration>'

Response:

HTTP/1.1 200 OK
Vary: Origin
X-Amz-Id-2: 1775204010
X-Amz-Request-Id: 1775204010
Date: Fri, 03 Apr 2026 08:13:30 GMT
Content-Length: 0

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Test CORS with OPTIONS Request

Note: CORS preflight requests are sent by browsers without credentials. Use plain curl (no auth).

Prerequisites: Create a bucket, upload an object, and configure CORS.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload test object
echo "Hello CB-Spider S3 Test!" > /tmp/local-file.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/local-file.txt -H "Content-Type: text/plain"
# Set CORS configuration
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?><CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod><AllowedMethod>PUT</AllowedMethod><AllowedHeader>*</AllowedHeader><MaxAgeSeconds>3000</MaxAgeSeconds></CORSRule></CORSConfiguration>'

Request:

curl -u admin:your-secure-password -X OPTIONS "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  -H "Origin: https://example.com" \
  -H "Access-Control-Request-Method: PUT" \
  -H "Access-Control-Request-Headers: Content-Type" \
  -v

Response:

*   Trying 127.0.0.1:1024...
* Connected to localhost (127.0.0.1) port 1024 (#0)
* Server auth using Basic with user 'admin'
> OPTIONS /spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01 HTTP/1.1
> Host: localhost:1024
> Authorization: Basic YWRtaW46eW91ci1zZWN1cmUtcGFzc3dvcmQ=
> User-Agent: curl/7.81.0
> Accept: */*
> Origin: https://example.com
> Access-Control-Request-Method: PUT
> Access-Control-Request-Headers: Content-Type
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 204 No Content
< Access-Control-Allow-Headers: Content-Type
< Access-Control-Allow-Methods: GET,HEAD,PUT,PATCH,POST,DELETE
< Access-Control-Allow-Origin: *
< Allow: OPTIONS, DELETE, GET, HEAD, POST, PUT
< Vary: Origin
< Vary: Access-Control-Request-Method
< Vary: Access-Control-Request-Headers
< Date: Fri, 03 Apr 2026 08:18:24 GMT
<
* Connection #0 to host localhost left intact

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Delete CORS Configuration

Prerequisites: Create a bucket and configure CORS first.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Set CORS first
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01" \
  -H "Content-Type: application/xml" \
  -d '<?xml version="1.0" encoding="UTF-8"?><CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod><AllowedHeader>*</AllowedHeader></CORSRule></CORSConfiguration>'

Request:

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1775204460
X-Amz-Request-Id: 1775204460
Date: Fri, 03 Apr 2026 08:21:00 GMT

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

CORS Configuration Elements

Element	Description	Required	Example
`AllowedOrigin`	Specifies the origins allowed to access the bucket	Yes	`*` or `https://example.com`
`AllowedMethod`	Specifies the HTTP methods allowed	Yes	`GET`, `PUT`, `POST`, `DELETE`
`AllowedHeader`	Specifies the headers allowed in the request	No	`*`, `Content-Type`, `Authorization`
`ExposeHeader`	Specifies which headers are exposed to the browser	No	`ETag`, `x-amz-request-id`
`MaxAgeSeconds`	Specifies how long the browser can cache the preflight response	No	`3600` (1 hour)

CORS Configuration Examples

Allow All Origins (Development):

<CORSRule>
  <AllowedOrigin>*</AllowedOrigin>
  <AllowedMethod>GET</AllowedMethod>
  <AllowedMethod>PUT</AllowedMethod>
  <AllowedMethod>POST</AllowedMethod>
  <AllowedMethod>DELETE</AllowedMethod>
  <AllowedHeader>*</AllowedHeader>
  <MaxAgeSeconds>3000</MaxAgeSeconds>
</CORSRule>

Production Configuration (Specific Domains):

<CORSRule>
  <AllowedOrigin>https://example.com</AllowedOrigin>
  <AllowedOrigin>https://app.example.com</AllowedOrigin>
  <AllowedMethod>GET</AllowedMethod>
  <AllowedMethod>PUT</AllowedMethod>
  <AllowedHeader>Content-Type</AllowedHeader>
  <AllowedHeader>Authorization</AllowedHeader>
  <ExposeHeader>ETag</ExposeHeader>
  <MaxAgeSeconds>1800</MaxAgeSeconds>
</CORSRule>

Read-Only Access:

<CORSRule>
  <AllowedOrigin>*</AllowedOrigin>
  <AllowedMethod>GET</AllowedMethod>
  <AllowedHeader>Range</AllowedHeader>
  <ExposeHeader>Content-Length</ExposeHeader>
  <ExposeHeader>Content-Range</ExposeHeader>
  <MaxAgeSeconds>300</MaxAgeSeconds>
</CORSRule>

CORS Best Practices

Security: Use specific origins instead of * in production
Performance: Set appropriate MaxAgeSeconds to reduce preflight requests
Headers: Only allow necessary headers to minimize security risks
Methods: Only specify required HTTP methods
Testing: Use browser developer tools to verify CORS behavior

6. CB-Spider Special Features

Generate PreSigned URL (Download)

Prerequisites: Create a bucket and upload a sample object.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload object
echo "Hello CB-Spider S3 Test!" > /tmp/local-file.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/local-file.txt -H "Content-Type: text/plain"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/presigned/download/spider-test-bucket/test-file.txt?ConnectionName=aws-config01&expires=3600"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<PresignedURLResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <PresignedURL>https://spider-test-bucket.s3.dualstack.ap-southeast-2.amazonaws.com/test-file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=AKIAZH5OR44MIE5O2C6Y%2F20260325%2Fap-southeast-2%2Fs3%2Faws4_request&amp;X-Amz-Date=20260325T055247Z&amp;X-Amz-Expires=3600&amp;X-Amz-SignedHeaders=host&amp;X-Amz-Signature=a08f3ab8db012c6f983a4a46e0f8dfe39cc9184e17e29c94cae82ba0b3a44c30</PresignedURL>
  <Expires>3600</Expires>
  <Method>GET</Method>
</PresignedURLResult>

Optional Parameters:

expires: URL expiration time in seconds (default: 3600)
method: HTTP method for the URL (default: GET, also supports PUT)
response-content-disposition: Content-Disposition header for downloads

Usage Example:

# Download a file using the generated PreSigned URL
curl -X GET "{PresignedURL}" -o /tmp/downloaded-file.txt

Note: This feature is unique to CB-Spider and not part of AWS S3 standard.

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Generate PreSigned URL (Upload)

Prerequisites: Create a bucket first.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/presigned/upload/spider-test-bucket/test-file.txt?ConnectionName=aws-config01&expires=3600"

Response Example:

<?xml version="1.0" encoding="UTF-8"?>
<PresignedURLResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <PresignedURL>https://spider-test-bucket.s3.dualstack.ap-southeast-2.amazonaws.com/test-file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=AKIAZH5OR44MIE5O2C6Y%2F20260325%2Fap-southeast-2%2Fs3%2Faws4_request&amp;X-Amz-Date=20260325T055247Z&amp;X-Amz-Expires=3600&amp;X-Amz-SignedHeaders=host&amp;X-Amz-Signature=144d1fb8b01c1aba35d599ca3f1268f6cd934fef255cc03d781e1f6975ee2d7f</PresignedURL>
  <Expires>3600</Expires>
  <Method>PUT</Method>
</PresignedURLResult>

Optional Parameters:

expires: URL expiration time in seconds (default: 3600)

Usage Example:

# Create a file to upload
echo "Hello CB-Spider S3 SigV4 Test!" > /tmp/local-file.txt
# Upload a file using the generated PreSigned URL
curl -X PUT "{PresignedURL}" --data-binary "@/tmp/local-file.txt" -H "Content-Type: text/plain"

Note: This feature is unique to CB-Spider and not part of AWS S3 standard.

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Force Empty Bucket

Prerequisites: Create a bucket and upload some objects.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload objects
echo "file1 content" > /tmp/force-empty-file1.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/force-empty-file1.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/force-empty-file1.txt -H "Content-Type: text/plain"
echo "file2 content" > /tmp/force-empty-file2.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/force-empty-file2.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/force-empty-file2.txt -H "Content-Type: text/plain"

Request:

curl -u admin:your-secure-password -i -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?empty&ConnectionName=aws-config01"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1775205378
X-Amz-Request-Id: 1775205378
Date: Fri, 03 Apr 2026 08:36:18 GMT

JSON Error Response (when the ?empty parameter is not used):

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>BucketNotEmpty</Code>
  <Message>The bucket you tried to delete is not empty. It contains 2 objects. Use force=true parameter to force delete.</Message>
  <Resource>/spider-test-bucket</Resource>
  <RequestId>1775205462</RequestId>
</Error>

Verification:

# Check that bucket is now empty
curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Empty Bucket Response:

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>spider-test-bucket</Name>
  <Prefix></Prefix>
  <Marker></Marker>
  <MaxKeys>1000</MaxKeys>
  <IsTruncated>false</IsTruncated>
</ListBucketResult>

Note: This feature is unique to CB-Spider and not part of AWS S3 standard. The ?empty query parameter is required as a safety mechanism.

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Force Delete Bucket

Prerequisites: Create a bucket and upload some objects.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload objects
echo "file1 content" > /tmp/force-del-file1.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/force-del-file1.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/force-del-file1.txt -H "Content-Type: text/plain"
echo "file2 content" > /tmp/force-del-file2.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/force-del-file2.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/force-del-file2.txt -H "Content-Type: text/plain"

Request:

curl -u admin:your-secure-password -i -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force&ConnectionName=aws-config01"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1775206684
X-Amz-Request-Id: 1775206684
Date: Fri, 03 Apr 2026 08:58:04 GMT

Error Response (bucket not empty, without force):

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>BucketNotEmpty</Code>
  <Message>The bucket you tried to delete is not empty. It contains 2 objects. Use force=true parameter to force delete.</Message>
  <Resource>/spider-test-bucket</Resource>
  <RequestId>1775206551</RequestId>
</Error>

Verification:

# Check that bucket no longer exists
curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Deleted Bucket Response:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>NoSuchBucket</Code>
  <Message>Resource &#39;spider-test-bucket&#39; does not exist in connection &#39;aws-config01&#39;</Message>
  <Resource>/spider-test-bucket</Resource>
  <RequestId>1775206707</RequestId>
</Error>

Note: This feature forcefully deletes the bucket and all its contents. This feature is unique to CB-Spider and not part of AWS S3 standard. The ?force query parameter is required as a safety mechanism.

CB‐Spider Object Storage API Guide (Basic Auth, XML Format) - cloud-barista/cb-spider GitHub Wiki

Related Links

CB-Spider Object Storage API Guide (Basic Auth / XML Format)

Key Features

Authentication

API List

1. Bucket Management

2. Object Management

3. Multipart Upload

4. Versioning Management

5. CORS Management

6. CB-Spider Special Features

API Usage Examples

1. Bucket Management

List Buckets

Create Bucket

Get Bucket Info

Check Bucket Exists (HEAD)

Get Bucket Location

Delete Bucket

2. Object Management

Upload Object (From File)

Upload Object (Form Data)

Download Object

Get Object Info (HEAD)

Delete Object

Delete Multiple Objects

3. Multipart Upload

Initiate Multipart Upload

Upload Part

Complete Multipart Upload

Abort Multipart Upload

List Parts

List Multipart Uploads

Multipart Upload Technical Notes

4. Versioning Management

Get Bucket Versioning

Set Bucket Versioning

List Object Versions

Delete Versioned Object

Versioning Important Notes

5. CORS Management

Get CORS Configuration

Set CORS Configuration

Test CORS with OPTIONS Request

Delete CORS Configuration

CORS Configuration Elements

CORS Configuration Examples

CORS Best Practices

6. CB-Spider Special Features

Generate PreSigned URL (Download)

Generate PreSigned URL (Upload)

Force Empty Bucket

Force Delete Bucket

Related Links

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️