Performance Measurements with Google CS - christianbaun/ossperf GitHub Wiki
Some performance results
Tested from Germany on March 30th 2017 from a Lenovo X240 Thinkpad. Internet connection was ADSL with 16 MBps downstream and 1 MBps upstream. The services (buckets) tested were located in the US.
Google Cloud Storage service
These options need to be modified inside the file ~/.s3cfg
in order to interact with Google Cloud Storage.
access_key = insert here your access key
secret_key = insert here your secret access key
host_base = storage.googleapis.com
host_bucket = %(bucket).storage.googleapis.com
5 Files, 1 Byte (2^0 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 1 -p
...
Required time to create the bucket: 1.291s
Required time to upload the files: 2.096s
Required time to download the files: 2.245s
Required time to erase the objects: 3.931s
Required time to erase the bucket: 1.092s
Required time to perform all S3-related operations: 10.655s
5 Files, 2 Byte (2^1 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 2 -p
...
Required time to create the bucket: 1.702s
Required time to upload the files: 2.037s
Required time to download the files: 2.198s
Required time to erase the objects: 3.670s
Required time to erase the bucket: 0.966s
Required time to perform all S3-related operations: 10.573s
5 Files, 4 Byte (2^2 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 4 -p
...
Required time to create the bucket: 1.479s
Required time to upload the files: 1.897s
Required time to download the files: 1.901s
Required time to erase the objects: 3.766s
Required time to erase the bucket: 1.033s
Required time to perform all S3-related operations: 10.076s
5 Files, 8 Byte (2^3 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 8 -p
...
Required time to create the bucket: 1.623s
Required time to upload the files: 2.295s
Required time to download the files: 1.854s
Required time to erase the objects: 3.751s
Required time to erase the bucket: 0.980s
Required time to perform all S3-related operations: 10.503s
5 Files, 16 Byte (2^4 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 16 -p
...
Required time to create the bucket: 1.581s
Required time to upload the files: 2.240s
Required time to download the files: 1.724s
Required time to erase the objects: 3.908s
Required time to erase the bucket: 0.924s
Required time to perform all S3-related operations: 10.377s
5 Files, 32 Byte (2^5 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 32 -p
...
Required time to create the bucket: 1.399s
Required time to upload the files: 2.283s
Required time to download the files: 1.945s
Required time to erase the objects: 3.966s
Required time to erase the bucket: 1.139s
Required time to perform all S3-related operations: 10.732s
5 Files, 64 Byte (2^6 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 64 -p
...
Required time to create the bucket: 1.589s
Required time to upload the files: 2.354s
Required time to download the files: 1.839s
Required time to erase the objects: 3.973s
Required time to erase the bucket: 2.351s
Required time to perform all S3-related operations: 12.106s
5 Files, 128 Byte (2^7 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 128 -p
...
Required time to create the bucket: 1.372s
Required time to upload the files: 2.396s
Required time to download the files: 1.873s
Required time to erase the objects: 5.442s
Required time to erase the bucket: 0.948s
Required time to perform all S3-related operations: 12.031s
5 Files, 256 Byte (2^8 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 256 -p
...
Required time to create the bucket: 1.789s
Required time to upload the files: 2.221s
Required time to download the files: 1.693s
Required time to erase the objects: 4.064s
Required time to erase the bucket: 0.914s
Required time to perform all S3-related operations: 10.681s
5 Files, 512 Byte (2^9 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 512 -p
...
Required time to create the bucket: 1.507s
Required time to upload the files: 2.236s
Required time to download the files: 2.100s
Required time to erase the objects: 4.487s
Required time to erase the bucket: 1.226s
Required time to perform all S3-related operations: 11.556s
5 Files, 1 kb (2^10 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 1024 -p
...
Required time to create the bucket: 1.415s
Required time to upload the files: 2.096s
Required time to download the files: 2.088s
Required time to erase the objects: 3.893s
Required time to erase the bucket: 0.920s
Required time to perform all S3-related operations: 10.412s
5 Files, 2 kb (2^11 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 2048 -p
...
Required time to create the bucket: 1.575s
Required time to upload the files: 2.440s
Required time to download the files: 2.076s
Required time to erase the objects: 4.069s
Required time to erase the bucket: 1.142s
Required time to perform all S3-related operations: 11.302s
5 Files, 4 kb (2^12 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 4096 -p
...
Required time to create the bucket: 1.497s
Required time to upload the files: 2.072s
Required time to download the files: 1.739s
Required time to erase the objects: 4.112s
Required time to erase the bucket: 1.275s
Required time to perform all S3-related operations: 10.695s
5 Files, 8 kb (2^13 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 8192 -p
...
Required time to create the bucket: 1.593s
Required time to upload the files: 2.269s
Required time to download the files: 2.047s
Required time to erase the objects: 4.205s
Required time to erase the bucket: 1.171s
Required time to perform all S3-related operations: 11.285s
5 Files, 16 kb (2^14 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 16384 -p
...
Required time to create the bucket: 1.558s
Required time to upload the files: 2.514s
Required time to download the files: 2.042s
Required time to erase the objects: 4.017s
Required time to erase the bucket: 1.094s
Required time to perform all S3-related operations: 11.225s
5 Files, 32 kb (2^15 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 32768 -p
...
Required time to create the bucket: 1.540s
Required time to upload the files: 2.492s
Required time to download the files: 2.287s
Required time to erase the objects: 4.212s
Required time to erase the bucket: 1.137s
Required time to perform all S3-related operations: 11.668s
5 Files, 64 kb (2^16 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 65536 -p
...
Required time to create the bucket: 1.669s
Required time to upload the files: 2.843s
Required time to download the files: 3.323s
Required time to erase the objects: 4.483s
Required time to erase the bucket: 1.121s
Required time to perform all S3-related operations: 13.439s
5 Files, 128 kb (2^17 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 131072 -p
...
Required time to create the bucket: 1.525s
Required time to upload the files: 3.701s
Required time to download the files: 2.373s
Required time to erase the objects: 4.153s
Required time to erase the bucket: 1.034s
Required time to perform all S3-related operations: 12.786s
5 Files, 256 kb (2^18 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 262144 -p
...
Required time to create the bucket: 1.484s
Required time to upload the files: 4.552s
Required time to download the files: 2.303s
Required time to erase the objects: 4.189s
Required time to erase the bucket: 1.226s
Required time to perform all S3-related operations: 13.754s
5 Files, 512 kb (2^19 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 524288 -p
...
Required time to create the bucket: 1.605s
Required time to upload the files: 6.153s
Required time to download the files: 2.696s
Required time to erase the objects: 4.128s
Required time to erase the bucket: 1.130s
Required time to perform all S3-related operations: 15.712s
5 Files, 1 MB (2^20 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 1048576 -p
...
Required time to create the bucket: 1.592s
Required time to upload the files: 13.479s
Required time to download the files: 3.278s
Required time to erase the objects: 3.950s
Required time to erase the bucket: 1.119s
Required time to perform all S3-related operations: 23.418s
5 Files, 2 MB (2^21 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 2097152 -p
...
Required time to create the bucket: 1.592s
Required time to upload the files: 13.479s
Required time to download the files: 3.278s
Required time to erase the objects: 3.950s
Required time to erase the bucket: 1.119s
Required time to perform all S3-related operations: 23.418s
5 Files, 4 MB (2^22 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 4194304 -p
...
Required time to create the bucket: 1.445s
Required time to upload the files: 19.873s
Required time to download the files: 10.074s
Required time to erase the objects: 4.046s
Required time to erase the bucket: 0.949s
Required time to perform all S3-related operations: 36.387s
5 Files, 8 MB (2^23 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 8388608 -p
...
Required time to create the bucket: 1.469s
Required time to upload the files: 75.071s
Required time to download the files: 16.321s
Required time to erase the objects: 3.841s
Required time to erase the bucket: 1.223s
Required time to perform all S3-related operations: 97.925s
5 Files, 16 MB (2^24 Byte) each, parallel transfer
$ ./s3perf.sh -n 5 -s 16777216 -p
...
ERROR: S3 error: 403 (SignatureDoesNotMatch): The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.
ERROR: S3 error: 403 (SignatureDoesNotMatch): The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.
ERROR: S3 error: 403 (SignatureDoesNotMatch): The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.
ERROR: S3 error: 403 (SignatureDoesNotMatch): The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.
ERROR: S3 error: 403 (SignatureDoesNotMatch): The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.