Available Space versus Usable Space - cniackz/public GitHub Wiki

Objective:

How to calculate the actual usable capacity of a Tenant

Initial Configuration:

Tenant info:

root@ubuntu:/# mc admin info myminio 
●  celis-pool-0-0.celis-hl.celis.svc.cluster.local:9000
   Uptime: 2 minutes 
   Version: 2023-09-30T07:02:29Z
   Network: 3/3 OK 
   Drives: 4/4 OK 
   Pool: 1

●  celis-pool-0-1.celis-hl.celis.svc.cluster.local:9000
   Uptime: 2 minutes 
   Version: 2023-09-30T07:02:29Z
   Network: 3/3 OK 
   Drives: 4/4 OK 
   Pool: 1

●  celis-pool-0-2.celis-hl.celis.svc.cluster.local:9000
   Uptime: 2 minutes 
   Version: 2023-09-30T07:02:29Z
   Network: 3/3 OK 
   Drives: 4/4 OK 
   Pool: 1

Pools:
   1st, Erasure sets: 1, Drives per erasure set: 12

12 drives online, 0 drives offline

Initial values:

root@ubuntu:/# mc admin info myminio --json | jq . | grep -i availspace | cut -d':' -f2- | sed -e "s/,$//" | numfmt --to=iec-i --suffix=B --format="%9.2f"
1023.93MiB
1023.93MiB
1023.93MiB
1023.93MiB
1023.93MiB
1023.93MiB
1023.93MiB
1023.93MiB
1023.93MiB
1023.93MiB
1023.93MiB
1023.93MiB

https://www.wolframalpha.com/input?i=1023.93MiB+*+12
Total is 12287 MiB (mebibytes)
Total is 12 GiB (gibibytes)

root@ubuntu:/# mc admin info myminio --json | jq . | grep -i totalspace | cut -d':' -f2- | sed -e "s/,$//" | numfmt --to=iec-i --suffix=B --format="%9.2f"
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB

root@ubuntu:/# mc admin info myminio --json | jq . | grep -i usedspace | cut -d':' -f2- | sed -e "s/,$//" | numfmt --to=iec-i --suffix=B --format="%9.2f"
 68.00KiB
 68.00KiB
 68.00KiB
 68.00KiB
 60.00KiB
 60.00KiB
 60.00KiB
 60.00KiB
 68.00KiB
 68.00KiB
 68.00KiB
 68.00KiB

Total is 816 KiB (kibibytes)

We upload 1GiB file:

root@ubuntu:/# mc mb myminio/celis 
Bucket created successfully `myminio/celis`.
root@ubuntu:/# dd if=/dev/urandom of=sample.txt bs=1G count=1
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 4.96406 s, 216 MB/s
root@ubuntu:/# ls -all sample.txt 
-rw-r--r-- 1 root root 1073741824 Oct  5 18:13 sample.txt
root@ubuntu:/# mc cp sample.txt myminio/celis
/sample.txt:                         1.00 GiB / 1.00 GiB ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 293.97 MiB/s 3sroot@ubuntu:/# 

Values after the 1GiB Upload

root@ubuntu:/# mc admin info myminio --json | jq . | grep -i availspace | cut -d':' -f2- | sed -e "s/,$//" | numfmt --to=iec-i --suffix=B --format="%9.2f"
895.64MiB
895.64MiB
895.64MiB
895.64MiB
895.64MiB
895.64MiB
895.64MiB
895.64MiB
895.65MiB
895.65MiB
895.65MiB
895.65MiB

Total is 10748 MiB (mebibytes)
Total is 10.5 GiB (gibibytes)

root@ubuntu:/# mc admin info myminio --json | jq . | grep -i totalspace | cut -d':' -f2- | sed -e "s/,$//" | numfmt --to=iec-i --suffix=B --format="%9.2f"
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB
  1.00GiB

root@ubuntu:/# mc admin info myminio --json | jq . | grep -i usedspace | cut -d':' -f2- | sed -e "s/,$//" | numfmt --to=iec-i --suffix=B --format="%9.2f"
128.38MiB
128.38MiB
128.38MiB
128.38MiB
128.38MiB
128.38MiB
128.38MiB
128.38MiB
128.38MiB
128.38MiB
128.38MiB
128.38MiB

Total is 1541 MiB (mebibytes)
Total is 1.505 GiB (gibibytes)

Usable Capacity

Usable Space = (Available Space) * (Storage Efficiency)
Usable Space = (10.5 GiB) * (8/12) = 7 GiB
                   |           |     |
                   |           |     |____ Total Usable space
                   |           |
                   |           |___ (Total Usable Capacity/Total Raw Capacity)
                   |
                   |___ The available space after we upload an Object of 1 GiB

How usable capacity would change:

1 GB Object -> Occupies  1.5 GiB of actual space due to metadata -> 7 GiB are usable for data only
2 GB Object -> Occupies  3.0 GiB of actual space due to metadata -> 6 GiB are usable for data only
3 GB Object -> Occupies  4.5 GiB of actual space due to metadata -> 5 GiB are usable for data only
4 GB Object -> Occupies  6.0 GiB of actual space due to metadata -> 4 GiB are usable for data only
5 GB Object -> Occupies  7.5 GiB of actual space due to metadata -> 3 GiB are usable for data only
6 GB Object -> Occupies  9.0 GiB of actual space due to metadata -> 2 GiB are usable for data only
7 GB Object -> Occupies 10.5 GiB of actual space due to metadata -> 1 GiB are usable for data only
8 GB Object -> Occupies 12.0 GiB of actual space due to metadata -> 0 GiB are usable for data only