Benchmark TPCH TerarkDB vs RocksDB A - topling/PureOld-TerarkDB GitHub Wiki
We use the lineitem table in TPC-H dataset, and set the length of comment text field in the dbgen lineitem to 512 (from 27). So the average row length of the lineitem table is 615 bytes, in which, the key is the combination of the first 3 fields printed into decimal string from integers.
The total size of the dataset is 554,539,419,806 bytes
, with 897,617,396 lines
. The total size of the keys is 22,726,405,004 bytes
, the other part is values.
TPC-H dbgen generates raw data of strings, which we use directly in our test, without any transformation on the data.
Server A | |
---|---|
CPU Number | 2 |
CPU Type | Xeon E5-2680 v3 |
CPU Freq. | 2.5 GHz |
CPU Actual Freq. | 2.5 GHz |
Cores per CPU | 8 cores with 16 threads |
Cores in total | 16 cores 32 threads |
CPU Cache | 30M |
CPU bogomips | 4988 |
Memory | 64GB |
Memory Freq. | DDR4 2133Hz |
SSD Capacity | 2TB x 1 |
SSD IOPS | 20000 (Network virtual SSD, not local SSD) |
RocksDB | TerarkDB | |
---|---|---|
Level Layers | 4 | 4 |
Compression | Level 0 doesn’t compress Level 1~3 use Snappy |
All level uses Terark Compression |
Compact Type | Level based compaction | Universal compaction |
MemTable Size | 1G | 1G |
Cache Size | 16G | Do not need |
MemTable Number | 3 | 3 |
Write Ahead Log | Disabled | Disabled |
Compact Threads | 2~4 | 2~4 |
Flush Threads (MemTable Compression) |
2 | 2 |
Terark Compress threads | NA | 12 |
Working Threads | 25 in total 24 Random Read, 1 Write |
25 in total 24 Random Read, 1 Write |
Target_file_size_base | Default (64M) | 1G |
Target_file_size_multiplier | 1 (SST size in RocksDB do not influence performance) |
5 |
For both of them, we disable all write speed limits:
- options.level0_slowdown_writes_trigger = 1000;
- options.level0_stop_writes_trigger = 1000;
- options.soft_pending_compaction_bytes_limit = 2ull<<40
- options.hard_pending_compaction_bytes_limit = 4ull<<40
RocksDB (DB Cache 16GB) | TerarkDB | |
0~2 minutes | Read OPS 1M Write OPS 140K (Memory is enough) |
Read OPS 1.69M Write OPS 120K (Memory is enough) |
2~15 minutes | Read OPS 690K Write OPS 90K (Compaction started, memory pressure increases, but still enough to use) |
Read OPS 850K Write OPS 80K (Memory is enough) |
15~30 minutes | Read OPS down to around 10K Write OPS around 90K (Memory runs out) |
Read OPS fluctuates sharply between 500K~1M, Write OPS around 60K. CPU usage close to 100%, IOWait close to 0. (Memory is sufficient, but the compression thread in the background starts to compete with the read threads.) |
30~60 minutes | Read OPS declines all the time. Write OPS around 110K |
Read OPS fluctuates between 350K ~ 980K, with average 500K.CPU usage starts to go down. (Compaction thread hits the read performance) |
60~120 minutes | Read OPS declines all the time. | Read OPS fluctuates sharply, with average 450K. Write OPS keeps at around 55K. |
170 minutes | 550G data all written, with average write OPS 88K. SSD usage starts to decline. | |
173 minutes | Read OPS declines to lower than 10K | |
3 hours 30 minutes | 550G data finishes writing (The compression thread in the background impacts read performance greatly.) | |
3.5~11 hours | Read OPS declines to lower than 5000 | Read OPS keeps at around 50K (Data being compressed gradually, more data can be loaded into memory) |
8 hours 40 minutes | Full Compaction finishes, data size about 213GB | Read OPS increases to around 60K |
11 hours | Read OPS keeps at around 3K (Data doesn’t fully fit in memory) | Read OPS grows and keeps at around 180K |
>30 hours | Data compression completes, Read OPS keeps at 1.85M (Compaction completes, data size is 47GB after compression, can be fully loaded into memory) |