rocks db - animeshtrivedi/notes GitHub Wiki

RocksDB notes

Bookmarks

Other bookmarks:

Errors

open error: IO error: lock hold by current process, acquire time 1732119426 acquiring thread 372738: /home/animesh.trivedi/rocksdb-exp//ssd/LOCK: No locks available

Most likely you have --use_existing_db=false and then using an destroying benchmarking.

Configuration space

Chenxingyu Zhao, Tapan Chugh, Jaehong Min, Ming Liu, and Arvind Krishnamurthy. 2022. Dremel: Adaptive Configuration Tuning of RocksDB KV-Store. Proc. ACM Meas. Anal. Comput. Syst. 6, 2, Article 37 (June 2022), 30 pages. https://doi.org/10.1145/3530903

image

WAL options

https://github.com/facebook/rocksdb/wiki/basic-operations#purging-wal-files

image

Plan

  • install from source
  • test a run with a single location
  • read through the db_bench parameters
  • generate per-seconds ops timeline
  • separate the locations
  • clean release build
  • device pre-conditioning
  • setup NVMe namespaces
  • setup long-running experiments

Benchmarks and generating stats

use : fillrandom,stats,levelstats,memstats,sstables

Avoid seed reuse when --benchmarks has more than one test

How to generate per-second ops/s timeline

When does a flush happens from memtable

https://github.com/facebook/rocksdb/wiki/MemTable#flush

building rocksdb from src (2024-12)

sudo apt install autoconf automake libgflags-dev libtool autoconf-archive m4
sudo apt install libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev libzstd-dev
git tag 
git checkout v9.4.7 
#DEBUG_LEVEL=0 ROCKSDB_PLUGINS="s2fs" USE_RTTI=1 PREFIX=/home/$USER/local/ DISABLE_JEMALLOC=1 DISABLE_WARNING_AS_ERROR=1 make -j4 db_bench install
DEBUG_LEVEL=0 USE_RTTI=1 PREFIX=/home/$USER/local/ DISABLE_JEMALLOC=1 DISABLE_WARNING_AS_ERROR=1 make -j32 db_bench install

Compile for release:

PREFIX=/home/$USER/local/ make -j32 release install

Old

So there is class DB which is the virtual class implementing the database, and one single implementation DBImpl and many wrapper. It is a very large class to make sense out of the blue for now.

(old) Performance links