performance - GamerDuck123/copyparty GitHub Wiki

defaults are usually fine - expect 8 GiB/s download, 1 GiB/s upload

below are some tweaks roughly ordered by usefulness:

• disabling HTTP/2 and HTTP/3 can make uploads 5x faster, depending on server/client software

• -q disables logging and can help a bunch, even when combined with -lo to redirect logs to file

• --hist pointing to a fast location (ssd) will make directory listings and searches faster when -e2d or -e2t is set

  • and also makes thumbnails load faster, regardless of e2d/e2t

• --dedup enables deduplication and thus avoids writing to the HDD if someone uploads a dupe

• --safe-dedup 1 makes deduplication much faster during upload by skipping verification of file contents; safe if there is no other software editing/moving the files in the volumes

• --no-dirsz shows the size of folder inodes instead of the total size of the contents, giving about 30% faster folder listings

• --no-hash . when indexing a network-disk if you don't care about the actual filehashes and only want the names/tags searchable

• if your volumes are on a network-disk such as NFS / SMB / s3, specifying larger values for --iobuf and/or --s-rd-sz and/or --s-wr-sz may help; try setting all of them to 524288 or 1048576 or 4194304

• --no-htp --hash-mt=0 --mtag-mt=1 --th-mt=1 minimizes the number of threads; can help in some eccentric environments (like the vscode debugger)

• when running on AlpineLinux or other musl-based distro, try mimalloc for higher performance (and twice as much RAM usage); apk add mimalloc2 and run copyparty with env-var LD_PRELOAD=/usr/lib/libmimalloc-secure.so.2

  • note that mimalloc requires special care when combined with prisonparty and/or bubbleparty/bubblewrap; you must give it access to /proc and /sys otherwise you'll encounter issues with FFmpeg (audio transcoding, thumbnails)

• -j0 enables multiprocessing (actual multithreading), can reduce latency to 20+80/numCores percent and generally improve performance in cpu-intensive workloads, for example:

  • lots of connections (many users or heavy clients)
  • simultaneous downloads and uploads saturating a 20gbps connection
  • if -e2d is enabled, -j2 gives 4x performance for directory listings; -j4 gives 16x

  ...however it also increases the server/filesystem/HDD load during uploads, and adds an overhead to internal communication, so it is usually a better idea to don't

• using pypy instead of cpython can be 70% faster for some workloads, but slower for many others

  • and pypy can sometimes crash on startup with -j0 (TODO make issue)

client-side

when uploading files,

• when uploading from very fast storage (NVMe SSD) with chrome/firefox, enable [wasm] in the [⚙️] settings tab to more effectively use all CPU-cores for hashing

  • don't do this on Safari (runs faster without)
  • don't do this on older browsers; likely to provoke browser-bugs (browser eats all RAM and crashes)
  • can be made default-enabled serverside with --nosubtle 137 (chrome v137+) or --nosubtle 2 (chrome+firefox)

• chrome is recommended (unfortunately), at least compared to firefox:

  • up to 90% faster when hashing, especially on SSDs
  • up to 40% faster when uploading over extremely fast internets
  • but u2c.py can be 40% faster than chrome again

• if you're cpu-bottlenecked, or the browser is maxing a cpu core:

  • up to 30% faster uploads if you hide the upload status list by switching away from the [🚀] up2k ui-tab (or closing it)
    • optionally you can switch to the lightweight potato ui by clicking the [🥔]
    • switching to another browser-tab also works, the favicon will update every 10 seconds in that case
  • unlikely to be a problem, but can happen when uploading many small files, or your internet is too fast, or PC too slow