Development Vast_amount_of_torrents - gazpachoking/deluge GitHub Wiki
Current state of Deluge (v1.3)
Deluge v1.3 handles hundreds of torrents fairly well. The GTKUI uses a lot of CPU to render the torrent list, so to keep CPU usage at a minimum it is a good idea to run in deamon/client mode and only connect with to the daemon when you need.
When the torrent list reaches 1000 torrents, the CPU usage gets more and more noticable, the more torrents the slower it gets.
Improvements are needed for Deluge to handle thousands of torrents!
State of git-develop (To be v1.4)
git-develop has been tested and works well with 2-3000 torrents. Currently the biggest problem is that to make libtorrent v0.16.X handle that many active torrents, the settings active_dht_limit, active_tracker_limit, active_lsd_limit must be set. This is currently not implemented in git-develop, but applying the patch proposed in #2257 should solve this.
The following patches have already been applied which improves the speed of Deluge with many torrents greatly:
- Speed optimizations to the daemon (#2255), develop (Daemon)
- Speedup set_prioritize_first_last in torrent.py (#2246), develop (Daemon, mainly on startup)
- Speed optimizations to the torrentview in GTKUI (#2184), develop (GTKUI)
- Cache tracker icons in gtkui (#2248) (develop) (1.3-stable) (GTKUI)
- Small (but significant) speed optimizations to the torrentview in GTKUI (#2259), 1.3-stable (GTKUI, smaller version of #2184 for 1.3 branch)
- Speed optimizations to sessionproxy (#2285) (GTKUI)
- Application layer protocol for transfering RPC messages + utf8 decoding error (#2116), develop (RPC communication between daemon and clients)
- Move cleanup of prev_status from torrent.py to torrentmanager.py (#2379) (Daemon)
- Add caching to the data-funcs in ui/gtkui/torrentview_data_funcs.py (moved from torrentview.py/listview.py) to reduce unecessary computation (Commit and ticket)
Proposed patches:
- Speedup removing multiple torrents from core (#2250) (GTKUI)
- Allow setting of active_dht_limit, active_tracker_limit and active_lsd_limit. Fix bugs. (#2257) (Daemon)
- Speed optimizations to torrent.get_status (#2332) (Daemon)
- Speed up adding multiple torrents (#2406) (Core/GTKUI)
Other issues that need investigation:
- Pausing/resuming multiple torrents is very slow. This is probably because when a torrent changes state, a lot of operations are done (event listeners in GUI presumable), so with hundreds of torrents changing state the result is too much work needing to be done.
- CPU usage on an idle daemon can still be around 5% (#1490).
- Reducing the number of calls (executions) torrent.update_state during startup (loading torrents)
- Investigate if libtorent.file_progress is a bottleneck