eggroll.properties: Eggroll's Main Configuration File - FederatedAI/eggroll GitHub Wiki
Configuration File Style
eggroll.properties uses typical k=v style as inferred by its filename extension.
Naming Conventions
The key of each configuration item follows this naming convention:
eggroll.${module}.${conf_key_seperated_by_dot}
For example, eggroll.session.processors.per.node can be explained as this:
fixed: eggroll
module: session
conf_key: processors.per.node
The value of each configuration item is in free style. Actual data type will be explained and converted in usage. The default value can be found in conf_keys.py and ConfKeys.scala. If the default values from conf_keys.py and ConfKeys.scala differs, the actual value depends of the actual usage: whether it is used in python or jvm. If it happens, please give us a feedback in issue zone.
Common Configurations:
| # | key | default value | mandatory (Y/N) | description | comments |
|---|---|---|---|---|---|
| 1 | eggroll.logs.dir |
(N/A) | Y | Eggroll's logs dir, a path where logs generates | |
| 2 | eggroll.data.dir |
(N/A) | Y | Eggroll's data dir, a path where data locates | |
| 3 | eggroll.core.security.ca.crt.path |
(N/A) | N | The path of CA certificate to verify client certificate | secure channel |
| 4 | eggroll.core.security.crt.path |
(N/A) | N | The path of server certificate (used when accepting an RPC request) | secure channel |
| 5 | eggroll.core.security.key.path |
(N/A) | N | The path of server private key | secure channel |
| 6 | eggroll.core.security.client.ca.crt.path |
(N/A) | N | The path of CA certificate to verify server certificate | secure channel |
| 7 | eggroll.core.security.client.crt.path |
(N/A) | N | The path of client certificate (used when initiating an RPC request) | secure channel |
| 8 | eggroll.core.security.key.path |
(N/A) | N | The path of client private key | secure channel |
| 9 | eggroll.resourcemanager.clustermanager.host |
(N/A) | Y | Host of Cluster Manager | |
| 10 | eggroll.resourcemanager.clustermanager.port |
(N/A) | Y | Port of Cluster Manager | |
| 11 | eggroll.resourcemanager.nodemanager.port |
(N/A) | Y | Port of Node Manager | |
| 12 | eggroll.resourcemanager.process.tag |
(N/A) | N | A tag in command line to distinguish different instances of CM / NM / RS | |
| 13 | eggroll.session.start.timeout.ms |
20_000 | N | Timeout in ms to get or create a new session | overridden in default conf file with 180_000 |
| 14 | eggroll.session.stop.timeout.ms |
20_000 | N | Timeout in ms to stop a session | |
| 15 | eggroll.session.processors.per.node |
(N/A) | Y | Number of processors started of each node for a session | explicitly written to 4 in default conf file |
| 16 | eggroll.rollpair.transferpair.sendbuf.size |
250_000 | N | Batch size of data transfer WITHIN A CLUSTER | |
| 17 | eggroll.rollpair.eggpair.server.executor.pool.max.size |
5_000 | N | Max size of egg_pair's command server executor pool | |
| 18 | eggroll.rollpair.eggpair.data.server.executor.pool.max.size |
5_000 | N | Max size of egg_pair's data server executor pool | |
| 19 | eggroll.rollpair.transferpair.executor.pool.max.size |
5_000 | N | Max size of egg_pair's shuffle executor pool | |
| 20 | eggroll.rollpair.default.store.type |
ROLLPAIR_LMDB |
N | Default store type of RollPair | |
| 21 | eggroll.rollsite.pull.max.retry |
720 | N | Max retries of pulling data. Actual waiting time is this value * eggroll.rollsite.pull.interval.sec. Transfer time should not be greater than this time period |
|
| 22 | eggroll.rollsite.pull.interval.sec |
600 | N | Interval of each pulling attempt. Also acts as timeout of one pulling attempt | |
| 23 | eggroll.rollsite.pull.header.timeout.sec |
432_000 (720 * 600) | N | Timeout of pulling the first batch of data. Computing time in other parties should not be greater than this value | |
| 24 | eggroll.rollsite.pull.header.interval.sec |
300 | N | Interval of each pulling header attempt | |
| 25 | eggroll.rollsite.push.batches.per.stream |
10 | N | Stream batch size when pushing data | |
| 26 | eggroll.rollsite.push.max.retry |
3 | N | Max retries in logic layer when pushing data | |
| 27 | eggroll.rollsite.push.overall.timeout.sec |
600 | N | Timeout of waiting push to complete and session exits | |
| 28 | eggroll.rollsite.push.per.stream.timeout.sec |
300 | N | Timeout of pushing a single stream | |
| 29 | eggroll.rollsite.polling.server.enabled |
false |
N | Whether the RS runs in polling server mode | polling mode |
| 30 | eggroll.rollsite.polling.client.enabled |
false |
N | Whether the RS runs in polling client mode | polling mode |
| 31 | eggroll.rollsite.polling.concurrency |
50 | N | Number of polling threads | polling mode |
| 32 | eggroll.rollsite.polling.no.data.timeout.sec |
300 | N | Timeout of polling but no data arrives | polling mode |
| 33 | eggroll.rollsite.polling.exchanger.data.op.timeout.sec |
240 | N | Timeout of offering / polling data to / from q | polling mode |
| 34 | eggroll.rollsite.host |
127.0.0.1 | N | Host of rollsite | |
| 35 | eggroll.rollsite.port |
9370 | N | Port of rollsite | |
| 36 | eggroll.rollsite.party.id |
10001 | N | Rollsite party ID of myself | |
| 37 | eggroll.rollsite.route.table.path |
conf/route_table.json | N | file path of route table | |
| 38 | eggroll.rollsite.route.table.key |
123 | N | Key to verify when updating route table | overridden in default conf file |
| 39 | eggroll.rollsite.route.table.whitelist |
127.0.0.1 | N | Ip white list seperated by comma where route table update is allowed | |
| 40 | eggroll.rollsite.jvm.options |
(N/A) | N | jvm options used when rollsite starts | overridden in default conf file |
| 41 | eggroll.rollsite.adapter.sendbuf.size |
1_048_576 | N | Batch size when sending data TO ANOTHER SITE / AMONG CLUSTERS | |
| 42 | eggroll.rollsite.receive.executor.pool.max.size |
5_000 | N | Max executor size of roll site client |