beta2 changes - MaynardMiner/SWARM GitHub Wiki

SWARM BETA2 RELEASE CHANGES.

VERY IMPORTANT

SWARM now requires latest 7.0.0 version of powershell for upcoming version:

https://github.com/PowerShell/PowerShell/releases/tag/v7.0.0

Running install_linux will update powershell. HiveOS is done automatically. This is important, as the executable for powershell is different- From pwsh to pwsh-preview. You can have both version 6 and 7 installed at the same time, if you desire to rollback.

Argument Changes

-StatsInterval

  • Now is in minutes, not seconds. Default has changed to 3.

-Benchmark

  • Now is in minutes, not seconds. Default has changed to 3.

-Interval

  • Now is in minutes, not seconds. Default has changed to 5.

-Historical_Bias

  • Format is now [Penalty]:[Bonus]
  • Penalty sets the maximum penalty you wish to place on an incoming estimate based on historical returns.
  • Bonus sets the maximum bonus you wish to place on an incoming estimate based on historical returns.
  • Default is now -Historical_Bias 1:1

-Throttle

  • Default is 0 (AUTO)
  • Sets the maximum allowed parallel threads used for building database.
  • Ideally, cpu thread use should be N + 1, where N is the number of cores. This is what default is set to do in auto.
  • Depending on storage device, and other resources- setting it to a lower or higher value may improve speed.

-Stat_Algo and -Stat_Coin

  • New time periods:
    • Live
    • Minute_10_MA
    • Minute_10_EMA
    • Minute_15_MA
    • Minute_15_EMA
    • Minute_30_MA
    • Minute_30_EMA
    • Hour_MA
    • Hour_EMA
    • Hour_4_MA
    • Hour_4_EMA
    • Day_MA
    • Day_EMA
  • MA means "Moving Average", which is a simple moving average of each estimate pulled.
  • EMA means "Exponential Moving Average", which is a weighted moving average that favors more recent pricing.

Runtime Changes

  • SWARM will always build database every 300 seconds. -SWARM_MODE yes is based on every :05 increment in time, such as 12:05, 12:10, 12:15 as usual.
  • -StatsInterval, -Benchmark, and -Interval behave differently now.
    • Once SWARM has completed building the database, and updated stats, it will check if -Interval time has elapsed. If it has, it will load all available miners. If it has not- It will either load previous miners (if SWARM was able to retrieve data) or all available miners.
    • At the benchmarking phase, SWARM will check the miner process runtime. If it is equal to or exceeds the -StatsInterval parameter, or if there is no previous stat and it exceeds the -Benchmark parameter, it will record the running hashrate. If not then it will skip until next time.
  • Parallel build of database.
    • SWARM now gathers most pool data, and writes stats in parallel, based on the -Throttle parameter. For users using slower cpus/storage, this will likely not make much difference. For users using better equipment- This could increase database speed over 50%.
  • New statistics. SWARM AI has implemented a better statistics system, which includes:
    • Removing erroneous data when SWARM has issues connecting to pools, or being shut off. SWARM intelligently reacts to gaps in time.
    • SWARM resets stats if shutoff longer than 24 hours.
    • SWARM will migrate data during period in which there is gaps in time. An example: SWARM failed to connect to the pool for 10 minutes. This means that there is a gap of time in the Minute_10_EMA. However, because it was only 10 minutes, Minute_15_EMA is still valid stat. Therefor, SWARM will migrate the Minute_15_EMA to the minute_10_EMA value, and factor the Minute_15_EMA into the all moving averages + incoming estimate. Then continues on. We are aware there is a better way to handle this, and will implement in future.

New Statistics

   Live (Last price pull from pool)
   Minute_10_MA
   Minute_10_EMA
   Minute_15_MA
   Minute_15_EMA
   Minute_30_MA
   Minute_30_EMA
   Hour_MA
   Hour_EMA
   Hour_4_MA
   Hour_4_EMA
   Day_MA
   Day_EMA

"MA" means simple moving average. "EMA" means weighted moving average. (Smoothed, favors more recent price data in average).

  • -Historical_Bias Will work with coins now, but its data will not be accurate until 24 hours of data has been collected.

  • -Historical_Bias Will begin with using direct 24_hour pool data, as it normally has functioned. After 24 hours of it running, SWARM will begin to record week statistics, and -Historical_Bias will be factored on this.

  • Pool files are now outputted as .json, and their formats have changed.

  • Better memory Garbage collection. Runtime memory usage has been lowered greatly.

  • SWARM now uses threading to collect pool data, this reduces time to gather and sort data by at least 30% (Unless CPU use is high from a miner).

  • Fixed Switching_Threshold to work better for coins.

  • Miners using both -CoinExchange "ETH" and -Poolname "Whalesburg", SWARM will only grab ETH pricing once to help reduce IP bans from cryptocompare.

  • This is a sample of the new stat file for pools, which soon will be used for miners as well:

{
  "Live": 0.000000000100308600,
  "Actual": 0.000000000182153475,
  "Minute_10_EMA": 0.0000000001003086000000000000,
  "Minute_10_MA": 0.0000000001003086,
  "Minute_15_EMA": 0.0000000001003086000,
  "Minute_15_MA": 0.0000000001003086,
  "Minute_30_EMA": 0.0000000000992932875000000004,
  "Minute_30_MA": 0.0000000000988871625,
  "Hour_EMA": 0.0000000001012588466346153844,
  "Hour_MA": 0.00000000010143161875,
  "Hour_4_EMA": 0.0000000001950285525641026223,
  "Hour_4_MA": 0.00000000020014855,
  "Day_EMA": 0.0000000001950285525641026223,
  "Day_MA": 0.00000000020014855,
  "Avg_Hashrate": 2374828.0,
  "Pulls": 38,
  "Historical_Bias": -0.1020,
  "Start_Of_Day": "2020-02-26T18:59:45.9164036-05:00",
  "Locked": false,
  "Updated": "2020-02-27T05:13:38.0133603Z",
  "Daily_Values": [],
  "Daily_Actual_Values": [],
  "Daily_Hashrate_Values": [],
  "Live_Values": [
    0.000000000153053075,
    0.00000000015507135,
    0.000000000166373025,
    0.000000000166373025,
    0.00000000017581935,
    0.00000000017581935,
 ]
}

Live: The price SWARM gathered from the estimate/paying of pool at time of pull.

Active: Either 24_hourBTC for coins, or 24_hour_Actual for algorithms. PPS Pools Will always show -1. They are handled differently (nicehash and whalesburg).

MA: Simple moving average of the specified time period

EMA: Weighted moving average of the specified time period

Avg_Hashrate: Simple 288 period (day) moving average of pool hashrate

Pulls: Reference to the number of pull that have happened in the day.

Historical_Bias: Current deviation between daily estimate MA and 24_hour_actual. If weekly values are stored, it will use weekly MA rather than 24 hour. This value is percentage in decimal form. -1 is 100% while 0.134 is 13.4%

Start_Of_Day: Beginning of the day in UTC.

Locked: Whether or not stat is locked (future implementation)

Updated: Last time pool was successfully checked for this item

Daily_Values: container to store 7 periods (week) of Daily_MA estimates

Daily_Actual_Values: container to store 7 periods (week) of actual returns

Daily_Hashrate_Values: container to store 7 periods (week) of daily hashrate average.

Live_Values: container to store 288 periods (day) of Live estimates.