Regression Tests - xoto10/fishtest GitHub Wiki

All of the information below has been generated from the results of tests performed on the Fishtest framework and can be found here.

All single thread tests are performed over 60.000 games at a time control of 60 seconds + 0.6 seconds. All eight thread tests are performed over 40.000 games at a time control of 30 seconds + 0.3 seconds. Results have a confidence interval of +- 1 elo.

Stockfish 14+ Development Progress

Date Version Elo
1 Thread 		Elo
8 Threads
02/07/21 Stockfish 14 Stockfish Logo
26/07/21 Stockfish 14 vs Update default net to nn-26abeed38351.nnue +2.74
15/08/21 Stockfish 14 vs New NNUE architecture and net +9.31 +6.08
31/08/21 Stockfish 14 vs Update default net to nn-735bba95dec0.nnue +15.04
15/09/21 Stockfish 14 vs Update default net to nn-13406b1dcbe0.nnue +16.64 +12.39

Total Development Progress Since Stockfish 7

Development Progress Compared

1 Thread 8 Threads

30 Day Average Development Progress Compared

1 Thread 8 Threads

Draw Percentage vs Elo Compared

1 Thread 8 Threads

Fishtest Progression History

Stockfish 3 Development (21/09/12 - 30/04/13)
Date Version Elo
1 Thread
21/09/12 Stockfish 2.3.1 Stockfish Logo
04/03/13 Stockfish 2.3.1 vs Increase see prune depth +15.00
11/03/13 Stockfish 2.3.1 vs Check for easy move just once +13.42
16/03/13 Stockfish 2.3.1 vs Further increase SEE prune depth +17.77
18/03/13 Stockfish 2.3.1 vs Be more aggressive on trying to finish iterations +15.49
24/03/13 Stockfish 2.3.1 vs Update bestValue when futility pruning (2) +16.71
30/03/13 Stockfish 2.3.1 vs Set IID half way between d/2 and d-4 +18.76
03/04/13 Stockfish 2.3.1 vs Double Impact of Gain tables +15.44
06/04/13 Stockfish 2.3.1 vs Increase null verification threshold to 12 plies +17.11
07/04/13 Stockfish 2.3.1 vs Rescale UCI parameters to 100 +19.02
10/04/13 Stockfish 2.3.1 vs De-templetize Position::is_draw() +24.13
19/04/13 Stockfish 2.3.1 vs Skip a couple of popcount in previous patch +28.27
26/04/13 Stockfish 2.3.1 vs Fix a crash introduced few days ago +33.67
30/04/13 Stockfish 2.3.1 vs Temporary revert "Expose EvalInfo struct to search" +30.86
30/04/13 Stockfish 3 Stockfish Logo
Stockfish 4 Development (30/04/13 - 20/08/13)
Date Version Elo
1 Thread
30/04/13 Stockfish 3 Stockfish Logo
16/05/13 Stockfish 3 vs Use two counter moves instead of one +22.11
25/05/13 Stockfish 3 vs Bunch of 3 small patches +26.70
31/05/13 Stockfish 3 vs Passed pawn tuning +29.50
23/06/13 Stockfish 3 vs Fix some stale comments +35.47
03/07/13 Stockfish 3 vs Simplify aspiration window code +37.36
13/07/13 Stockfish 3 vs Fully qualify memset and memcpy +39.27
19/07/13 Stockfish 3 vs Halve king eval margin +39.83
26/07/13 Stockfish 3 vs Rewrite pawn shield and storm code +48.84
05/08/13 Stockfish 3 vs Streamline time computation +50.95
18/08/13 Stockfish 3 vs Further tweak movecount pruning +56.66
20/08/13 Stockfish 4 Stockfish Logo
Stockfish DD Development (20/08/13 - 29/11/13)
Date Version Elo
1 Thread
20/08/13 Stockfish 4 Stockfish Logo
29/08/13 Stockfish 4 vs Enable LMR for dangerous moves +16.18
02/09/13 Stockfish 4 vs Don't use lpthread for Android +19.30
05/09/13 Stockfish 4 vs Do not prune useless checks in QS +23.80
08/09/13 Stockfish 4 vs Remove unreachable values in mobility table +27.66
11/09/13 Stockfish 4 vs Extend checks more when below alpha +24.87
12/09/13 Stockfish 4 vs Revert "Move draw by material check" +24.34
13/09/13 Stockfish 4 vs Increase passed bonus for having more pieces +26.21
16/09/13 Stockfish 4 vs Fix time parameters for blitz games +31.56
23/09/13 Stockfish 4 vs Update disabled warnings for Intel compiler +34.03
29/09/13 Stockfish 4 vs Drop 'is' prefix from query functions +33.49
02/10/13 Stockfish 4 vs Add more depth/positions to bench +33.90
08/10/13 Stockfish 4 vs Use TT refined value to stand pat +36.58
08/10/13 Stockfish 4 vs Increase slowmover and reduce instability +38.91
09/10/13 Stockfish 4 vs Smoother transition for LMR +39.29
14/10/13 Stockfish 4 vs Double king safety weights +43.64
18/10/13 Stockfish 4 vs Score chain pawn also by rank +49.51
19/10/13 Stockfish 4 vs Re-add "Further increase safe checks bonus" +50.11
20/10/13 Stockfish 4 vs Further improve chain pawn evaluation +49.72
22/10/13 Stockfish 4 vs Tweak again chain pawn bonus +53.85
24/10/13 Stockfish 4 vs Retire mirror() +55.18
30/10/13 Stockfish 4 vs Tweak bishop pair and knight weight +52.84
01/11/13 Stockfish 4 vs Set timer to a fixed interval +59.73
09/11/13 Stockfish 4 vs Futility pruning simplification +60.68
11/11/13 Stockfish 4 vs Remove opposed flag for doubled pawns +61.23
12/11/13 Stockfish 4 vs Simplify generate +63.85
30/11/13 Stockfish 4 vs Add support for PPC 64bit on Linux +67.44
29/11/13 Stockfish DD Stockfish Logo
Stockfish 5 Development (29/11/13 - 31/05/14)
Date Version Elo
1 Thread
29/11/13 Stockfish DD Stockfish Logo
09/12/13 Stockfish DD vs Research at intermediate depth if LMR is very high +7.94
20/12/13 Stockfish DD vs Faster and simplified threat eval +11.35
24/12/13 Stockfish DD vs Loosened trigger condition for king safety +18.62
30/12/13 Stockfish DD vs Retire asymmThreshold +19.54
02/01/14 Stockfish DD vs Ensure move_importance() is non-zero +25.36
09/01/14 Stockfish DD vs Position::gives_check - use ci.ksq +29.85
14/01/14 Stockfish DD vs Introduce 'follow up' moves +29.84
21/01/14 Stockfish DD vs Small simplification to Position::see +32.49
29/01/14 Stockfish DD vs Tweak bishop PSQT tables +36.66
11/02/14 Stockfish DD vs Faster handling of king captures in Position::see +38.63
27/02/14 Stockfish DD vs Dynamic draw value +39.25
27/02/14 Stockfish DD vs Fix a warning with Intel compiler +38.93
14/03/14 Stockfish DD vs Depth dependant aspiration window delta +40.85
25/03/14 Stockfish DD vs Simplify TT replace strategy +43.70
08/04/14 Stockfish DD vs Restrict queen mobility to safe squares +47.70
12/04/14 Stockfish DD vs Move args parsing to UCI::loop +49.21
21/04/14 Stockfish DD vs Reset DrawValue[] before new search +54.53
26/04/14 Stockfish DD vs Speed up picking of killers +57.08
08/05/14 Stockfish DD vs Revert dynamic contempt +53.27
15/05/14 Stockfish DD vs Drop to qsearch at low depth in razoring +57.15
26/05/14 Stockfish DD vs Fix a warning with Intel compiler +53.28
26/05/14 Stockfish DD vs Fix an off-by-one bug in extract_pv_from_tt +55.26
31/05/14 Stockfish 5 Stockfish Logo
Stockfish 6 Development (31/05/14 - 27/01/15)
Date Version Elo
1 Thread
31/05/14 Stockfish 5 Stockfish Logo
03/06/14 Stockfish 5 vs Symmetric King Safety: take 2 +5.35
15/06/14 Stockfish 5 vs Simplify pawn threats and merge into ThreatenedByPawn[] +8.30
29/06/14 Stockfish 5 vs Fix Singular extension condition to handle mate scores +10.78
23/07/14 Stockfish 5 vs Outpost tuning +19.63
06/08/14 Stockfish 5 vs Remove insufficient material rule +19.68
13/09/14 Stockfish 5 vs Small tweak to idle_loop() +15.90
28/09/14 Stockfish 5 vs Remove use of half-ply reductions +22.80
17/10/14 Stockfish 5 vs Document why initing eval tables +27.58
01/11/14 Stockfish 5 vs Merge pull request #89 from official-stockfish/pull_no_pretty +31.00
12/11/14 Stockfish 5 vs Profile Build with Hash=16 +36.21
29/11/14 Stockfish 5 vs Bitbase index() from ADD to OR +39.02
09/12/14 Stockfish 5 vs Simpler PRNG and faster magics search +41.42
27/12/14 Stockfish 5 vs Prefer names to numbers in storm code +46.26
08/01/15 Stockfish 5 vs Assorted formatting and comment tweaks in position.h +52.10
18/01/15 Stockfish 5 vs Stockfish 6 Release Candidate 1 +51.71
27/01/15 Stockfish 6 Stockfish Logo
Stockfish 7 Development (27/01/15 - 02/01/16)
Date Version Elo
1 Thread
27/01/15 Stockfish 6 Stockfish Logo
09/02/15 Stockfish 6 vs Pawn Center Bind Bonus +7.50
20/03/15 Stockfish 6 vs Retire ConditionVariable +15.92
29/03/15 Stockfish 6 vs Remove some difficult to understand C++11 constructs +19.02
10/04/15 Stockfish 6 vs Allow Position::init() to be called more than once +22.12
09/05/15 Stockfish 6 vs Smart TT save +28.82
07/06/15 Stockfish 6 vs Simplify outpost evaluation +27.85
19/07/15 Stockfish 6 vs Fix formatting of previous patch +30.74
30/07/15 Stockfish 6 vs Simplify IID depth formula +34.04
05/10/15 Stockfish 6 vs File based passed pawn bonus +44.23
26/10/15 Stockfish 6 vs Use atomics instead of volatile +61.76
28/12/15 Stockfish 6 vs Stockfish 7 Beta 1 +62.62
02/01/16 Stockfish 7 Stockfish Logo
Stockfish 8 Development (02/01/16 - 01/11/16)
Date Version Elo
1 Thread
02/01/16 Stockfish 7 Stockfish Logo
01/02/16 Stockfish 7 vs Time management simplification +4.93
11/03/16 Stockfish 7 vs Add follow up moves history for move ordering +12.83
08/04/16 Stockfish 7 vs Small passed pawn simplification +15.27
21/05/16 Stockfish 7 vs More detailed dependence of time allocation +29.44
11/06/16 Stockfish 7 vs Stat Formula Tweak +36.29
24/07/16 Stockfish 7 vs Allow null pruning at depth 1 +49.73
18/08/16 Stockfish 7 vs Remove a stale assignment +52.59
08/09/16 Stockfish 7 vs Refactor previous patch +66.53
09/10/16 Stockfish 7 vs Optimisation of Position::see and Position::see_sign +76.60
01/11/16 Stockfish 8 Stockfish Logo
Stockfish 9 Development (01/11/16 - 31/01/18)
Date Version Elo
1 Thread
01/11/16 Stockfish 8 Stockfish Logo
31/12/16 Stockfish 8 vs Small eval cleanup and renaming +5.21
29/01/17 Stockfish 8 vs Simplify TT penalty stat (#980) +8.82
09/03/17 Stockfish 8 vs Helper functions to count material for both sides +10.84
20/04/17 Stockfish 8 vs simplify logic for history based pruning +15.17
08/05/17 Stockfish 8 vs Linear Protector bonus by distance +20.25
22/06/17 Stockfish 8 vs Increase reduction if tt-move is a capture +27.41
26/08/17 Stockfish 8 vs Improve multi-threaded mate finding +29.32
02/10/17 Stockfish 8 vs Good bishops on the main diagonals +32.61
03/11/17 Stockfish 8 vs Introduce capture history table for capture move sorting +35.18
03/12/17 Stockfish 8 vs Use constexpr when makes sense +43.31
23/01/18 Stockfish 8 vs Contempt 20 +57.25
31/01/18 Stockfish 9 Stockfish Logo
Stockfish 10 Development (31/01/18 - 29/11/18)
Date Version Elo
1 Thread
31/01/18 Stockfish 9 Stockfish Logo
28/02/18 Stockfish 9 vs Reintroduce depth 2 razoring (with additional margin) +5.16
07/03/18 Stockfish 9 vs Simplification: use Arctan for the optimism S-curve +6.94
14/03/18 Stockfish 9 vs Use intrinsics only for LSB/MSB +13.03
27/03/18 Stockfish 9 vs Make kingRing always 8 squares +13.77
05/04/18 Stockfish 9 vs Remove the Queen from the mobility area of minor pieces +16.59
08/04/18 Stockfish 9 vs Reset negative statScore on fail high +16.63
23/04/18 Stockfish 9 vs Alternative formula for dynamic contempt +15.90
29/04/18 Stockfish 9 vs Always scale using pawn contribution +14.61
04/05/18 Stockfish 9 vs Tweak the connected[] array value for pawns on rank 5 +18.52
15/05/18 Stockfish 9 vs Update search.cpp +24.92
24/05/18 Stockfish 9 vs LMR Capture Tweak +26.72
06/06/18 Stockfish 9 vs Call cycle detection before qsearch() +28.16
11/06/18 Stockfish 9 vs Optimize an expression in endgame.cpp +29.72
23/06/18 Stockfish 9 vs Another set of tuned values after one million games +31.98
19/07/18 Stockfish 9 vs Better check evasion move sorting +36.70
27/07/18 Stockfish 9 vs Simplify cmh pruning +37.45
28/07/18 Stockfish 9 vs Increase the mg->eg gradient for the PawnlessFlank malus +35.84
31/07/18 Stockfish 9 vs Small tweaks to recent code changes +37.67
08/08/18 Stockfish 9 vs First check threshold in space evaluation +37.78
12/08/18 Stockfish 9 vs Combo of several promising parameter tweaks +40.88
14/08/18 Stockfish 9 vs Double weight of capture history +41.69
17/08/18 Stockfish 9 vs Use an affine formula to mix stats and eval +43.15
28/08/18 Stockfish 9 vs Tweak stat bonus formula +44.20
31/08/18 Stockfish 9 vs Remove PawnsOnBothFlanks +42.37
04/09/18 Stockfish 9 vs Re-introduce "keep pawns on both flanks" +46.46
11/09/18 Stockfish 9 vs Tweak opposite colored bishops endgame scaling +45.47
27/09/18 Stockfish 9 vs Fix two typos in comments +46.93
14/10/18 Stockfish 9 vs Simplify check extensions +49.01
26/10/18 Stockfish 9 vs On main thread: reduce depth after fail high +52.82
01/11/18 Stockfish 9 vs Fix issues from using adjustedDepth too broadly +51.68
08/11/18 Stockfish 9 vs Update list of top CPU contributors +50.43
19/11/18 Stockfish 9 vs Stockfish 10-beta +53.77
27/11/18 Stockfish 9 vs Simplify casting extension +54.21
29/11/18 Stockfish 10 Stockfish Logo
Stockfish 11 Development (29/11/18 - 17/01/20)
Date Version Elo
1 Thread 		Elo
8 Threads
29/11/18 Stockfish 10 Stockfish Logo
06/12/18 Stockfish 10 vs Revert "pseudo_legal() and MOVE_NONE" +4.32
13/12/18 Stockfish 10 vs A combo of parameter tweaks +8.06 +12.70
17/12/18 Stockfish 10 vs Use stronglyProtected +7.85
04/01/19 Stockfish 10 vs Check tablebase files +7.20
10/01/19 Stockfish 10 vs Remove pvExact +13.14 +16.17
22/01/19 Stockfish 10 vs Simplify TrappedRook +13.98
03/02/19 Stockfish 10 vs Less king danger if we have a knight near by to defend it +17.71 +19.77
15/03/19 Stockfish 10 vs Increase thread stack for OS X (#2035) +16.58
01/04/19 Stockfish 10 vs Assorted trivial cleanups 3/2019 (#2030) +16.58 +24.33
24/04/19 Stockfish 10 vs Remove useless initializations (#2115) +16.39
05/05/19 Stockfish 10 vs LMR for captures not cracking alpha +16.65
15/05/19 Stockfish 10 vs Update failedHighCnt rule #2063 +19.76 +28.93
09/06/19 Stockfish 10 vs Remove depth condition for ttPv (#2166) +19.87
20/06/19 Stockfish 10 vs More bonus for free passed pawn +24.06 +30.76
27/06/19 Stockfish 10 vs Bonus for double attacks on unsupported pawns +22.75
11/07/19 Stockfish 10 vs Assorted trivial cleanups June 2019 +24.39
25/07/19 Stockfish 10 vs Tweak of SEE pruning condition +25.72 +37.49
14/08/19 Stockfish 10 vs Tweak unsafe checks +32.24
26/08/19 Stockfish 10 vs Tweak Late Move Reduction at root +35.63 +44.52
12/09/19 Stockfish 10 vs Scale down complexity for almost unwinnable endgames +39.10
17/09/19 Stockfish 10 vs Raise stack size to 8MB for pthreads +37.63 +46.57
26/09/19 Stockfish 10 vs Increase weight for supported pawns +38.97
06/10/19 Stockfish 10 vs Introduce separate counter-move tables for captures +41.96 +51.76
18/10/19 Stockfish 10 vs Current capture for Counter-Move history +44.63
05/11/19 Stockfish 10 vs Rook PSQT Tuned +42.20 +52.90
14/11/19 Stockfish 10 vs Prune before extension +43.12
21/11/19 Stockfish 10 vs Do lmr for more captures +46.56 +53.93
03/12/19 Stockfish 10 vs UnblockedStorm tuned +44.88
10/12/19 Stockfish 10 vs Refine improving-logic +47.27 +56.62
07/01/20 Stockfish 10 vs Tuned nullmove search +51.50 +58.15
18/01/20 Stockfish 10 vs Stockfish 11 +53.59 +58.07
17/01/20 Stockfish 11 Stockfish Logo
Stockfish 12 Development (17/01/20 - 02/09/20)
Date Version Elo
1 Thread 		Elo
8 Threads
17/01/20 Stockfish 11 Stockfish Logo
28/01/20 Stockfish 11 vs More bonus for bestMoves on past PV nodes -2.47
31/01/20 Stockfish 11 vs Revert 5 recent patches +0.85
27/02/20 Stockfish 11 vs Weak queen protection +1.33 +0.51
20/03/20 Stockfish 11 vs Adjust singular extension search depth +2.94
07/04/20 Stockfish 11 vs Introduce capture history pruning +5.74 +6.49
16/04/20 Stockfish 11 vs Remove one condition in probcut TTmove skipping +11.33
02/05/20 Stockfish 11 vs Fishtest Tuning Framework +15.21 +16.71
21/05/20 Stockfish 11 vs Tweak knight mobility +15.97
06/06/20 Stockfish 11 vs Use lowply-history also on low depths +19.45 +23.70
13/06/20 Stockfish 11 vs Tuned values for search constants +20.91 +24.86
29/06/20 Stockfish 11 vs Tweak single queen endgame scaling +25.67
17/07/20 Stockfish 11 vs Do not overwrite valuable TT data after probcut +26.44 +30.71
30/07/20 Stockfish 11 vs Tweak cutnode reduction +25.49 +32.39
06/08/20 Stockfish 11 vs Add NNUE evaluation Stockfish Logo +83.42 +86.10
08/08/20 Stockfish 11 vs LMR search tweak +106.20
11/08/20 Stockfish 11 vs This commit enables a mixed bench +125.60 +111.78
18/08/20 Stockfish 11 vs Fix Makefile typo +121.54 +111.40
30/08/20 Stockfish 11 vs Update parameters in classical evaluation +130.96
02/09/20 Stockfish 11 vs Stockfish 12 +133.65 +117.62
02/09/20 Stockfish 12 Stockfish Logo
Stockfish 13 Development (02/09/20 - 18/02/21)
Date Version Elo
1 Thread 		Elo
8 Threads
02/09/20 Stockfish 12 Stockfish Logo
08/09/20 Stockfish 12 vs Double probability of using classical eval +6.49
21/09/20 Stockfish 12 vs Add large page support for NNUE weights +13.88 +8.97
28/09/20 Stockfish 12 vs Include pawns in NNUE scaling +23.15
18/10/20 Stockfish 12 vs Do more reductions for late quiet moves +24.09 +21.17
01/11/20 Stockfish 12 vs Update default net to nn-cb26f10b1fd9.nnue +28.02
15/11/20 Stockfish 12 vs Rook Mobility Tweak +29.99 +25.49
29/11/20 Stockfish 12 vs Update default net to nn-62ef826d1a6d.nnue +30.61
14/12/20 Stockfish 12 vs Increase reduction in case of stable best move +32.09 +27.50
31/12/20 Stockfish 12 vs WeakUnopposed penalty for backwards on file A or H +33.71
13/01/21 Stockfish 12 vs Optimize generate_moves +32.40 +28.29
16/02/21 Stockfish 12 vs Small trivial clean-ups, February 2021 +36.03 +29.08
18/02/21 Stockfish 13 Stockfish Logo
Stockfish 14 Development (18/02/21 - 02/07/21)
Date Version Elo
1 Thread 		Elo
8 Threads
18/02/21 Stockfish 13 Stockfish Logo
26/02/21 Stockfish 13 vs Introduce DistanceFromPV +1.34
24/03/21 Stockfish 13 vs Small cleanups (march 2021) +0.61
15/04/21 Stockfish 13 vs Use classical eval for Bishop vs Pawns +1.47 +0.89
22/05/21 Stockfish 13 vs Sometimes change the (materialist, positional) balance +10.12
14/06/21 Stockfish 13 vs Update default net to nn-8e47cf062333.nnue +21.80 +16.96
18/06/21 Stockfish 13 vs Make net nn-50144f835024.nnue the default +25.56
29/06/21 Stockfish 13 vs Update Top CPU Contributors +30.27 +22.62
02/07/21 Stockfish 14 Stockfish Logo

External Links

There are several pages on the web run by chess engine fans. Some collect progress of Stockfish over previous stable versions and development builds. Some compare Stockfish progress to other chess engines. Here is a collection of some useful links in that regard.

Next Chess Move
SF Versions CCRL Blitz
SF Versions CCRL 40/2 FRC
SF Versions CCRL 40/15
Ipman Chess
SPCC Stockfish Testing
SPCC Stockfish Regression Testing
FastGMs Rating Lists (FGRL)

Add a custom footer
⚠️ **GitHub.com Fallback** ⚠️