Statistical Reference Datasets (StRD) from NIST
National Institute of Standards and Technology (U.S.)
provides datasets for assessment of software accuracy for a variety of
statistical methods. 27 datasets are provided for nonlinear regression testing:
http://www.itl.nist.gov/div898/strd/nls/nls_main.shtml .
This page describes how the program was tested and presents the results.
Fityk gives more accurate results than two other programs that we also
tested. This is a bit surprising, because, like most of the curve fitting
programs, it uses (by default) Levenberg-Marquardt method and double precision
for floating point numbers. In all the cases sum of squared residuals
(Residual Sum of Squares in StRD)
is exactly the same as certified values, so it is not listed below.
Fityk can read the data directly from Misra1a.dat, automatically skipping
description, but since the file has y in the first column and x in the
second, we need to swap x and y . Also, we ensure that the standard
deviation of all y 's is 1:
@0 < Misra1a.dat
Y=x, X=y, S=1
After swapping the axes, you may click the Zoom All button in the GUI (or
type plot [] in the CLI). Since this dataset has only a few points, it is
a good idea to increase the size of the points (in GUI) to make them more
visible.
The certified values in the NIST files are presented in the scientific format
with 11 decimal places. It is possible to set the same format in Fityk:
set numeric_format='%.10E'
Now let's prepare the model with the initial values from Start 1 :
define Misra1a(b1, b2) = b1*(1-exp(-b2*x))
F = Misra1a(~500, ~0.0001)
To fit the data with a very high precision we change the stopping criteria:
=-> set lm_stop_rel_change=1e-20
=-> fit
Fitting 2 (of 2) parameters to 14 points ...
In L-M method: lambda=1e+15 > 1e+15, stopped.
levenberg_marquardt: 100 iterations, 101 evaluations, 0.01 s. of CPU time.
WSSR: 1.2455138894E-01 (-99.9988%)
=-> i errors
Standard errors:
$_1 = 2.3894212918E+02 +- 2.7070075241E+00
$_2 = 5.5015643181E-04 +- 7.2668688436E-06
If Fityk was compiled with the option to use the long double type for
calculations (USE_LONG_DOUBLE set to 1 in
fityk.h )
we get exactly the same numbers as the certified values.
With the double precision used (which is the default) the parameters and
standard deviations differ at 10th or 11th decimal place.
All the results shown below are obtained with Fityk 1.0.0 compiled
with (only) double precision.
Parameter
Standard Deviation
b1 certified
2.3894212918E+02
2.7070075241E+00
b1 fityk/double
2.3894212923E+02
2.7070075253E+00
b2 certified
5.5015643181E-04
7.2668688436E-06
b2 fityk/double
5.5015643168E-04
7.2668688435E-06
define Chwirut2(b1, b2, b3) = exp(-b1*x)/(b2+b3*x)
F = Chwirut2(~0.1, ~0.01, ~0.02)
Again, we get exactly the certified results using the long double type:
=-> i errors
Standard errors:
$_1 = 1.6657666537E-01 +- 3.8303286810E-02
$_2 = 5.1653291286E-03 +- 6.6621605126E-04
$_3 = 1.2150007096E-02 +- 1.5304234767E-03
And the only difference when using double is in b2 on 11th place.
Parameter
Standard Deviation
b1 certified
1.6657666537E-01
3.8303286810E-02
b1 fityk/double
1.6657666537E-01
3.8303286810E-02
b2 certified
5.1653291286E-03
6.6621605126E-04
b2 fityk/double
5.1653291285E-03
6.6621605127E-04
b3 certified
1.2150007096E-02
1.5304234767E-03
b3 fityk/double
1.2150007096E-02
1.5304234767E-03
define Chwirut1(b1, b2, b3) = exp(-b1*x)/(b2+b3*x)
F = Chwirut1(~0.1, ~0.01, ~0.02)
Parameter
Standard Deviation
b1 certified
1.9027818370E-01
2.1938557035E-02
b1 fityk/double
1.9027818358E-01
2.1938557029E-02
b2 certified
6.1314004477E-03
3.4500025051E-04
b2 fityk/double
6.1314004468E-03
3.4500025053E-04
b3 certified
1.0530908399E-02
7.9281847748E-04
b3 fityk/double
1.0530908402E-02
7.9281847747E-04
define Lanczos3(b1, b2, b3, b4, b5, b6) = b1*exp(-b2*x) + b3*exp(-b4*x) + b5*exp(-b6*x)
F = Lanczos3(~1.2, ~0.3, ~5.6, ~5.5, ~6.5, ~7.6)
Parameter
Standard Deviation
b1 certified
8.6816414977E-02
1.7197908859E-02
b1 fityk/double
8.6816415134E-02
1.7197908832E-02
b2 certified
9.5498101505E-01
9.7041624475E-02
b2 fityk/double
9.5498101593E-01
9.7041624152E-02
b3 certified
8.4400777463E-01
4.1488663282E-02
b3 fityk/double
8.4400777500E-01
4.1488663264E-02
b4 certified
2.9515951832E+00
1.0766312506E-01
b4 fityk/double
2.9515951842E+00
1.0766312488E-01
b5 certified
1.5825685901E+00
5.8371576281E-02
b5 fityk/double
1.5825685896E+00
5.8371576235E-02
b6 certified
4.9863565084E+00
3.4436403035E-02
b6 fityk/double
4.9863565087E+00
3.4436403007E-02
define Gauss1(b1, b2, b3, b4, b5, b6, b7, b8) = b1*exp(-b2*x) + b3*exp(-(x-b4)^2 / b5^2) + b6*exp(-(x-b7)^2 / b8^2)
F = Gauss1(~97.0, ~0.009, ~100.0, ~65.0, ~20.0, ~70.0, ~178.0, ~16.5)
This is the first dataset that gives all 11 digits in all results the same
as the NIST certified values.
Parameter
Standard Deviation
b1 certified
9.8778210871E+01
5.7527312730E-01
b1 fityk/double
9.8778210871E+01
5.7527312730E-01
b2 certified
1.0497276517E-02
1.1406289017E-04
b2 fityk/double
1.0497276517E-02
1.1406289017E-04
b3 certified
1.0048990633E+02
5.8831775752E-01
b3 fityk/double
1.0048990633E+02
5.8831775752E-01
b4 certified
6.7481111276E+01
1.0460593412E-01
b4 fityk/double
6.7481111276E+01
1.0460593412E-01
b5 certified
2.3129773360E+01
1.7439951146E-01
b5 fityk/double
2.3129773360E+01
1.7439951146E-01
b6 certified
7.1994503004E+01
6.2622793913E-01
b6 fityk/double
7.1994503004E+01
6.2622793913E-01
b7 certified
1.7899805021E+02
1.2436988217E-01
b7 fityk/double
1.7899805021E+02
1.2436988217E-01
b8 certified
1.8389389025E+01
2.0134312832E-01
b8 fityk/double
1.8389389025E+01
2.0134312832E-01
define Gauss2(b1, b2, b3, b4, b5, b6, b7, b8) = b1*exp(-b2*x) + b3*exp(-(x-b4)^2 / b5^2) + b6*exp(-(x-b7)^2 / b8^2)
F = Gauss2(~97.0, ~0.009, ~100.0, ~65.0, ~20.0, ~70.0, ~178.0, ~16.5)
Parameter
Standard Deviation
b1 certified
9.9018328406E+01
5.3748766879E-01
b1 fityk/double
9.9018328406E+01
5.3748766879E-01
b2 certified
1.0994945399E-02
1.3335306766E-04
b2 fityk/double
1.0994945399E-02
1.3335306766E-04
b3 certified
1.0188022528E+02
5.9217315772E-01
b3 fityk/double
1.0188022528E+02
5.9217315773E-01
b4 certified
1.0703095519E+02
1.5006798316E-01
b4 fityk/double
1.0703095519E+02
1.5006798316E-01
b5 certified
2.3578584029E+01
2.2695595067E-01
b5 fityk/double
2.3578584028E+01
2.2695595067E-01
b6 certified
7.2045589471E+01
6.1721965884E-01
b6 fityk/double
7.2045589471E+01
6.1721965883E-01
b7 certified
1.5327010194E+02
1.9466674341E-01
b7 fityk/double
1.5327010194E+02
1.9466674342E-01
b8 certified
1.9525972636E+01
2.6416549393E-01
b8 fityk/double
1.9525972637E+01
2.6416549394E-01
define DanWood(b1, b2) = b1*x^b2
F = DanWood(~1, ~5)
All digits the same.
Parameter
Standard Deviation
b1 certified
7.6886226176E-01
1.8281973860E-02
b1 fityk/double
7.6886226176E-01
1.8281973860E-02
b2 certified
3.8604055871E+00
5.1726610913E-02
b2 fityk/double
3.8604055871E+00
5.1726610913E-02
define Misra1b(b1, b2) = b1 * (1-(1+b2*x/2)^(-2))
F = Misra1b(~500, ~0.0001)
Again, we get exactly the certified results.
Parameter
Standard Deviation
b1 certified
3.3799746163E+02
3.1643950207E+00
b1 fityk/double
3.3799746163E+02
3.1643950207E+00
b2 certified
3.9039091287E-04
4.2547321834E-06
b2 fityk/double
3.9039091287E-04
4.2547321834E-06
define Kirby2(b1, b2, b3, b4, b5) = (b1 + b2*x + b3*x^2) / (1 + b4*x + b5*x^2)
F = Kirby2(~2, ~-0.1, ~0.003, ~-0.001, ~0.00001)
Parameter
Standard Deviation
b1 certified
1.6745063063E+00
8.7989634338E-02
b1 fityk/double
1.6745063047E+00
8.7989634326E-02
b2 certified
-1.3927397867E-01
4.1182041386E-03
b2 fityk/double
-1.3927397859E-01
4.1182041376E-03
b3 certified
2.5961181191E-03
4.1856520458E-05
b3 fityk/double
2.5961181182E-03
4.1856520445E-05
b4 certified
-1.7241811870E-03
5.8931897355E-05
b4 fityk/double
-1.7241811882E-03
5.8931897332E-05
b5 certified
2.1664802578E-05
2.0129761919E-07
b5 fityk/double
2.1664802573E-05
2.0129761914E-07
define Hahn1(b1, b2, b3, b4, b5, b6, b7) = (b1+b2*x+b3*x^2+b4*x^3) / (1+b5*x+b6*x^2+b7*x^3)
F = Hahn1(~10, ~-1, ~0.05, ~-0.00001, ~-0.05, ~0.001, ~-0.000001)
Parameter
Standard Deviation
b1 certified
1.0776351733E+00
1.7070154742E-01
b1 fityk/double
1.0776351730E+00
1.7070154741E-01
b2 certified
-1.2269296921E-01
1.2000289189E-02
b2 fityk/double
-1.2269296919E-01
1.2000289189E-02
b3 certified
4.0863750610E-03
2.2508314937E-04
b3 fityk/double
4.0863750606E-03
2.2508314935E-04
b4 certified
-1.4262662514E-06
2.7578037666E-07
b4 fityk/double
-1.4262662508E-06
2.7578037665E-07
b5 certified
-5.7609940901E-03
2.4712888219E-04
b5 fityk/double
-5.7609940903E-03
2.4712888218E-04
b6 certified
2.4053735503E-04
1.0449373768E-05
b6 fityk/double
2.4053735501E-04
1.0449373767E-05
b7 certified
-1.2314450199E-07
1.3027335327E-08
b7 fityk/double
-1.2314450196E-07
1.3027335327E-08
Dataset Nelson has 2 predictors (x1 and x2), Fityk does not handle 3D data,
we skip this test.
define MGH17(b1, b2, b3, b4, b5) = b1 + b2*exp(-x*b4) + b3*exp(-x*b5)
F = MGH17(~0.5, ~1.5, ~-1, ~0.01, ~0.02)
Parameter
Standard Deviation
b1 certified
3.7541005211E-01
2.0723153551E-03
b1 fityk/double
3.7541005211E-01
2.0723153551E-03
b2 certified
1.9358469127E+00
2.2031669222E-01
b2 fityk/double
1.9358469126E+00
2.2031669217E-01
b3 certified
-1.4646871366E+00
2.2175707739E-01
b3 fityk/double
-1.4646871365E+00
2.2175707733E-01
b4 certified
1.2867534640E-02
4.4861358114E-04
b4 fityk/double
1.2867534640E-02
4.4861358109E-04
b5 certified
2.2122699662E-02
8.9471996575E-04
b5 fityk/double
2.2122699662E-02
8.9471996569E-04
define Lanczos1(b1, b2, b3, b4, b5, b6) = b1*exp(-b2*x) + b3*exp(-b4*x) + b5*exp(-b6*x)
F = Lanczos1(~1.2, ~0.3, ~5.6, ~5.5, ~6.5, ~7.6)
Parameter
Standard Deviation
b1 certified
9.5100000027E-02
5.3347304234E-11
b1 fityk/double
9.5100000027E-02
5.3291941815E-11
b2 certified
1.0000000001E+00
2.7473038179E-10
b2 fityk/double
1.0000000001E+00
2.7444527387E-10
b3 certified
8.6070000013E-01
1.3576062225E-10
b3 fityk/double
8.6070000013E-01
1.3561973349E-10
b4 certified
3.0000000002E+00
3.3308253069E-10
b4 fityk/double
3.0000000002E+00
3.3273686649E-10
b5 certified
1.5575999998E+00
1.8815731448E-10
b5 fityk/double
1.5575999998E+00
1.8796204982E-10
b6 certified
5.0000000001E+00
1.1057500538E-10
b6 fityk/double
5.0000000001E+00
1.1046025358E-10
define Lanczos2(b1, b2, b3, b4, b5, b6) = b1*exp(-b2*x) + b3*exp(-b4*x) + b5*exp(-b6*x)
F = Lanczos2(~1.2, ~0.3, ~5.6, ~5.5, ~6.5, ~7.6)
Parameter
Standard Deviation
b1 certified
9.6251029939E-02
6.6770575477E-04
b1 fityk/double
9.6251029939E-02
6.6770575433E-04
b2 certified
1.0057332849E+00
3.3989646176E-03
b2 fityk/double
1.0057332849E+00
3.3989646154E-03
b3 certified
8.6424689056E-01
1.7185846685E-03
b3 fityk/double
8.6424689056E-01
1.7185846674E-03
b4 certified
3.0078283915E+00
4.1707005856E-03
b4 fityk/double
3.0078283915E+00
4.1707005829E-03
b5 certified
1.5529016879E+00
2.3744381417E-03
b5 fityk/double
1.5529016879E+00
2.3744381401E-03
b6 certified
5.0028798100E+00
1.3958787284E-03
b6 fityk/double
5.0028798100E+00
1.3958787275E-03
define Gauss3(b1, b2, b3, b4, b5, b6, b7, b8) = b1*exp(-b2*x) + b3*exp(-(x-b4)^2 / b5^2) + b6*exp(-(x-b7)^2 / b8^2)
F = Gauss3(~97.0, ~0.009, ~100.0, ~65.0, ~20.0, ~70.0, ~178.0, ~16.5)
Parameter
Standard Deviation
b1 certified
9.8940368970E+01
5.3005192833E-01
b1 fityk/double
9.8940368970E+01
5.3005192832E-01
b2 certified
1.0945879335E-02
1.2554058911E-04
b2 fityk/double
1.0945879335E-02
1.2554058910E-04
b3 certified
1.0069553078E+02
8.1256587317E-01
b3 fityk/double
1.0069553078E+02
8.1256587330E-01
b4 certified
1.1163619459E+02
3.5317859757E-01
b4 fityk/double
1.1163619459E+02
3.5317859756E-01
b5 certified
2.3300500029E+01
3.6584783023E-01
b5 fityk/double
2.3300500027E+01
3.6584783020E-01
b6 certified
7.3705031418E+01
1.2091239082E+00
b6 fityk/double
7.3705031424E+01
1.2091239080E+00
b7 certified
1.4776164251E+02
4.0488183351E-01
b7 fityk/double
1.4776164250E+02
4.0488183353E-01
b8 certified
1.9668221230E+01
3.7806634336E-01
b8 fityk/double
1.9668221231E+01
3.7806634338E-01
define Misra1c(b1, b2) = b1 * (1-(1+2*b2*x)^(-.5))
F = Misra1c(~500, ~0.0001)
Parameter
Standard Deviation
b1 certified
6.3642725809E+02
4.6638326572E+00
b1 fityk/double
6.3642725809E+02
4.6638326572E+00
b2 certified
2.0813627256E-04
1.7728423155E-06
b2 fityk/double
2.0813627256E-04
1.7728423155E-06
define Misra1d(b1, b2) = b1*b2*x*((1+b2*x)^(-1))
F = Misra1d(~500, ~0.0001)
Parameter
Standard Deviation
b1 certified
4.3736970754E+02
3.6489174345E+00
b1 fityk/double
4.3736970754E+02
3.6489174345E+00
b2 certified
3.0227324449E-04
2.9334354479E-06
b2 fityk/double
3.0227324449E-04
2.9334354479E-06
define Roszman1(b1, b2, b3, b4) = b1 - b2*x - atan(b3/(x-b4))/pi
F = Roszman1(~0.1, ~-0.00001, ~1000, ~-100)
Parameter
Standard Deviation
b1 certified
2.0196866396E-01
1.9172666023E-02
b1 fityk/double
2.0196866395E-01
1.9172666023E-02
b2 certified
-6.1953516256E-06
3.2058931691E-06
b2 fityk/double
-6.1953516245E-06
3.2058931692E-06
b3 certified
1.2044556708E+03
7.4050983057E+01
b3 fityk/double
1.2044556708E+03
7.4050983056E+01
b4 certified
-1.8134269537E+02
4.9573513849E+01
b4 fityk/double
-1.8134269538E+02
4.9573513852E+01
define Enso(b1, b2, b3, b4, b5, b6, b7, b8, b9) = b1 + b2*cos(t/12) + b3*sin(t/12) + b5*cos(t/b4) + b6*sin(t/b4) + b8*cos(t/b7) + b9*sin(t/b7) where t=2*pi*x
F = Enso(~11.0, ~3.0, ~0.5, ~40.0, ~-0.7, ~-1.3, ~25.0, ~-0.3, ~1.4)
Parameter
Standard Deviation
b1 certified
1.0510749193E+01
1.7488832467E-01
b1 fityk/double
1.0510749192E+01
1.7488832468E-01
b2 certified
3.0762128085E+00
2.4310052139E-01
b2 fityk/double
3.0762128089E+00
2.4310052139E-01
b3 certified
5.3280138227E-01
2.4354686618E-01
b3 fityk/double
5.3280138256E-01
2.4354686618E-01
b4 certified
4.4311088700E+01
9.4408025976E-01
b4 fityk/double
4.4311088678E+01
9.4408025781E-01
b5 certified
-1.6231428586E+00
2.8078369611E-01
b5 fityk/double
-1.6231428614E+00
2.8078369522E-01
b6 certified
5.2554493756E-01
4.8073701119E-01
b6 fityk/double
5.2554493176E-01
4.8073701153E-01
b7 certified
2.6887614440E+01
4.1612939130E-01
b7 fityk/double
2.6887614416E+01
4.1612939012E-01
b8 certified
2.1232288488E-01
5.1460022911E-01
b8 fityk/double
2.1232286294E-01
5.1460022934E-01
b9 certified
1.4966870418E+00
2.5434468893E-01
b9 fityk/double
1.4966870451E+00
2.5434468712E-01
define MGH09(b1, b2, b3, b4) = b1*(x^2+x*b2) / (x^2+x*b3+b4)
F = MGH09(~25, ~39, ~41.5, ~39)
Parameter
Standard Deviation
b1 certified
1.9280693458E-01
1.1435312227E-02
b1 fityk/double
1.9280693477E-01
1.1435312166E-02
b2 certified
1.9128232873E-01
1.9633220911E-01
b2 fityk/double
1.9128232455E-01
1.9633220578E-01
b3 certified
1.2305650693E-01
8.0842031232E-02
b3 fityk/double
1.2305650614E-01
8.0842030241E-02
b4 certified
1.3606233068E-01
9.0025542308E-02
b4 fityk/double
1.3606232875E-01
9.0025541048E-02
define Thurber(b1, b2, b3, b4, b5, b6, b7) = (b1 + b2*x + b3*x^2 + b4*x^3) / (1 + b5*x + b6*x^2 + b7*x^3)
F = Thurber(~1000, ~1000, ~400, ~40, ~0.7, ~0.3, ~0.03)
Parameter
Standard Deviation
b1 certified
1.2881396800E+03
4.6647963344E+00
b1 fityk/double
1.2881396799E+03
4.6647963334E+00
b2 certified
1.4910792535E+03
3.9571156086E+01
b2 fityk/double
1.4910792548E+03
3.9571155198E+01
b3 certified
5.8323836877E+02
2.8698696102E+01
b3 fityk/double
5.8323836966E+02
2.8698695459E+01
b4 certified
7.5416644291E+01
5.5675370270E+00
b4 fityk/double
7.5416644466E+01
5.5675368991E+00
b5 certified
9.6629502864E-01
3.1333340687E-02
b5 fityk/double
9.6629502951E-01
3.1333340084E-02
b6 certified
3.9797285797E-01
1.4984928198E-02
b6 fityk/double
3.9797285837E-01
1.4984927908E-02
b7 certified
4.9727297349E-02
6.5842344623E-03
b7 fityk/double
4.9727297607E-02
6.5842343110E-03
define BoxBOD(b1, b2) = b1*(1-exp(-b2*x))
F = BoxBOD(~100, ~0.75)
Parameter
Standard Deviation
b1 certified
2.1380940889E+02
1.2354515176E+01
b1 fityk/double
2.1380940890E+02
1.2354515178E+01
b2 certified
5.4723748542E-01
1.0455993237E-01
b2 fityk/double
5.4723748534E-01
1.0455993235E-01
define Rat42(b1, b2, b3) = b1 / (1+exp(b2-b3*x))
F = Rat42(~100, ~1, ~0.1)
Parameter
Standard Deviation
b1 certified
7.2462237576E+01
1.7340283401E+00
b1 fityk/double
7.2462237576E+01
1.7340283401E+00
b2 certified
2.6180768402E+00
8.8295217536E-02
b2 fityk/double
2.6180768402E+00
8.8295217536E-02
b3 certified
6.7359200066E-02
3.4465663377E-03
b3 fityk/double
6.7359200066E-02
3.4465663377E-03
define MGH10(b1, b2, b3) = b1 * exp(b2/(x+b3))
F = MGH10(~0.02, ~4000, ~250)
Parameter
Standard Deviation
b1 certified
5.6096364710E-03
1.5687892471E-04
b1 fityk/double
5.6096364710E-03
1.5687892478E-04
b2 certified
6.1813463463E+03
2.3309021107E+01
b2 fityk/double
6.1813463463E+03
2.3309021118E+01
b3 certified
3.4522363462E+02
7.8486103508E-01
b3 fityk/double
3.4522363462E+02
7.8486103544E-01
define Eckerle4(b1, b2, b3) = b1 * exp(b2/(x+b3))
F = Eckerle4(~1.5, ~5, ~450)
Parameter
Standard Deviation
b1 certified
1.5543827178E+00
1.5408051163E-02
b1 fityk/double
1.5543827179E+00
1.5408051165E-02
b2 certified
4.0888321754E+00
4.6803020753E-02
b2 fityk/double
4.0888321764E+00
4.6803020765E-02
b3 certified
4.5154121844E+02
4.6800518816E-02
b3 fityk/double
4.5154121844E+02
4.6800518828E-02
define Rat43(b1, b2, b3, b4) = b1 / ((1+exp(b2-b3*x))^(1/b4))
F = Rat43(~100, ~10, ~1, ~1)
Parameter
Standard Deviation
b1 certified
6.9964151270E+02
1.6302297817E+01
b1 fityk/double
6.9964151266E+02
1.6302297809E+01
b2 certified
5.2771253025E+00
2.0828735829E+00
b2 fityk/double
5.2771253072E+00
2.0828735841E+00
b3 certified
7.5962938329E-01
1.9566123451E-01
b3 fityk/double
7.5962938380E-01
1.9566123468E-01
b4 certified
1.2792483859E+00
6.8761936385E-01
b4 fityk/double
1.2792483872E+00
6.8761936428E-01
define Bennett5(b1, b2, b3) = b1 * (b2+x)^(-1/b3)
F = Bennett5(~-2000, ~50, ~0.8)
Parameter
Standard Deviation
b1 certified
-2.5235058043E+03
2.9715175411E+02
b1 fityk/double
-2.5235058043E+03
2.9715175183E+02
b2 certified
4.6736564644E+01
1.2448871856E+00
b2 fityk/double
4.6736564644E+01
1.2448871760E+00
b3 certified
9.3218483193E-01
2.0272299378E-02
b3 fityk/double
9.3218483193E-01
2.0272299222E-02