sample_error - bruno-beloff/scs_analysis GitHub Wiki

docs > software repositories > scs_analysis > commands > filtering and aggregating data

DESCRIPTION

The sample_error utility is used to compute either the difference between or the ratio of a reported value and a reference value for a given path is a stream of JSON documents.

Input is a sequence of JSON documents on stdin, containing both reported and reference values. If the specified REFERENCE_PATH and REPORTED_PATH are not present, the utility terminates. If either value cannot be interpreted as a floating-point value, the document is ignored. If the specified ERROR_PATH is already present in the document, it is overwritten.

SYNOPSIS

sample_error.py { -l | -s } [-p PRECISION] [-v] REFERENCE_PATH REPORTED_PATH ERROR_PATH

Options
--version	show program's version number and exit
-h, --help	show this help message and exit
-l, --linear	error is REPORTED - REFERENCE
-s, --scaling	error is REPORTED / REFERENCE
-p PRECISION, --prec=PRECISION	precision (default 3 decimal place)
-v, --verbose	report narrative to stderr

EXAMPLES

csv_reader.py -v Pi-R1-joined-2019-10-15min.csv | sample_error.py -s -v "fidas.PM25 Converted Measurement" r1.val.pm2p5 error.pm2p5 | sample_error.py -s -v "fidas.PM10 Converted Measurement" r1.val.pm10 error.pm10 | csv_writer.py -v Pi-R1-joined-2019-10-15min-error.csv

DOCUMENT EXAMPLE - INPUT

{"rec": "2019-09-24T13:15:00Z", "fidas": {"Site": "Heathrow LHR2", "SiteCode": "LHR2", "PM10 Processed Measurement": 6.3, "PM10 Converted Measurement": 6.3, "PM10 Data Status": "Provisional", "PM25 Processed Measurement": 3.447, "PM25 Converted Measurement": 3.2519, "PM25 Data Status": "Provisional"}, "r1": {"val": {"mtf1": 32.4, "pm1": 2.1, "mtf5": 34.1, "pm2p5": 6.3, "sht": {"hmd": 29.5, "tmp": 33.0}}, "tag": "scs-pb1-3", "src": "R1"}}

DOCUMENT EXAMPLE - OUTPUT

{"rec": "2019-09-24T13:15:00Z", "fidas": {"Site": "Heathrow LHR2", "SiteCode": "LHR2", "PM10 Processed Measurement": 6.3, "PM10 Converted Measurement": 6.3, "PM10 Data Status": "Provisional", "PM25 Processed Measurement": 3.447, "PM25 Converted Measurement": 3.2519, "PM25 Data Status": "Provisional"}, "r1": {"val": {"mtf1": 32.4, "pm1": 2.1, "mtf5": 34.1, "pm2p5": 6.3, "sht": {"hmd": 29.5, "tmp": 33.0}}, "tag": "scs-pb1-3", "src": "R1"}, "error": {"pm2p5": 1.937, "pm10": 1.889}}