sample_nullify - bruno-beloff/scs_analysis GitHub Wiki
docs > software repositories > scs_analysis > commands > filtering and aggregating data
DESCRIPTION
The sample_nullify utility is used to remove values where that value - or some other value in the same JSON document - is outside one or two bounding values. If bounds are present, values must be numeric. If not present, the JSON document is passed to stdout without alteration.
Evaluation follows the rule:
lower bound <= value < upper bound
Both upper and lower bounds are optional. If both are present, then the lower bounding value must be less than the upper bounding value.
SYNOPSIS
sample_nullify.py -t TARGET_PATH -s SOURCE_PATH [-l LOWER] [-u UPPER] [-v]
| Options | |
|---|---|
| --version | show program's version number and exit |
| -h, --help | show this help message and exit |
| -t TARGET, --target=TARGET | field to be nullified |
| -s SOURCE, --source=SOURCE | field providing the test value |
| -l LOWER, --lower=LOWER | lower bound |
| -u UPPER, --upper=UPPER | upper bound |
| -v, --verbose | report narrative to stderr |
EXAMPLES
csv_reader.py -v scs-bgx-405-corrected-2019-04-1min.csv | sample_nullify.py -v -u 80.1 -s meteo.val.hmd -t proc_PM10 | csv_writer.py -v scs-bgx-405-corrected-2019-04-1min-r80.csv
DOCUMENT EXAMPLE - INPUT
{"rec": "2019-04-27T02:07:00Z", "opc": {"tag": "scs-bgx-405", "src": "N2", "val": {"per": 9.9, "pm1": 2.6, "pm2p5": 3.9, "pm10": 7.0}}, "meteo": {"tag": "scs-bgx-405", "val": {"hmd": 87.6, "tmp": 20.0, "bar": ""}}, "error_model": 4.684, "proc_PM10": 1.5, "proc_PM2p5": 0.8, "proc_PM1": 0.6}
DOCUMENT EXAMPLE - OUTPUT
{"rec": "2019-04-27T02:07:00Z", "opc": {"tag": "scs-bgx-405", "src": "N2", "val": {"per": 9.9, "pm1": 2.6, "pm2p5": 3.9, "pm10": 7.0}}, "meteo": {"tag": "scs-bgx-405", "val": {"hmd": 87.6, "tmp": 20.0, "bar": ""}}, "error_model": 4.684, "proc_PM10": "", "proc_PM2p5": 0.8, "proc_PM1": 0.6}