csv_writer - bruno-beloff/scs_analysis GitHub Wiki
docs > software repositories > scs_analysis > commands > file-based data management
DESCRIPTION
The csv_writer utility is used to convert from JSON format to comma-separated value (CSV) format.
The path into the JSON document is used to name the column in the header row: dictionary fields are separated from their container by a period ('.') character, and array members are separated by a colon (':') character.
The --quote-all flag forces quote ('"') characters around all cell values. This is useful when long numeric codes such as IMEI numbers are included in the data, such codes should not be interpreted as floats by spreadsheet applications. The default behaviour is to quote only where necessary.
DEFAULT MODE:
All the leaf nodes of the first JSON document are included in the CSV. If subsequent JSON documents in the input stream contain fields that were not in this first document, these extra fields are ignored. If subsequent JSON documents do not contain a field that is in the header, then this field is given the null value.
HEADER-SCAN MODE:
All input documents are scanned in order to build an inclusive hearer row. Any documents that do not contain the header fields are assigned a null value for that field. Any values bound to paths that become internal nodes are discarded.
Warning: the header-scan mode requires memory proportional to the size of its input.
SYNOPSIS
csv_writer.py [{ -a | -x | -s }] [-l LIMIT] [-q] [-e] [-v] [FILENAME]
| Options | |
|---|---|
| --version | show program's version number and exit |
| -h, --help | show this help message and exit |
| -a, --append | append rows to existing file |
| -x, --exclude-header | do not write the header row to stdout |
| -s, --header-scan | scan all documents before building the header row |
| -l LIMIT, --limit=LIMIT | output a maximum of LIMIT rows |
| -q, --quote-all | wrap all CSV cell values in quotes |
| -e, --echo | echo stdin to stdout |
| -v, --verbose | report narrative to stderr |
EXAMPLES
socket_receiver.py | ./csv_writer.py temp.csv -e
DOCUMENT EXAMPLE - INPUT
{"tag": "scs-ap1-6", "rec": "2018-04-04T14:50:27.641+00:00", "val": {"hmd": 59.6, "tmp": 23.8}}
DOCUMENT EXAMPLE - OUTPUT
tag,rec,val.hmd,val.tmp
scs-ap1-6,2018-04-04T14:50:38.394+00:00,59.7,23.8
SEE ALSO
scs_analysis/csv_collator
scs_analysis/csv_segmentor
scs_analysis/csv_join
scs_analysis/csv_writer