Troubleshooting - dathere/qsv GitHub Wiki
Tier: Beginner
Common errors and fixes. When in doubt, also check the Discussions Q&A category — many issues have been hashed out there.
Important
The advice to "switch to the portable qsvp variant" is obsolete. The p-suffixed binaries were retired in 22.0.1 and are in no release archive. They were redundant — -C target-cpu=native had already been removed from the x86_64 publish targets, so the regular prebuilts are the portable ones.
A SIGILL from a current prebuilt is therefore a real bug — please file an issue with your CPU model (lscpu on Linux, Get-WmiObject Win32_Processor on Windows). As a workaround, build for your own CPU:
cargo install qsv --locked --features all_featuresApple Silicon / Windows-on-ARM / IBM Power / s390x are unaffected.
Warning
New in 24.0.0: the x86_64 prebuilts (every variant except qsvdp) require AVX2/FMA/BMI2 (#4649). On a CPU without them, Illegal instruction is the expected result, not a bug — check with grep -o avx2 /proc/cpuinfo | head -1 (Linux) and, if it prints nothing, build from source as above or stay on a 23.0.x release. If qsv --update already put you on a binary that won't start, it can't update itself back: re-download 23.0.x from the releases page. Details: Installation.
Easy Installer v1.1.1 and earlier extract a hardcoded qsvp.exe — the portable binary retired in 22.0.1 — and its update path returns no error, so it reports success even when nothing was installed.
Fix: install Easy Installer v1.1.2 or later and re-run it, then verify with qsv --version. Or skip the installer entirely: download the x86_64-pc-windows-msvc archive from releases and extract qsv.exe to a directory on your PATH.
Fixed in 23.0.1. Most commands created the --output writer before opening the input reader, so naming the input as the output truncated it to zero bytes — the command then read an empty file and exited 0 with no warning. qsv fmt big.csv -o big.csv silently destroyed the data.
qsv now detects that the input and output are the same file (by file identity, not by string comparison, so ./x.csv and x.csv are caught too) and errors out instead. If you are on an older release, always write to a different path and rename afterwards.
qsv binaries are signed with an ad-hoc signature. Either:
xattr -d com.apple.quarantine qsv…or right-click the binary → Open → confirm in the Gatekeeper dialog. See Apple's Gatekeeper docs.
The MSI Easy Installer adds qsv to your PATH — but only for new terminals. Close and reopen your terminal after install. If still not found, the installer may have failed; check the install log.
The binary's enabled features depend on the build. Run qsv --version to see them — package-manager builds differ from the official prebuilt binaries. For the full feature set, install the prebuilt or build from source with -F all_features. See Binary Variants.
Not in the prebuilt — musl builds are cross-compiled and can't statically link Luau. Build from source on your musl distro (Alpine, Void, …) with cargo build --release --locked --bin qsv -F all_features.
The input isn't UTF-8. Two paths:
# Path A: Lossy normalization (substitute � for invalid bytes)
qsv input --encoding-errors replace messy.csv > utf8.csv
# Path B: Proper transcoding (recommended)
iconv -f ISO-8859-1 -t UTF-8 messy.csv -o utf8.csv
qsv input utf8.csv > clean.csvqsv input is lossy, not transcoding-correct. Use iconv first for known-source encodings. See Transform & Reshape → input.
Mojibake in a fetch/fetchpost response column is a different problem: the server sent latin-1/windows-1252 text without a charset in its Content-Type, so it was decoded as UTF-8. New in 24.0.0: pass --default-encoding windows-1252 (see HTTP & Web).
# Explicit
qsv stats --delimiter ';' data.ssv
# Or: tell qsv to sniff every input
export QSV_SNIFF_DELIMITER=1
qsv stats data.weird-delimFor a one-off normalization to comma:
qsv fmt --out-delimiter ',' data.weird > data.csvThe CSV has rows of inconsistent width. Fix with:
qsv fixlengths ragged.csv > fixed.csvOr pass --flexible to commands that accept it (count --flexible).
New in 24.0.0: stats and frequency accept --flexible too. Without it, a ragged record used to kill them silently with no output (exit 133 / 139) — undefined behavior in the record reader, not a clean error; they now report it (#4614).
Strip with:
qsv input --trim-headers bom.csv > clean.csvFor generating Excel-friendly CSVs with a BOM:
QSV_OUTPUT_BOM=1 qsv stats data.csv --output stats_excel.csvqsv index --force data.csvOr auto-refresh by setting QSV_AUTOINDEX_SIZE=<bytes> — see Performance Tuning.
Smart commands (frequency, validate, schema, …) consume <file>.stats.csv.data.jsonl, not <file>.stats.csv. Regenerate with:
qsv stats --stats-jsonl <file>Or set QSV_STATSCACHE_MODE=force. See Stats Cache & Caching.
Polars commands (sqlp, joinp, pivotp) inferred a column as Int64 but a later row has 12.5. Fix:
qsv schema --polars --force data.csv
# Hand-edit data.pschema.json: change the column's dtype to Float64Polars commands pick up the corrected .pschema.json automatically.
New in 24.0.0: several Polars SQL behaviors changed (details):
-
no table or struct column named '<t>' foundmeans you aliased a table (FROM t AS x) but still refer to it ast.col. Usex.col. -
could not find an appropriate format to parse datesis the new wording of an old error: aCASTto a date/time type hit a value it couldn't parse. That was already fatal on 23.0.1 and earlier. UseTRY_CAST, orSTRPTIME(col, '%m/%d/%Y')for US-style dates.
In-memory commands (🤯) — sort, dedup, reverse, table, transpose, pragmastat, stats --everything — can OOM on huge files. Three fixes:
-
Switch to the streaming variant:
-
Enable the pre-flight memory check:
export QSV_MEMORY_CHECK=1
export QSV_FREEMEMORY_HEADROOM_PCT=10- Constrain
stats/frequencychunk memory:
export QSV_STATS_CHUNK_MEMORY_MB=2048
export QSV_FREQ_CHUNK_MEMORY_MB=1024See Recipe: Larger-than-RAM CSV.
Pre-populate the stats cache so frequency knows the column is ALL_UNIQUE and short-circuits it:
qsv stats --cardinality --stats-jsonl data.csv
qsv frequency data.csvOr switch to the heavy-hitters mode:
qsv frequency --sketch-method frequent_items --sketch-map-size 16384 data.csv--infer-dates only checks columns whose names match the --dates-whitelist patterns (default: sniff). To force date inference on a column:
qsv stats --infer-dates --dates-whitelist 'mydate,timestamp,created_at' data.csvRegex is case-sensitive by default. Add -i:
qsv search -i 'brooklyn' data.csvFor non-ASCII characters, enable Unicode mode:
QSV_REGEX_UNICODE=1 qsv search 'café' data.csv(Unicode mode is slower; off by default.)
Likely a rate limit. Check the report:
# --report takes d|s and writes a sidecar (data.csv.fetch-report.tsv), not stdout
qsv fetch URL data.csv --report d --new-column response > out.csv
qsv frequency --select qsv_fetch_status data.csv.fetch-report.tsvAdd --rate-limit N to throttle, or --disk-cache to dedupe repeated calls. See Recipe: Fetch & Cache.
Recent qsv versions fix several Excel edge cases. Run qsv --version and update if you're on an older release. The 20.0.0 and 20.1.0 changelogs have the full list of excel fixes.
The luau feature isn't enabled in qsvlite / qsvdp and isn't available in musl prebuilds. Install the standard qsv or qsvpy* variant, or build from source. See Binary Variants.
qsv --version
qsv --envlistPaste both in your Discussions / Issues post.
export QSV_LOG_LEVEL=debug
export QSV_LOG_DIR=./qsv-logs
qsv stats data.csv
ls qsv-logs/See docs/Logging.md.
- Questions / how-to: Discussions Q&A
- FAQ-worthy: Discussions FAQ
- Confirmed bugs: Issues
- Security: see SECURITY.md
Always include qsv --version and qsv --envlist output, plus a minimal reproducer.