Troubleshooting - dathere/qsv GitHub Wiki

Troubleshooting

Tier: Beginner

Common errors and fixes. When in doubt, also check the Discussions Q&A category — many issues have been hashed out there.

Install & launch

Illegal instruction (core dumped) / SIGILL on x86_64

Important

The advice to "switch to the portable qsvp variant" is obsolete. The p-suffixed binaries were retired in 22.0.1 and are in no release archive. They were redundant — -C target-cpu=native had already been removed from the x86_64 publish targets, so the regular prebuilts are the portable ones.

A SIGILL from a current prebuilt is therefore a real bug — please file an issue with your CPU model (lscpu on Linux, Get-WmiObject Win32_Processor on Windows). As a workaround, build for your own CPU:

cargo install qsv --locked --features all_features

Apple Silicon / Windows-on-ARM / IBM Power / s390x are unaffected.

Warning

New in 24.0.0: the x86_64 prebuilts (every variant except qsvdp) require AVX2/FMA/BMI2 (#4649). On a CPU without them, Illegal instruction is the expected result, not a bug — check with grep -o avx2 /proc/cpuinfo | head -1 (Linux) and, if it prints nothing, build from source as above or stay on a 23.0.x release. If qsv --update already put you on a binary that won't start, it can't update itself back: re-download 23.0.x from the releases page. Details: Installation.

Windows: the Easy Installer said "Successfully installed" but qsv is missing or stale

Easy Installer v1.1.1 and earlier extract a hardcoded qsvp.exe — the portable binary retired in 22.0.1 — and its update path returns no error, so it reports success even when nothing was installed.

Fix: install Easy Installer v1.1.2 or later and re-run it, then verify with qsv --version. Or skip the installer entirely: download the x86_64-pc-windows-msvc archive from releases and extract qsv.exe to a directory on your PATH.

qsv <cmd> file.csv -o file.csv produced an empty file

Fixed in 23.0.1. Most commands created the --output writer before opening the input reader, so naming the input as the output truncated it to zero bytes — the command then read an empty file and exited 0 with no warning. qsv fmt big.csv -o big.csv silently destroyed the data.

qsv now detects that the input and output are the same file (by file identity, not by string comparison, so ./x.csv and x.csv are caught too) and errors out instead. If you are on an older release, always write to a different path and rename afterwards.

macOS: "qsv cannot be opened because the developer cannot be verified"

qsv binaries are signed with an ad-hoc signature. Either:

xattr -d com.apple.quarantine qsv

…or right-click the binary → Open → confirm in the Gatekeeper dialog. See Apple's Gatekeeper docs.

Windows: "qsv is not recognized"

The MSI Easy Installer adds qsv to your PATH — but only for new terminals. Close and reopen your terminal after install. If still not found, the installer may have failed; check the install log.

qsv --version reports unexpected features

The binary's enabled features depend on the build. Run qsv --version to see them — package-manager builds differ from the official prebuilt binaries. For the full feature set, install the prebuilt or build from source with -F all_features. See Binary Variants.

Want the luau feature on musl Linux?

Not in the prebuilt — musl builds are cross-compiled and can't statically link Luau. Build from source on your musl distro (Alpine, Void, …) with cargo build --release --locked --bin qsv -F all_features.

Encoding & format

"invalid UTF-8 sequence" / mojibake characters

The input isn't UTF-8. Two paths:

# Path A: Lossy normalization (substitute � for invalid bytes)
qsv input --encoding-errors replace messy.csv > utf8.csv

# Path B: Proper transcoding (recommended)
iconv -f ISO-8859-1 -t UTF-8 messy.csv -o utf8.csv
qsv input utf8.csv > clean.csv

qsv input is lossy, not transcoding-correct. Use iconv first for known-source encodings. See Transform & Reshape → input.

Mojibake in a fetch/fetchpost response column is a different problem: the server sent latin-1/windows-1252 text without a charset in its Content-Type, so it was decoded as UTF-8. New in 24.0.0: pass --default-encoding windows-1252 (see HTTP & Web).

Wrong delimiter detected

# Explicit
qsv stats --delimiter ';' data.ssv

# Or: tell qsv to sniff every input
export QSV_SNIFF_DELIMITER=1
qsv stats data.weird-delim

For a one-off normalization to comma:

qsv fmt --out-delimiter ',' data.weird > data.csv

"CSV record has X fields, but expected Y" (ragged rows)

The CSV has rows of inconsistent width. Fix with:

qsv fixlengths ragged.csv > fixed.csv

Or pass --flexible to commands that accept it (count --flexible).

New in 24.0.0: stats and frequency accept --flexible too. Without it, a ragged record used to kill them silently with no output (exit 133 / 139) — undefined behavior in the record reader, not a clean error; they now report it (#4614).

BOM in the first column header

Strip with:

qsv input --trim-headers bom.csv > clean.csv

For generating Excel-friendly CSVs with a BOM:

QSV_OUTPUT_BOM=1 qsv stats data.csv --output stats_excel.csv

Index & cache

Index is stale; please rebuild it

qsv index --force data.csv

Or auto-refresh by setting QSV_AUTOINDEX_SIZE=<bytes> — see Performance Tuning.

Stats cache not being picked up

Smart commands (frequency, validate, schema, …) consume <file>.stats.csv.data.jsonl, not <file>.stats.csv. Regenerate with:

qsv stats --stats-jsonl <file>

Or set QSV_STATSCACHE_MODE=force. See Stats Cache & Caching.

Polars schema drift / "schema inference error"

Polars commands (sqlp, joinp, pivotp) inferred a column as Int64 but a later row has 12.5. Fix:

qsv schema --polars --force data.csv
# Hand-edit data.pschema.json: change the column's dtype to Float64

Polars commands pick up the corrected .pschema.json automatically.

sqlp query that worked before now fails

New in 24.0.0: several Polars SQL behaviors changed (details):

  • no table or struct column named '<t>' found means you aliased a table (FROM t AS x) but still refer to it as t.col. Use x.col.
  • could not find an appropriate format to parse dates is the new wording of an old error: a CAST to a date/time type hit a value it couldn't parse. That was already fatal on 23.0.1 and earlier. Use TRY_CAST, or STRPTIME(col, '%m/%d/%Y') for US-style dates.

Memory

OOM killer / qsv exited unexpectedly on a big file

In-memory commands (🤯) — sort, dedup, reverse, table, transpose, pragmastat, stats --everything — can OOM on huge files. Three fixes:

  1. Switch to the streaming variant:

    • sort → extsort
    • dedup → extdedup
    • stats with --cardinality-method approx --quantile-method approx, plus --mode-cardinality-cap under --everything/--mode (modes keep an exact table of every distinct value; only the cap bounds it)
  2. Enable the pre-flight memory check:

export QSV_MEMORY_CHECK=1
export QSV_FREEMEMORY_HEADROOM_PCT=10
  1. Constrain stats / frequency chunk memory:
export QSV_STATS_CHUNK_MEMORY_MB=2048
export QSV_FREQ_CHUNK_MEMORY_MB=1024

See Recipe: Larger-than-RAM CSV.

frequency runs out of memory on an ID column

Pre-populate the stats cache so frequency knows the column is ALL_UNIQUE and short-circuits it:

qsv stats --cardinality --stats-jsonl data.csv
qsv frequency data.csv

Or switch to the heavy-hitters mode:

qsv frequency --sketch-method frequent_items --sketch-map-size 16384 data.csv

Specific commands

qsv stats: "infer_dates is set but no date-like columns found"

--infer-dates only checks columns whose names match the --dates-whitelist patterns (default: sniff). To force date inference on a column:

qsv stats --infer-dates --dates-whitelist 'mydate,timestamp,created_at' data.csv

qsv search returns no matches but you can see them

Regex is case-sensitive by default. Add -i:

qsv search -i 'brooklyn' data.csv

For non-ASCII characters, enable Unicode mode:

QSV_REGEX_UNICODE=1 qsv search 'café' data.csv

(Unicode mode is slower; off by default.)

qsv fetch returns errors against a known-good API

Likely a rate limit. Check the report:

# --report takes d|s and writes a sidecar (data.csv.fetch-report.tsv), not stdout
qsv fetch URL data.csv --report d --new-column response > out.csv
qsv frequency --select qsv_fetch_status data.csv.fetch-report.tsv

Add --rate-limit N to throttle, or --disk-cache to dedupe repeated calls. See Recipe: Fetch & Cache.

qsv excel: "case sensitivity" / "negative sheet index" failures

Recent qsv versions fix several Excel edge cases. Run qsv --version and update if you're on an older release. The 20.0.0 and 20.1.0 changelogs have the full list of excel fixes.

qsv luau not available

The luau feature isn't enabled in qsvlite / qsvdp and isn't available in musl prebuilds. Install the standard qsv or qsvpy* variant, or build from source. See Binary Variants.

Diagnostics

Quick env dump for bug reports

qsv --version
qsv --envlist

Paste both in your Discussions / Issues post.

Enable verbose logging

export QSV_LOG_LEVEL=debug
export QSV_LOG_DIR=./qsv-logs
qsv stats data.csv
ls qsv-logs/

See docs/Logging.md.

Where do I report bugs?

Always include qsv --version and qsv --envlist output, plus a minimal reproducer.

See also

⚠️ **GitHub.com Fallback** ⚠️