Geospatial - dathere/qsv GitHub Wiki

Geospatial

Tier: Intermediate Commands covered: geocode, geoconvert

Note

Per-command flag reference lives in /docs/help/. This page is the workflow layer β€” when to reach for each command and how they compose.

Two commands, big footprint. geocode handles forward geocoding, reverse geocoding, city suggestions, IP geolocation, and country info β€” all against a local Geonames index plus an optional MaxMind GeoLite2 database. No network round-trip per row, 360,000 records / sec with caching and multithreading. It can also geocode online via the OpenCage API (the opencage / opencagenow subcommands) when you need true street-address geocoding that the local city index can't do. For New York City, the geoclient / geoclientnow subcommands use NYC's own Geoclient API, which also returns BBLs, BINs, council districts, census tracts and other NYC attributes. For any US address or coordinate, the census / censusnow subcommands use the free, keyless US Census Geocoder, returning the state, county, tract, block group and block GEOIDs you need to join Census statistics.

geoconvert flips between CSV/SVG and spatial formats (GeoJSON, GeoJSONL, SHP).

Quick decision table

If you want to… Use Notes
Find lat/lon from a city name geocode suggest Jaro-Winkler fuzzy matching
Resolve small-town place names the default index misses geocode index-load 1000 New in 23.0.1 β€” the denser cities1000 prebuilt (details)
Find the nearest city to a lat/lon geocode reverse Local index; no API rate limits
Find country code from an IP geocode iplookup Needs MaxMind GeoLite2-City.mmdb
Enrich with US state/county FIPS codes geocode suggest / reverse with FIPS dyncols Census data prep
Geocode a full street address (online) geocode opencage OpenCage API; needs an API key
Reverse-geocode a coordinate to a full address (online) geocode opencage Same subcommand; mode auto-detected
Geocode a New York City address, BBL or BIN (online) geocode geoclient NYC Geoclient API; needs a free NYC API key (details)
Geocode a US address or coordinate to Census tract/block GEOIDs (online) geocode census US Census Geocoder; free, no key; batch for large files (details)
Manage / prune the OpenCage, Geoclient and Census result caches geocode cache-* cache-clear, cache-prune, cache-info
CSV β†’ GeoJSON for QGIS / web maps geoconvert Specify --geometry for WKT or lat/lon
GeoJSON / SHP β†’ CSV geoconvert Drops to a flat tabular form
Plot lat/lon points on an interactive map viz map Tile basemap; point or --density heatmap
Plot lat/lon on an offline projection map viz geo ScatterGeo projection β€” no tiles or token; --projection

geocode

Twenty-one subcommands; the big four are suggest, suggestnow, reverse, reversenow. Plus iplookup / iplookupnow, countryinfo / countryinfonow, the online opencage / opencagenow, geoclient / geoclientnow and census / censusnow subcommands, the geocode index-* subcommands for managing the local Geonames cities index, and the geocode cache-* subcommands for managing the on-disk OpenCage, Geoclient and Census result caches.

Setting up the index

# Downloads the prebuilt Geonames cities15000 index (~26k cities, pop > 15k)
qsv geocode suggest --help    # first run triggers the download

# Manage the local index
qsv geocode index-check                    # show index metadata; check for updates
qsv geocode index-update                   # rebuild from the latest Geonames data
qsv geocode index-load my_index.rkyv       # load a custom-built .rkyv index file
qsv geocode index-load 1000                # NEW in 23.0.1 β€” fetch the denser cities1000 prebuilt
qsv geocode index-reset                    # restore the default prebuilt cities15000 index

Old indexes are cleaned up automatically (25.0.0, #4776). The index file is named for the qsv release (qsv-24.0.0-geocode-index.rkyv), so every upgrade downloads a new one, and older releases never removed the previous ones. A long-lived ~/.qsv-cache/ could hold gigabytes of them. Now, when qsv downloads a new release's index, it keeps the index files of the 5 most recent releases and deletes the rest, printing what it removed. Only files named the way qsv names them are touched. Backups (.bak) and other files stay, and the active index (including one picked with QSV_GEOCODE_INDEX_FILENAME) is always kept. Run qsv geocode index-reset to clean up an existing cache right away.

index-update keeps your index's city set and languages (25.0.0, #4786). --cities-url and --languages used to default to cities15000 and en, so a plain index-update on a cities1000 or multi-language index rebuilt it as cities15000 with English only. They now default to what the current index was built with. Passing a different value forces a rebuild. --languages is case-insensitive and ignores spaces (EN, fr works), and --force can build an index when none exists yet.

No more blank region names (25.0.0, #4786). Region (admin1), county (admin2) and country names used to print blank when the index had no translation for them in the requested language, as the city name never did. "Maputo, MZ" is now "Maputo, Maputo City MZ". This happened even on the default English index, for about 5% of cities in a world-wide sample. A -l/--language the index wasn't built with now prints a warning that says how to rebuild for it.

Parallel runs on a fresh install no longer break the index (25.0.0, #4793). With no index in the cache dir yet, geocode downloaded the index and decompressed it straight into its final file. Any geocode run that started meanwhile, such as a parallel worker or a second terminal, could load it half-written and fail with errors like On load index file: failed to fill whole buffer or Error initializing Engine: unaligned pointer. The index is now written to a temporary file beside it and renamed into place in one step, so other runs see either no index or a complete one. This also applies to index-load <N>. If you hit those errors on 24.0.0 or earlier, rerun after the first download finishes, or run qsv geocode index-reset.

For IP-based lookups, manually download GeoLite2-City.mmdb from your MaxMind account (free) and copy it to ~/.qsv-cache/ (or set QSV_GEOIP2_FILENAME).

Choosing a population floor

The default index is cities15000 β€” every populated place with a population over 15,000. That floor is usually the binding constraint when a place name fails to resolve: the town exists in Geonames, just not in this index.

New in 23.0.1: index-load accepts a published population floor as a shortcut, and 1000 now joins 15000 as a published prebuilt. So reaching the denser index is one command instead of a full local rebuild.

Floor Places Prebuilt index Download How to get it
15000 (default) ~26k worldwide 22.6 MiB β€” ships by default; geocode index-reset restores it
1000 ~140k worldwide 82.7 MiB (32 MiB compressed) one command qsv geocode index-load 1000

index-load <floor> downloads a prebuilt, English-only index built for your qsv version. Contrast with qsv geocode index-update --cities-url 1000, which is the old route: it pulls the ~200 MB Geonames source dump and rebuilds from scratch β€” still the right call when you need non-English place names (its --languages option), but slow and unnecessary if you only want more coverage in English.

Important

The global default is deliberately unchanged. index-load swaps the index for every subsequent geocode run, and a denser index costs roughly 4Γ— the index size and gives different suggest results β€” a small town can now outrank the large city you were matching before. Load 1000 because your data is full of small places, not as a blanket upgrade.

It is also not a fix for cross-country mis-resolution (a US city name matching a same-named place abroad). That is a country-filter problem, not a density problem, and it reproduces identically against a cities1000 index β€” constrain the lookup with --country instead.

A floor this binary knows about can still be missing from the release it was built from. When that happens the download failure now names the release and points at the local-rebuild alternative instead of surfacing a bare 404.

index-load fixes in 23.0.1

Three user-visible bugs in index-load were fixed in 23.0.1. The headline: on 22.0.1 and earlier, index-load did not work at all.

  • index-load no longer rejects every index as invalid β€” including qsv's own published prebuilt. Both forms failed, and the command contradicted itself within a single run, printing Valid: /tmp/15000.rkyv and then a usage error saying that same file was in an unsupported format. The two code paths were asking different questions of the same file; validation now goes through the same metadata API that index-check uses, against the resolved path. The premature Valid: line is gone β€” it came from a helper that only checked the file's extension and existence, so it now reports Found index file: and validity is reported only once the index has actually loaded. This was never a recent regression: the bad gate shipped in 37 releases starting with 10.0.0, unnoticed because the subcommand had no success-path test. It has one now.
  • The numeric shortcut no longer litters the working directory. qsv geocode index-load 1000 downloaded to its bare argument as a relative path, dropping files named 1000 and 1000.rkyv into whatever directory you happened to run it from. The payload is now staged in the geocode cache dir under a descriptive name (cities1000.rkyv.sz), where it doubles as a warm copy for a later re-install. Any stray 1000, 1000.rkyv, 15000 or 15000.rkyv files left behind in old working directories are safe to delete.
  • index-load installs a working index, not a corrupt one it reports as a success. Having accepted a valid index, it then re-serialized the loaded engine in order to install it, producing a file that came out short and could not be loaded at all (Error initializing Engine: subtree pointer overran range) β€” while the command printed a success message. It now copies the file, which is exact and cannot corrupt it, and a failed copy is an error rather than being silently ignored. This defect was only reachable after the validity fix above, so no released version could hit it β€” nothing to clean up retroactively.

Caution

A failed download is no longer written to disk as if it were the file. Fixed in 23.0.1, and it reaches well past geocode. util::download_file streamed whatever the server returned and never checked the HTTP status, so a 404 page became the payload β€” an HTML "Not Found" body saved as a Geonames index, a JSON Schema, or a CSV, with the failure surfacing much later as a baffling parse error about the contents. It now fails on any non-2xx response and names the status. This affected every download path: geocode index fetches, validate's remote JSON Schema, and sample / snappy remote inputs. Remediation: check ~/.qsv-cache/ (or $QSV_CACHE_DIR) for an implausibly small index file β€” an HTML error body is a few hundred bytes where a real index is tens of megabytes.

Forward geocoding (suggest)

Example: enrich a list of city names with coordinates

# input has a "city" column with values like "Brooklyn", "san jose", "PARIS"
qsv geocode suggest city \
  --new-column 'lat,lon' \
  --formatstr "%latitude,%longitude" \
  cities.csv > cities_with_coords.csv

Example: US-only suggestion that also pulls FIPS codes (for Census joins)

qsv geocode suggest city_col --country US -f \
  "%dyncols: {geocoded_city:name},{state:admin1},{county:admin2},\
{state_fips:us_state_fips_code},{county_fips:us_county_fips_code}" \
  voters.csv -o voters_with_fips.csv

The %dyncols: directive expands to multiple new columns named on the left of each : pair.

Restricting to a state or province (--admin1)

# a column of Florida city names
qsv geocode suggest city --admin1 US.FL -f "{name}, {admin1}" fl_cities.csv

New in 24.0.0 (#4687):

  • --admin1 is a strict filter. Only a place inside the listed admin1s is returned. A value with no match there is left unchanged (or set to --invalid-result), and stderr reports how many. Before, suggest fell back to the best match anywhere, so Inverness under US.FL came back as Inver Grove Heights, Minnesota. Small towns missing from the default index are the usual cause of a miss, and index-load 1000 finds more of them (choosing a population floor).
  • US territories work: US.PR, US.VI, US.GU, US.AS and US.MP. Geonames files them as countries of their own, so --admin1 US.PR used to find no Puerto Rico place at all (Ponce came back as Ponca City). An explicit --country US admits them too.
  • Mixed code lists infer every country, so --admin1 US.NY,US.PR needs no --country. Admin1 names still need --country.
  • Admin1 codes are case-insensitive (25.0.0, #4786): --admin1 us.ca works like US.CA. Before, only the territory codes (us.pr) were accepted in lowercase. Matching is by prefix, and a few full Geonames codes are prefixes of others: KZ.12 also matches KZ.12510143.
  • Exact names win. The fuzzy search treats any name that merely starts with the value as a perfect match and breaks ties by population, so "Davis" used to return San Diego. A place whose own name is the value now wins, with or without --admin1.

Reverse geocoding (reverse)

Example: reverse-geocode NYC 311 lat/lon to Borough / NTA

qsv geocode reverse 'Location' \
  --new-column 'nearest_city,admin1,admin2' \
  --formatstr "%name,%admin1,%admin2" \
  --country US \
  NYC_311_SR_2010-2020-sample-1M.csv > nyc311_reversed.csv

Location is the source column in "lat, lon" or "(lat, lon)" format.

Example: reverse-geocode FIPS codes for downstream Census tract joins

qsv geocode reverse coordinate_col --country US -f \
  "%dyncols: {city:name},{state:admin1},{county:admin2},\
{state_fips:us_state_fips_code},{county_fips:us_county_fips_code}" \
  events.csv -o events_with_fips.csv

Census joins need the 5-digit county GEOID. us_county_fips_code is the 3-digit county part (003), unique only within a state β€” prefix it with us_state_fips_code (e.g. -f '{us_state_fips_code}{us_county_fips_code}' β†’ 42003). 24.0.0 and earlier (#4771): any county code β‰₯ 100 came out with its first digit dropped (510 β†’ 010), usually naming a different real county. This is fixed in 25.0.0; regenerate files enriched by older versions. See US Census Integration.

One-shot variants (suggestnow, reversenow, countryinfonow)

For ad-hoc one-off lookups from the command line (not a CSV):

qsv geocode suggestnow "Brooklyn"
qsv geocode reversenow "40.6782, -73.9442"
qsv geocode countryinfonow US

IP lookup (iplookup)

Needs the MaxMind GeoLite2-City.mmdb, which you download yourself (a free MaxMind account is required) and copy to ~/.qsv-cache/, or point to with QSV_GEOIP2_FILENAME.

GeoLite2 finds the IP's city, and qsv then returns that city's record from the local Geonames index. So iplookup takes the same --formatstr formats as suggest/reverse, and its default output is %city-admin1-country (e.g. San Francisco, California US). If the IP can't be located, or its city isn't in your Geonames index (a small town missing from the default cities15000, for example), the IP itself is returned. A URL is resolved to its host's IP first.

qsv geocode iplookup -f %country --new-column country ip_col logs.csv > logs_with_country.csv
qsv geocode iplookupnow 208.67.222.222       # San Francisco, California US: 37.77493, -122.41942

Country info (countryinfo)

qsv geocode countryinfo country_code visitors.csv > with_country_info.csv
# Adds columns: name, capital, area, population, languages, currency, ...

Online geocoding with OpenCage (opencage, opencagenow)

suggest / reverse are city-level and offline β€” fast, but they can't resolve a full street address. When you need real address geocoding, opencage calls the online OpenCage API. One subcommand does forward (address β†’ coordinates) and reverse (coordinates β†’ address) geocoding β€” the mode is auto-detected per row (pass --reverse to force reverse).

Compared with more popular commercial geocoders such as Google Maps, OpenCage is built on open data, chiefly OpenStreetMap, so its results come with far fewer strings (details):

  • you may store results as long as you like, even after you stop being a customer, which is what lets qsv keep them in a persistent cache;
  • results can be displayed on any map;
  • results may be used commercially, behind a firewall or publicly;
  • pricing is a flat monthly or annual fee;
  • bad results can be fixed at the source by correcting OpenStreetMap.

Requires an OpenCage API key β€” set it with --api-key or the QSV_OPENCAGE_API_KEY environment variable. A free trial key (https://opencagedata.com/users/sign_up) allows 2,500 requests/day at 1 request/second, for testing only: OpenCage's FAQ asks that production use go on a paid plan.

# forward-geocode a column of street addresses to coordinates
qsv geocode opencage address --new-column coordinates -f %location addresses.csv

# reverse-geocode "(lat, lon)" coordinates to a full formatted address (the default format)
qsv geocode opencage Location --reverse --new-column address events.csv

# pull individual OpenCage components with dotted dynamic formatting
qsv geocode opencage address -f '{components.city}, {components.country}' addresses.csv

# add multiple columns in one pass with the %dyncols: directive
qsv geocode opencage address \
  -f '%dyncols: {city:components.city}, {postcode:components.postcode}, {tz:annotations.timezone.name}' \
  addresses.csv

# one-off lookups from the shell
qsv geocode opencagenow "1600 Pennsylvania Ave NW, Washington DC"
qsv geocode opencagenow "40.71427, -74.00597"

OpenCage's terms of service permit caching, so results are kept in a persistent on-disk cache (separate from the Geonames index, in {cache-dir}/geocode-opencage_v1) β€” re-runs and duplicate queries never re-hit the API (--cache-ttl sets the TTL; --no-cache disables it). Rows are processed sequentially behind a rate limiter (--rate-limit, default 1 request/sec β€” the free-tier limit), so reach for opencage for enrichment quality and for the offline subcommands for raw throughput. Inspect and clean up that cache with the cache-* subcommands.

Hardening (25.0.0). Your OpenCage key no longer ends up in the qsv log. OpenCage takes it as a key= URL parameter, and with QSV_LOG_LEVEL=warn or more verbose, failed requests logged the full URL; the key is now replaced with key=REDACTED (#4788). If you ran geocode opencage with logging on, check old qsv log files for your key. OpenCage responses over 4 MiB are now refused rather than read into memory, and the row counts as a failed lookup (#4791).

OpenCage --formatstr options: %formatted (default β€” the full formatted address), %lat-long, %location, %city, %state, %county, %country, %country_name, %postcode, %confidence, %json, %pretty-json; plus dotted dynamic keys {components.<name>} and {annotations.<dotted.path>}. The %dyncols: directive works here too β€” %dyncols: {city:components.city}, {tz:annotations.timezone.name} adds several columns to the output CSV in one pass (it cannot be combined with --new-column).

NYC geocoding with Geoclient (geoclient, geoclientnow)

New (25.0.0, #4789).

For New York City data, geoclient geocodes through the city's own Geoclient v2 API, a REST wrapper around the Department of City Planning's Geosupport. Beyond coordinates it returns the NYC identifiers and districts that general geocoders don't have: BBL, BIN, community district, city council district, census tracts (2010 and 2020), NTA, ZIP code, and many more fields. It only geocodes locations in the five boroughs.

The public API needs a free subscription key: sign up at https://api-portal.nyc.gov and subscribe to the "Geoclient V2 User" product. Pass it with --api-key or the QSV_GEOCLIENT_API_KEY environment variable. qsv sends the key as a request header, so it never appears in a URL or a log. To use a self-hosted Geoclient instead, set QSV_GEOCLIENT_API_URL to its base URL (e.g. http://localhost:8080/geoclient/v2). A self-hosted instance needs no key and has no quota.

--endpoint picks the kind of lookup:

Endpoint Input
search (default) A free-form location in the column: 314 W 100 St Manhattan, a landmark, an intersection, a BBL or a BIN
bin A Building Identification Number
bbl A Borough-Block-Lot: 1000670001, 1-67-1, 1/00067/0001, or a spreadsheet float such as 1000670001.0
address, addresspoint, intersection, place, blockface Several columns, mapped with --params
# free-form NYC addresses -> "lat, lon" (the default format)
qsv geocode geoclient address --api-key YOURKEY nyc_311.csv

# BBLs (e.g. from PLUTO) -> separate latitude/longitude columns
qsv geocode geoclient bbl --endpoint bbl -f '%dyncols: {lat:latitude}, {lng:longitude}' pluto.csv

# structured address columns -> BBL and council district in new columns
qsv geocode geoclient Street --endpoint address \
  --params "houseNumber={HouseNo},street={Street},borough={Borough}" \
  -f '%dyncols: {bbl:bbl}, {council:cityCouncilDistrict}, {tract:censusTract2020}' \
  permits.csv

# one-off lookups
qsv geocode geoclientnow "350 5th Ave Manhattan"
qsv geocode geoclientnow --endpoint bbl -f '%pretty-json' 1008350041

--params maps Geoclient parameter names to input columns (name={Column}) or to fixed values (name=value). Parameter names are case-sensitive, and are checked before any request is sent:

  • address, addresspoint: houseNumber, street, and borough or zip
  • intersection: crossStreetOne, crossStreetTwo, borough, plus optional boroughCrossStreetTwo and compassDirection
  • place: name, and borough or zip
  • blockface: onStreet, crossStreetOne, crossStreetTwo, borough, plus optional boroughCrossStreetOne, boroughCrossStreetTwo and compassDirection

A borough can be a name, an abbreviation (MN, BX, BK, QN, SI) or a number from 1 to 5. With a structured endpoint, the <column> argument is not sent to the API. It only names the column that is replaced with the result, or that --rename applies to. Use -c to add the result as a new column instead. For search, --params can set fixed search options such as exactMatchMaxLevel=5.

Formats: %+ / %lat-long (default), %location, %bbl, %bin, %json, %pretty-json, {field} templates with any result field (e.g. "{bbl} - CD {communityDistrict}"), and %dyncols:. Geosupport pads some fields with spaces, such as censusTract2020; qsv trims them. A field missing from a result renders empty. bbl and bin lookups return no street-front coordinate, so for those latitude/longitude are the building's internal label point.

Throughput and quotas: there is no batch endpoint, so every distinct location is one request. The public API allows 100 requests/second per IP, 2,500/minute and 500,000/day. qsv sends --jobs requests at a time behind a rate limiter. The limiter defaults to 40 requests/second, just under the per-minute cap; set it with --rate-limit. Repeated locations within a batch are looked up once, results are cached on disk, and a location Geoclient can't geocode is left unchanged or set to --invalid-result. Rate-limited and server-error responses are retried up to 3 times. If 10 requests in a row still fail, the run stops with a message about the quota rather than grinding through the rest of the file. A bad key stops it at the first request, and the error includes the gateway's own reason (e.g. "invalid subscription key", or "missing subscription key"), which tells you which to fix.

Ambiguous searches and reject reasons: a free-form search without a borough is tried in all five boroughs, and a common address can match in several. For example, 100 Broadway exists in Manhattan, Brooklyn, Queens and Staten Island. qsv treats such a row as not geocoded, rather than silently picking one. Add the borough or ZIP code to the input. If exactly one of the matches is an exact match, it is used. A warning at the end reports how many rows were ambiguous. To see why a row didn't geocode, add a %dyncols column whose key starts with _reject.. These keys are filled only on rows that failed. _reject.message gives Geosupport's own reason (e.g. ADDRESS NUMBER OUT OF RANGE), or the boroughs an ambiguous search matched, with [street only] marking a borough where only the street's address range matched, not a property. Other keys include _reject.reasonCode, _reject.message2 and _reject._resultCount. Successful results also carry _resultCount and, for search, _level. (25.0.0.)

qsv geocode geoclient Address addresses.csv \
  -f '%dyncols: {lat:latitude}, {lng:longitude}, {why:_reject.message}'

Ready-made output formats: -f %nyc-districts adds 11 columns: borough, community_district, council_district, assembly_district, state_senate_district, congressional_district, police_precinct, school_district, nta2020, census_tract2020 and zip_code. They are filled for address, intersection, place and search results; BBL, BIN and address-point lookups return only the borough. -f %normalized-address prints a single-spaced line such as 314 WEST 100 STREET, NEW YORK, NY 10025, for results with a house number. (25.0.0, #4798.)

qsv geocode geoclient Address complaints.csv -f %nyc-districts

Standardizing street names: --endpoint normalize returns a street name in <column> in Geosupport's normalized form (w 100 st β†’ WEST 100 STREET, bway β†’ BROADWAY). It only reformats and doesn't check that the street exists. --endpoint streetcode turns a 6-, 8- or 11-digit Geosupport street code (e.g. 11361001) into the street's name. An unknown code's reason is available as _reject.message. Both output the name by default and work with geoclientnow. --params "format=C" drops the sort padding (WEST 1 STREET instead of WEST 1 STREET), and length=10 abbreviates names to fit (W 100 ST). (25.0.0, #4797.)

qsv geocode geoclient Street --endpoint normalize --params format=C streets.csv
qsv geocode geoclientnow --endpoint streetcode 11361001

US Census geocoding (census, censusnow)

New (25.0.0, #4804).

census geocodes US addresses and coordinates through the US Census Geocoder. It is free and needs no API key. Besides the coordinates and the matched address, it returns the Census geographies a location falls in, with their GEOIDs: state, county, tract, block group and block, plus places, county subdivisions, congressional and state legislative districts, urban areas, ZCTAs and more. Those GEOIDs are the keys for joining Census statistics to your data.

There are three kinds of input:

  • a one-line address in <column> (the default), e.g. 4600 Silver Hill Rd, Washington, DC 20233;
  • an address in several columns, mapped with --params to the parts street, city, state and zip (name={Column} or a fixed name=value). Census needs a street plus a ZIP code, or a street plus city and state. A row without them is not sent and is left unchanged (or set to --invalid-result);
  • a coordinate (lat, long) with --reverse, which returns the geographies at that point.
# one-line addresses -> the five Census GEOIDs, in five new columns
qsv geocode census Address -f %census-geoids addresses.csv

# address columns -> "lat, lon" in a new column
qsv geocode census Address \
  --params "street={Street},city={City},state={State},zip={Zip}" -c latlong file.csv

# coordinates -> census tract and congressional district
qsv geocode census LatLong --reverse \
  -f '%dyncols: {tract:flat.Census Tracts.GEOID}, {cd:flat.120th Congressional Districts.GEOID}' points.csv

# one-off lookups
qsv geocode censusnow -f %pretty-json "4600 Silver Hill Rd, Washington, DC 20233"
qsv geocode censusnow --reverse -f %census-geoids "38.84601, -76.92744"

Output formats: %lat-long (the default), %location, %matched-address, %census-geoids (columns census_state, census_county, census_tract, census_blockgroup and census_block, e.g. 24, 24033, 24033802405, 240338024052, 240338024052004), and %json / %pretty-json. Every result has the same fields: matchedAddress, coordinates.x (longitude) and coordinates.y (latitude), tigerLine.*, addressComponents.*, Census's own geographies verbatim, and flat.<Layer>.<attribute>, the first feature of each layer, e.g. flat.Census Tracts.GEOID. qsv adds _resultCount (how many addresses matched; the first is used), _matchType (Exact or Non_Exact, batch lookups only) and _source (single or batch). Layer names contain spaces, so flat.* fields work in %dyncols: but not in {field} templates. Layer names also vary by vintage: run censusnow -f %pretty-json to see them.

Single and batch lookups: Census has a single-record endpoint (everything, one request per location, 5 per second by default) and a batch endpoint (up to 10,000 rows per upload, much faster). Batch results carry only the matched address, coordinates, TIGER line and match type, and the state, county, tract, block group and block. With --census-mode auto (the default), qsv batches a read chunk when your --formatstr only needs those fields and at least 50 of its locations aren't cached yet. It uploads 2,000 rows at a time, one upload at a time. --census-mode batch always batches (and refuses an output it can't produce, such as %json), and single never does. Output is the same either way: qsv builds the batch result in the same shape as the single one. Batch lookups are only used with Current_* vintages. An address the batch endpoint matches to several addresses (a Tie) is left ungeocoded and counted in a warning. Connecticut batch matches are looked up again one by one: in Current vintages, Connecticut's counties are its 2022 planning regions (e.g. 09110), but a batch result only carries the 2020 county (09003). Coordinates from the two endpoints can differ in their last digit (about 1e-12 degrees), because Census rounds them differently. Batch matching also varies with the load on the Census servers, so batch no-matches and ties are not cached, and a rerun tries them again.

Benchmark and vintage: --benchmark (default Public_AR_Current) picks the address ranges and --vintage (default Current_Current) the geographies, e.g. --vintage Census2020_Current for 2020 Census geographies or ACS2026_Current for the ones the ACS uses (ACS vintages have no blocks). See the vintages list.

Errors, retries and caching work like geoclient: retries with backoff for rate limits and server errors, an abort after 10 failures in a row, and a persistent result cache (--cache-ttl, --no-cache). An HTTP 400 from Census, such as a bad --vintage, stops the run. To use a caching proxy that speaks the same API, such as ucgeocoder, set QSV_CENSUS_GEOCODER_URL to its base URL (e.g. http://localhost:8080/geocoder) and pass --no-cache. The output is the same as against census.gov.

Managing the online result caches (cache-*)

The opencage, geoclient and census subcommands keep their results in persistent on-disk caches: {cache-dir}/geocode-opencage_v3.redb, {cache-dir}/geocode-geoclient_v3.redb and {cache-dir}/geocode-censusgeocoder_v3.redb (the default cache dir is ~/.qsv-cache/). Only those subcommands populate them; the Geonames cities index is managed separately with index-*. Three subcommands manage the caches, and each one applies to all of them. (25.0.0, #4789: before Geoclient, they covered the OpenCage cache only. cache-info now prints one JSON line per cache, with a provider field.)

# wipe the entire OpenCage, Geoclient and Census caches
qsv geocode cache-clear

# delete entries older than a relative age (s/m/h/d/w suffix) or an absolute date
qsv geocode cache-prune --older-than 30d
qsv geocode cache-prune --older-than 2025-01-01

# report the cache dir, entry count, on-disk size and oldest/newest timestamps (JSON)
qsv geocode cache-info

cache-prune is the tool for honoring data-retention policies without discarding the whole cache; cache-info emits a JSON summary you can pipe into other commands.

See also: /docs/help/geocode.md, Geonames, MaxMind GeoLite2, OpenCage API, NYC Geoclient, US Census Geocoder, Recipe: Geographic Enrichment, Lookup Tables β€” for fallback when geocode misses.

geoconvert

Convert between CSV/SVG and spatial formats. Input formats: geojson, shp (Shapefile), csv. Output formats: csv, svg, geojson, geojsonl β€” the exact set depends on the input format (see qsv geoconvert --help).

Example: GeoJSON β†’ CSV (for SQL analysis)

qsv geoconvert nta_boundaries.geojson geojson csv > nta.csv
qsv sqlp nta.csv "SELECT borough, COUNT(*) FROM nta GROUP BY borough"

Example: CSV with WKT geometry β†’ GeoJSON for a web map

qsv geoconvert parcels.csv csv geojson --geometry geometry > parcels.geojson

Example: CSV with latitude/longitude columns β†’ GeoJSON, or β†’ a WKT column

qsv geoconvert stations.csv csv geojson --latitude lat --longitude lon > stations.geojson
qsv geoconvert stations.tsv csv csv --latitude lat --longitude lon > stations_wkt.csv   # TSV in, WKT column out

CSV input skips rows it can't place (25.0.0, #4800). A row with an empty, unparseable or out-of-range latitude/longitude (beyond Β±90/Β±180), or empty or invalid WKT, is left out. One stderr warning gives the counts and the line of the first one, e.g. Skipped 4 of 6 rows with unusable geometry (1 empty, 2 invalid coordinates, 1 out of range; first at line 2). If every row is skipped, geoconvert fails instead: the columns then likely hold projected coordinates such as State Plane feet, or are swapped. Swapped columns are only caught when the swapped values fall out of range (e.g. US West Coast longitudes near βˆ’122); where both are within Β±90, as in New York (40.7, βˆ’74), they are not detected. Before, the first bad row stopped the whole conversion, and NaN/inf produced invalid GeoJSON. CSV input is now streamed (a 500k-row lat/lon file went from 857 MB–1.8 GB of memory to ~28 MB), reads .tsv/.tab files and takes -d, --delimiter, and a misnamed column is reported before anything is written.

Example: SHP β†’ CSV (drops to tabular form)

qsv geoconvert nyc_boroughs.shp shp csv > nyc_boroughs.csv

A shapefile needs its .shp, .shx and .dbf side by side; a missing one is reported by name. geoconvert does not reproject: if the .prj describes a projected coordinate system (e.g. NY State Plane), it warns, because the coordinates are output unchanged and GeoJSON requires WGS-84 longitude/latitude. Reproject first, e.g. ogr2ogr -t_srs EPSG:4326 out.shp in.shp.

Fixed (25.0.0, #4800): SHP β†’ CSV used to panic (CSV features must all have the same column names) on any shapefile with two or more attribute fields, because the fields arrived in a random order per record. Columns now follow the .dbf's field order (a null is an empty cell), and GeoJSON properties get the same stable order. SHP β†’ SVG is now supported.

Example: pick a file via dialog, then convert

qsv prompt -m 'Choose a GeoJSON file' -F geojson \
  | qsv geoconvert - geojson csv > result.csv

Example: combine geocode + geoconvert to enrich and map

# 1. Reverse-geocode NYC 311 to get the borough column
qsv geocode reverse 'Location' --new-column 'borough' \
  --formatstr "%admin2" --country US nyc311.csv > step1.csv

# 2. Convert the lat/lon pairs straight to GeoJSON (rows without coordinates are skipped)
qsv geoconvert step1.csv csv geojson --latitude Latitude --longitude Longitude > nyc311.geojson

See also: /docs/help/geoconvert.md, geocode, Conversion & I/O.

See also

⚠️ **GitHub.com Fallback** ⚠️