postgres pg_bulkload - ghdrako/doc_snipets GitHub Wiki

• https://ossc-db.github.io/pg_bulkload/pg_bulkload.html
• https://www.highgo.ca/2021/05/31/getting-started-with-pg_bulkload/
• https://github.com/ossc-db/pg_bulkload

pg_bulkload is a high-speed data loading tool for PostgreSQL, compared with the copy command. The biggest advantage is speed. In the direct mode of pg_bulkload, it will skip shared buffer and WAL buffer, writing files directly. It also includes the data recovery function to recover in case of import failure.

pg_bulkload mainly includes two modules: reader and writer.

• The reader is responsible for reading files and parsing tuples, and
• the writer is responsible for writing the parsed tuples to the output source.

create extension pg_bulkload;

pg_bulkload -i dstat.csv -O dstat -l dstat.log -P dstat_bad.txt -o "TYPE=CSV" -o "DELIMITER=," -d bldemo -U postgres
pg_bulkload -i dstat.csv -O dstat -l dstat.log -P dstat_bad.txt -o "TYPE=CSV" -o "DELIMITER=," -o "TRUNCATE=YES" -d bldemo -U postgres