postgres index create - ghdrako/doc_snipets GitHub Wiki

Tunning

Most operations have to do some sorting or memory allocation of some kind. The administrative ones, such as the CREATE INDEX clause, don’t rely on the work_mem variable and use the maintenance_work_mem variable instead.

SET maintenance_work_mem = '40GB'; -- zwiekszenie pamięć dostępną do budowy indeksów, co przyspieszy operację i pozwoli uniknąć problemów z pamięcią.
--SET work_mem = '1GB'; -- więcej pamięci na operacje sortowania przy selectach - nie jest uzywana przy budowie indeksow

PostgreSQL 11+ - build btree indexes in parallel, which can dramatically speed up the indexing of large tables.

 SHOW max_parallel_maintenance_workers;

As for every parallel operation, PostgreSQL will determine the number of workers based on table sizes. When indexing large tables, index creation can see drastic improvements.

Keep in mind that we use memory:

max_parallel_maintenance_workers x maintenance_work_mem

Another factot to speed up index is data types. The numeric is worsts typ Much faster index is create on int4,int8,integer.

If checkpoints and I/O have started to become a limiting factor we can

checkpoint_timeout = 120min
max_wal_size = 50GB
min_wal_size = 80GB

SELECT pg_reload_conf();

Monitoring (Postgres 12+)

SELECT * FROM pg_stat_progress_create_index;
  • jest odpowiednikiem widoku pg_progress_vacuum
  • Column 'phase' in example 'building index: scanning table' or building index: loading tuples in tree
  • Columns tuples_total and tuples_done give a rough estimate of the percentage of work done.