postgres locks - ghdrako/doc_snipets GitHub Wiki

• https://www.postgresql.org/docs/current/view-pg-locks.html
• https://wiki.postgresql.org/wiki/Lock_Monitoring
• https://postgres-locks.husseinnasser.com/
• https://www.postgresql.org/docs/current/explicit-locking.html#LOCKING-TABLES

PostgreSQL generally uses pessimistic locking (although it’s also possible to use optimistic locking), which means lockable resources like tables or rows are locked upfront before they’re modified. When queries try to access lockable resources, depending on the query type, it will require either a shared lock or exclusive lock.

Locks can be created:

• Explicit: https://www.postgresql.org/docs/current/explicit-locking.html
• Implicit: Mapping lock to command - https://pglocks.org/

A “lock” or “mutex” (short for “mutual exclusion”) ensures only one client can do something dangerous at a time.

how they conflict (X means they are conflicting):

For example consider the following for a single table:

For example consider the following for a single table:

Statements requesting access for a lock type are put into a queue where they wait in order. If transaction B is waiting to acquire the lock type that transaction A holds, B must wait for A. The pids for the backend processes are logged, including the statement holding a lock type and the statements waiting to acquire it. Some helpful lock parameters to set are log_lock_waits to on and setting a value for deadlock_timeout to gain visibility into locks or blocked queries. When these timeouts cause cancellations, the events and queries will be logged to postgresql.log.

log_lock_waits (boolean) # Controls whether a log message is produced when a session waits longer than deadlock_timeout to acquire a lock. This is useful in determining if lock waits are causing poor performance. The default is off. Only superusers and users with the appropriate SET privilege can change this setting.

log_lock_waits = on:

pgBadger organizes lock-related query information from your postgresql.log. The lock information is put into categories like “Most frequent waiting queries” and “Queries that waited the most,” which can help your investigations.Using it is straightforward; just run it and supply it with the path to your postgresql.log file.

1 SELECT
2 pg_stat_activity.pid, 
3 pg_class.relname, 
4 pg_locks.transactionid, 
5 pg_locks.granted , 
6 age(now() ,pg_stat_activity.query_start) AS "age" 
7 FROM pg_stat_activity,pg_locks left 
8 OUTER JOIN pg_class 
9 ON (pg_locks.relation = pg_class.oid) 
10 WHERE pg_stat_activity.query. <> '<insufficient privilege>' 
11 AND pg_locks.pid = pg_stat_activity.pid 
12 AND pg_stat_activity.pid <> pg_backend_pid() 
13 ORDER BY age DESC LIMIT 10; 

• https://xata.io/blog/migrations-and-exclusive-locks
• https://joinhandshake.com/blog/our-team/postgresql-and-lock-queue/

SELECT *, pg_sleep(30) FROM users; -- simulate a long-running query that acquires an ACCESS SHARE lock on the users table
ALTER TABLE users ADD COLUMN AGE integer --  DDL statement blocks while trying to acquire an ACCESS EXCLUSIVE lock on the same table

This ALTER TABLE statement attempts to acquire an ACCESS EXCLUSIVE lock on the users table but is unable to do so until the SELECT statement completes and releases its ACCESS SHARE lock.

The problem is that any other statements that require a lock on the users table are now queued behind this ALTER TABLE statement, including other SELECT statements that only require ACCESS SHARE locks.

This means that the table is effectively blocked for reads and writes until the ALTER TABLE statement completes. SELECTs and UPDATEs will queue up behind it, unable to execute. If there is a long-running query that prevents the ALTER TABLE from acquiring the lock, then reads and writes will be blocked for the duration of that query.

To identify this we can use the pg_blocking_pids function in combination with pg_backend_pid to find the process ID of the blocked processes.

Postgres provides the lock_timeout setting to control how long statements should wait to acquire locks before giving up.

By setting a lock_timeout on the ALTER TABLE statement it's possible to prevent other queries from queueing behind it for an unacceptable length of time

SET lock_timeout TO '1000ms'

ALTER TABLE users ADD COLUMN age INTEGER

DDL statements in migration sessions should always set lock_timeout to an appropriate value for the application; values of less than 2 seconds are common. This ensures that reads and writes won't queue behind a blocked DDL statement and cause application downtime.

-- Session 1 

-- start a transaction    
BEGIN ;   
LOCK  trips  IN  ACCESS EXCLUSIVE  MODE ;

-- session 2
-- set a transaction level lock_timeout    
BEGIN ;   
SET   LOCAL  lock_timeout =  '5s' ;   
-- Run the modification    
-- It should hang since the table is locked for exclusive access    
-- But it should get canceled after 5s    
ALTER   TABLE  trips  ADD   COLUMN  city_id  INTEGER ;    
-- In psql2 notice the statement is canceled    
-- ERROR: canceling statement due to lock timeout
ROLLBACK;

With the timeout in place, the statement will be canceled after waiting the maximum amount of time. This adds a safeguard that prevents your transactions from waiting forever.

• https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-STATEMENT-TIMEOUT

The statement_timeout[116] can be set from the client application to set a maximum allowed time for statements. When the time is reached, the statements are canceled.

Number of locks that are not granted currently

SELECT COUNT(*) FROM pg_locks WHERE NOT granted

if is lot of locks currently being held - identify which connection is causing the problems:

SELECT pid, pg_blocking_pids(pid), wait_event, wait_event_type, query
FROM pg_stat_activity
WHERE backend_type = 'client backend'
AND wait_event_type ~ 'Lock'

SELECT * FROM pg_locks;
select relation::regclass, * from pg_locks;
SELECT pid,  MODE,  locktype,  relation::regclass,  page,  tuple  FROM pg_locks  WHERE pid in ('508499',  '508335');

select pid, state, usename, query, query_start 
from pg_stat_activity 
where pid in (  select pid from pg_locks l 
  join pg_class t on l.relation = t.oid 
  and t.relkind = 'r'  where t.relname = 'log_operacji');

select nspname, relname, l.* 
from pg_locks l 
    join pg_class c on (relation = c.oid) 
    join pg_namespace nsp on (c.relnamespace = nsp.oid)
where pid in (select pid 
              from pg_stat_activity
              where datname = current_database() 
                and query != current_query());

• https://github.com/postgres/postgres/blob/master/src/backend/storage/lmgr/README Lightweight locks and these are typically used for controlling access to data structures in shared memory. Shared memory is the memory area in Postgres that's shared between different connections.It is helpful to understand which resource in Postgres is busy, and which part the system is doing a lot of work.

LWLock DataRead,BufferContent LWLock - Postgres internal locks