GTDB metadata is stored in a PostgreSQL DB currently running on Nectar.

Note: commands should be run from /srv/db/gtdb/metadata//processing

GTDB DB information can be found in the GTDB Bitwarden account.

1. Create a duplicate of the current PostgreSQL database (on Nectar):

CREATE DATABASE <database_name> (e.g. database_name = gtdb_r237_dev)

2. Populate the newly created database with data from the previous version (e.g. gtdb_r232):

pg_dump -h <hostname> -p <port> -U <user> gtdb_<prev_release> -f ~/pg_dumps/gtdb_<prev_release>_duplicate.sql
psql -h <hostname> -p <port> -U <user> -d gtdb_<release>_dev -f ~/pg_dumps/gtdb_r<prev_release>_duplicate.sql

3. [Optional] Create PostgreSQL config file .pg_service.conf and password file .pgpass for accessing the DB by name:

.pg_service.conf

[gtdb_<release>_dev]
host=<host>
port=<port>
dbname=gtdb_<release>_dev
user=<user>

.pgpass

<host>:<port>gtdb_<release>_dev:<user>:<db_password>

4. Update the genomes table in the DB to reflect new, updated, and removed genomes in the latest release:

gtdb_migration_tk update_db --hostname <host> -u <user> -d <db_name> -p <password> -g ../../../genomes/ncbi/<release>/genome_dirs.tsv --report ../../../genomes/ncbi/<release>/report.log -f 2026-09-05 -o update_db -c 64 -l ./logs/update_db.log

Alternatively, if the PostgreSQL config and password files have been generated:

gtdb_migration_tk update_db --db_service gtdb_r<new>_dev -g ../../../genomes/ncbi/<release>/genome_dirs.tsv --report ../../../genomes/ncbi/<release>/report.log -f 2026-09-05 -o update_db -c 64 -l ./logs/update_db.log

The -f (--ftp_download_date) should reflect the data NCBI genome and assembly summary files were downloaded.

1. Import CheckM v1 and v2 estimates into the metadata_genes table:

gtdb_migration_tk update_checkm_db --db_service gtdb_r237_dev -f ./checkm/checkm.profiles.tsv.gz -q ./checkm/checkm.qa_sh100.tsv.gz -n ./checkm/checkm_not_assessed.tsv -o ./update_checkm_dbm -l ./logs/update_checkm_db.log

gtdb_migration_tk update_checkm2_db --db_service gtdb_r237_dev -f ./checkm2/checkm2.quality_report.tsv.gz -n ./checkm2/checkm2_not_assessed.tsv -o ./update_checkm2_dbm -l ./logs/update_checkm2_db.log

2. Create tables with metadata for all genomes:

gtdb_migration_tk create_tables -g ../../../genomes/ncbi/<release>/genome_dirs.tsv -o metadata_tables -v 138.2 -c 64 -l ./logs/create_tables.log

This creates the following tables:

• metadata_nt.tsv.gz
• metadata_gene.tsv.gz
• metadata_ssu_silva.tsv.gz
• metadata_lsu_silva.tsv.gz
• metadata_lsu_5S.tsv.gz
• metadata_ssu_silva_count.tsv.gz
• metadata_lsu_silva_count.tsv.gz
• metadata_lsu_5S_count.tsv.gz
• metadata_trna_count.tsv.gz

3. Create NCBI specific metadata tables:

gtdb_migration_tk parse_ncbi_assemblies -n ../ncbi/assembly_summary_*.txt.gz -o metadata_tables -l ./logs/parse_ncbi_assemblies.log

This creates the ncbi_assembly_summary.tsv.gz metadata file.

gtdb_migration_tk parse_ncbi_dir -g ../../../genomes/ncbi/<release>/genome_dirs.tsv -o metadata_tables -c 64 -l ./logs/parse_ncbi_dir.log 

This creates the ncbi_assembly_metadata.tsv.gz metadata file.

gtdb_migration_tk parse_ncbi_genome_category -g ../../../genomes/ncbi/release237/genome_dirs.tsv -n ../ncbi/assembly_summary_*.txt.gz -o metadata_tables -c 64 -l ./logs/parse_ncbi_genome_category.log

This creates the ncbi_genome_category.tsv.gz metadata file.

gtdb_migration_tk ncbi_strains -g ../../../genomes/ncbi/<release>/genome_dirs.tsv -n ../ncbi/assembly_summary_*.txt.gz -o metadata_tables
-c 64 -l ./logs/ncbi_strains.log

This creates the strain_summary_file.tsv.gz metadata file along with substrains.tsv which lists identified substains, but is not directly used.

4. Import all metadata tables into the database:

gtdb_migration_tk update_metadata_db --db_service gtdb_r237_dev -i metadata_tables -l ./logs/update_metadata_db.log

5. Import additional type material metadata:

gtdb_migration_tk update_metadata_db --db_service gtdb_r237_dev --metadata_table ../strain_table/gtdb_type_strain_summary.tsv.gz --metadata_table_desc <git directory>/gtdb-migration-tk/gtdb_migration_tk/data_files/table_description/metadata_type_material.desc.tsv -l ./logs/update_metadata_db-type_material.log

gtdb_migration_tk update_metadata_db --db_service gtdb_r237_dev --metadata_table ../seqcode/seqcode_table.tsv --metadata_table_desc <git directory>/gtdb-migration-tk/gtdb_migration_tk/data_files/table_description/metadata_seqcode.desc.tsv -l ./logs/update_metadata_db_seqcode.tsv

Standardized and unfiltered NCBI taxonomy files are created during the initial data sync with NCBI. This information must be added to the GTDB DB:

gtdb_migration_tk update_ncbi_tax_db --db_service gtdb_r237_dev -n ../ncbi/taxonomy/standardised_taxonomy/ncbi_<release>_organism_names.tsv --filtered ../ncbi/taxonomy/standardised_taxonomy/ncbi_<release>_standardized.tsv --unfiltered ../ncbi/taxonomy/standardised_taxonomy/ncbi_<release>_unfiltered_taxonomy.tsv -o update_ncbi_tax_db -l ./logs/update_ncbitax_db.log

The final GTDB taxonomy from the previous release should be propagated to this releases. This can be done as follows:

1. Get final GTDB metadata from previous release:

• wget https://data.ace.uq.edu.au/public/gtdb/data/releases/latest/ar53_metadata.tsv.gz -O gtdb_<prev_release>_ar53_metadata.tsv.gz
• wget https://data.ace.uq.edu.au/public/gtdb/data/releases/latest/bac120_metadata.tsv.gz -O gtdb_<prev_release>_bac120_metadata.tsv.gz

2. Determine genomes in current release that should be assigned a GTDB classification from the previous release:

gtdb_migration_tk propagate_gtdb_taxonomy --db_service gtdb_r237_dev --gtdb_metadata_prev gtdb_r232_ar53_metadata.tsv.gz gtdb_r232_bac120_metadata.tsv.gz -o propagate_gtdb_taxonomy -log ./logs/propagate_gtdb_taxonomy.log

This command recognizes when genomes have moved between RefSeq and GenBank, or have had a genome assembly version bump. It produces the files gtdb_taxonomy_propagated.tsv.gz and `gtdb_sp_reps_propagated.tsv.gz.

3. Add the propagated taxonomy information to the DB:

gtdb_migration_tk update_propagated_tax --db_service gtdb_r237_dev -i ./propagate_gtdb_taxonomy  -l ./logs/update_propagated_tax.log

The command sets the GTDB taxonomy for all genomes that are shared between releases and sets the GTDB species representative flag for those genomes previously identified as a representative genome. This command should be run before assigning new genomes to representatives.

Marker genes for the following sets must be aligned with HMMER v3.4 as they are used for phylogenetic inference:

• bac120, 120 bacterial marker genes (set 1)
• ar53, 53 archaeal marker genes (set 19)
• rp2, 23 universal ribosomal proteins (set 11)
• rp1_bac, 16 bacterial ribosomal proteins (set 12)
• rp1_ar, 16 archaeal ribosomal proteins (set 13)

gtdb_migration_tk align_marker_genes --db_service gtdb_r237_dev --marker_set_ids 1 19 11 12 13 -g ../../../genomes/ncbi/release237/genome_dirs.tsv --new_genomes -o align_marker_genes -c 96 -l ./logs/align_marker_genes

New genomes will not have a designated domain under the GTDB taxonomy. Setting the GTDB domain is critical since this field is used for filtering (e.g., to create archaeal and bacterial specific trees). A genome is assigned to the domain with the highest percentage of bac120 (d__Bacteria) or ar53 (d__Archaeal) marker genes. The exception is if both marker sets have <10% of the genes, then the domain specified by the NCBI classification is used.

gtdb_migration_tk set_gtdb_domain --db_service gtdb_r237_dev -o set_gtdb_domain -l ./logs/set_gtdb_domain.log

NCBI holds a growing number of surveillance genomes. We don't analyse these genomes, but we store these genomes in the DB to display on the website.

gtdb_migration_tk add_surveillance_genomes --db_service gtdb_r237_dev -n ../ncbi/assembly_summary_*.txt.gz -l ./logs/add_surveillance_genomes.log