Integrations GEDCOM Portability - alerum68/Theogony GitHub Wiki
type: concept title: GEDCOM Portability & Data Interchange description: Comprehensive reference on GEDCOM 5.5.1 and GEDCOM 7 import/export, complete database backups (.theb), lossless data packages (.tgpkg), and vendor extension preservation. tags: [gedcom, porting, backup, interchange, data-portability, vendors, gedzip, tgpkg, theb] sources:
- id: openwiki-source-63e7659a6ca9bd405fc03001 resource: repo://theogony-app/src/commands/backup.rs
- id: openwiki-source-2563c4d5acf89385cce3d2c1 resource: repo://theogony-app/src/commands/interchange/assemble.rs
- id: openwiki-source-d5eebb2df5921528f12b3fdc resource: repo://theogony-app/src/commands/interchange/package.rs
- id: openwiki-source-ddf01269e6202b120e06d4bd resource: repo://theogony-app/src/commands/interchange/plain_ged/mod.rs
- id: openwiki-source-d5bede6a3c1f1e60c5bc7f38 resource: repo://theogony-gedcom/src/mapper/export.rs
- id: openwiki-source-bdb8e62eaee2feaa6b2f05e1 resource: repo://theogony-gedcom/src/mapper/import.rs
- id: openwiki-source-50d45df9034cad9755689e9c resource: repo://theogony-gedcom/src/mapper/naming.rs
- id: openwiki-source-3cae5790683d51118c7597cf resource: repo://theogony-gedcom/src/vendor/mod.rs generated: { by: "openwiki/0.5.1", at: "2026-09-28T00:00:28.276Z" }
Theogony implements a robust, multi-tier data portability architecture designed to ensure zero data loss when moving genealogical data between Theogony and other desktop software, web services, or collaborative partners. Interchange spans four primary mechanisms:
-
Standard GEDCOM 5.5.1 and GEDCOM 7 Export & Import (
.ged): Plain textual genealogy files. -
Lossless Data Packages (
.tgpkg): GEDZIP-compliant ZIP archives containingtree.ged,evidence.sqlite(complete evidentiary provenance),manifest.json, and media attachments (media/). -
Complete Database Backups (
.theb): Compressed tar+gzip archives (tree.sqliteplusmanifest.json) created via SQLite's Online Backup API. - Vendor Extension Tag Preservation: Intelligent multi-tier mapping that detects source vendor software signatures and maps proprietary extension tags into canonical fact types.
Portability is divided into three architectural layers to maintain clean separation of concerns:
-
theogony-gedcom: A pure parsing, serialization, and mapping crate. It parses GEDCOM text intoGedcomDocumentstructures, runs vendor signature detection, computes fact-type drafts for extension tags, and maps between domain export shapes and GEDCOM records. It contains no database access and knows nothing about SQLite or Tauri. -
theogony-db-sqlite: Handles SQLite storage, schema migrations (migrations::ALL), and online backup/snapshot routines (export_theb_snapshot,export_snapshot), as well as importing match trees and evidentiary provenance structures (such as DNA match tree imports referencing shared builtin fact types likeBirthviatheogony-db-sqlite/src/dna_import.rs). -
theogony-app: Tauri command handlers (crate::commands::interchange,crate::commands::backup, and source/bibliography report generators incrate::commands::reports) that bridge the live database (TreeRepository) with pure export/import documents and analytical views, orchestrating transaction boundaries, file locks, and temp-file renaming (repo://theogony-app/src/commands/reports.rs).
graph TD
UI[Tauri UI / Dialogs] --> App[theogony-app Commands]
App --> DB[theogony-db-sqlite Repository]
App --> Ged[theogony-gedcom Parser & Mappers]
Ged --> Vendor[Vendor Profile Registry]
DB --> Snap[SQLite Online Backup & Snapshots]
Plain GEDCOM interchange supports both legacy GEDCOM 5.5.1 and modern GEDCOM 7 (repo://theogony-app/src/commands/interchange/plain_ged.rs).
-
Document Assembly:
gather_export_documentqueries the activeTreeRepositoryto collect individuals, family units, and source documents, translating internal structures intoExportDocument(repo://theogony-app/src/commands/interchange/assemble.rs). -
Sex Normalization: Translates UI-level
"Male"and"Female"strings into standard GEDCOM1 SEX Mand1 SEX Ftags (repo://theogony-app/src/commands/interchange/assemble.rs#L24-L30). -
Custom Fact Fallback: If a fact type lacks a standard
gedcom_tag, it is exported as a generic GEDCOM event (1 EVEN) with a2 TYPE <FactTypeName>substructure rather than being dropped (repo://theogony-gedcom/src/mapper/export.rs#L17-L22). -
Serialization:
theogony_gedcom::mapper::export::build(&export_doc).serialize()emits strictly formatted GEDCOM text.
-
Parsing: Reads the source file and parses it via
GedcomDocument::parse(&text)(repo://theogony-app/src/commands/interchange/plain_ged.rs#L79-L80). -
Vendor Detection & Fact Drafting: Detects the originating software vendor and generates required
FactTypeDraftdefinitions for any unrecognized extension tags before creating the target tree database (repo://theogony-app/src/commands/interchange/plain_ged.rs#L85-L91). -
Unsourced Persona Routing: Facts and assertions lacking explicit source citations are routed through a synthetic "Imported GEDCOM, no source cited" persona to maintain evidentiary auditability (matching the transactional granularity seen across interchange and DNA match tree imports in
theogony-db-sqlite/src/dna_import.rs).
.tgpkg files are real ZIP archives (fully GEDZIP-compliant) designed for peer-to-peer tree sharing where evidentiary rigor must be preserved alongside genealogical structure (repo://theogony-app/src/commands/interchange/package.rs).
-
manifest.json: Carriesschema_version,app_version,tree_lineage_id,tree_copy_id,exported_at,contributor_label, andkind(PackageKind::FullorPackageKind::Delta). -
tree.ged: Standard GEDCOM 7 representation of individuals and families (enabling non-Theogony genealogy software to read the tree). -
evidence.sqlite: A complete, self-contained SQLite snapshot of the evidentiary database (sources, repositories, citations, assertions, and agent confidence ratings). Per the GEDZIP specification, tools unaware of Theogony's evidentiary schema simply ignore this file. -
media/: Directory for attached media files and digital artifacts.
-
Upfront Validation: Archives are inspected via
zip_entriesto verify required entries (manifest.json,evidence.sqlite,tree.ged) (repo://theogony-app/src/commands/interchange/package.rs#L163-L200). -
Schema Compatibility: Schema migrations are inspected (
migrate::inspect) on the temporary evidence snapshot before touching the user's destination path, ensuring rejected imports leave no residue (repo://theogony-app/src/commands/interchange/package.rs#L231). -
Full vs. Delta Handling: Full packages instantiate a new tree database at
dest_path. Delta packages contain incremental assertion updates relative to a baselinetree_copy_idand display a preview (PackagePreview) summarizing individual and source document counts.
.theb files are compressed tar+gzip archives providing instantaneous, lossless snapshots of a Theogony database (repo://theogony-app/src/commands/backup.rs).
-
Online Backup API:
repo.export_theb_snapshot(&temp_sqlite)uses SQLite's Online Backup API to safely capture a live database snapshot without locking out active readers or writers (repo://theogony-app/src/commands/backup.rs#L74). -
Manifest Inclusion: Bundles
tree.sqlitewithmanifest.jsoncontainingschema_version,app_version,tree_lineage_id,tree_copy_id, andexported_at. -
Atomic Replacement: Archives are written to a temporary staging file (
.theb.tmp) and renamed into place only upon successful completion. A failed disk write never overwrites or corrupts an existing valid backup.
Restore (RestoreTheb) validates the archive manifest and forward schema compatibility before initializing the destination path, guarding against database corruption or incompatible schema versions.
Genealogy software frequently uses proprietary extension tags (e.g., Ancestry, RootsMagic, Family Tree Maker, Gramps). Theogony preserves these through a multi-tier resolution strategy (repo://theogony-gedcom/src/vendor/mod.rs and repo://theogony-gedcom/src/mapper/naming.rs):
-
Tier 1 (Standard Tags): Standard GEDCOM 5.5.1 / 7 tags (
BIRT,DEAT,MARR, etc.) map directly to built-in fact types. -
Tier 2 (Vendor Signature & Tag Tables):
detect_vendorinspectsHEAD.SOURand child tags (NAME,VERS,CORP) against a prioritizedREGISTRYof known vendor profiles (repo://theogony-gedcom/src/vendor/mod.rs#L57-L71). Matched extension tags (such as RootsMagic's_MILTor custom military tags) resolve to canonical fact types (canonical_names::MILITARY_SERVICE,HAS_PHOTO, etc.) via centralized mapping tables. -
Tier 3 (Event Type Fallback): Unrecognized
EVENorFACTtags with a2 TYPE <Value>substructure adopt the specified type name (repo://theogony-gedcom/src/mapper/naming.rs#L43-L50). -
Tier 4 (Mechanical Fallback): Any remaining custom extension tags (e.g.,
_DIT_NAME,_UID) are processed viamechanical_name_for_tag: leading underscores are stripped, remaining underscores and dots are converted to word breaks, and words are title-cased (_DIT_NAME→"Dit Name"). This guarantees zero data loss: every custom tag becomes a valid fact type rather than falling back to unsearchable raw notes (repo://theogony-gedcom/src/mapper/naming.rs#L14-L22).
Complementing direct import and export, theogony-app/src/commands/reports.rs provides analytical source and citation reports that track evidentiary usage across the database:
-
Source Index (
build_source_index): Lists all source documents in the tree along with title-case-insensitive filtering (title_filter) and citation counts per source (repo://theogony-app/src/commands/reports.rs). -
Source Usage (
build_source_usage): Inspects all fact links and assertion chains citing a given source document, reporting fact type, subject display name, subject kind (individualor family), and page reference (repo://theogony-app/src/commands/reports.rs). -
Individual Bibliography (
build_bibliography): Gathers citation footnotes and evidentiary documentation for a specified individual or ancestor line (repo://theogony-app/src/commands/reports.rs). -
Unsourced Facts (
build_unsourced_facts): Identifies facts and assertions lacking supporting citations, filterable by owner type, ensuring research gaps are clearly highlighted (repo://theogony-app/src/commands/reports.rs). -
Privacy Redaction: All report builders enforce living-person privacy protection (
is_private), redacting names and sensitive metadata across source usage and bibliographic views (repo://theogony-app/src/commands/reports.rs).