Gramplets - dsblank/gramps-connect GitHub Wiki

๐ŸŒ Also available in: Deutsch ยท Espaรฑol ยท Franรงais ยท Nederlands ยท ็ฎ€ไฝ“ไธญๆ–‡

Gramplets and the Add-on Store

Gramplets are Gramps Connect's add-on system: small Python programs that query the tree and render a result โ€” a table, a chart, plain text, or HTML. They come in two kinds: Gramplets, which you run from the Gramplets menu in a window of their own, and View Gramplets, which sit in a panel under a list (Person, Family, or any other object type) and follow what you select there. What makes them unusual is where they run: each one is real Python, executing directly in your browser tab under Pyodide (CPython compiled to WebAssembly), against locally-built wheels of Gramps' own gramps.gen.lib data model. Nothing is installed on your machine, and nothing runs on a server โ€” see Architecture.

That also means every Gramplet is editable. Open one from inside the app's Gramplet editor, and you're looking at (and can change) its actual source โ€” not a black box behind a settings panel. Change how a chart looks, tweak what a lookup searches for, or write a new one from scratch, all without leaving the app.

Who can use Gramplets

Anyone logged in to the tree can run its Gramplets, from the Gramplets menu or in a list's View Gramplets panel. Only the tree's Owner or a site Admin can write them: create one, edit one's code, or install, update, or remove one through the Gramplet Store. For everyone else, the Store, New Grampletโ€ฆ and the edit pencils don't appear at all.

Role Run Gramplets Write / install Gramplets
Guest โœ“ (except private ones) โ€”
Member โœ“ โ€”
Contributor โœ“ โ€”
Editor โœ“ โ€”
Owner โœ“ โœ“
Admin โœ“ โœ“

Writing a Gramplet takes more than an Editor's usual rights because its code runs in the browser of everyone who uses it, so changing it affects the tree for everyone, not just your own edits. A Gramplet is stored as an ordinary Media object tagged Gramplet, so the usual privacy rules apply: if it's marked private, Guests won't see it.

The same goes for Allow data changes (see Changing data): only an Owner or Admin can turn it on for a Gramplet. Anyone can do a dry run of such a Gramplet, which saves nothing; saving its changes takes Editor rights or above (the server's own add, edit and delete permissions), so for Guests, Members and Contributors the window's Run button is disabled.

Browsing and installing

The Gramplet Store is a browsable catalog inside the app โ€” open it from Gramplets โ†’ Gramplet Storeโ€ฆ. It has two tabs, Gramplets and View Gramplets; install, update, or remove any entry with a click. An installed Gramplet appears straight away in the Gramplets menu (grouped by its category); an installed View Gramplet is added to a list with the + in that list's View Gramplets panel. It's published independently of Gramps Connect's own releases โ€” the catalog is static content (gramplet-store/ in the repo) fetched by the running app at runtime, not bundled into the app build, so new Gramplets can appear in the Store without anyone upgrading Gramps Connect itself.

Examples already in the catalog include these Gramplets, which run from the Gramplets menu:

Gramplet What it does
Age at Death by Decade Chart of the median age at death for each decade of birth or death
Mark Unsourced People Private Marks everyone with no source at all as private โ€” changes data, so opening it is a dry run that lists who would change, and Run saves it (see Changing data)
People Without Sources Everyone with no citation on themselves or any of their events (a whole-tree scan with a progress bar)
Possible Duplicate People Same-named people born within a few years of each other, to check and merge
Tag People Adds a tag you pick from a list to everyone in the current People filter โ€” changes data, so it's a dry run until you click Run

and these View Gramplets:

View Gramplet What it does
Age-at-Death Histogram Interactive plotly histogram of age at death across the tree (or the current filtered list)
All Relationships Relationship calculations across the tree
Attributes / Backlinks / Children / Citations / Coordinates / Events / Gallery / Notes Desktop-Gramplet-style detail panels for the selected record
Born In Lookup by birthplace
Crossing Relationships Finds relationships that cross generations in unusual ways
GOQL Query Type a raw GOQL where clause, and pick field expressions as columns
Hello Table The minimal starting example
People Search A live, as-you-type search of People by name, gender and death date
Phonebook Surname Sort Accent-folding surname sort (e.g. "Mรผller" sorts next to "Muller")
Relationship / Relationships How two selected people are related
Selected Record Reacts to whichever record is currently open
To-Do / Todo Tracker Tracks research to-do items across the tree
Tree Statistics A dashboard of record counts and derived percentages

On each list view (People, Families, Events, โ€ฆ), the View Gramplets you've added to that view show up as tabs in a collapsible View Gramplets panel under the table. The panel starts collapsed, and a View Gramplet only runs while the panel is open: expanding it runs the selected tab. The first run in a session also loads the in-browser Python runtime, so it can take a few seconds; later runs are quick. Keeping the panel collapsed costs nothing, since no Python is loaded until you open it.

Gramplets in their own window

A Gramplet (as opposed to a View Gramplet) isn't tied to any list. Pick one from the Gramplets menu โ€” grouped into submenus by category once there's more than one category โ€” and it opens in a floating window:

  • It runs when it opens. A Gramplet that needs input first โ€” a surname, a number of generations โ€” draws its own fields and a Run button, the same widgets View Gramplets use.
  • Gramplets that change data (marked โœŽ in the menu) do a dry run when they open and have Dry run / Run buttons and Undo โ€” see Changing data.
  • It looks at the list behind it. If a list is open in the main view, the Gramplet sees its selected record and its current filter, as they were when the run started; navigating elsewhere afterwards doesn't change a run in progress.
  • Progress and Cancel. A longer Gramplet shows a progress bar while it works; Cancel stops it right away and keeps whatever it had produced so far.
  • Keep working meanwhile. Minimize a window (โ–พ) and it becomes a small chip at the bottom of the screen showing how far along it is; the run carries on. When a minimized Gramplet finishes you get a notification with a Show button. Several Gramplets can run at the same time (up to five windows), each independently, and they stay open as you move between views.
  • Move, resize, maximize. Drag a window by its title bar, resize it from its corner, or maximize it (โคข, or double-click the title). Each Gramplet remembers its window's size and position in your browser.
  • Editing. For the tree's Owner or a site Admin (see Who can use Gramplets), the window's title bar has a pencil (โœŽ) that opens the Gramplet editor; saving restarts the window with your changes.
  • Closing. To run a Gramplet again from the start, close its window and pick it from the menu again. Closing a window that's still running asks first, and so does leaving or reloading the page โ€” closing the browser tab stops every run.

Results are shown in the window only; they aren't saved anywhere yet.

Writing a Gramplet

A Gramplet's code runs in a sandbox with a set of functions and objects already available โ€” no imports needed to reach the tree. The gramplet_examples/ directory in the repo is a numbered tour through the API, from simplest to most advanced; the same reference is available from inside the app itself via the (i) "Writing a Gramplet" help button in the Gramplet editor.

The smallest useful Gramplet

matches = people(
    "gender == Person.MALE and birth.date.sortval >= Date('Jan 1, 1900')",
    limit=25,
)

columns("Person")
for person in matches:
    row(person)
  • people(where=None, what=None, order=None, limit=50) fetches People matching a query, as full records, in one call โ€” or, with what=["gramps_id", "primary_name"], just those fields โ€” and there's a families()/events()/places()/repositories()/sources()/ citations()/media()/notes()/tags() for every other object type, same signature, just a different table.
  • where= is written in GOQL โ€” the exact same syntax as the search box on any list view, including reaching across relationships (birth.date.sortval) even when the field you're filtering on isn't one you're displaying.
  • No await needed even though this is a real network call under the hood โ€” it's inserted for you automatically.
  • row(*values) adds one row to the table. Passing a whole Person/Event/Place/... object (rather than a hand-picked field of it) renders it as a clickable link that already shows its full name/title and Gramps ID โ€” clicking it opens a popup to view that record in List, Map, Graph, or Timeline.
  • name_of(person) is a person's name as text โ€” given name, then surnames ("Root Ancestor"), the way row() labels them โ€” for a heading or a sentence. It also takes one of their names, e.g. name_of(person.alternate_names[0]).

Every record returned by people()/families()/etc. is a Gramps "DataDict" โ€” a plain dict of the object's fields, but with dot access on top: person.primary_name.first_name is the same value as person["primary_name"]["first_name"]. Reach for this whenever you want to compute something from a field or show one row()'s own rendering doesn't surface. row(), html(), and print() can each be called as many times as you like, in any order โ€” everything shows up in the order you added it.

Printing a record never fails: print(person) or f"{person}" shows a whole record the way row() labels it ("Root Ancestor [I0001]"), a date as a date, and anything else โ€” a filter() result, person.primary_name โ€” as its plain fields.

Cheap counting without downloading the tree

total_people = db.get_number_of_people()
women = db.get_number_of("person", where="gender == Person.FEMALE")
no_birth_date = db.get_number_of("person", where="birth.date.sortval is None")

db.get_number_of_<type>() is a cheap, whole-tree count โ€” the total comes back in a response header, no records are actually downloaded. db. get_number_of(object_type, where=...) is the conditional equivalent, for when the count itself needs a filter.

Reaching across relationships in a query, and back out for display

A where= condition can reach across a relationship to filter on โ€” a person's birth event, a family's father, an event's place โ€” even though people()/families()/etc. always hand back a whole, unrelated object with no way to ask for a related field back directly:

born_and_died_same_place = people(
    "birth.place.title == death.place.title and birth.place.title is not None",
    order=[{"column": "surname", "direction": "asc"}],
    limit=50,
)

Getting a related field back for display (rather than just filtering on it) is a separate, by-hand lookup through db's own methods โ€” db.get_event_from_handle(...), and so on โ€” following the same handle indirection Gramps' own data model uses internally.

Collections work too: any(c.given_name == 'Steve' for c in children) matches a family if any child satisfies a condition, and len([c for c in children if c.gender == Person.MALE]) > 1 counts just the sons. See GOQL for the full collections story, including its current limit (one hop into a collection, no further chaining past what's inside it).

Reacting to the selected record

record = get_selected()
if record is not None:
    if isinstance(record, Person):
        ...

get_selected() returns whichever record is currently open in the view's own detail pane โ€” None when nothing is selected, or when running from the standalone editor preview (which has no view context at all). One network fetch on first call in a run, then memoized for the rest of it. isinstance(record, Person) (etc.) tells you what kind of record you got, which matters for a Gramplet attached to "All" views. For this to update live as you click through the list, turn on "Re-run automatically when the selected record changes" in the Gramplet's own editor โ€” left off (the default), get_selected() still works, but only reflects whatever was selected the last time the Gramplet happened to run for some other reason.

Reacting to the active filter

A Gramplet can layer its own query on top of whatever filter is currently applied to the view โ€” either typed into the view's own search box or applied through the "Filters" picker (a saved filter or Custom Rule) โ€” so filtering the list down to one branch of the tree narrows the Gramplet's own output the same way:

rows = filter(
    "person",
    where=and_filters(get_filter(), "birth.date.sortval is not None and death.date.sortval is not None"),
)

This needs views: [...] set in the manifest (get_filter() only hands back a where_expr matching the view's own object type) and listensToFilter: true so the Gramplet actually re-renders when the filter changes.

Charts

pygal, matplotlib, and plotly are all pre-bundled โ€” a plain import (or, for plotly, from plotly.subplots import make_subplots) works offline, no %pip install needed. print(fig) on a plotly Figure, an SVG string from pygal, or a matplotlib figure is recognized automatically and rendered as a chart โ€” no manual to_html()/embedding required.

For chart data, pass what=[...] when you only need one or two fields โ€” people() on its own downloads every field of every record, which is wasteful for a chart that only plots, say, age at death.

Side-by-side layout

col1, col2 = st.columns([2, 1])
with col1:
    row("Wider column")
with col2:
    row("Narrower column")

st.columns(spec) lays out spec regions side by side โ€” an int for that many equal-width columns, or a list of weights (st.columns([2, 1])) for proportional widths โ€” and hands back one region per column. Anything written inside with col: (row()/html()/print(), or another st.* call, including a nested st.columns()) lands in that column instead of at the top level; col.write(x) works the same way without a with block.

A Gramplet in its own window

Gramplets โ†’ New Grampletโ€ฆ opens the Gramplet editor ready for this kind (in an existing one, set Kind to Gramplet โ€” opens in a window from the Gramplets menu). In place of the View Gramplet settings there's a Category field, which groups it with others in the Gramplets menu (e.g. chart); blank puts it under Other.

The code runs as soon as the window opens. To ask for input first, draw it with the st.* widgets and do the real work behind an st.button. They follow Streamlit's names and arguments: st.button, st.text_input, st.number_input, st.checkbox, st.selectbox, st.radio, st.columns, st.progress, the message boxes st.info / st.success / st.warning / st.error, and st.session_state. They work the same in View Gramplets. For a long-running Gramplet, show a bar with st.progress(value, text), as in Streamlit: value is 0โ€“100 (or 0.0โ€“1.0), and the returned bar's .progress(...) moves it in place and .empty() removes it โ€” cheap enough to call once per row. It works in View Gramplets too. people(..., limit=None) (and the other list functions) return every match rather than the first 50:

surname = st.text_input("Surname", "Smith")
if st.button("Run"):
    everyone = people(f"primary_name.surname_list[0].surname == '{surname}'", limit=None)
    bar = st.progress(0, text="Checking people...")
    for i, person in enumerate(everyone):
        bar.progress((i + 1) / len(everyone))
        ...
    bar.empty()

A Gramplet can run for as long as it needs, and a page left open for hours can still run them: every request a Gramplet makes uses your current sign-in, renewed in the background like the rest of the app. Only if the sign-in itself has ended does a run stop, with Your session has expired. Sign in again, then rerun this Gramplet.

Installing a third-party package

%pip install unidecode
from unidecode import unidecode

%pip install is Jupyter-magic syntax, rewritten into a real await micropip.install([...]) call before your code runs โ€” it must be its own line, at the top, exactly like in a notebook (writing it inside an if, for instance, won't work). This only works for pure-Python packages with no compiled/C-extension code; pygal/matplotlib/plotly don't need it because they're pre-bundled, but most small pure-Python utilities on PyPI install fine this way.

Changing data

A Gramplet (not a View Gramplet โ€” those never change data) can change and delete records once Allow data changes is turned on in the Gramplet editor. The API is the one Gram.py Script uses on Gramps desktop: change a record the way you'd change any Python object.

for person in people("len(citation_list) == 0", limit=None):
    person.private = True

The Store's Mark Unsourced People Private is a complete example: it finds people cheaply with filter() first, then fetches and changes only the ones that need it, inside one named begin_changes() group. Tag People shows adding to a list (person.tag_list.append(tag.handle) โ€” records store tags as handles, not names) and an st.selectbox() you must choose from before anything changes.

  • record.field = value changes a field, nested ones too (person.primary_name.first_name = "Chris"). So do list edits (.append(), .pop(), ...) and any set_*() method, e.g. person.set_privacy(True).
  • delete(record) deletes a record. It doesn't remove references to it from other records.
  • begin_changes("message") / end_changes() group the changes in between under that message. Optional: without them the whole run is one group, named after the Gramplet.

Records from people() and the other plural functions, get_raw_object(), get_object(), get_selected() and db.* can be changed. filter() results can't, since they hold only some of a record's fields. Asking for the same record twice in a run gives back the same object, with your changes in it.

Dry run first. In the Gramplet editor every run is a dry run, and opening the Gramplet's window from the Gramplets menu does a dry run too. Nothing is saved; below the output, a yellow Dry run box lists each record that would change, with every changed field's old and new value, and each record that would be deleted, with how many other records still refer to it. Each changed record is first checked against its Gramps schema, the same check the server makes; an invalid value stops the run and names the field. One way a dry run can differ from the real thing: searches (filter(), count(), where=) run on the server, so they still see the tree as it was before the run's changes.

Then Run. The Gramplet's window has Dry run and Run buttons (the menu marks such Gramplets with โœŽ). Run saves each group of changes as the code closes it โ€” at end_changes(), or when the run ends โ€” as one transaction, so it shows up as one entry in the tree's history. While the result of a real run is shown, the title bar says Saving changes, and the box below the output turns green and marks each group saved. If the server refuses a group โ€” most likely because someone else changed one of its records after the Gramplet read it โ€” none of that group is saved, the run stops with an error, and groups saved before it stay saved. Widgets in the output (st.button("Apply") and so on) rerun the code in whichever mode the current result came from, so a Gramplet can show what it would change in a dry run and apply it after Run.

Undo. After a real run the window shows how many groups it saved and an Undo button, which takes them back, last first. Undo refuses a group whose records someone has changed since, rather than overwriting those edits. It's only offered for the window's current result: running the Gramplet again, or closing its window, ends it.

Publishing to the Store

One folder per Gramplet in gramplet-store/, named by its id (the folder name and the manifest's own id field must match):

gramplet-store/
  <slug>/
    manifest.json   # required
    code.py         # required -- the Gramplet's Python source
    icon.png        # optional (png/jpg/jpeg/svg/webp)

manifest.json fields:

Field Required Meaning
id yes Must match the folder name.
name yes Shown as the entry's title in the Store.
description yes Shown under the name in the Store.
version yes Plain semver, e.g. "1.0.0" โ€” bump it whenever code.py or the manifest changes, so installed copies can detect an update is available.
author yes Shown in the Store.
category yes A short tag ("example", "detail", "utility", "chart", ...) used to group/filter entries.
views no Which object-type views ("person", "family", ...) this Gramplet can be added to. Omit for "every type".
listensToSelection no Whether this Gramplet should re-run when the selected record changes.
listensToFilter no Whether this Gramplet should re-run when the active filter changes (search box or Filters picker).
kind no "view" (the default) for a View Gramplet; "window" for a Gramplet run from the Gramplets menu. views/listensTo* only apply to View Gramplets.
allowChanges no true to install it with Allow data changes on (the Store shows a Changes your data badge). Only allowed with "kind": "window".

After adding or editing an entry, regenerate catalog.json โ€” the single file the app actually fetches:

npm --prefix app run build:gramplet-catalog

This validates every entry (required fields present, id matches its folder, version looks like semver, code.py non-empty, kind valid) and fails loudly on the first problem rather than publishing a broken catalog.

What Gramplets can't do yet

Gramplets can change and delete records (see Changing data) but can't create new ones yet. There's also no equivalent yet for other Gramps add-on types (tools, reports). See Roadmap and Known Limitations for the current state of these.

โš ๏ธ **GitHub.com Fallback** โš ๏ธ