Operation under Sqlite‐WASM - CameronD73/529renew GitHub Wiki
The conversion from WebSQL to Sqlite-WASM was anything but trivial, as the restrictions imposed by each module meant a simple translation layer was not possible. WebSQL is entirely asynschronous. WASM calls are synchronous, but the process is made asynch by forcing wasm to run in a worker. But workers cannot use callbacks to non-worker code, so we need a system of forward and return messages.
This is not documentation meant for users. It summarises the message passing process just for the benefit of programmers. It is very incomplete
The Results Tab handles message passing to and from the DB worker.
Main Page (DNA Relatives)
The main content code attached to the relatives list is in scan_relatives.js, which runs at document_idle.
It calls get_dna_cache() to preload some data already stored in the DB. It uses the getICW code message passing code, but the return values are not yet used in 529REnew (Mar 2026).
After about 1 second it sends a message "requestTriangTable". This returns a table of already known ICW, including flags whether each profile-relative pair has had ICW and/or triangulation scans done.
It creates the 529renew buttons, but cannot yet place them since the paginator element is not attached to the page. Consequently the add529Buttons() routine loops at half second delays, waiting for the paginator to appear.
Before displaying anything, 23andMe's own js code loads data for all DNA relatives (1500 to 5000) via several Ajax calls. It then applies display filters locally. The 529 code cannot access this data and so it has to reload the entire list (1500 to 5000) by duplicating the Ajax requests. To avoid clashing with the 23andMe calls, it waits until it knows that has finished (appearance of paginator). The json data from the ajax returns are processed into:
- relativesMap ; indexed by relativeID: most data related to the match between profile kit and this relative. It includes some data not saved to DB.
- notesMap ; indexed by relativeID: the user-notes created about this relative (specific to each profile kit)
- messagesMap ; indexed by relativeID: table of any messages exchanged between relative and tester or manager (not sure).
The function fill_relative_details() is called when Ajax download completed, in order to adjust the display of visible relatives (background colour, added notes). That function is also called when the display changes (pagination, filtering, order change)
The 529-Gather button updates the DB with content of DNA matches. Then the scan-ICWs is used to load general data data for relatives as well as loading ICWs and relative cM matching.
529-Gather
This button will store any useful data in DB
- pass message "process_relatives" to results tab, with relatives data (It was a Map, but that cannot be passed across messages, so done as array)
- forward same message to DBworker.
- in worker.js, "process_relatives" message passes args to DBwasm.processRelatives()
- in dbcode.js,
- inserts(ignore) profile if a new one is found
- inserts(ignore) idalias table with name and date of the profile person.
- for each relative,
- inserts(ignore) alias table
- inserts(ignore) DNArelatives table with side, notes
- if DNArelatives entry already exists then side, notes are updated if they were previously emepty
- insert(ignore) DNAmatches summary between profile and relative (pct chared, cM, n segs) returns a count of updates
Scan ICWs button
normal mode
- calls
run_ICW_scan()
shift-click mode (force rescan)
Initially I was just going to set code to unconditionally rescan every relative. Then I realise that if anything went wrong then the only options were start again from the beginning or, more likely, user would not realise some fraction had not been rescanned and carry on.
So, it requires another flag in the DB for each specific relative in the current list to the profile person. And more complicated code...
- sends "force_ICW_rescan" message to resuls tab, thence to db-worker and DB code, which simply sets bit in DB.
- on return need to refresh _triangMap _ before rejoining main path.
run_ICW_scan() function
- for every DNA relative (in relativesMap) create a queue of relatives to scan
- check value in _triangMap _- these flags determine if need to scan this relative
- start
next_relative_scan().
next_relative_scan: Scan one relative
This function prepares the icwQueue with content for the next relative. The queue includes content to construct the url for the required ajax calls:
- ...//family_background/ - returns only a family tree url
- haplogroup - only if DNA sharing is enabled, otherwise the request returns a 403
- .../relatives_in_common/ - this retrieves data on all in-common relatives, rather than 10 at a time.
- call
launch_next_icw_ajax_query()which processes each entry in the icwQueue in turn- the callbacks when each part of the ajax completes save data for processing later. Each of those functions calls
launch_next_icw_ajax_query()to start the next step
- the callbacks when each part of the ajax completes save data for processing later. Each of those functions calls
- processing data from the relatives_in_common request passes the response to
load_icw_list(), which prepares the icwMap, as objects indexed by the match's ID
Completing all requests in the icwQueue leads to fill_ICW_details(), which:
- saves any haplogroup to DB (using message "update_haplogroups")
- saved family tree url, if any ( using message "update_familytree")
- passes the message "update_ICWs", along with the ICW array (which includes the pct shared and predicted relationship.)
- calls DBwasm.updateICW()
After programmed delay, call next_relative_scan() to initiate the next relative's scan.
DNA Match page
These pages have /profile/xxxx in the url, so they are sometimes referred to as profile pages, however the term "profile" is reserved in this code for the DNA testers that are managed by this account. These pages are for a relative who matches the specified profile person.
The main content code attached to the relative profile page is in save_segments.js (even though there may not be segments available)
Match page startup
When the page code is loaded and run, it extracts the profile and Match UID and name and then
runs the get_ICW code
get_ICW code
This preloads a cache with content that we already have in the DB. It is called from two pages, in one of two different modes, and in each case profileID defines the test profile:
- matchID is null: this loads all known matches
- matchID defines the match person: this loads only the data relevant to the defined match, and includes extra segment ICW counts.
- sends a "get_ICW_prelude" message to the results tab.
- this passes details to the DB worker with the same message code
- the worker calls DBwasm.getICWPrelude()
- DBwasm.getICWPrelude does the following using profileID and (optionally) matchID
- updates new profiles
- get table of all in DNAmatch who match the profileID
- if matchID is not null:
- get table of all in DNAmatch who match the matchID
- get all entries in ICWsets between profile, match and 3rd person.
- returns the 3 tables
- the worker then passes a message type "ICWPrelude_return" with those tables from the DB lookup
- it sends a message back to the originating page, which is either the scan_relatives or the save_segments code. The message listener in either case calls load_match_cache(payload)...
The load_match_cache() Function
Takes the returned tables/objects and loads the content into:
- profileMatchesMap, contains objects of profileMatch data indexed by the Relative's ID
- matchMatchesMap, contains objects of testers who are in common with with both the Profile person and the relative.data indexed by the Relative's ID, and
- sharedSegMapMap, contains objects of profileMatch data indexed by the Relative's ID
Triangulate into 529Renew
At this time (Mar 2026) the chromosome segments are available to selected customers.
Start with the page showing the (up to) 10 ICWs after manually clicking on "Find Relatives in Common"
The process is undertaken by clicking on "Triangulate into 529Renew", which triggers tr_el.onclick... .
- process_settings_then_compare()
- with the list of first 10 matches
- runComparison() on the page contents, reading data on up to 10 ICWs
- add each one with DNA results to the queue (skip hidden or "connect to share" matches).
- runComparison() on the page contents, reading data on up to 10 ICWs
- run launch_next_IBD_query() to start collecting.
- this calls run_query(), which sends message "checkIfInDatabase" to DB worker
- returns DB message "return_DBcheck"
- which passes message "returnNeedCompare" to calling tab indicating if we already have segment data for this ICW match.
- if we do not have segs (needtocompare is true) then
- compose ajax request and assign callback function makeSegmentSaver() to handle reply.
- this creates array of matching segments and then
- sends message "storeSegments" to DB which calls function storeSegments()
- this builds message and posts "insertNewAliasAndSegs" message to DB worker.This adds content to db tables:
- DNARelatives
- DNAmatches
- ibdsegs - half and full (if any)
- loop back to next IBD_query
- repeat while next page button is available.
- Finally, update ICWScanned value for the pair.