FreeREG Content and Statistics

Part of FreeREG-Core-Records.

Models: app/models/freereg_content.rb, site_statistic.rb, search_statistic.rb, open_county.rb, open_names_per_place.rb, plus the counter fields on Place/Church/Register/Freereg1CsvFile · Controllers: freereg_contents_controller.rb, site_statistics_controller.rb, search_statistics_controller.rb, open_controller.rb

This page covers everything that answers "what's in the database?" rather than "find me a person".

Denormalised counters

Freereg1CsvFile, Register, Church and Place each carry the same summary fields:

Field	Type	Meaning
records	String	number of entries
datemin / datemax	String	earliest / latest year
daterange	Array (file) / Hash of Arrays keyed ba/ma/bu/total (register and up)	record counts per decade bucket
transcribers	Hash	who transcribed what, for credits
last_amended	String ("%d %b %Y")	most recent change below this level
data_present	Boolean (Place only)	anything transcribed here at all

They're rebuilt bottom-up:

Freereg1CsvFile#calculate_distribution      (from its entries)
  → Register#calculate_register_numbers     (sums its files)
    → Church#calculate_church_numbers       (sums its registers)
      → Place#calculate_place_numbers       (sums its churches; sets data_present)

Freereg1CsvFile#update_freereg_contents_after_processing runs the last three after each processed batch. Online entry create and delete do the same, but entry update doesn't (see Batch-Records-Freereg1CsvFile-and-Entry). The full rebuild is:

rake freereg:calculate_freereg_content    # FreeregContent.calculate_freereg_content: every register, church, place

It walks every register, church and place with no_timeout, so it takes a long time on production data. last_amended is maintained separately by Place#recalculate_last_amended_date, which runs from Freereg1CsvFile's after_save.

Year bounds come from FreeregValidations::YEAR_MIN/YEAR_MAX. Years outside them are ignored for datemin/datemax, and a file whose entries have no usable year gets zero_entries: true.

Database Contents pages (FreeregContentsController)

The public "Database Contents" section. It has nothing to do with search: it reads the gazetteer and the counters above.

• new / create: pick a county (stored in session[:chapman_code]) or jump straight to a place. FreeregContent is only a form object here, though it does persist a document per submission.
• index: county page: coordinator name (County.coordinator_name), county record total, list of places with data. Large counties are split alphabetically (FreeregOptionsConstants::ALPHABETS/ALPHABET_SELECTION_LIST).
• show/place, show_church/church, show_register/register: drill-down pages with counts, date ranges, decade histograms and transcriber credits from the counter fields. The register page also shows image-server progress (Register.image_transcriptions_calculation, see Image-Server-System).
• gaps_and_embargoes: the register's Gaps and EmbargoRules (Embargoes-and-Gaps).
• unique_place_names, unique_church_names, unique_register_names: the distinct surname/forename lists stored in PlaceUniqueName/ChurchUniqueName/RegisterUniqueName (populated by rake reports:extract_collection_unique_names[0], run from lib/tasks/scripts/update_unique_names.sh).
• recent_additions: places in a county sorted by last_amended, newest first.
• send_request_email: "volunteer to transcribe this" from an image group on the register page. Emails the group's syndicate coordinator if the group has allocatable images, otherwise the first volunteer_coordinator.

If a contents page shows the wrong count or date range, the counters are stale: run freereg:calculate_freereg_content, or reprocess the batch. A stale PlaceCache affects only the search form's place picker, not these pages.

Site and search statistics

• SiteStatistic is one document per day. rake freereg:calculate_site_statistics → SiteStatistic.calculate counts records by type (by iterating every Freereg1CsvFile and counting its entries), computes the day's additions by diffing against the previous day's document, and stores per-county counts in county_stats. archive_site_statistics.rake archives old ones. The admin view is site_statistics_controller.
• SearchStatistic is per-day search volume and timing, aggregated from SearchQuery documents by rake freereg:calculate_search_queries (also run by freereg:clean_search_queries). See Search-Query-System.

Open Data (/open)

A browse-by-surname section (/open → county → place → surname → records), built for search-engine crawling and backed by OpenCounty/OpenNamesPerPlace documents precomputed by rake myopic_vicar:rebuild_open. On master it requires login and nothing links to it.