Error Management - FreeUKGen/MyopicVicar GitHub Wiki

Error Management

Part of FreeREG-Core-Records (shared with FreeCEN and FreeBMD).

Controllers: app/controllers/errors_controller.rb, app/controllers/application_controller.rb (plus per-controller rescue_from) ยท Config: config/initializers/errbit.rb, config/initializers/airbrake_final_fix.rb, config/errbit.example.yml, config/environments/production.rb, config/initializers/public_files.rb ยท Views: app/views/errors/internal_server_error.html.erb, public_site_specific/<site>/404.html, 422.html, 500.html

This page covers what happens when something goes wrong in a web request: how errors are reported to developers (Errbit), what users see, and the local error handling conventions in controllers. It also points at where background (rake) failures end up.

Overview

flowchart TD
    EX["Exception in a request"] --> RF{"rescue_from in the<br/>controller?"}
    RF -->|yes| H["handler: flash + redirect<br/>(user stays in the app)"]
    RF -->|no| AB["Airbrake middleware<br/>โ†’ Errbit (if configured)"]
    AB --> EA["config.exceptions_app = routes<br/>(production)"]
    EA --> E404["/404 โ†’ ErrorsController#not_found<br/>public/404.html"]
    EA --> E422["/422 โ†’ #unprocessable_entity<br/>public/422.html"]
    EA --> E500["/500 โ†’ #internal_server_error<br/>page with URL, userid, IP, host<br/>+ '500 ERROR:' log line"]
Loading

Error reporting: Errbit via Airbrake

  • The airbrake gem (Gemfile) is the notifier. Reports go to FreeUKGen's self-hosted Errbit server, which speaks the Airbrake API.
  • config/initializers/errbit.rb loads config/errbit.yml (not committed; copy config/errbit.example.yml) for the current environment and calls Airbrake.configure with project_key = api_key, host, project_id = 1, environment. test is ignored, and remote config, performance stats and query stats are all off.
  • If api_key or host is blank, the app logs "โš  Errbit not configured for " and carries on. Nothing is reported, but nothing breaks either. That's the normal state in development.
  • If config/errbit.yml is missing, YAML.load_file raises inside the initializer's outer rescue, which logs "Failed to initialize Errbit" and continues. The app still boots (see Development-Setup for the setup step that creates it).
  • The same initializer, and airbrake_final_fix.rb, monkey-patch Airbrake::Notifier#stop_polling / Airbrake.close into no-ops. They work around an airbrake-ruby shutdown error. If you upgrade the airbrake gem, check whether these patches are still needed.
  • config/errbit.config.yml is a committed placeholder file with empty hosts. The initializer doesn't read it.

There are no explicit Airbrake.notify calls in app/ or lib/. Only exceptions that reach the middleware (i.e. aren't rescued) are reported. Anything swallowed by a rescue => e in a controller or the CSV processor appears only in the Rails log.

The Errbit dashboard is the main source of bug reports for the live sites. About 26 commits on master mention it ("errbit error: โ€ฆ").

What the user sees

In production config.consider_all_requests_local = false and config.exceptions_app = routes, so unhandled exceptions are re-dispatched to the app's own routes:

Route Action Renders
/404 ErrorsController#not_found public/404.html, status 404
/422 #unprocessable_entity public/422.html, status 422 (e.g. CSRF failures)
/500 #internal_server_error app/views/errors/internal_server_error.html.erb, status 500

ErrorsController inherits from ActionController::Base, not ApplicationController, so none of the app's before_actions (login, stats, message flags) run while an error page is being rendered. That's deliberate: an error page must not raise an error of its own.

The 500 page asks the user to copy or screenshot the URL, their userid ("Not logged in" if none), client IP and server hostname when reporting the problem. The same values plus the request id are written to the log as a single line:

500 ERROR: URL=https://www.freereg.org.uk/... USERID=jsmith CLIENT_IP=... SERVER_HOSTNAME=... REQUEST_ID=...

To go from a user's report to the stack trace: grep the production log for 500 ERROR: with that URL/userid, take the REQUEST_ID, then look for the matching exception in Errbit or in the log lines tagged with that request.

The static pages (404.html, 422.html, 500.html, favicons, robots.txt, ads.txtโ€ฆ) are site-specific. They live in public_site_specific/freereg|freecen|freebmd/ and are copied into public/ at boot by config/initializers/public_files.rb, which first deletes the old top-level files in public/. public/ is git-ignored apart from the FreeCEN newsletters. Edit the copies in public_site_specific/; changes made directly in public/ are lost on the next restart.

Handling errors inside controllers

There's no central error-handling module. The conventions are:

  • rescue_from per controller for errors the controller expects:
    • ApplicationController: ActionController::UnknownFormat โ†’ missing_template (flash "unsupported format", redirect to search).
    • PlacesController, ChurchesController, RegistersController (and Freecen2PlacesController): Mongoid::Errors::DeleteRestriction โ†’ record_cannot_be_deleted, Mongoid::Errors::Validations โ†’ record_validation_errors. These turn the restrict_with_error associations described in Gazetteer-Place-Church-Register into a friendly message.
    • SearchQueriesController: Mongo::Error::OperationFailure and Timeout::Error โ†’ search_taking_too_long, Mongoid::Errors::DocumentNotFound โ†’ missing_document. Any MongoDB failure during a search (including a bad index hint) is shown to the user as "taking too long". See Search-Query-System.
    • SearchRecordsController: Mongo::Error::OperationFailure โ†’ catch_error.
  • [success, message] tuples instead of exceptions in model methods (see FreeREG-Core-Records). The controller shows the message with redirect_back(fallback_location: โ€ฆ, notice: message).
  • "Link validation" guards at the top of most batch/entry actions, e.g. Freereg1CsvEntry.valid_freereg1_csv_entry?, Freereg1CsvFile.valid_freereg1_csv_file?. They check the entry โ†’ file โ†’ register โ†’ church โ†’ place chain and show "The entry was incorrectly linked. Have your coordinator contact the web master" rather than raising on a nil. The model-side helpers log a FREEREG:LOCATION:VALIDATION or โ€ฆ_ERROR warning.
  • protect_from_forgery with: :reset_session: a CSRF failure resets the session instead of raising, so the user is effectively logged out rather than seeing a 422.
  • strip_string_params (prepended before_action) trims whitespace from all string params. Many "not found" issues caused by trailing spaces in form input never become errors at all.

Logging conventions

Search the logs by these prefixes:

Prefix Source
500 ERROR: ErrorsController#internal_server_error
FREEREG:CSV_PROCESSING:, FREEREG:CSV_PROCESSOR_FAILURE: the CSV processor (CSV-Processing-Pipeline)
FREEREG:LOCATION:VALIDATION, FREEREG:PLACE_ERROR, โ€ฆ:CHURCH_ERROR, โ€ฆ:REGISTER_ERROR broken gazetteer links
FREEREG:SEARCH_ERROR, FREEREG:SEARCH_HINT, โ€ฆ:DATE_SEARCH_TIMING, โ€ฆ:PERSIST_RESULTS_TIMING search (Search-Query-System)
FREEREG:EMBARGO_PROCESSING embargo rake runs
FREEREG:ACCESS ISSUE reject_access (a member tried an action their role doesn't allow)
FREEREG:PHC WARNING log_missing_document: a document expected to exist was not found
UCF: / [UCF Check] UCF list maintenance
FREEREG::USER sign-in and session setup

(For FreeCEN the prefix is FREECEN:. Most messages are built with appname_upcase.)

Production runs at log_level :info, so logger.debug lines (e.g. most UCF detail) don't appear there.

Background jobs

Rake tasks spawned from the web or run by cron (FreeREG-Operations-and-Rake-Tasks) aren't covered by the Airbrake middleware. Their failures go to:

  • email: the CSV processor emails the transcriber (batch_processing_failure) with the exception message; embargo processing emails the coordinator who started it;
  • their own log files under log/ (update_freereg_messages_*.log, delete_list_processing.txt, add_embargo_records_*.log, โ€ฆ);
  • stdout/stderr of the spawned process (p "FREEREG:โ€ฆ" lines), which for Kernel.spawn ends up wherever the web server's output goes;
  • the cron scripts' own trace output. update_freereg2_production.sh also re-enables search (searchctl.sh enable) in its fail trap.

rake extract_errors_log[production.log] (run by the cron script delete_orphan_files.sh, despite the name) appends lines beginning ENTRY ERROR from the given log to log/errors.txt.

โš ๏ธ **GitHub.com Fallback** โš ๏ธ