Database - HelpRefugees/project-flamingo GitHub Wiki

Flamingo uses MongoDB to store all of its data.

Database URL

The logic for selecting a database URL is kept in scripts/utils.js. It works as follows:

  • Search for a flamongo-db service in the VCAP_SERVICES environment variable, which will be available if running in Cloud Foundry (see Deployment).

  • If not found, search for a DATABASE_URL environment variable, commonly used by deployment platforms like Heroku*. This is how we inject the appropriate connection URL in the Docker configuration, for example.

  • Finally, if neither was available, assume mongodb://localhost:27017/flamingo. This is what we use for local development and on Travis CI.

Connection

The connection is managed via server/db.js, which is used by everything that accesses the database. This module provides two methods:

  • connect(url[, callback]): if there is already a connection in the state, provide it. Otherwise, create a new connection to the specified URL, store it in the state, then provide it.

  • close([callback]): close the connection and clear the state.

Both methods return promises, so you can use await/.then instead of providing callbacks if you prefer. Note that only one connection is stored at a time, so additional calls to connect with a different url will not work as you might expect.

Collections

Below are the collections you will find in the database.

Note: we've adopted a convention of using a prefixed underscore _ to indicate system collections.

  • reports: the central purpose of the tool is to share monthly reports, stored in this collection. To see the expected structure of a new report, see scripts/reports-generator.js.

  • users: the various users of the tool. Reports are owned by a user with the role of "implementing-partner", users with the role of "help-refugees" can see any submitted report.

  • _migrations: stores information about which migrations (see below) have been run in the database.

  • _sessions: stores the session tokens generated by express-session.

Migrations

Although Mongo doesn't have a schema in the same way that an SQL database does, we've still found it useful to have migrations updating the state of our database. We're using mongodb-migrations to manage this. The migration files are stored in server/migrations. Two commands are available:

  • yarn mm migrate: run the migrations against the current database (selected according to the rules above).

  • yarn mm create [name]: create a new migration file. Note that these are run in numerical order - there should only be one migration for each number otherwise you might run into issues, so be careful when merging parallel changes.

These migrations are run as part of the yarn start:server command, so will be run before each E2E test run and when deploying.

Databases

Currently up to three databases will be used when connecting on localhost:

  • flamingo: the main database, used for local development and the E2E tests

  • scripts: used by the report generation tests (configured in scripts/jest.config.js)

  • test: used by the server integration tests (configured in server/tests/jest.config.js)

You may need to update these if any conflict with existing local databases on your machine.

* This should also be set on Cloud Foundry, but we've had some issues with it...