Standalone Waffle Server - Gapminder/waffle-server GitHub Wiki

Run WS (standalone)

  1. Install docker-compose

  2. Clone (or download as a ZIP and uncompress) WS to the directory of choice.

  3. Enter directory with uncompressed contents.

  4. Generate an ssh key with a dev name (the name is important) in the WS directory (location is important). Use Docker QuickStart shell for key generation. Don't provide any passphrase - just press Enter.

  • Make sure to add the ssh key to ssh-agent. It may matter in which terminal the action is performed. Previous problems on Windows were solved by adding it while in the Docker QuickStart shell.
  1. Starting Waffle Server

    • For Linux/MacOS run docker-compose -f docker-compose.yml -f docker-compose.tools.yml up in a terminal opened in the WS directory.

    • Windows 10 (below Pro) users have to run following command in the Docker QuickStart shell opened in the WS directory:

      WAFFLE_SERVER_HOST=$(docker-machine ip default) docker-compose -f docker-compose.yml -f docker-compose.windows.yml -f docker-compose.tools.yml up

    • Windows 10 Pro users should use PowerShell:

      docker-compose -f docker-compose.yml -f docker-compose.windows.yml -f docker-compose.tools.yml up

Note: If you are using Windows 10, double-check your edition, because all versions lesser than Pro are missing necessary virtualization features

  1. Access Vizabi tools page on http://localhost:8080/tools
    • Windows 10 (below Pro) users can acquire an IP address of virtual machine by executing docker-machine ip default in the Docker QuickStart shell, then using this IP, access Vizabi tools page on http://${DEFAULT_DOCKER_MACHINE_IP}:8080/tools.

Standalone Waffle Server:

  • Standalone Waffle Server will be accessible on http://localhost:3001, hence its API can be invoked from there. To add a new endpoint in Waffle Server Import CLI use http://localhost:3001;

    • Windows 10 (below Pro) users can acquire an IP address of virtual machine by executing docker-machine ip default in the Docker QuickStart shell. By using this IP user can access WS on port 3001: http://${DEFAULT_DOCKER_MACHINE_IP}:3001. Also, this address should be used as an endpoint in Waffle Server Import CLI.

  • Default Waffle Server credentials: Username: [email protected], Password: 123.

  • WS comes without any data in it, therefore desired dataset has to be imported using Waffle Server Import CLI and Github repository with DDF dataset (i.e., ddf--gapminder--systema_globalis). Don't forget to mark imported dataset as default in WS Import CLI because otherwise, there won't be any data.

  • Importing requires a powerful machine. Configuration was tested on following setups:

    • 16GB RAM machine on Linux
    • 8 GB RAM machine on macOS
    • 8 GB RAM (for Virtual Machine) on Windows

All artifacts generated and used by WS:

All the artifacts can be found in ~/waffle-server directory:

  • cloned datasets: ~/waffle-server/repositories;
  • diffs generated for dataset update: ~/waffle-server/diffs;
  • MongoDB database files: ~/waffle-server/mongodata;
  • Redis cache files: ~/waffle-server/redisdata;
  • WS logs: ~/waffle-server/logs

MongoDB artifacts on Windows (all versions)

There are some issues (#1, a bit more detailed explanation at #2) with mounting docker volumes on Windows host machines for MongoDB, this leads to the fact that artifacts generated on Windows for MongoDB will be stored in VirtualBox virtual machine. In order to look at them, open a virtual machine, and access generated artifacts from inside the VM.

If there is a need to copy those artifacts to Windows host:

  1. Get a list of available Docker volumes by executing docker volume ls in the Docker QuickStart shell.

  2. In the printed list of volumes - find waffleserver_mongodata.

  3. Get more information regarding waffleserver_mongodata volume by executing docker volume inspect waffleserver_mongodata. After the execution, the output will be similar to:

       [
       {
           "Driver": "local",
           "Labels": {
               "com.docker.compose.project": "waffleserver",

               // Name of the volume as it is defined in docker-compose.yml
               "com.docker.compose.volume": "mongodata"
           },
           
           // Volume location
           "Mountpoint": "/var/lib/docker/volumes/waffleserver_mongodata/_data",
           "Name": "waffleserver_mongodata",
           "Options": {},
           "Scope": "local"
       }
   ]

  4. Then create a shared folder on a local virtual machine and copy all the data to this folder from "Mountpoint" path (see input from step #3).

If there is a need to remove MongoDB data, just remove the volume in which it is stored: docker volume rm waffleserver_mongodata.

Update (rebuild) a service in the Docker container

Docker container contains a number of services: vizabi-tools-page, waffle-server, redis, mongodb To rebuild one of them, for example, because their version got updated run following commands:

  • Windows 10 (below Pro) syntax
  1. WAFFLE_SERVER_HOST=$(docker-machine ip default) docker-compose -f docker-compose.yml -f docker-compose.windows.yml -f docker-compose.tools.yml build <SERVICE NAME>
  2. WAFFLE_SERVER_HOST=$(docker-machine ip default) docker-compose -f docker-compose.yml -f docker-compose.windows.yml -f docker-compose.tools.yml rm
  3. WAFFLE_SERVER_HOST=$(docker-machine ip default) docker-compose -f docker-compose.yml -f docker-compose.windows.yml -f docker-compose.tools.yml up
  • Windows 10 Pro syntax
  1. docker-compose -f docker-compose.yml -f docker-compose.windows.yml -f docker-compose.tools.yml build <SERVICE NAME>
  2. docker-compose -f docker-compose.yml -f docker-compose.windows.yml -f docker-compose.tools.yml rm
  3. docker-compose -f docker-compose.yml -f docker-compose.windows.yml -f docker-compose.tools.yml up
  • Linux/Mac syntax
  1. docker-compose -f docker-compose.yml -f docker-compose.tools.yml build <SERVICE NAME>
  2. docker-compose -f docker-compose.yml -f docker-compose.tools.yml rm
  3. docker-compose -f docker-compose.yml -f docker-compose.tools.yml up