Local development environment - HearstCorp/rover-wiki GitHub Wiki

Prerequisites

A quay.io account and access to the Hearst org (ask a DevOps engineer for access)
When you receive the invitation link. Click Create Account then you can connect your GitHub to your new account. Note: Do not sign in with your GitHub.

Quick Start

Install Docker for Mac from https://www.docker.com/products/docker#/mac. Don't use brew as that results in many misconfigurations.
Login to quay.io
1. Generate an encrypted password by logging in at https://quay.io, clicking your username in the top right corner, selecting account settings and finally clicking the "Generate Encrypted Password" button.
Login with docker - supply the encrypted password you generated in the previous step when prompted
```
 docker login quay.io
```

    docker login -u="your_username" -p="your_encrtyped_password" quay.io

Start the docker services
```
 docker-compose up -d
```

Setup the dinghy http proxy:

Add the following lines to the file /etc/resolver/docker (creating the /etc/resolver dir if it doesn't exist)

 nameserver 127.0.0.1
 port 19322

Install and start the dinghy http proxy container

 docker run -d --restart=always -v /var/run/docker.sock:/tmp/docker.sock:ro -p 80:80 -p 443:443 -p 19322:19322/udp -e DNS_IP=127.0.0.1 -e CONTAINER_NAME=http-proxy --name http-proxy codekitchen/dinghy-http-proxy

You will now have a container called http-proxy that you can start and stop with

 docker start/stop http-proxy

You should now be able to access rover at http://rover.docker/

Customizing your containers

You can add a local docker-compose.override.yml file at the root of your repo to export ports or rename containers as needed.

Running unit tests

# running the entire suite
docker-compose run --rm rover-test python manage.py test
# running specific module
docker-compose run --rm rover-test python manage.py test content.tests.test_api
# running specific tests
docker-compose run --rm rover-test python manage.py test content.tests.test_api:ContentTests.test_create_adcategory

You can pass the --keepdb flag when running tests to reuse the test DB and avoid having to wait for migrations to run. However, be sure to run the test suite without this flag when you're done to clean up the test DB (unless for some reason your tests clean up after themselves or don't care about existing data.)

Settings

When using Docker for local development, settings will use the docker.py settings, and NOT dev.py

Authentication

To enable full authentication for local development, set the ENV variable (case matters):

ROVER_AUTHENTICATION=True

To set up Postman to authenticate to rover, see Rover authentication with Postman

Useful commands

Shell access
```
  docker exec -it rover-only sh
```
SMTP server for testing

Log in to shell as shown in command above and run the following to get smtp output in stdout
```
  python -m smtpd -n -c DebuggingServer localhost:25
```

Delete all docker containers that are not running

  docker rm -v $(docker ps -a -q -f "status=exited")

Delete all orphaned docker images

  docker rmi $(docker images -a -q -f "dangling=true")

Delete all orphaned docker volumes

  docker volume rm $(docker volume ls -q -f "dangling=true")

Nuke the entire docker universe (will have to restart dinghy, rebuild/re-pull and re-run all containers and reload all data)
```
  docker rm -f -v $(docker ps -a -q) && docker rmi -f $(docker images -q) && docker volume rm $(docker volume ls -q)
```

Troubleshooting

Some basic commands for troubleshooting

Are all dinghy services running?
```
  dinghy status
```
Are all containers running?
```
  docker ps
```
Is there any information in the logs?
```
  docker-compose logs
```
Do you have the latest versions of the images? (Be aware that this might overwrite your postgres container)
```
  docker-compose pull
```
Has a new pip dependency been added, and docker-compose up fails?
```
  docker-compose pull
  docker-compose up
```
Are you unable to run tests via docker-compose run --rm rover-test python manage.py test?
```
  docker-compose build --no-cache rover-test
```

Fun with the REPL

Quick REPL access is available via:

docker exec -it rover-only-test python manage.py shell

However, the ipython history isn't saved by default. You can work around this by making rover-test a long-lived container, and by mapping its ipython dir to your host machine.

Add the following to your docker-compose.override.yml:

  rover-test:
    volumes:
      - /Users/${USER}/.docker.ipython:/root/.ipython
    command: tail -f /dev/null

Your ipython history should now persist even after you kill the rover-test containers and rebuild them.

DB and data

Initial brand data

These can be imported using:

./manage.py loaddata brands

Importing data from another Rover instance

Rather than loading the brands fixtures, you can replace your DB with a copy of the Rover dev DB:

# export ROVER_DB_HOST=<Rover dev DB host> PGPASSWORD=<Rover dev DB password>
$ pg_dump -h $ROVER_DB_HOST -O -U mediaos mediaos | gzip > dev_dump.sql.gz
$ docker cp dev_dump.sql.gz rover-only-postgres:/tmp/dev_dump.sql.gz
$ docker exec -it rover-only-postgres bash
# export PGPASSWORD=password
# dropdb -U postgres postgres
# createdb -U postgres postgres
# zcat /tmp/dev_dump.sql.gz | psql postgres postgres

You'll get some errors because the DB user in Docker isn't called mediaos; ignore them.

Resetting the database

There may be times where you want to reset the database to a clean slate (i.e. remove all data). A django command is available for this: https://docs.djangoproject.com/en/1.9/ref/django-admin/#flush

./manage.py flush

Database schema

Rover schema