1. Setting up the website

1.1 Using PostgreSQL

The website uses PostgreSQL as the database server.

Installing postgresql

Before proceeding, please install according to your respective operating system and distro.

• Arch Linux
• Ubuntu
• Windows

Set up databases

Firstly, enter the postgresql shell. The example shown below is for Unix users.

user@host      $ sudo -iu postgres
postgres@host  $ psql

Afterwards, enter the following commands into the postgresql shell:

CREATE ROLE cosmos_website_tester WITH
	LOGIN
	SUPERUSER
	NOCREATEDB
	NOCREATEROLE
	INHERIT
	NOREPLICATION
	CONNECTION LIMIT -1
    PASSWORD '2020123';

CREATE DATABASE cosmos_website_test WITH 
    OWNER = postgres
    ENCODING = 'UTF8'
    CONNECTION LIMIT = -1;

GRANT ALL PRIVILEGES ON DATABASE cosmos_website_test TO cosmos_website_tester;

ALTER USER cosmos_website_tester CREATEDB;

\q

1.2 Install Redis

The website uses Redis as a cache, and for celery.

Installing Redis

Before proceeding, please install according to your respective operating system and distro.

• Arch Linux
• Other Distros

No additional configuration is required.

1.3 Downloading the project

The project is stored in this git repository. To download either use a GUI application for git, which is highly recommended in general as git is difficult to use from the command line, or run the following command:

git clone https://github.com/CosmosTUe/Cosmos.git

Afterwards, copy secrets.json.template into the following folder (depending on the platform):

• Linux: /etc/secrets.json
• Windows: C:\Users\{Account folder}\secrets.json

Ensure that the database credentials are as configured in step 1.1 by ensuring DATABASES.DEFAULT.USER has the value cosmos_website_tester.

1.4 Setting up python

To handle all dependencies we use pipenv, so make sure to install it on top of python. Either use your package manager to install it if you are using linux or run the following command to try to install the package globally:

pip install pipenv

To install the python packages required for this project, without interfering with other projects packages, we use a python virtual environment. To create the python virtual environment and install, run the following command (on Windows, we recommend setting up the virtual environment in the same folder as your code:

cd Cosmos
mkdir .venv
pipenv install --python 3.8.1

To activate the virtual environment run the following command:

pipenv shell

If you want to deactivate the virtual environment for whatever reason, simply close the shell or run the following command:

exit

1.5 Setting up npm-packages

Be sure to have npm package installed. Afterwards, run inside of the terminal the following:

npm install

1.6 Installing the project

To populate the database with the correct structure, django uses migrations, to apply these migrations, run:

python manage.py migrate

In order to be able to modify the website, an admin user is required, to create one run the following commandL

python manage.py createsuperuser

2. Setting up the development tools

The recommended editor to use is visual studio code. Other editors will also work fine, however this guide will expect vscode.

2.1 Configuring IDE

vscode

First make sure that the python plugin for vscode is installed. Then open the plugin and make sure that the correct Python environment is selected. To change the currently selected python environment press Ctrl + Shift + P and enter 'Python: Select Interpreter' and then select the one that you just created in the virtual environment.

PyCharm

If you decide to work on PyCharm, be sure to exclude the folder .venv by right-clicking on it and selecting Mark Directory as -> Excluded. This will ensure the folder is excluded in project-wide searches and modifications.

2.2 Linting

In order to check the code for errors, we use flake8 as linter. To add additional capabilities to flake8, we install several plugins:

• flake8-django gives django specific linting,
• pep8-naming gives warnings on non pep8 compliant function and variable names, and
• isort with flake8-isort makes sure that import statements are sorted in a consistent way.

The configuration for flake8 and isort can be found in the tox.ini config file. To install flake8, isort and all plugins, and all other development requirements, run the following command:

pipenv install --python 3.8.1 --dev

vscode

To integrate flake8 into vscode select the linter by pressing Ctrl + Shift + P and enter 'Python: Select Linter' and select flake8, or set the following options in the .vscode/settings.json config file (on Windows, this file can be accessed via the VSCode settings):

"python.linting.pylintEnabled": false,
"python.linting.flake8Enabled": true,
"python.linting.enabled": true

PyCharm

To integrate flake8 into PyCharm, go to File -> Settings -> Tools -> External Tools -> Add (the + icon) and enter the following values on the respective fields:

Name: Flake8
Program: $PyInterpreterDirectory$/python
Arguments: -m flake8 --max-complexity 10 --ignore E501 $FilePath$
Output Filters: $FILE_PATH$\:$LINE$\:$COLUMN$\:.*

From here, linting an opened file in the editor window is done by selecting Tools -> External Tools -> Flake8. Additionally, it is also possible to set a shortcut for running the linter at Settings -> Keymap -> External Tools -> External Tools - Flake8.

For more accessibility, the linter can be made to run automatically after a file is modified in the IDE. For this, install JetBrain's File Watcher plugin first. Then go to Settings -> Tools -> File Watchers -> Add (+) and enter the following parameters:

Name: Flake8-watcher

Program: $PyInterpreterDirectory$/flake8
Arguments: $FileDir$/$FileName$
...
[x] Auto-save edited files to trigger the watcher
[x] trigger the watcher on external changes
...
Show console: Never
Output filters: $FILE_PATH$:$LINE$:$COLUMN$: $MESSAGE$

The next step is to get PyCharm to highlight the issues detected from flake8. Go to Editor - Inspections - File Watchers - File Watcher Problem and change the Severity value from Warning (Yellow Icon) to Error (Red Icon)

2.3 Formatting

For formatting we use black, which strictly enforces a pep8-compliant code style. The configuration for black can be found in the pyproject.toml config file. To install black and all other development requirements run:

pipenv install --dev

To integrate black into vscode set the following option in the .vscode/settings.json config file (on Windows, this file can be accessed via the VSCode settings):

"python.formatting.provider": "black"

To format python files, open them, and then press Ctrl + Shift + I to automatically format them. To sort the import statement using isort press Ctrl + Shift + P and enter 'Python Refactor: Sort Imports'

To integrating black into PyCharm, follow the official guide by black.

2.4 Testing

For testing we use tox, which manages all the testing environments and runs all the tests. The configuration for tox can be found in the tox.ini config file. To install tox, and all other development requirements, run the following command:

pipenv install --dev

To run all tests via tox simply run:

python -m tox

3. Setting up the git hooks

Pull requests can only be accepted if the linting, formatting and testing give no errors. In order to catch small mistakes before you create a commit, a pre-commit hook can be installed. This hook checks the linting and formatting and will give an error if they are not correct. To install this, first install pre-commit, and then install the hook itself by running the following commands:

pip install pre-commit
pre-commit install

4. Running the website locally

To run the website locally first you must start up the database server, and then start the django debug server. If you want to run the website on a server visit Setting up the server. To start the database server run:

sudo systemctl start postgresql.service

To then run the server, run the following command:

python manage.py runserver

If everything went right, the website should be running now at localhost:8000.

PyCharm

If you would like to take advantage of PyCharm's debugger, you need to run the server within the PyCharm environment.

Debugging the server

• Open Run > Edit Configurations...
• Click + > Django Server
• Fill in as follows:

Name: server (simply a label, call it as you want)
Host: 127.0.0.1
Port: 8000
Environment variables: PYTHONUNBUFFERED=1;DJANGO_SETTINGS_MODULE=cosmos.settings
Python interpreter: Project Default

Debugging the tests

• Open Run > Edit Configurations...
• Click + > Django tests
• Fill in as follows:

Target: <class notation eg. tests.user.forms.registration.RegisterUserFormTest>

Testing Celery

• Install celery and redis

  • Arch Linux: sudo pacman -S python-celery redis

• Run celery and rabbitmq`

sudo systemctl start redis
celery -A cosmos worker -B -l INFO