This first section contains the basic commands to set up a new aiida virtual environment for work (which should be the most common task). You must already have all the following parts installed:

• Basics: PSQL, RabitMQ, git, and an adequate python and pip versions.
• Extras: Virtual-Env and Virtual-Env-Wrapper.
• AiiDA: A local AiiDA git repo.

If you already have all of this set up, then you can create and configure your virtual environment like this:

python -m pip venv ~/.virtualenvs/aiida_env # This is wrong? why does it have the pip?
python -m venv ~/.virtualenvs/aiida_env
workon aiida_env
cd ~/aiida_repo
pip install -e .[rest,docs,tests,pre-commit]
pip install -e .[rest,docs,notebook,pre-commit,tests] # NICER
pre-commit install
deactivate
echo '' >> ~/.virtualenvs/aiida_env/bin/activate
echo 'eval "$(_VERDI_COMPLETE=source verdi)"' >> ~/.virtualenvs/aiida_env/bin/activate

MOVE TO TESTING:

workon aiida_python2
pre-commit run --all
verdi -p test_django devel test [selection:db.orm.etc]
verdi -p test_sqla devel test [selection:db.orm.etc]

Repeat the same for aiida_python3 environment and if everything works then you are ready!

NOTE 1: documentation depends on the current profile and can only compile using a django database.

NOTE 2: to run stuff you need to have rabbitqm installed and running and also setup the computer.

After you have cloned your fork of the aiida-core repository, it is a good idea to set an upstream remote pointing to the main distribution (1). Here we include also commands to check your remotes (2), change their names (3) or adresses (4), and delete them (5) and remove branches associated with deleted remotes (6), all of these using upstream as an example.

git remote add upstream https://[email protected]/aiidateam/aiida-core.git
git remote -vv
git remote rename vps7r34m upstream
git remote set-url upstream https://[email protected]/aiidateam/aiida-core.git
git remote remove upstream
git remote prune

All branches for developing new features should be started from the current develop branch. The following set of commands will make sure you start from this branch (1) when you make a new feature branch (2).

git checkout develop
git checkout -b feature

If changes are made to develop, you should either merge this changes back in your feature branch or rebase your branch to the new develop (use the first if this is something many people are working on or the second if you are the only one working in this branch).

git checkout develop
git pull --ff-only upstream develop
git push origin develop
git checkout feature
git merge develop
git rebase develop

There are a couple of environment managers out there, but the most relevants to us seem to be venv, virtualenv and virtualenvwrapper. For use with the latest releases of python 3 it is recommended to use venv instead of virtualenv, and although virtualenvwrapper was designed to be used with virtualenv, it can also work with venv. This recommendation however seems to be only based in not needing the additional dependency on virtualenv, since venv is directly incorporated into python, but since we are anyways going to be using virtualenvwrapper then there is not much difference.

To create a new environment using virtualenv:

virtualenv --python python3 ~/.virtualenvs/aiida_name

If you want to use venv instead, just run:

python -m pip venv ~/.virtualenvs/aiida_env
# NOTE: some people will want to put the virtualenvs and venvs in different folders
# python -m pip venv ~/.venvs/aiida_env

To make sure that virtualenvwrapper recognizes your environments, make sure that WORKON_HOME points at the correct directory:

# Python Environment Handling
export WORKON_HOME=$HOME/.virtualenvs
# export WORKON_HOME=$HOME/.venvs
# Default name changed from virtualenv to highlight I am using python3 -m venv (aka pyvenv)
export PROJECT_HOME=$HOME/dev
source /usr/local/bin/virtualenvwrapper.sh 
# symlinked to /Library/Frameworks/Python.framework/Versions/3.7/bin/virtualenvwrapper.sh

If you need to delete an old environment, just delete the corresponding folder:

rm -rf .virtualenvs/env_name/

Relevant resources:

• A stackoverflow response of why venv is better here
• A stackoverflow thread on how to use virtualenvwrapper with venv here

First make sure you have an psql role/user (you can use the same for all your databases). Then you need to create databases for both django and sqlalchemy. These need to start with test_xxx. We show commands for the django database, these should be repeated for the sqlachemy one (the last command tests the database).

username@pcbox:~$ sudo su - postgres
postgres@pcbox:~$ psql
postgres=# CREATE USER aiida_dbuser WITH PASSWORD '<password>';
postgres=# CREATE DATABASE test_dbdjango OWNER aiida_dbuser ENCODING 'UTF8' LC_COLLATE='en_US.UTF-8' LC_CTYPE='en_US.UTF-8' TEMPLATE=template0;
postgres=# GRANT ALL PRIVILEGES ON DATABASE test_dbdjango to aiida_dbuser;
postgres=# \q
postgres@pcbox:~$ psql -h localhost -d test_dbdjango -U aiida_dbuser -W

Then you need to setup the respective profiles for aiida, which will also need to start with test_xxx. You can do this with any of the python environments, and as long as you didn't isolate them changing the AIIDA_PATH in the activation script, it will work for both.

workon aiida_python3
verdi setup
> Info: enter "?" for help
> Info: enter "!" to ignore the default and set no value
> Profile name: test_django
> User email [[email protected]]: 
> First name [yourname]: 
> Last name [yourname]: 
> Institution [yourdefault]: 
> Database engine (postgresql_psycopg2) [postgresql_psycopg2]: 
> Database backend (django, sqlalchemy) [django]: django
> Database hostname [localhost]: 
> Database port [5432]: 
> Database name: test_dbdjango
> Database username: aiida_dbuser
> Database password: **********
> Repository directory [/home/username/.aiida/repository/test_django]: 

To quickly load a database information into a newly created profile, you can run:

verdi import https://object.cscs.ch/v1/AUTH_b1d80408b3d340db9f03d373bbde5c1e/marvel-vms/tutorials/aiida_tutorial_2019_05_perovskites_v0.3.aiida

• fork the aiida_core repository

• git clone your fork to have a local copy.

• Check out the right branch and create a new one for your development::

  git checkout develop git checkout -b my-new-addition

The AiiDA pre-commit hooks help you write clean code by running

• code formatting
• syntax checking
• static analysis
• checks for missing docstrings
• checks for consistency of dependencies and version numbers
• ...

locally at every commit you make. We currently use yapf_ and prospector_, but more tools may follow.

Set up the hooks as follows::

   cd aiida-core
   pip install -e .[pre-commit]
   pre-commit install
   pre-commit run  # test running the hooks
   # from now on, hooks will check the *changed* files on every commit

Besides of pre-commit, you may want to take a look at other complements you can also install such as tests and docs (take a look at the "extras_require" inside of the setup.json).

Note:

• If you work in a conda environment, you may need to conda install virtualenv to avoid problems with virtualenv inside conda.
• On Ubuntu-18.04 you may need to sudo apt install ruby-dev

• Use pre-commit run to run the checks without committing
• If you ever need to commit a 'work in progress' you may skip the checks via git commit --no-verify. Yet, keep in mind that the pre-commit hooks will also run (and fail) at the continuous integration stage when you push them upstream.
• For historical reasons, some code files are excluded from pre-commit checks in aiida. When touching such a file, please try to remove them from the exclude list .pre-commit-config.yaml.

• In some cases of very long strings, yapf will not be able to split the string and then prospector will complain. In this case just split the string manually. See discussion in #2114_.

.. _#2114: https://github.com/aiidateam/aiida_core/issues/2114

For speed considerations WSL 2 is recommended. WSL 2 introduces a full-blown VM for your Linux distributions, meaning faster I/O-operations and no need for Windows "support" services (like RabbitMQ). In other words, WSL 2 should provide a more self-contained installation and running experience.

Note, however that it is only available through the Windows Insider Program, using Windows 10 builds 18917 and higher. For more information, see the WSL documentation <https://docs.microsoft.com/en-us/windows/wsl/wsl2-install>_.

Tip:

Consider using Visual Studio Code with the Remote WSL extension for a full IDE experience, if you're not using in-terminal IDEs. See the VS Code WSL documentation <https://code.visualstudio.com/docs/remote/wsl>_ for more information.

.. _yapf: https://github.com/google/yapf .. _prospector: https://prospector.readthedocs.io/en/master/