While you should refer to the Prefect documentation, this page provides some solutions in
cases where things to not work as expected.

Setup a Prefect.io on services like AWS, with Linux CentOs 7
Need python version >= 3.6 (2021-06-17) (in some cases I needed python version >= 3.7)

On the machine that run the Prefect server, check what is the name of the Docker postgres container name: docker ps
add to crontab the following:
0 8 * * * docker exec -it temp_data_processing_postgres_1 sh -c "psql PGPASSWORD=test-passwordand -U prefect -d prefect_server -c \"DELETE FROM log WHERE timestamp < NOW() - INTERVAL '14 days';\" "

for more info on this: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/config.toml
Other issues related to tenants and authentication docs

from prefect.executors import LocalDaskExecutor

# Some code

with Flow("parallel tasks", schedule=schedule, executor=LocalDaskExecutor()) 

Using instruction from:

• https://docs.prefect.io/core/getting_started/installation.html
• https://docs.prefect.io/api/latest/cli/server.html#start

pip3 install prefect
pip3 install "prefect[aws, google, dev, redis, viz]"

Docker and Docker-compose are needed to run Prefect server

• Based on: https://docs.docker.com/engine/install/centos/
• (Uninstalling docker: https://docs.docker.com/engine/install/centos/#uninstall-docker-engine)
• Post install steps: https://docs.docker.com/engine/install/linux-postinstall/
• Allow running docker not as root: https://docs.docker.com/engine/security/rootless/

sudo yum install -y yum-utils

sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
# Returns: repo saved to /etc/yum.repos.d/docker-ce.reposudo
sudo yum install docker-ce docker-ce-cli containerd.io

# If you get errors for missing components
cd tmp
wget http://mirror.centos.org/centos/7/extras/x86_64/Packages/fuse-overlayfs-0.7.2-6.el7_8.x86_64.rpm
sudo yum localinstall fuse-overlayfs-0.7.2-6.el7_8.x86_64.rpm  
wget http://mirror.centos.org/centos/7/extras/x86_64/Packages/slirp4netns-0.4.3-4.el7_8.x86_64.rpm
sudo yum localinstall slirp4netns-0.4.3-4.el7_8.x86_64.rpm

sudo yum install docker-ce docker-ce-cli containerd.io
   
# Start Docker
sudo systemctl start docker
# Verify Doscker
sudo docker run hello-world

# Configure to start on boot
sudo systemctl enable docker.service
sudo systemctl enable containerd.service

# Create docker users group
sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker
   
# install Docker composer
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod 755 /usr/local/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
sudo mount /tmp -o remount,exec
sudo docker-compose --version

# Add to .bash_profile
alias docker-compose="TMPDIR=${HOME}/tmp docker-compose"

# Starting server
prefect backend server
# if you need to adjust options like the UI port, check https://docs.prefect.io/api/latest/cli/server.html#start 
prefect server start --detach  



# Run the Docker daemon as a non-root user (Rootless mode)
# You must install newuidmap and newgidmap on the host.  
These commands are provided by the uidmap package on most distros.

/etc/subuid and /etc/subgid should contain at least 65,536 subordinate UIDs/GIDs for the user.
In the following example, the user testuser has 65,536 subordinate UIDs/GIDs (231072-296607).

# Add user.max_user_namespaces=28633 to /etc/sysctl.conf (or /etc/sysctl.d) and run sudo sysctl --system.
systemctl --user does not work by default. Run dockerd-rootless.sh directly without systemd.  

This will include

1. installing Prefect Server on a VM
2. installing agent on each of the machines you want to connect to the VM

create config.toml file in the .prefect folder with:

cd .prefect/
ls -l
vi config.toml
# add to config file the following:
[server]
host = "http://<server ip>"
port = "4200"
host_port = "4200"

[server.ui]
apollo_url="http://<server ip>:4200/graphql"

# Save and exit config.toml

# restart server (https://docs.prefect.io/api/latest/cli/server.html#start)
prefect server stop
prefect agent docker start --detach
prefect backend server

# Start agent
prefect agent docker start

# Ctrl-C might be  needed to close window

In your browser:

• go to http://:8080/home
• click on "Getting started" → "PREFECT SERVER" and in step 2 enter "http://<server_ip>:4200/graphql" and click "Connect"
• Refresh the page and click, in the top right side of the page, verify that it show connection to the Prefect remote server
• Next if you click on "Dashboard" → "AGENTS", you should see one agent show up

One every machine you should install:

1. install prefect
2. setup backend.toml (with backend = "server")
3. setup the config.toml with Prefect server ip
4. run "prefect backend server"
5. start agent:
   • "prefect agent docker start" (if docker is installed)
   • "prefect agent local start" (if docker is installed)
   • Might need to add "--agent-address http://<Server_IP>:8080"
   • AFTER RUNNING LOOK AT THE UI, IT WILL SHOW THE LABEL YOUR COMPUTER CREATES ADD "--label " to the agent start command
6. See below how to make the agent persist
7. You should not do "prefect server start"
8. install docker if your flows depend on it
9. run prefect agent local install

If you have root permissions use supervisord to set persistent agent on you machine

• https://docs.prefect.io/orchestration/agents/local.html#using-with-supervisor
• http://supervisord.org/introduction.html

If you do not, use

nohup prefect agent local start --label <computer label> > ~/tmp/prefect_agent.log &   
# nohup prefect agent local start --key "API_KEY" # setup config.toml isntead

To ensure that the agent runs after restart, run crontab -e and add:

@reboot nohup prefect agent local start --label <computer label> > ~/tmp/prefect_agent.log &    
 
# or, if using supervisorctl   
# @reboot supervisorctl   

• It is a good practice to use labels in the flow.register function
• to run code that is not packaged, use flow.run_config = LocalRun(working_dir=working_dir)

Log into the server you want to use prefect on andin create .prefect directory

mkdir ~/.prefect
cd ~/.prefect
mkdir results
chmod -R 776 ~/.prefect/results/

Add to ~/.prefect the files with the proper content:

• auth.toml
• backend.toml
• config.toml

The code folder, where your flows resides, should also have chmod 755 or chmod 776 permissions

When several teams are working on the same machine, and each team have different user group permissions, we
might need to have different users run their own instance.

On a single server: User 1 - Start agent

prefect agent local start -l team_1

User 2 - Start agent

prefect agent local start -l team_2

Then to register the flow
User - 1

flow.register(project_name="some project", labels=['team_1'])

User - 2

flow.register(project_name="some other project", labels=['team_2'])

This is true for both on-prem and cloud situation