Celery - Tirrilee/TechTalk GitHub Wiki

Celery

Pythonμ—μ„œ 비동기 λ™μž‘μ„ μœ„ν•΄ μ‚¬μš©ν•˜λŠ” 라이브러리

Celery version 4.3 runs on, Python (2.7, 3.4, 3.5, 3.6, 3.7), PyPy2.7 (6.0), PyPy3.5 (6.0)

Celery 4.0 supports Django 1.8 and newer versions. Please use Celery 3.1 for versions older than Django 1.8


Celery μ‚¬μš©λ²•


1. Install Celery

$ pip install celery==4.3.0 $ brew install redis


2. Setting Celery Configuration

Project Layout

- Django Project
	- project
		- settings 
			- local
			- dev
			- prod

project/setting/celery.py

파일 κ²½λ‘œκ°€ μ€‘μš”ν•©λ‹ˆλ‹€!!

from __future__ import absolute_import, unicode_literals
from django.conf import settings

from celery import Celery
import os


# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings.dev')

app = Celery('project', broker=settings.CELERY_BROKER_URL) # ν”„λ‘œμ νŠΈ 이름 & broker μ„€μ •ν•˜μ§€ μ•ŠμœΌλ©΄ μ•ˆλ¨

# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY') # Version 4 이상

# Load task modules from all registered Django app configs.
# a common practice for reusable apps is to define all tasks in a separate tasks.py module,
# and Celery does have a way to auto-discover these modules
app.autodiscover_tasks() # Version 4 이상

project/init.py

from __future__ import absolute_import, unicode_literals

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .settings.celery import app as celery_app


__all__ = ('celery_app',)

project/apps/tasks.py

from __future__ import absolute_import

from project.settings.celery import app

@app.task
def say_hello():
    print("hellow world!")

Shell

Shell을 μ‹€ν–‰ν•˜λŠ” κ²½λ‘œλŠ” project의 μœ„μ˜ λ‹¨κ³„μ˜ directoryμ—μ„œ μ‹€ν–‰

$ celery -A project worker -l info


Ubuntu

1. redis

$ sudo apt-get install redis-server

$ sudo systemctl start redis

2. μ‹€ν–‰νŒŒμΌ 생성

$ sudo vi /etc/init.d/celeryd

celery repo 볡뢙

3. Config 파일 생성

$ sudo vi /etc/default/celeryd

# Names of nodes to start
#   most people will only start one node:
CELERYD_NODES=1

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/home/ubuntu/movedot/venv/bin/celery"

# App instance to use
# comment out this line if you don't use an app
CELERY_APP="project"

# Where to chdir at start.
CELERYD_CHDIR="/home/ubuntu/movedot/api-v1/"
# CELERYD_MULTI="$CELERYD_CHDIR/manage.py celeryd_multi"

# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"

# CELERYD_MULTI="multi"
# CELERY_CONFIG_MODULE="project.settings.dev"

# %N will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%N.log"
CELERYD_PID_FILE="/var/run/celery/%N.pid"
CELERYD_LOG_LEVEL="INFO"
# Workers should run as an unprivileged user.
#   You need to create this user manually (or you can choose
#   a user/group combination that already exists, e.g. nobody).
CELERYD_USER="celery"
CELERYD_GROUP="celery"

# If enabled pid and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1

export DJANGO_SETTINGS_MODULE="project.settings.dev"
export PYTHONPATH="/home/ubuntu/movedot/venv/bin/python"

3. κΆŒν•œ μ„€μ •

$ sudo chmod 755 /etc/init.d/celeryd

$ sudo chown root:root /etc/init.d/celeryd

$ sudo /etc/init.d/celeryd start

$ sudo sh -x /etc/init.d/celeryd start : celery history

4. Celery Error

였λ₯˜! > Permission 였λ₯˜λŠ” log에 보이지 μ•ŠλŠ”λ‹€

$ sudo su
$ C_FAKEFORK=1 sh -x /etc/init.d/celeryd start
⚠️ **GitHub.com Fallback** ⚠️