celery-director

Simple and rapid framework to build workflows with Celery

Showing:

Popularity

Downloads/wk

0

GitHub Stars

301

Maintenance

Last Commit

11d ago

Contributors

13

Package

Dependencies

28

License

BSD

Categories

Readme

Celery Director logo

Tests Python versions License Code style: black

Celery Director


Director is a simple and rapid framework used to manage tasks and build workflows using Celery.

The objective is to make Celery easier to use by providing :

  • a WebUI to track the tasks states,
  • an API and a CLI to manage and execute the workflows,
  • a YAML syntax used to combine tasks into workflows,
  • the ability to periodically launch a whole workflow,
  • and many others.

See how to use Director with the quickstart and guides in the documentation.

Installation

Install the latest version of Director with pip (requires Python 3.6 at least):

pip install celery-director

Usage

Write your code in Python

# tasks/orders.py
from director import task
from .utils import Order, Mail

@task(name="ORDER_PRODUCT")
def order_product(*args, **kwargs):
    order = Order(
      user=kwargs["payload"]["user"],
      product=kwargs["payload"]["product"]
    ).save()
    return {"id": order.id}

@task(name="SEND_MAIL")
def send_mail(*args, **kwargs):
    order_id = args[0]["id"]
    mail = Mail(
      title=f"Your order #{order_id} has been received",
      user=kwargs["payload"]["user"]
    )
    mail.send()

Build your workflows in YAML

# workflows.yml
product.ORDER:
  tasks:
    - ORDER_PRODUCT
    - SEND_MAIL

Run it

You can simply test your workflow in local :

$ director workflow run product.ORDER '{"user": 1234, "product": 1000}'

And run it in production using the director API :

$ curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"project": "product", "name": "ORDER", "payload": {"user": 1234, "product": 1000}}' \
  http://localhost:8000/api/workflows

Read the documentation to try the quickstart and see advanced usages of Celery Director.

Project layout

.env                # The configuration file.
workflows.yml       # The workflows definition.
tasks/
    example.py      # A file containing some tasks.
    ...             # Other files containing other tasks.

Commands

  • director init [path] - Create a new project.
  • director celery [worker|beat|flower] - Start Celery daemons.
  • director webserver - Start the webserver.
  • director workflow [list|show|run] - Manage your project workflows.

License

See https://github.com/ovh/celery-director/blob/master/LICENSE

Rate & Review

Great Documentation0
Easy to Use0
Performant0
Highly Customizable0
Bleeding Edge0
Responsive Maintainers0
Poor Documentation0
Hard to Use0
Slow0
Buggy0
Abandoned0
Unwelcoming Community0
100
No reviews found
Be the first to rate

Alternatives

No alternatives found

Tutorials

No tutorials found
Add a tutorial