Celery

This is the standard for async task queues in Python, and for good reason. It handles the whole distributed job workflow: broker setup with Redis or RabbitMQ, worker management, retries with exponential backoff, periodic scheduling, and task chaining. The progressive disclosure here walks through real broker configs, task routing patterns, and production concerns like monitoring and rate limiting. Use it when you need to offload email sending, report generation, or any long-running work that would block a web request. The overhead is real though. If you just need simple background jobs and don't care about distribution, RQ or even basic asyncio might be easier to reason about.

skills.sh installs: 470
GitHub stars: 79
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add bobmatnyc/claude-mpm-skills --skill celery --agent claude-code

Installs into .claude/skills of the current project.

Files

Celery: Distributed Task Queue

Summary

Celery is a distributed task queue system for Python that enables asynchronous execution of background jobs across multiple workers. It supports scheduling, retries, task workflows, and integrates seamlessly with Django, Flask, and FastAPI.

When to Use

Background Processing: Offload long-running operations (email, file processing, reports)
Scheduled Tasks: Cron-like periodic jobs (cleanup, backups, data sync)
Distributed Computing: Process tasks across multiple workers/servers
Async Workflows: Chain, group, and orchestrate complex task dependencies
Real-time Processing: Handle webhooks, notifications, data pipelines
Load Balancing: Distribute CPU-intensive work across workers

Don't Use When:

Simple async I/O (use asyncio instead)
Real-time request/response (use async web frameworks)
Sub-second latency required (use in-memory queues)
Minimal infrastructure (use simpler alternatives like RQ or Huey)

Quick Start

Installation

# Basic installation
pip install celery

# With Redis broker
pip install celery[redis]

# With RabbitMQ broker
pip install celery[amqp]

SKILL.md on GitHub

Details

Repository: github.com/bobmatnyc/claude-mpm-skills
Categories: Backend & APIs, Python
First seen: Jun 3, 2026