Background Tasks with Celery

Celery is a distributed task queue that runs slow or unreliable work in separate worker processes, so a Django view can enqueue a job and return immediately instead of making the user wait for it.

Learn Background Tasks with Celery in our free Django course — a beginner-friendly interactive lesson with worked examples, a practice exercise and a quick…

Part of the free Django course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

In this lesson you'll define a task with @shared_task, enqueue it with .delay(), understand the broker and result backend, run a worker that executes queued tasks, and add retries and periodic schedules with Beat.

You mark a function as a Celery task with the @shared_task decorator. The crucial idea is that calling task.delay(args) does not run the function in your web process — it serializes the arguments into a message, puts that message on the broker, and returns instantly. A separate worker runs it later. This is how a view replies to the user in milliseconds while the slow work happens in the background.

Three pieces make Celery work. The broker (Redis or RabbitMQ) holds the queued messages. A worker is a separate process you start with celery worker ; it pulls messages off the broker and runs the tasks. The optional result backend stores each task's return value and status so you can look it up later by task id.

Background work fails sometimes — an API times out, a payment gateway hiccups. With bind=True and self.retry you re-enqueue a failing task up to max_retries times, optionally after a delay. For work that should run on a schedule, Celery Beat is a scheduler that puts tasks on the broker at fixed times defined in beat_schedule ; the regular workers still execute them.

Replace the blank with .delay so the task is queued instead of running right now.

You enqueued it with .delay() but no worker is running.

✅ Fix: start a worker with celery -A myproject worker -l info .

❌ I passed a whole model object and got a serialization error

Task arguments are serialized to the broker, and full objects don't serialize cleanly.

✅ Fix: pass the primary key (e.g. user.id ) and re-fetch inside the task.

No result backend is configured, so results aren't stored.

✅ Fix: set CELERY_RESULT_BACKEND , then read result.get() by task id.

Build a small Celery-like system with a task registry, a broker, a worker, and a result backend. Enqueue two tasks with .delay() , run the worker, and fetch the results.

Lesson complete — slow work runs in the background!

You defined tasks with @shared_task, enqueued them with .delay(), traced a task from the broker through a worker to the result backend, and added retries for flaky work plus periodic schedules with Beat.

🚀 Up next: Checkpoint: Production Django — combine everything into a production-ready feature.

Practice quiz

Which decorator marks a function as a Celery task?

  • @shared_task
  • @task_queue
  • @background
  • @celery_job

Answer: @shared_task. @shared_task turns a function into a Celery task you can enqueue.

What does calling task.delay(args) do?

  • Runs the task immediately and blocks
  • Deletes the task
  • Pauses the worker
  • Enqueues a message on the broker and returns instantly

Answer: Enqueues a message on the broker and returns instantly. .delay() serializes the arguments, puts a message on the broker, and returns an AsyncResult.

What is the broker's job?

  • It stores return values
  • It holds queued messages until a worker runs them
  • It renders templates
  • It schedules nothing

Answer: It holds queued messages until a worker runs them. The broker (Redis or RabbitMQ) is the message queue that holds tasks for workers.

What does .delay() return?

  • The task's final result
  • None
  • An AsyncResult with a task id
  • The worker process

Answer: An AsyncResult with a task id. .delay() returns an AsyncResult almost instantly; the work happens later in a worker.

What runs the queued tasks?

  • A worker process started with celery worker
  • The web server
  • Celery Beat alone
  • The result backend

Answer: A worker process started with celery worker. A separate worker process pulls messages off the broker and executes the tasks.

What stores a task's return value for later lookup?

  • The broker
  • The web request
  • Celery Beat
  • The result backend

Answer: The result backend. The optional result backend stores each task's status and return value by task id.

How do you retry a failing task up to a limit?

  • Call task.delay() in a loop
  • Use bind=True and self.retry with max_retries
  • Restart the worker
  • Set timeout=None

Answer: Use bind=True and self.retry with max_retries. With bind=True you call self.retry(exc=exc), re-enqueuing up to max_retries.

What is Celery Beat used for?

  • Running workers
  • Storing results
  • Scheduling periodic tasks onto the broker
  • Serializing arguments

Answer: Scheduling periodic tasks onto the broker. Beat is a scheduler that enqueues tasks on a schedule; workers still execute them.

Why pass a primary key instead of a full model object to a task?

  • PKs are faster to type
  • Tasks cannot take arguments
  • Workers reject objects
  • Arguments are serialized to the broker and full objects don't serialize cleanly

Answer: Arguments are serialized to the broker and full objects don't serialize cleanly. Pass the PK and re-fetch inside the task to avoid serialization problems.

If you call .delay() but never start a worker, what happens?

  • The message waits on the broker and nothing runs
  • The task runs in the web process
  • An error is raised immediately
  • Beat runs it instead

Answer: The message waits on the broker and nothing runs. Without a worker, the queued message just sits on the broker and is never executed.