When I first built renee.ovh, MFA (Multi-Factor Authentication) wasn’t part of the initial release. Later, I decided to add a simple layer of security through email-based OTP verification on each login. I deliberately avoided TOTP (Time-based One-Time Password) to keep things straightforward for now, at least.
The logic was simple:
-
If the user logs in with the correct email and password,
-
Generate an OTP,
-
Send it to their email.
To avoid delaying the user’s login experience, I didn’t await the email sending function. Instead, I used my old friend asyncio.create_task() a classic “fire and forget” move. No error handling. No logging. Just trust the code and lord.
And it worked… until it didn’t.
Eventually, I started noticing a problem: some OTPs never reached the user’s inbox. No errors, no clues. It was as if they were sent straight to Mars. Since the email task wasn’t awaited, and there was no error capturing or retry mechanism, I had no way of knowing what went wrong. I was too busy with other parts of the project, so I let it slide until I read about Celery.
I had heard of Celery before but never really paid attention. This time, though, it clicked.
After reading a few blog posts, I realized Celery was exactly what I needed: a robust solution for managing background tasks like sending emails, without blocking the main app.
Let me break it down simply:
-
Celery: A task queue for running asynchronous jobs (sending emails, processing images, etc.)
-
RabbitMQ/Redis: Message brokers that hold tasks in a queue until a worker is ready to process them.
The process looks like this:
-
My app says:
→ “Hey Celery, please send this OTP email.” -
Celery passes the task to RabbitMQ.
-
RabbitMQ queues it like a to-do list.
-
A Celery worker, always listening, picks up the task.
-
The worker sends the email and reports success/failure if we need.
Simple. Efficient. Reliable.
Why Celery is a Game Changer
-
✅ Built-in retries for failed tasks.
-
✅ Can run multiple workers on multiple servers.
-
✅ Handles I/O-bound tasks beautifully.
-
✅ Support for concurrency (e.g., using
eventletfor lightweight, scalable processing). -
✅ You can even store task results if needed.
This architecture ensures that my main app stays fast and responsive, while all the heavy lifting happens quietly in the background.
At first, I hit a snag: I was using an async email library to send OTPs, which worked great with asyncio.create_task. But Celery wasn’t happy with it. Celery tasks run in a synchronous environment by default, and mixing async code in a sync context caused unexpected issues.
The fix? I swapped out the async library for a synchronous email sender, the built in smtplib library used and it worked like a charm.
Did I spend hours debugging the async issue?
Nope. Not yet.
Maybe tomorrow.
Maybe after another bug.
Maybe when I’m older and wiser. 😅
Don’t fire-and-forget critical tasks like MFA emails. Use Celery with RabbitMQ/Redis to handle background jobs the right way with retries, scalability, and peace of mind.
