I've also been struggling to wrap my head around durable execution hype and whether my workload would benefit, maybe after sleeping on this post it will be clear.
A bit off-topic, but I recently switched from Celery to Hatchet. I haven't even fully explored everything it can do, but the change has already made me a fan. Overall simplified my stack and made several features easier to implement.
Some takeaways from my experience
1. Streaming — My application provides real-time log streaming to users (similar to GitHub Actions or AWS CodeBuild). With Celery, I had to roll my own solution using Supabase Realtime. Hatchet’s streaming is straightforward: my frontend now connects to a simple SSE endpoint in my API that forwards the Hatchet stream.
2. Dynamic cron scheduling — Celery requires a third-party tool like RedBeat for user-defined schedules. Hatchet supports this natively.
3. Logs — Hatchet isolates logs per task out of the box, which is much easier to work with.
4. Worker affinity — Hatchet’s key-value tags on workers and workflows allow dynamic task assignment based on worker capabilities. For example, a customer requiring 10 Gbps networking can have tasks routed to workers tagged {'network_speed': 10}. This would have required custom setup in Celery.
5. Cancellation — Celery has no graceful way to cancel in-flight tasks without risking termination of the entire worker process (Celery docs note that terminate=True is a “last resort” that sends SIGTERM to the worker). Hatchet handles cancellation more cleanly.
A bit off-topic, but I recently switched from Celery to Hatchet. I haven't even fully explored everything it can do, but the change has already made me a fan. Overall simplified my stack and made several features easier to implement.
Some takeaways from my experience
1. Streaming — My application provides real-time log streaming to users (similar to GitHub Actions or AWS CodeBuild). With Celery, I had to roll my own solution using Supabase Realtime. Hatchet’s streaming is straightforward: my frontend now connects to a simple SSE endpoint in my API that forwards the Hatchet stream.
2. Dynamic cron scheduling — Celery requires a third-party tool like RedBeat for user-defined schedules. Hatchet supports this natively.
3. Logs — Hatchet isolates logs per task out of the box, which is much easier to work with.
4. Worker affinity — Hatchet’s key-value tags on workers and workflows allow dynamic task assignment based on worker capabilities. For example, a customer requiring 10 Gbps networking can have tasks routed to workers tagged {'network_speed': 10}. This would have required custom setup in Celery.
5. Cancellation — Celery has no graceful way to cancel in-flight tasks without risking termination of the entire worker process (Celery docs note that terminate=True is a “last resort” that sends SIGTERM to the worker). Hatchet handles cancellation more cleanly.