Someone posed a question on the falcon mailing list the other day. They asked "How do I handle a long running-request > 15 minutes, but ensure termination when the client kills the http connection?"

The response was: Don't

Instead of having the client wait on one request for the 15 minute plus answer, try a polling style request instead:

Polling Style Request

  1. Accept the job, and queue it, returning a 202 and a transaction/request id
  2. Have another end-point to be able to query the status (working, completed, canceled, etc) of the transaction/request id
  3. Have a third end-point to get the result

Courtesy Ben Meyer

Libraries and Tools for Long Running Tasks

As suggested on the thread.

  • RabbitMQ is a message library. Check out this tutorial on work queues. Tutorial

  • Celerey is an asynchronous task queue. Hook this up with RabbitMQ.