ERP Tech Talk #10: Odoo Celery Basic Implementation

Odoo is an all-in-one management software that offers a range of business applications for small and mid-size businesses that form a complete suite of enterprise management applications, it offers community and enterprise versions. It designs to be easy both for the developer to extend the capability and practical for users.

To realize convenience from the developer side, Odoo uses ORM (object-relational mapping) which acts as a bridge between programming language and database. The ORM helps to execute SQL queries without writing them explicitly. The developer just focuses on how to manipulate the object and Odoo will take the rest. However, this feature comes with a price. Accessing data through ORM doesn’t go as fast as using direct SQL. If you want to know what the SQL looks like for a certain ORM operation, please read the article written by my colleague: ERP Tech Talk #8: High-Performance Odoo.

Back again to ORM. Writing operation through ORM still the common way instead of rewriting it to SQL syntax due to its benefit [³]. When a user posts a request to the server, the user needs to wait until the process is accomplished. This happens because Odoo uses a synchronous process to force everything to run serially.

Because the process in Odoo takes place serially, for certain conditions such as processing a lot of data, it will take a long time and this forces the user to wait until the process is complete. Users will be presented with a message to wait without certainty when it’s finished.

To avoid users waiting for the process to finish, this behavior must be re-engineered differently. The trick is to make the processes not run synchronously but forced to run in the background asynchronously. The big benefit is that users can focus and move on to the next job.

The celery comes into account. Celery is one of the common and famous tools in the python world to implement distributed task queues. Celery was chosen because[⁴]:

• Very simple, easy to use, and maintain
• Very fast with a sub-millisecond round trip latency that uses the standard protocol for message queue (AMQP)
• Manageable task (error handling, task prioritization, etc)
• Traceable process

Basically, when the user sends the request through a reverse proxy. The reverse proxy will reroute to the Odoo instance that is available to work. The Odoo instance will prepare any prerequisite parameters and forward them to the broker. In this case, Celery by default using RabbitMQ as broker messenger and puts it into queue block. The consumer will consume the data in the broker’s queue as soon as possible. Even it took so much time, the user will not be disturbed by the consumer. The number of consumers instance can be more than 1. When the consumer received the data/parameter from the broker agent it will post it to Odoo worker instance. This worker only receives the request from the consumer, not from the user. To inform the user that they are currently queuing, we can create an intermediate state so that the user knows the current state of the document.

Implementation

Time to make our hands dirty. Let’s say we want to invoice validate that contains more than 500 lines. By extending the state field (line 9), the user will know the invoice status after clicking the validate button. Any requested parameters to call call_task are prepare in the method action_invoice_open_queue. If there is something went wrong, the state could be a rollback to the previous state (line 33). The consumer in the celery will call _action_invoice_open_queue in the Odoo worker.

Register a new button that calls method action_invoice_open_queue when the user clicks the validate button. This button only visible when the state is draft. Otherwise, it will invisible. Then, the original button that calls method action_invoice_open will force to invisible.

Feature

Even though the design is quite simple, it is assumed that all goes well. However, there are times when unexpected conditions are encountered. To cover it, just extend celery.task model.

• If there is a user error exception, the task will be canceled and the state of the document being processed will be returned to the previous state-defined when creating the task. For example, if the invoice state changes from draft to queue by validation process and it forms an unbalanced entry journal, then the invoice status will be returned to its initial condition. After that, the user can check the cause of the problem if they have had sufficient time (line 132).
• If there is a TransactionRollbackError (concurrent update) exception, the task will be automatically re-requested without being limited by the number of repetitions. Normally, this number of repetitions is limited to avoid wasted processing (line 145).
• Normally, canceling tasks manually will influence to the original document. By customization, the task cancellation process can force the document’s status back to its previous state (line 41).

Conclusion

Implementation of the queuing system becomes very easy with celery. This will also increase the user confidence level. The user does not need to be bothered with an annoying display that does not have direct benefits for it. With this queuing system, the need to upgrade the system exponentially can be avoided and will ultimately save the cost of purchasing new hardware.

Reference

1. IT Paragon’s ERP Tech Talk #10 by Hendra Priyana Mirantika
2. ERP Tech Talk #8: High-Performance Odoo
3. https://www.cybrosys.com/blog/orm-in-odoo
4. https://docs.celeryproject.org/en/stable/getting-started/introduction.html#celery-is
5. https://apps.odoo.com/apps/modules/11.0/celery/