Transactional behavior - camunda-community-hub/vanillabp-camunda7-adapter GitHub Wiki

Camunda 7 runs inside your application's JVM and on your application's database. So the engine could do everything inside your transaction. This adapter did that, until it hit the one thing this model cannot offer: an operation which loses a conflict with a concurrent change cannot be repeated.

  1. What happens when you call VanillaBP Which operations run after your commit, and which ones still answer inside it.
  2. Why it is worth the change The concurrency conflict which cannot be repeated inside the caller's transaction.
  3. What this means for you What your code has to expect from an operation which happens after the commit.
  4. What did NOT change The parts of the old behaviour which are still exactly as they were.
  5. What an engine on its own datasource changes here The second transaction that mode introduces, and what follows from it.

What happens when you call VanillaBP

Some operations move a workflow forward: starting it, completing or cancelling a task, correlating a message, broadcasting a signal, pushing the changed aggregate. They run after your transaction has committed, through VanillaBP's phase-two outbox. Your call writes the outbox entry in the same transaction as your business data. Right after the commit, the entry is sent on. (why)

What your call still does at once is ask. An embedded engine answers from your own transaction, exactly and at no cost. So these calls still fail where you made them, and not later in a log file:

You call What is checked before your transaction may commit
completeTask / cancelTask the task (the parked execution) still exists
completeUserTask / cancelUserTask the user task still exists
correlateMessage an execution of this workflow waits for that message, and with a correlation id: that it expects exactly this one
startWorkflowByMessage the process of your ProcessService has a message start event of that name
startWorkflow nothing, there is no state yet to check

Why it is worth the change

An application which answered an open task while a timer job of the same workflow ran used to get this:

OptimisticLocking ENGINE-03005 Execution of 'UPDATE ExecutionEntity[63]' failed.
Entity was updated by another transaction concurrently.

Camunda's own job executor survives such a collision. It runs the whole command again in a transaction of its own. VanillaBP could not do that inside your transaction. Every engine command joins your transaction, so a failing command marks it rollback-only. VanillaBP could repeat only the engine part in a transaction of its own. But then the process would move forward while your data rolls back, a ghost progress. With the outbox, the operation is simply repeated until it succeeds.

What this means for you

  • You need an outbox. Configure one: JDBC/gruelbox on Spring Boot, JDBC or MongoDB on Quarkus. If VanillaBP cannot find one, the application says so while booting.
  • "The engine has done it when my method returns" is no longer true. When your method returns, the operation is scheduled. It happens right after the commit. Code which read engine state right after such a call has to wait for the workflow instead. It always had to do that on every other BPMS.
  • @Transactional keeps its meaning. Your aggregate and the outbox entry commit together, and the workflow only moves forward if that commit happened.
  • Repeating is at-least-once. Every operation is idempotent, so running it twice does no harm. A start skips an existing instance with the same business key and tenant. Completing or cancelling checks that the task is still there. Correlating checks the subscription. A failure which the engine rejects as a wrong request (BadUserRequestException) is not repeated at all. The outbox entry is blocked at once, and the log says why.
  • A repeated start looks only at workflows started since you called startWorkflow. Before VanillaBP repeats a start, it asks the engine's history whether the workflow exists already. It counts only the workflows of the aggregate which started at or after the moment the start was planned. So where the first workflow of an aggregate ended and you start a second one, a failed first attempt is repeated, and the ended workflow does not count as the new one.

What did NOT change

While the engine shares your datasource, it still delivers tasks inside its own transaction. Your @WorkflowTask method, the aggregate it changes and the engine's state commit or roll back together. If the engine retries a job, that proves nothing was committed. That is why nothing is deduplicated in this mode, see Configuration. (why)

VanillaBP still writes a record of such a delivery, but the record does not answer a repetition. It says which BPMS holds the task and which kind of id the task has. A later call about that task reads this. See what a record of a delivery is for. (why)

What an engine on its own datasource changes here

An engine with a data-source-name of its own cannot join your transaction. So the section above is no longer true for calls from the engine to your code. Your handler runs in a transaction which VanillaBP opens on your datasource. That transaction commits before the engine's job does. A crash in between leaves the aggregate changed and the job unfinished. So the engine hands the same job out again. This is the at-least-once delivery which every remote BPMS has:

sequenceDiagram
  autonumber
  participant E as Engine (its own datasource)
  participant V as VanillaBP
  participant H as Your @WorkflowTask method
  participant DB as Your datasource
  E->>V: execute job 42
  V->>DB: BEGIN
  V->>H: invoke(aggregate)
  V->>DB: save aggregate + record delivery 'job 42'
  V->>DB: COMMIT
  E--xE: the job transaction fails
  E->>V: execute job 42 again
  V->>DB: read the record of 'job 42'
  V-->>E: the recorded outcome, your method is not entered again

Step 4 of the picture makes the difference. The adapter reports the id of the job as the identity of the delivery. The engine keeps that id while it counts down the job's retries. The core records the processed delivery under that id, in the same transaction which saves your aggregate. The repeated delivery finds that record and reports the outcome it holds, and your method is not called again. If no delivery log is available, your application says so once while booting. The message names both ways out: a TaskDeliveryLog bean of your own, or the switch which turns deduplication off.

A user-task notification still runs again

A user task gets no job of its own. One transaction creates every user task the token reaches. So the job would name several notifications, and the second one would look like a repetition of the first. Whatever is unique per task is created together with the task. It does not survive the rollback which causes the repetition. So a handler which is told about a created or cancelled user task has to cope with the repetition itself, in both datasource modes.

A record of that delivery is written anyway. It deduplicates nothing. It tells a later caller that the id belongs to a user task. If somebody calls completeTask with the id of a user task, the answer names the method they should have called, instead of a list of guesses. The record lives as long as vanillabp.delivery.retention allows, counted from the moment your handler ran, because no new delivery arrives to extend it. If a user task stays open longer than that, it loses its record, and a wrong call about it gets the list of guesses again.

What happens outside your transaction stays yours

A mail or a payment happens while the handler runs, before any record exists. So if a handler crashed halfway, the second delivery sends it again. Base such decisions on the state of the workflow aggregate. This is the price of the mode, and the reason why the shared datasource is the default.