Data integrity ‐ Stale browser data - civiform/civiform GitHub Wiki

In a "stale tab" or "lost update" integrity issue, if an entity is being updated in multiple browser tabs, the second update can overwrite the firsts' changes without ever knowing there had been other changes.

This can happen when two users are editing the same Question or Program at the same time or when a single user has the same edit page open in two tabs, and submits one after the other; even days later. In both cases, the state stored in the second tab is "stale" with respect to the database but the application may apply its old data as if they are intended new changes. This is further exacerbated if a publish action has occurred in the mean time and the system allows published data to be changed.

This data integrity issue occurs when state is stored on the client, such as on the question edit page: It stores the question text, question help text, question options, and other pieces of the QuestionDefinition in form fields on the client and builds a new QuestionDefinition with those fields when the form is submitted. If the contents of the database changed since the page was loaded, the tab isn't updated so the user never knows.

For example, when an admin user is updating the options to a multi-option question, the question options are held on the client as values in the form. If two admin users are concurrently updating the question options, we have this scenario:

  1. Admin User A loads the question edit page with options [spoon, knife].
  2. Admin User B loads the question edit page with options [spoon, knife].
  3. Admin B adds the option fork and submits the form with options [spoon, knife, fork].
  4. The database now has options [spoon, knife, fork].
  5. Admin A adds the option chopsticks and submits the form with options [spoon, knife, chopsticks].
  6. Admin A's changes overwrote Admin B's, so the database now has options [spoon, knife, chopsticks], with no fork.

Preventing stale tab integrity issues

This issue is prevented by having a concurrency_token field on the entity in the database, which is just a UUID field that is updated every time the entity is updated.

We pass this token to the frontend form and persist it as hidden data. When the form is submitted the token is passed back to the server along with the other data. In the process of saving the new form data we check that the concurrency_token has not changed before saving any data. If it hasn't changed we set it to a new UIUD and persist the changes. If it has changed it means another user edited the entity while we had the form open; the update should fail and the user should be notified to refresh and try again.

Note: It's important that the concurrency_token be checked and updated in the same transaction that saves the data in case there are simultaneous update operations.

Example update flow

In practice, this looks like:

  1. Admin User A loads the question edit page with options [spoon, knife] and concurrency_token ending in 1a2f56.
  2. Admin User B loads the question edit page with options [spoon, knife] and concurrency_token ending in 1a2f56.
  3. Admin B adds the option fork and submits the form with options [spoon, knife, fork] and concurrency_token ending in 1a2f56.
    1. Before persisting the changes, the server checks if the question's row in the database still has concurrency_token ending in 1a2f56.
    2. The concurrency_token matches, so a new token is generated ending in 32f1e50 and the update is persisted.
  4. The database now has options [spoon, knife, fork] and concurrency_token ending in 32f1e50.
  5. Admin A adds the option chopsticks and submits the form with options [spoon, knife, chopsticks] and concurrency_token ending in 1a2f56.
    1. Before persisting the changes, the server checks if the question's row in the database still has concurrency_token ending in 1a2f56.
    2. The concurrency_token does not match (it's now 32f1e50) so the server throws a ConcurrentUpdateException.
  6. The server reloads a fresh form showing Admin B's changes with options [spoon, knife, fork], so that Admin A can try their update again without overwriting Admin B's changes.

In CiviForm

In CiviForm, (as of #9834) the concurrency token for Questions is validated in the model's @PreUpdate handler. This prevents services and repositories from having to deal with concurrency token creation and validation. The rest of the application only needs to worry about passing the token back and forth between the client and the server and handling concurrency failures.

As of #9834, only the Question is guarded with concurrency tokens. They still need to be added to other entities, such as Programs.

Potential issues when using concurrency tokens:

[!CAUTION] Always be sure to use the concurrency token received from the client when persisting updates to the database.

If the server's update flow loads the entity from the database into memory, the entity in memory may have a newer concurrency token than the one the client passed back. If you don't first set the concurrency token on the entity to the one received from the client, the update is no longer guarded because it's guaranteed to have the same concurrency token as the one in the database. For example, this flow AdminQuestionTranslationsController.update() first retrieves the current draft QuestionDefinition from the database and then applies the localization changes to it, rather than building an entire QuestionDefinition from scratch from the form contents, so we must be sure to also add the concurrency token to the QuestionDefinition before persisting it.

[!CAUTION] If an update fails due to a concurrent update, always reload a fresh form so the user sees the changes that were made by the other user.

If you reload the form with the user's attempted changes, as we do for other validation errors, they'll never know there were concurrent changes and they won't be able to submit their changes because the server will never have retrieved the other user's updates or the newer concurrency token from the database.