1.12 Replication enhancements - rbasch/krb5 GitHub Wiki

Two key enhancements to the Kerberos 1.12 iprop native replication have been implemented:
1. Implement tree-model replication strategy to facilitate better scale-out.
2. Implement "notify" method to facilitate closer to real-time replication.

Tree model replication was implemented by having the slave track ulog updates received by kpropd and committing them to the local ulog while performing ulog_replay. Fixes were required in kproplog because the circular ulog buffer may not have content at the beginning on slaves since their ulog will only begin populating since the last full resync.

kpropd can now be started with an alternate "admin server" from which to poll for database updates. kadmind can be started on slaves in a mode which only answers kpropd requests (-proponly).

https://github.com/rbasch/krb5/commit/0f4b84669c345cb81e60b26372fa32bb9834c093 (ulog tracking on slaves) https://github.com/rbasch/krb5/commit/023602cc7934c96ab7130aa19481228958473749 (hierarchical support)

Change notification was implemented atop the tree replication patches. When kadmind receives an update, it will notify any slave it has seen poll it with an iprop get_updates request of pending changes. The notification is performed by opening a connection to the slave kpropd on the kprop port (normally used for full resync) and sending a generic Kerberos error message. On older clients, this has no ill effect. With this patch, newer kpropd implementations will send a USR1 to the other kpropd process (the iprop fork) which resets its timer and causes an immediate re-polling for updates.

Since notification should trickle down to downstream slaves in a tree replication model and since kadmind receives no updates on the slaves, an additional IPC was required. When kpropd applies updates, it will then attempt to connect to the local kadmind (if it is running), invoking the NULLPROC process. This will also cause kadmind to notify any slaves it has seen poll about pending updates.

Once kadmind has notified a slave about pending updates, it forgets about the slave for future notifications until such time as the slave performs another iprop get_updates.

https://github.com/rbasch/krb5/commit/de078a1f49a8d40ba5a5cb7fdbe22c593f30509a (slave notification)

Note: There is a comment in the patch suggesting it might be possible for contention to arise resulting in slaves receiving an UPDATE-BUSY response. However, after further examination of the 1.12 code base, locks are blocking operations and kpropd no longer has any ability to generate an UPDATE-BUSY response. However, the original code submission introduced such a condition, and a patch is required.

https://github.com/rbasch/krb5/commit/a055a47f28afcb882bfd9c7ad3aa5a8a5327059e (patch - UPDATE_BUSY/lock issue)