Is there a way to say that CDBs are synchronized between the participating nodes?
Let’s say Node A is the master, B and C are slaves and I’m not going to change the master’s configuration for the moment.
Can I somehow know in node B and C that they are up-to-date?
I tried looking in the notification: CONFD_NOTIF_HA_INFO
Or CONFD_NOTIF_HA_INFO_SYNC – with confd_sync_ha_notification()
But couldn’t understand if it exceeds my expectations.
Thanks
When CDB replication is enabled (which it is when high-availability mode is enabled, see /confdConfig/ha in the ConfD config file, default called confd.conf):
For configuration data , replication is done on a per-transaction basis to all the slaves in parallel. When the transaction through for example NETCONF has gone through successfully, all the slaves have been successfully updated too
For operational data (e.g. statistics), replication can be configured to be done asynchronously (best performance) or synchronously in step with the transaction (most secure). With synchronous mode you will know that the participating nodes are in sync for operational data (and always for configuration data).
For more on the ConfD HA functionality see ConfD UG “High Availability” chapter.
You can play with different HA scenarios using the ConfD HA Example, see $CONFD_DIR/examples.confd/ha/dummy/ See README, Makefile, and ctrl.c files.
So, you mean that if I config /confdConfig/cdb/operational/replicationMode to sync, “confd_ha_beslave()” would block?
(not only operational data?)
Thanks
If ‘/confdConfig/cdb/operational/replicationMode’ is set to ‘sync’ the transaction, e.g. over NETCONF, will not go through until all slaves have been updated (in a transaction within that transaction from the master to the slaves).
confd_ha_beslave() behaviour is not affected.
Actually, the relevant confd.conf parameter for config/transaction replication is /confdConfig/cdb/replication - /confdConfig/cdb/operational/replication pertains only to operational data in CDB. Please refer to the confd.conf(5) manual page.
Sorry, slight typo - while /confdConfig/cdb/operational/replication also pertains only to operational data in CDB, I meant to write /confdConfig/cdb/operational/replicationMode, which was the parameter being discussed. (The naming of the parameters is less than ideal, for “historical reasons”.)
Thanks for catching my syntax error Per. Didn’t double check the config parameter. Too quick on the trigger that time. Edited my initial answer for correctness for future reference
Is there a way to know in the slave that the transaction is over? Some kind of notification I can look for? for example, can I wait for CONFD_NOTIF_COMMIT_SIMPLE, and when I get it I know the slave is synchronized with the master?
I mean, as another process which is the HAFW, I want to be able to know if a slave is finished synchronizing with the master.
Thanks.
Exactly, you can subscribe to CONFD_NOTIF_COMMIT_SIMPLE to be notified of when a transaction has gone through, so for your use-case, when a transaction has gone through at the standby/slave node, the standby node knows that the ConfD CDB database has been updated by the active/master node.
A very good ConfD notifications starting point is the
$CONFD_DIR/examples.confd/misc/notifications
example
Quick demo where I subscribe for CONFD_NOTIF_COMMIT_SIMPLE notifications:
In terminal 1:
$ pwd
/Users/conny/tailf/confd-basic-5.4/examples.confd/misc/notifications
$ make start START_FLAGS=-c
...
/Users/conny/tailf/confd-basic-5.4/bin/confd -c ./confd.conf --addloadpath /Users/conny/tailf/confd-basic-5.4/etc/confd
./confd_notifications -c
Waiting for event notifications...
In terminal 2:
$ netconf-console -i
* Enter a NETCONF operation, end with an empty line
<edit-config>
<target>
<running/>
</target>
<config xmlns="http://tail-f.com/ns/config/1.0">
<root xmlns="http://tail-f.com/ns/example/noti">
<bar>32100</bar>
</root>
</config>
</edit-config>
<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="2">
<ok/>
</rpc-reply>
In terminal 1 we now see that our application received the CONFD_NOTIF_COMMIT_SIMPLE notification:
Waiting for event notifications...
commit on db 2 from user admin usid 12
See ConfD User Guide called “Notifications” for more info.
Note that if your goal is for the applications in the standby node to be notified of new configuration and react, we recommend that you use the more powerful CDB subscriber interface.
See ConfD Use Guide Chapter called CDB subscriptions and CDB subscription and Intro examples.
@cohult - The notifications and subscriptions feature will tell you that secondary has been updated by primary or not, but is there a way that the secondary can access the last trans-id of the primary or other nodes, so that it can check whether a secondary is completely upto date.
The use-case is:
P is primary and S1,S2 are secondaries
Let’s say connection between P and S1 is broken and something is committed in P (tid=t1)
Now, the replication mode by default is ‘sync’, but since S1 connection is broken so P would not know about S1.
So end result is S1 is out-of-sync (does not have t1) but S2 is in-sync (has t1)
Now, P dies and S1 becomes Primary so it will replicate to other nodes and the t1 is lost(config is lost)
I want to check if I can compare transaction-ids across nodes or any other way where I am able to determine that S1 is out-of-sync so when HAFW chooses S1 to be primary, I can do these additional checks in the code and reject the ‘primary’ status since the instance is out-of-sync.
I have read about Active-Active and tried the demo as referenced in Multiple confD instances running standalone shared CDB - Installation - ConfD User Community (tail-f.com) but I want to solve this problem with Active-Standup setup.
Hi, See the ConfD UG “High Availability” chapter under “Mode of operation”. If a secondary loses the connection, and while away, the primary is updated, the secondary will copy the entire configuration from the primary when reconnecting to get back into sync.
Replied here for same discussion - Multiple confD instances running standalone shared CDB - #19 by navjot
Hello there,
To ensure a secondary node is up-to-date with the primary in a high availability setup, you can subscribe to “CONFD_NOTIF_COMMIT_SIMPLE” notifications to confirm synchronization. For checking the transaction ID across nodes to verify sync status, ConfD does not provide a direct method. However, if a secondary loses connection and the primary updates, the secondary will copy the entire configuration from the primary upon reconnection to resynchronize.
Hope you find this informative!