Subscription replay from an arbitrary old tid

CDB and CDB API
1

Hi,
Is it possible to use cdb_replay_subscriptions() to reply subscription events from an arbitrary old transaction id.

The UG says

The caller specifies the transaction id of the last transaction that the application has completely seen and acted on.

This verifies that the application has only missed (part of) the last transaction. 

If a different (older) transaction ID is specified, an error is returned and no subscriptions will be triggered.

If the transaction id is the latest transaction ID (i.e. the caller is already up to date) nothing is triggered and CONFD_OK is returned.

What is the scope of this older transaction id? If we want to trigger subscription from an old transaction id (say 5-10 transactions behind current tid), do we need to reconnect with confd? Currently if we attempt this (w/o reconnecting) confd throws error like DEBUG internal error - Failed to restore replay information .

2

Hi,

As the confd_lib_cdb(3) man page for cdb_replay_subscriptions() says, it is only possible to replay the subscription events for the last configuration (i.e. transaction) using cdb_replay_subscriptions().
If the subscriber missed more than that, best practice is to resync all of the config that it subscribes to. The most common way to do that is to use cdb_trigger_subscriptons() for just that subscriber, i.e. for its subscription id.

3

Thanks Cohult for confirming this.

What caused my confusion is the description of API cdb_get_replay_txids().

When the subscriptionReplay functionality is enabled in confd.conf this function returns the list of available transactions that CDB can replay. The current transaction id will be the first in the list, the second at txid[1] and so on.

So here it is talking about multiple txids being available, which led me to the confusion that perhaps we can replay from arbitrary txids.

Regarding the last transaction only, is it configurable by any means? It felt like this information is maintained in replay.cdb. I tried to look for any settings to control its size (so that we can tell confd to maintain last say 10 transactions replay information). But did not find much.

Our use-case is, in our HA setup, active card dataplane gets configuration from CDB (via subscription) and then synchronizes it to standby card dataplane (by its custom clustering method). Since it is distributed in parallel to confd’s cdb synchronization, at the time of switchover, dataplane state could be out-of-sync with CDB state. So post switchover, we audit with CDB state by triggering all subscriptions by cdb_trigger_subscriptons() . But it turns out, if database was very large, then this auditing takes a lot of time.

So our plan to reduce it was to have a notion of which cdb_txid our standby card has last finished successfully and then trigger replay beyond that.

4

Hi,

Regarding the last transaction only, is it configurable by any means?

The man entry for cdb_get_replay_txids() should be updated. In the confd.conf(5) man page under ‘/confdConfig/cdb/subscriptionReplay/enabled’ it is more clear. CDB will save only one previous transaction in a separate file, here replay.cdb.

I read that you use ConfD’s HA functionality to synchronize the active card CDB with the standby card CDB. Is that correct?

5

Yes, CDB is being synced using ConfD’s synchronous replication functionality.

6

How do the subscribers read their config? Using cdb_get_,modifications(), cdb_diff_itter() or with cdb_get_*() calls?

7

Currently subscribers populate change-sheet by recursive handling in using cdb_diff_iter() on CDB_SUB_PREPARE msgs on subscription sockets.

8

Is it possible to get the transaction id value (cdb_txid), for which one subscription event has been generated (from cdb_diff_iterate context).
I had following in subscription socket event

  cdb_start_session2(readsock, CDB_RUNNING, CDB_LOCK_REQUEST | CDB_LOCK_WAIT);

So I added a code there to obtain tid

  cdb_get_txid(readsock, &txid);

But this is giving me the last saved tid (which kind of makes sense too. Since it is a new cdb session.) So somehow I have to attach to current cdb session to obtain the proper txid value. Is it possible from subscription context?

9

To get the transaction id from subscriber context you can use MAAPI:

maapi_get_elem(sock, tid, &v, "/confd-state/internal/cdb/datastore{running}/transaction-id");
1 Like