External dB performace

Hi,

I’m working on performance right now and looking at how confd interacts with an external dB.

then when I create a entry in a list I get the following in the trace log:

TRACE CALL trans init(thandle=15,mode="rw",db=running)
TRACE CALL data get_elem(thandle=15,/x/y{z}/key)
 (NOEXISTS)
TRACE CALL trans_lock(thandle=15)
TRACE CALL write_start(thandle=15)
TRACE CALL data create(thandle=15,/x/y{z})
TRACE CALL data set_elem(thandle=15, /x/y{z}/a, 1)
TRACE CALL trans prepare(thandle=15)
TRACE CALL trans commit(thandle=15)
TRACE CALL trans finish(thandle=15)

Which is fine, not much to say about that.

If I now do a edit_config to my running dB and change the a leaf value to 4 via netconf-console I get.

TRACE CALL trans init(thandle=17,mode="rw",db=running)
TRACE CALL data get_elem(thandle=17,/x/y{z}/key)
(z)
TRACE CALL data get_elem(thandle=17,/x/y{z}/a)
(1)
TRACE CALL trans_lock(thandle=17)
TRACE CALL trans init(thandle=18,mode="r",db=running)
TRACE CALL data get_object(thandle=18,/x/y{z})
TRACE CALL trans finish(thandle=18)
TRACE CALL write_start(thandle=17)
TRACE CALL data set_elem(thandle=17, /x/y{z}/a, 4)
TRACE CALL trans prepare(thandle=17)
TRACE CALL trans commit(thandle=17)
TRACE CALL trans finish(thandle=17)

Why do I get the transaction 18? This is very costly, confd has enugh information in transaction 17 to perform the operation.

The real objects in our system are quite large and costly to fetch so the transaction 18 will take more time than the transaction 17 in total.

Why do confd start a new transaction and read the whole object? which just waste time
If this would be a part of the validate phase of the transaction all the read operations should have been on the transaction 17.

Pls explain why we see the transaction 18 and how we can get rid of it.

Hi Kackola,

Did you got the reason for the above call ?

@All,
Can anyone explain the above call with transaction 18 ?

Do you have the rollback feature enabled in confd.conf? If you disable it you won’t get this behavior.

The second read transaction is used to read the “object” from Running post locking the database so that we have the latest old value so a rollback file can be created, and to also make sure that the object still exists or not (in which case ConfD needs to call the appropriate callbacks such as set_elem() for ‘a’ and/or create() for the list ‘y’).

See user guide, section 9.7. External configuration data with transactions.

I think the read post trans_lock() is clear. This helps serialize transactions.

If we read this value using the same first transaction, we would read the new value and not the one we needed to save as rollback. This is why we had to start a new read transaction to the still unmodified running.

If you want to avoid this, I think you can disable rollbacks in confd.conf.

If you enable the trace level of the developer log: in confd.conf, you will see more details on what ConfD is doing.

Example:

12-Aug-2020::20:46:01.051 NMICHRAF-M-26ZL confd[19438]: confd commit progress db=running usid=17 thandle=58: applying transaction…

12-Aug-2020::20:46:01.051 NMICHRAF-M-26ZL confd[19438]: confd commit progress db=running usid=17 thandle=58: entering validate phase for running…

12-Aug-2020::20:46:01.051 NMICHRAF-M-26ZL confd[19438]: confd commit progress db=running usid=17 thandle=58: grabbing transaction lock…

12-Aug-2020::20:46:01.051 NMICHRAF-M-26ZL confd[19438]: confd commit progress db=running usid=17 thandle=58: grabbing transaction lock ok

12-Aug-2020::20:46:01.051 NMICHRAF-M-26ZL confd[19438]: confd commit progress db=running usid=17 thandle=58: creating rollback file…

12-Aug-2020::20:46:01.056 NMICHRAF-M-26ZL confd[19438]: devel-c new_trans request daemon id: 0 thandle: 59

12-Aug-2020::20:46:01.056 NMICHRAF-M-26ZL confd[19438]: devel-c new_trans succeeded daemon id: 0 session id: 59 worker id: 4

12-Aug-2020::20:46:01.056 NMICHRAF-M-26ZL confd[19438]: devel-c get_elem request for callpoint simplecp path /smp:servers/server{ssh}/port

12-Aug-2020::20:46:01.057 NMICHRAF-M-26ZL confd[19438]: devel-c get_elem succeeded for callpoint simplecp path /smp:servers/server{ssh}/port

12-Aug-2020::20:46:01.057 NMICHRAF-M-26ZL confd[19438]: devel-c get_elem request for callpoint simplecp path /smp:servers/server{ssh}/port

12-Aug-2020::20:46:01.057 NMICHRAF-M-26ZL confd[19438]: devel-c get_elem succeeded for callpoint simplecp path /smp:servers/server{ssh}/port

12-Aug-2020::20:46:01.058 NMICHRAF-M-26ZL confd[19438]: devel-c close_trans request daemon id: 0 session id: 59

12-Aug-2020::20:46:01.058 NMICHRAF-M-26ZL confd[19438]: devel-c close_trans succeeded daemon id: 0 session id: 59

12-Aug-2020::20:46:01.059 NMICHRAF-M-26ZL confd[19438]: confd commit progress db=running usid=17 thandle=58: creating rollback file ok

Thanks nabil.
After disabling the rollback feature , get calls are not seen for running db in “r” mode.