Multiple threads

[RishiRaj]

Hi ,

We wanted to increase the system responsiveness by having multiple threads in the application.
We are thinking of implementing all operational commands in a different threads context and all configuration in another thread’s context.
So we want that when init() callback is invoked ( i.e. transaction has started ) , we want to deduce that the transaction is for stats or configuration.
Is there anything available in the transaction context to derive this so that we can then do set_fd() to worker sockets accordingly.
We also tried to use the cb_opaque pointers for this but they are NULL in the init() callback and are only set in the get_elem() callbacks.

Thanks ,
Rishi

[greg]

There is no way to tell that the information that will be requested is configuration or operational ahead of time. In fact, if you have a mix of configuration and operational data, for example, in a list entry, then when you do a NETCONF (this will get both config and operational data in one request), ConfD will use the same transaction/worker thread to do a get_elem for both config and operational data. So here is a case where a worker thread may have to get both kinds of data.

If you think about it, from ConfD’s perspective the data, either config or operational, is just external to ConfD. ConfD doesn’t really make a distinction when the data is needed because of a request from a northbound interface, it just goes to the registered data provider to get what it needs.

[RishiRaj]

Hi Greg,

Thanks for the answer. In CLI the user cannot mix configuration and operational data commands since he needs to be in 2 different modes for that - in config mode he can feed in config and in terminal mode/operational mode he can issue show commands. ( this we checked in 5-c_stats example). May be in NETCONF this is not possible.

So the follow-up query is that to increase the system’s responsiveness , what to do ??

For example , if a northbound user issues a command to “show all stats xyz” which obviously is going to take considerable time , and if we have a single thread of execution then further configuration transactions will be blocked until the operation command processing is completed.

From your experience , can you let me know what is generally done to take care of these situations when using CONFD.

Thanks,
RIshi

[cohult]

@RishiRaj

In CLI the user cannot mix configuration and operational data commands since he needs to be in 2 different modes for that - in config mode he can feed in config and in terminal mode/operational mode he can issue show commands. ( this we checked in 5-c_stats example)

> configure
Entering configuration mode private
[ok]

[edit]
% run show all 

@RishiRaj

So the follow-up query is that to increase the system’s responsiveness , what to do ??

I recommend you take a look at the ConfD example $CONFD_DIR/examples.confd/intro/9-c_threads which show you an example of how to use threads in C for a data provider.

In addition to the 5-c_stats example these previous posts can be useful to speed up your data provider:

How to add the support of the get_object() callback to the 5-c_stats example?

In order to improve the performance of the data provider API callbacks, for both external configuration as well as operational data, when dealing with lists with many leaf elements in them, it is recommended to provide the implementation of the get_object() API. This will reduce the number of round trip calls from the ConfD daemon process for retrieving the leaf elements of each instance in your list as they will all be fetched with a single API call as opposed to via multiple get_elem() calls. …

How to add the support of the get_next_object() callback to the 5-c_stats example?

EDIT 2021 - an updated data provider performance demo is available here: As this post explains for the get_object() callback: In order to improve the performance of the data provider API callbacks, for both external configuration as well as operational data, when dealing with lists with many leaf elements in them, it is recommended to provide the implementation of the get_object() API. This will reduce the number of round trip calls from the ConfD daemon process for retrieving the leaf ele…

How to add the support of the find_next_object() callback to the 5-c_stats example?

EDIT 2021 - an updated data provider performance demo is available here: Implementing the find_next_object() callback can be a significant optimization for large lists in cases where a northbound agent, e.g. over NETCONF, requests a specific part of the list. Similar to the get_next_object() that we saw in action in this post, the find_next_object() callback combines find_next() and get_object() into a single callback and optimizes cases where ConfD wants to start a list traversal at some ot…