Hello,
suppose there’s a ConfD with RUNNING and OPERATIONAL stores and a bunch of long-running applications that read and write to it. Forget for a moment about hooks and validators and subscriptions.
Each application opens one or more connections to ConfD (sockets) and keep them open all the time.
Consider two models for applications:
MODEL A: Whenever an application (actually a thread) needs to read/write CDB it opens a new session on a socket, does its thing and closes the session.
MODEL B: The application (actually each thread) opens two long running sessions (separate sockets required?), one to RUNNING, one to OPERATIONAL and it tries to reuse it when it needs to do stuff with ConfD during its “sessions”.
Is MODEL B feasible at all? I’m aware of locking (it would need to lock/unlock on every “session”, at least in case of configurational store), but is it enough?
What if a commit takes place (by user or by another application). Is the long running session aware of the changes when the application gets to start its “session”?
Is the long-running session going to time out if the application’s “sessions” are spread in time? Any keep-alive or cdb_set_timeout needed?
Is the application going to notice immediately if connection is broken in case ConfD is shut down or restarted?
Is ConfD bothered in any way if such a long-running session exist?
And last but not least, would there be any gains (considerably less stress on ConfD, less CPU load) in MODEL B vs MODEL A?
“A CDB data session” means that you first do a cdb_connect(my_socket, CDB_DATA_SOCKET, ... , ...), then a cdb_start_session()/cdb_start_session2(). You can reuse the same socket for another CDB data session, but you must then end the first session and create another session using cdb_start_session(). So you need separate sockets for your two “long-running” CDB sessions.
You need one socket per thread. No sharing of sockets between threads, please.
From ConfD UG chapter The Protocol and a Library Threads Discussion:
ConfD API functions are thread-safe as such, but multiple threads using them with the same socket will have unpredictable results, just as multiple threads using the read() and write() system calls on the same file descriptor in general will. In the ConfD case, one thread may end up getting the response to a request from another, or even a part of that response, which will result in errors that can be very difficult to debug.
If you want to share sockets, another option is for the application to use MAAPI for reading(/writing) config, and reading/writing operational data. Using MAAPI you can start multiple transactions towards different datastores from the same “user session” (could be system context or a dedicated user) as transactions are separated by the transaction id.
When a config transaction is committed and has entered the commit phase, the new configuration will be visible to applications/clients reading the configuration when the transaction has entered the commit phase.
See /confdConfig/cdb/clientTimeout in the confd.conf(5) man page, and as you mention, cdb_set_timeout() in the confd_lib_cdb(3) man page.
The “normal” Linux kernel will close an application’s open sockets when it (e.g. ConfD) terminates. So for example, a read, e.g. poll()/select() from the closed socket will then return an EOF.
No, but you want to make sure you have a good strategy for config write transaction locking, knowing when to take it, release it, when not to take a config write transaction lock, when the config transaction lock is already taken for you (e.g. during config subscription notifications), and that there are no transaction locks for the CDB operational datastore.
For any intelligent performance discussion, you need to know the data for your particular use-case. For example, if there very frequent small read and write operations, how does the wall-clock time of those operations compare to the open/close socket + cdb_connect() + cdb_start_session() operations?
Say each read and write operation take 100x longer than it takes to open/close the socket + start-session, then the 1% overhead of socket and session operations are insignificant and an unnecessary optimization. Instead, you then want to adhear to for example the KISS principle and not introduce complex optimization that doesn’t give any significant benefit.
As always when optimizing, looking at the big picture and keeping, for example, Elon Musk’s words on this in mind: “The biggest mistake smart engineers make: optimizing a thing that shouldn’t exist” https://www.youtube.com/watch?v=cIQ36Kt7UVg&t=288s
Know (including measure) your options, e…g locks, CDB API, MAAPI API, and the performance + other implications of your CDB/MAAPI operations when running your use-case, and make intelligent design decisions based on that measured data.
Theorizing will only get you part of the way, i.e. avoid unnecessary time-consuming performance measurements of bad designs. Collecting performance data based on a couple or so promising design prototypes will enable that intelligent design decision.
Another use-case where MAAPI may be of benefit is If you have the confd --smp option enabled (see the confd(1) man page), and if spreading the load of reading larger amounts of data across multiple cores will help your use-case, then for those larger reads of data using a northbound interface such as MAAPI, NETCONF, CLI, etc. may help.
The CDB API will not use multiple cores, but again, spreading the load across multiple cores may just be a more complex solution instead of benefitting use-case.
@cohult, Conny, to clarify my original question: by long running session I meant:
cdb_connect
cdb_start_sesion2 (flags=0 /no lock/)
<< now here anything may happen: another thread’s or user’s commit
<< eventually when my thread needs to read or write it does:
maapi_lock()
<< my thread reads or writes
maapi_unlock()
<< now here anything may happen: another thread’s or user’s commit
<< this session isn’t closed as long as connection to ConfD is ok
If you are asking if maapi_lock() will ensure a safe read from the running datastore, then yes, maapi_lock() on the running datastore will take “global write lock” on running. The “global write lock” will ensure that there is no ongoing transaction writing to CDB and that no new write transaction can be started towards CDB until you unlock.