Is the ConfD built-in database modular and easily replaced?

The ConfD CDB (Configuration Database) is designed modular and easily replaceable. For most use cases, the built-in CDB is used, but other databases can be plugged-in, or even some simple storage in flat files could be built. As CDB implements transactions and multiple datastores, i.e. running, candidate, and operational, the level of integration with another database management system can vary from just the operational datastore to replacing all three CDB datastores with or without transaction support.
If you are implementing NETCONF, transaction and candidate support is likely desired.

ConfD Datastores
----------------

 +-----------+     +-----------+     +-----------+
 |  Startup  |     | Candidate |     |  Running  |
 +-----------+     +-----------+     +-----------+

Each datastore is a full copy of the configuration
Each datastore can be mapped to CDB or external database
Running can be split across CDB and several databases
5 combinations (S+Rro, S+R, R, R+C, Rro+C)

 +---------------+
 |  Operational  |
 +---------------+

Operational data store
Store status, performance or application data

The API for integrating your DMS with ConfD with full support for configuration data and transactions is described below in the “transaction state + external database API” picture:

                  +-------+
                  | START |
                  +-------+
                      | init()
                      |
                      v
       get_elem() +------+          finish()
         ------>  | READ | --------------------> START
                  +------+
                    ^  |
     trans_unlock() |  | trans_lock()
                    |  v
      get_elem() +----------+       finish()
         ------> | VALIDATE | -----------------> START
                 +----------+
                      | write_start()
                      |
                      v
       set_elem() +-------+          finish()
         -------> | WRITE | -------------------> START
create() remove() +-------+
                      | prepare()
                      |
                      v
                 +----------+   commit()   +-----------+
                 | PREPARED | -----------> | COMMITTED |
                 +----------+              +-----------+
                      | abort()                  |
                      |                          |finish()
                      v                          |
                  +---------+                    v
                  | ABORTED |                  START
                  +---------+
                      | finish()
                      |
                      v
                    START

With the addition of optional get_* and set_* calls for improved performance.

Example integrations comes with the ConfD example set and are described in the ConfD user guide.

For operational data, i.e. status information, the applications themselves register as data providers most of the time. There is usually no point in storing this kind of potentially rapidly changing data in a database. The values are simply computed when an operator wants to see them. For certain types of operational data, e.g. performance and alarm data, it may make sense to store this in a database. The CDB Database has a separate Operational Datastore available for this purpose, should you wish to use it.

There are users of ConfD that has replaced the CDB, with their own database management. Note that most of them have done so to preserve their legacy applications, not for performance reasons. So far, on the few occasions when CDB did not live up to the performance expectations, it has been for the use case where the operational data store in CDB has been used to collect massive amounts of statistics and monitor data. Here, for example PostgreSQL has been used to store operational data, while CDB has been used to store configuration data in running and candidate datastores.

For transactional configuration data, CDB has so far been proven performant enough… When you enable NETCONF using ConfD and e.g. for legacy reasons want to store configuration data in your own database management system, if you implement candidate commit with NETCONF, you can still use CDB for the candidate datastore, while your database management implements the running datastore part.

The key features of CDB are:

Transactions: the data store and the ConfD transaction engine are
tightly connected to enable NETCONF and network wide transactions. This
removes the burden from application developers to understand and implement transactional integrity.
Persistent embedded in-memory database with optimized performance for hierarchical data. The in-memory model makes all validation, CLI tab-completion etc. execute quickly. The CDB journals are stored to disk.
Automatic schema management: the embedded data store removes all traditional work around databases. CDB uses YANG as native schema language, so the schema management is transparent. Nor is there any need for run-time configuration or administration of the database.
Schema upgrade/downgrade: the schema (YANG model) can be changed in run- time. CDB will automatically manage migrating the existing stored data. In case of conflicting schema and instance data, upgrade hooks can be registered.
Subscription-based programming model: programmers can subscribe to changes to CDB and react upon those. This in contrast to stub-based approaches. It also enables easy addition of logic.
High-availability: CDB can run in high-availability mode, where hot-standby nodes are replicated from the master node.
Support for candidate and running data stores
Fast and light-weight: CDB is optimized for embedded applications, the foot-print is small and response times are quick.

Hi,

Are there any other options to connect to external database apart from using callpoints?
If we use external database for storing authorization rules(ietf-netconf-acm.yang and tailf extension), still can we use confd built-in authorization module.

Thanks & Regards,
Azeem

Hi,

See confd_lib_dp(3) man page, section AUTHORIZATION CALLBACKS, (also available in the UG as an appendix).
The callbacks can partially or completely replace the access checks done within the AAA subsystem. See “CONFD_ACCESS_RESULT_DEFAULT” for more.

Note that typically many access checks are done during the processing of commands etc, and using these callbacks can thus have a significant performance impact.

Hi,

Thanks for the quick reply, I will look into the section that you have suggested.
The first part of the question, whether we can connect to external database without using callpoints?

Thanks & Regards,
Azeem

Hi,

The authorization callback does not require you to add callpoints to any YANG model. You enable the authorization callback in confd.conf as described in the man page.

Regards

Hi,

Okay, for other config data to store in the external database we need callpoints?

Thanks & Regards
Azeem

Either callpoints, or even better, use CDB mirroring. There is an app note on the topic:
http://info.tail-f.com/confd-integration-with-legacy-systems-with-CDB-API

Okay, thanks.
I will have a look and get back to you in case of further queries.

Thanks & Regards,
Azeem

Just for completeness, I’d like to point out that it’s perfectly possible to provide the data also for the ietf-netconf-acm, tailf-aaa, and/or tailf-acm models via callpoints and external data providers (or perhaps the “CDB mirroring”, if it turns out to be a better fit), just as for any other data models. The authorization callbacks are for a different scenario, basically where you want to use a different authorization mechanism than the one defined by these data models. It is also possible to have the AAA data be “config false”, whether it is stored in CDB or externally - for the external data provider case, this is a significant simplification.

Hi,

If I understand correctly, CDB mirroring is used to store configuration data from CDB to external database but not the other way around, I mean configuration data from external database to CDB.

Can you please comment.

Thanks & Regards,
Azeem

Hi,

You typically synchronize both ways using the ConfD CDB API and MAAPI.
A ConfD CDB API subscriber using for example cdb_get_modifications() replicate the updated ConfD CDB data to the external DB.
If the external DB data is updated “out-of-band” its configuration data is replicated to ConfDs’ CDB using MAAPI.
Any operational (e.g. status) data is replicated to CDB using the CDB API or MAAPI

Thanks for the reply cohult!

In our management element if we need to synchronize data both ways, which solution ConfD recommends, callpoint or CDB mirroring together with MAAPI - which one will be better in terms of performance in a typical configuration data?

Thanks & Regards,
Azeem