Accessing nested list

[biswajit]

Hi,
I have a nested list(list within list).Inside low-level-tx-endpoints list there is number-of-prb-per-scs inner list having scs as key of inner list.

  < low-level-tx-endpoints >
    < name >LLTE-DCH-0 < /name >
     < cp-length-other > 72 < /cp-length-other >
     < number-of-prb-per-scs >
      < scs>KHZ_15 < /scs >
      < number-of-prb > 273 < /number-of-prb >
     < /number-of-prb-per-scs >
  </low-level-tx-endpoints>

I have subscribed to outer list through cdb_subscribe, so that any modification in any leaf my call back is getting invoked.
I am able to read all the leaf nodes of outer list.But i am not able to read any leaf node of inner list.

static void read_db_key_based_TxEndpoint(int cdbsock, confd_value_t *key)
{

    int index = 0;
    int pos = -1;
    char tmp_key[30];
    char *ptr;
    confd_value_t key1;
    /* which position should we overwrite */
    for (index=0; index< MAX_ENDPOINT; index++) {
        ptr=(char *)CONFD_GET_BUFPTR(key);
        strncpy(tmp_key,ptr,CONFD_GET_BUFSIZE(key));
        if (strcmp(tmp_key,Tx_EndPoints[index].KeyName)==0) {
            pos = index;
            break;
        }
    }
    if (pos == -1) { /* pick first */
        for (index=0; index < MAX_ENDPOINT; index++) {
            if (!Tx_EndPoints[index].inuse) {
                pos = index;
                break;
            }
        }
    }
    alwaysLog("Picking %d\n", pos);
    TxEndPoints *hp = &Tx_EndPoints[pos];
    if (cdb_cd(cdbsock, "/user-plane-configuration/low-level-tx-endpoints{%x}", key) != CONFD_OK)
    {
        alwaysLog("Failed to cd");
    }
    ptr=(char *)CONFD_GET_BUFPTR(key);
    strcpy(hp->KeyName ,ptr);
    hp->inuse = 1;

      
    if (cdb_get_int32(cdbsock,&(hp->OffsetabsCenterFreq),"offset-to-absolute-frequency-center") != CONFD_OK)
        alwaysLog("offset-to-absolute-frequency-center Read data failed");
  
   int maxInstance = cdb_num_instances(cdbsock,"/user-plane-configuration/low-level-tx-endpoints{%x}/number-of-prb-per-scs",key);
    alwaysLog("found prb instances:%d",maxInstance);
for(index=0; index< maxInstance; index++) {
        if (cdb_get(cdbsock, &key1, "/user-plane-configuration/low-level-tx-endpoints{%x}/number-of-prb-per-scs[%d]/scs", key,index) !=
                CONFD_OK) alwaysLog("Can't get key");
        else    alwaysLog("PRB FOUND Key****");
   if (cdb_cd(cdbsock, "/user-plane-configuration/low-level-tx-endpoints{%x}/number-of-prb-per-scs{%x}", key,key1) != CONFD_OK)
           {
                       alwaysLog("Failed to cd scs");
           }
 if (cdb_get_enum_value(cdbsock,(int32_t *)&(hp->TxprbConfig[index].scs),"scs") != CONFD_OK)
             alwaysLog("SCS Read data failed");
             }

}

OUTPUT:
On multiple change of only scs the number of instances are keep increasing even if i am modifying against same key value of outer list

Not sure what went wrong.
It will be really helpful if you can figure out my mistake.

Regards,
Biswajit

[josephm]

it’s quite hard to identify issue in longer code - there are unknown global variables etc.

try to narrow down your question to which one procedure call specifically is problem - what result/return value you see, and what did you expect instead?

double check with printout of actual input parameters - to verify that you actually run the “correct” cdb_ request …

[biswajit]

Hi,
Sorry for long code.I would like to explain with the small example:
list tx_endpoint {
key name [20];
leaf leaf_endpoint;
list array-carrier {
key name_inner_list[20];
leaf 1;
leaf2;
}
}
I am able to access leaf of outer list i.e leaf_endpoint.But when i am trying to read number of instances of inner list, even though i am just change only leaf1 value of inner list with same key,the number of instances keep increasing on each change.

Steps followed:
1.Find number of instances of outer list using cdb_num_instances
2.find the key of outer list by cdb_get(cdbsock, &key, “/tx_endpoint[%d]/name”, index)
3.Read the all the leaf of corresponding key of outer list.
i.move to corresponding index of outer list using cdb_cd(cdbsock, “/tx_endpoint{%x}”, key)
ii.Read all the leaf of outer list
iii.Find number of instances of inner list using
cdb_num_instances(cdbsock,"/tx_endpoints{%x}/array-carrier",key)
iv. iterate all the instances
1.Get the key of inner list by cdb_get(cdbsock, &key1, “/tx_endpoints{%x}/array-carrier[%d]/scs”, key,index)–>key is outerlist key & index is for inner list iteration .
2. move to index of inner list by cdb_cd(cdbsock, “/tx_endpoints{%x}/array-carrier{%x}”, key,key1) -->key is outerlist key & key1 is for inner list
3.Read the fields of inner list

Let me know if i am making any mistake.

Regards,
Biswajit

[josephm]

Hmm, the sequence of steps you describe should be fine, though you may have problems with e.g. read lock of the session, depending on where your cdbsock comes from (how it was created, with what options), and how you edit the config (while the subscriber executes?) …

E.g. there’s a mention in confd_lib_cdb manual page regarding cdb_num_instances when working with CDB_RUNNING datastore (and consistency of cdb reading in general):

   CDB_RUNNING
       CDB_LOCK_SESSION obtains a read lock for the complete session, i.e. using this flag alone is equivalent to calling cdb_start_session(). CDB_LOCK_REQUEST
       obtains a read lock only for the duration of each read request. This means that values of elements read in different requests may be inconsistent with each
       other, and the consequences of this must be carefully considered. In particular, the use of cdb_num_instances() and the [n] "integer index" notation in
       keypaths is inherently unsafe in this mode. Note: The implementation will not actually obtain a lock for a single-value request, since that is an atomic
       operation anyway. The CDB_LOCK_PARTIAL flag is not allowed.

[biswajit]

Hi,
The calling function creates the dataSock as below:

if ((dataSock = socket(PF_INET, SOCK_STREAM, 0)) < 0)
alwaysLog(“Failed to create socket”);

if (confd_load_schemas((struct sockaddr*)&addr,
            sizeof (struct sockaddr_in)) != CONFD_OK)
    alwaysLog("Failed to load schemas from confd\n");

if (cdb_connect(dataSock, CDB_DATA_SOCKET, (struct sockaddr *)&addr,
            sizeof(struct sockaddr_in)) != CONFD_OK)
    alwaysLog("Failed to connect to ConfD");

The datasock is passed to function as argument and received as cdbsock.

Let me know if anyother option need to provide to solve the issue.

Regards,
Biswajit

[josephm]

You should also be calling cdb_start_session() or cdb_start_session2() depending on what you want to do…
Try to compare your codebase to existing examples, e.g. examples.confd/cdb_subscription/iter_c - and see if you miss some steps or do confd-lib related things differently…

[biswajit]

Hi,
I am calling at the initial stage like : if ((ret = cdb_start_session(cdbsock, CDB_RUNNING)) != CONFD_OK)
alwaysLog(“Cannot start session\n”);

Then i am calling int maxInstance = cdb_num_instances(cdbsock, daemonPath); to get the number of instances of outer list.Then i am doing outer key based iteration.
Basically i have taken the reference of the one you have suggested.
If you can provide any example of nested list then it will be really helpful.

Regards,
Biswajit

[josephm]

i tried editing the basic example examples.confd/cdb_subscription/iter_c, and it seems to work as expected (tested with confd 7.2.0.1), at laest with the basic scenario i tried;

i tried to do it in similar way like you described, at least how i understood it:

• i iterate all parent lists, and use [] index to get real key value

for each parent key, I do:

• use retrieved key value, to iterate it’s nested lists via [] index again
• for each nested list, i invoke cdb_cd(), and some cdb_get() to get its nested data

I also DID NOT update original subscriber code, so the new nested list data is not mirrored into C application and it’s “external db” memory structs, and it does not appear in all old/original printouts etc…

The changes i made in ConfD’s example YANG model:

+++ b/examples.confd/cdb_subscription/iter_c/root.yang
@@ -35,6 +35,16 @@ module root {
         leaf dn {
           type int64;
         }
+        list nested {
+          key nestkey;
+          leaf nestkey {
+            type int64;
+          }
+          leaf payload {
+            type string;
+            default "nested-hello";
+          }
+        }
         leaf sector-id {
           type string;

and in subscriber application:

+++ b/examples.confd/cdb_subscription/iter_c/cdbl.c
@@ -294,6 +294,40 @@ static void dump_db()
     fprintf(stderr, "---------- \n");
 }

+void dump_nested_of_head(int cdbsock, confd_value_t *rfHeadKey) {
+    // print requested top list key
+    char buffer[BUFSIZ];
+    confd_pp_value(buffer, BUFSIZ, rfHeadKey);
+    fprintf(stderr, "\t==== dumping nested lists of parent key \"%s\"\n", buffer);
+    // get number of nested list entries
+    int countNested = cdb_num_instances(cdbsock, "/root/node-b/rf-head{%x}/nested", rfHeadKey);
+    fprintf(stderr, "\t\t==== nested count: %d\n", countNested);
+    // iterate all nested entries
+    int i;
+    for (i = 0 ; i < countNested; i++) {
+        confd_value_t val;
+        CONFD_SET_NOEXISTS(&val);
+        // get nested list entry data
+        cdb_cd(cdbsock, "/root/node-b/rf-head{%x}/nested[%d]", rfHeadKey, i);
+        cdb_get(cdbsock, &val, "payload");
+        // print it
+        confd_pp_value(buffer, BUFSIZ, &val);
+        fprintf(stderr, "\t\t==== nested index %d: \"%s\"\n", i, buffer);
+    }
+}
+
+void dump_all_cdb_parents(int cdbsock) {
+    int parentCount = cdb_num_instances(cdbsock, "/root/node-b/rf-head");
+    fprintf(stderr, "==== parent (rf-head) count: %d\n", parentCount);
+    int i;
+    for (i = 0 ; i < parentCount; i++) {
+        confd_value_t val;
+        // get parent list key value
+        cdb_get(cdbsock, &val, "/root/node-b/rf-head[%d]/dn", i);
+        // and pass it to print its nested data
+        dump_nested_of_head(cdbsock, &val);
+    }
+}

 int main(int argc, char **argv)
 {
@@ -386,6 +420,8 @@ int main(int argc, char **argv)
                 if ((status = cdb_set_namespace(sock, root__ns)) != CONFD_OK)
                     confd_fatal("Cannot set namespace\n");

+                dump_all_cdb_parents(sock);
+
                 cdb_diff_iterate(subsock, sub_points[0], iter,
                                  ITER_WANT_PREV, (void*)&sock);
                 cdb_end_session(sock);
@@ -425,4 +461,3 @@ int main(int argc, char **argv)

then i tried playing with CLI and checked the printouts:
(there are more lines printed by original example app, but i show only the new ones added by us here)

admin connected from 10.0.2.2 using ssh on dev
admin@dev> configure
admin@dev% set root node-b rf-head 111 nested 1 payload pp1
[ok][2020-07-02 15:41:36]
[edit]
admin@dev% commit
Commit complete.
[ok][2020-07-02 15:41:43]
[edit]

here, the console with running subscriber prints:

*** Config updated
==== parent (rf-head) count: 1
        ==== dumping nested lists of parent key "111"
                ==== nested count: 1
                ==== nested index 0: "pp1"

then i continue with more commands:

admin@dev% set root node-b rf-head 111 nested 2 payload pp2
[ok][2020-07-02 15:42:12]
[edit]
admin@dev% set root node-b rf-head 222 nested 333
[ok][2020-07-02 15:42:25]
[edit]
admin@dev% commit
Commit complete.
[ok][2020-07-02 15:42:27]

and i get output:

*** Config updated
==== parent (rf-head) count: 2
        ==== dumping nested lists of parent key "111"
                ==== nested count: 2
                ==== nested index 0: "pp1"
                ==== nested index 1: "pp2"
        ==== dumping nested lists of parent key "222"
                ==== nested count: 1
                ==== nested index 0: "nested-hello"

We can discuss e.g. small updates/differences of example code or details easily here,
but its really hard to debug applications of different author as a whole…

So you have the hard task to identify potential differences or small indexing/memory bugs (if any exist) - to go step by step and verify your parameters passed to cdb calls, plus expected results vs real results…

Or, if there are any differences or sequence of configuration commands, try to map them to be reproducible with this (extended) example we have here…

Or maybe it’s a different running scenarios (e.g. configuration edits running in parallel and affecting your subscriber printouts? as we don’t handle read locks in example code or my extension)

---

edit: to sum up potential issues that happened to me frequently when i started using confd:

• actual confd key values vs [] indices in keypaths (%d vs %s vs %x, etc.)
• socket used - cdb socket vs subscription socket
• make sure cdb_start_session() is invoked before we try to work with cdbsocket
• no parallel session conflicts - resp. using read locks when starting cdb read sessions, when i don’t want my example to be limited to single user scenario and keep consistency of data…