I have large lists / tables of operational data. How can I optimize replying to requests for that data over the ConfD DP API?

CDB and CDB API
1

For a use case where you want to traverse large lists / tables and reply fast you want to implement ConfD’s get_next_object() and find_next_object() DP API callbacks.

Here is a short example on how to optimize a your replies for data when you want to traverse large lists / tables fast.

We start out with a very simple YANG data model that implements a list / table called “port” which has two keys and a three more leafs in it:

list port {
  key "slot port”;
  config false;
  tailf:callpoint ps {
    tailf:cache true;
  }
  leaf slot {
    type int8;
  }
  leaf port {
    type int8;
  }
  leaf status {
    type enumeration {
      enum "disabled";
      enum "active";
      enum "error";
    }
    mandatory true;
  }
  leaf extra {
    type int32;
  }
  leaf flag {
    type empty;
  }
}

This is the desired end result, here shown from the ConfD CLI:

admin@ncsvm> show port
SLOT  PORT  STATUS    EXTRA  FLAG  
-----------------------------------
1     0     disabled  1      -     
1     1     active    2      X     
1     2     error     3      -     
2     1     disabled  4      X     
2     2     active    5      -     
4     0     error     6      X     
4     2     disabled  7      -

In this example we register our data callbacks for our callpoint from the YANG data module in the main() function of our program:

int main(int argc, char *argv[])
{
...
  struct confd_data_cbs data;
...
  data.get_object = get_object;
  data.get_next_object = get_next_object; /* We implement these callbacks to be able to traverse a list/table very fast */
  data.find_next_object = find_next_object; 
...
  strcpy(data.callpoint, ports__callpointid_ps);
  confd_register_data_cb(dctx, &data);
...
}

Our get_next_object() data callback then return the whole list / table in one swoop:

static int get_next_object(struct confd_trans_ctx *tctx,
                           confd_hkeypath_t *keypath, long next)
{

  int pos, ival, i;
  confd_value_t *v;
  struct confd_next_object *obj;
...   
  if (next == -1) {
    pos = 0; /* ConfD wants to get the whole table / list from the beginning */ 
  } else if (next < n_list_entries) {
    pos = next; /* ConfD wants the table / list from a specific index */
  } else { /* next == n_list_entries, return nothing */
    confd_data_reply_next_object_array(tctx, NULL, -1, -1);
    return CONFD_OK;
  }
...
  obj = malloc(sizeof(struct confd_next_object) * (n_list_entries + 2));
  v = (confd_value_t *) malloc(sizeof(confd_value_t) * n_list_entries * N_LEAFS);
...
  for (i = 0; pos + i < n_list_entries; i++) { /* Collect all the rows in the table / list entries */
    obj[i].v = &v[i * N_LEAFS];   
...
    CONFD_SET_INT8(&(obj[i].v[0]), ival); /* List key leaf slot int8 */
...
    CONFD_SET_INT8(&(obj[i].v[1]), ival); /* List key leaf port int8 */
...
    CONFD_SET_ENUM_HASH(&(obj[i].v[2]), ival); /* Leaf status enumeration */
...
    CONFD_SET_INT32(&(obj[i].v[3]), ival); /* Leaf extra int32 */
...
    if (ival) /* Leaf flag */
      CONFD_SET_XMLTAG(&(obj[i].v[4]), ports_flag, ports__ns);
    else
      CONFD_SET_NOEXISTS(&(obj[i].v[4]));
      
    obj[i].n = N_LEAFS;
    obj[i].next = (long)pos + i + 1; /* Tell ConfD where next object is, if we did not get all of the rows in the table / list entries */
  }
  if (pos + i == n_list_entries)
    obj[i++].v = NULL;
  confd_data_reply_next_object_arrays(tctx, obj, i, 0); /* Reply with an array of object arrays. I.e. the whole table / list */
  free(v);
  free(obj);
  return CONFD_OK;
}

With this implementation of get_next_object() ConfD will return the whole table / list in one single swoop. I.e. there will be only one call to a data callback, in this example the get_next_object() callback. This is useful if you have large tables / lists of data that need to be fetched and want to tune your performance for that use case. If users normally just wants a single entry or just a list entry / table row, using get_elem() or get_object() and find_next() will provide better performance.

You can learn more about ConfD data provider callpoints and associated data callbacks in Volume 3 of the ConfD man-pages under the confd_lib_dp section confd_register_data_cb() function.

1 Like