i want to CONFD return data of confd_tag_value_t type, which API should i should in client? as i know , maapi_get_objects just get confd_value_t
We are getting the size of one list entry / object / row here, not the number of list entries / instances / objects / rows
Try maapi_get_values() / cdb_get_values().
@cohult, Do you know how confd only trigger get_next_object for Netconf request and confd still trigger get_next/get_elem for CLI. We use the external data provider and need to support 8k onts. Since get_next_object back-end code will stuck some while, we don’t want cli stuck. So we want to know how to trigger get_next_object only for Netconf request.
Much appreciated.
I think it is recommended not to return huge amount of objects (8k) in get_next_object. You should limit it to cca 100 elements (make it perhaps configurable and try to find best value). Then CLI will be responsive as well. In several cases (e.g. TAB completion) ConfD can choose itself to call get_next instead of get_next_object
Thanks @mnovak. Yes, we don’t return all the instances in one swoop and are trying out to find the best cca value. But actually, there are no customers complaining cli performance issue, so we don’t want to impact on CLI. We only want to support get_next_object for netconf request. I see there are “bulk hint” mentioned in confd_user_guide_6.7.4.pdf, but there are no further detail description. So I want to confirm whether confd can only trigger get_next_object for netconf somehow.
Much appreciated.
HI @mnovak, I also have another block issue. Take a simple example: list ont
and list ont has nested-list vlan.
Since nested-list vlan can’t be returned to confd in the get_next_object for list ont, confd will trigger get_next_object for nested-list vlan. So there are large numbers of get_next_object for nested-list vlan, which is not efficient and consumes lots of time, right? So how to improve the loop of nested-list? By the way, “container children nodes” can be returned in get_next_object for list onts, and I use confd_data_reply_next_object_tag_value_arrays() and external data provider.
Much appreciated.
Hello,
unfortunately I do not know know how to force get_next_object just on NETCONF. Usually ConfD should choose best method. I suggest you implement get_next_object returning cca 100 records and test the implementation together with get_next (and maybe also find_next and find_next_object if needed). CLI performance should not be affected.
I thing for get_next_object same restriction applies as for get_object (see user guide - description of get_object):
However, although the tagged value array format can represent nested lists, these must not be passed via this function, since the get_object() callback only pertains to a single entry of one list. Nodes representing sub-lists must thus be omitted from the array, and ConfD will issue separate get_object() invocations to retrieve the data for those.
HI @mnovak, thanks for your confirm. Now, I hit another block issue. There are “when statement” and “must statement” under some nodes in list, such as “/config/product/machine!= 'E3-16F”. Since there are substantial amounts of list instances, confd trigger lots of get-elem() request to validate “when/must statement”. I want to know whether confd can cache the “when/must statement” result and only trigger the first time for the same xpath.
Please refer to the below fragment:
DEBUG> 25-Aug-2019::20:15:12.961 Jessica-is-using confd[6524]: devel-c get_elem request for callpoint config_cp path /exa:config/product/machine
25-Aug-2019::20:15:12.963 Jessica-is-using confd[6524]: devel-c get_elem succeeded for callpoint config_cp path /exa:config/product/machine
25-Aug-2019::20:15:12.967 Jessica-is-using confd[6524]: devel-c get_elem request for callpoint config_cp path /exa:config/product/machine
25-Aug-2019::20:15:12.969 Jessica-is-using confd[6524]: devel-c get_elem succeeded for callpoint config_cp path /exa:config/product/machine
25-Aug-2019::20:15:12.971 Jessica-is-using confd[6524]: devel-c get_elem request for callpoint config_cp path /exa:config/product/machine
25-Aug-2019::20:15:12.974 Jessica-is-using confd[6524]: devel-c get_elem succeeded for callpoint config_cp path /exa:config/product/machine
25-Aug-2019::20:15:12.976 Jessica-is-using confd[6524]: devel-c get_elem request for callpoint config_cp path /exa:config/product/machine
25-Aug-2019::20:15:12.979 Jessica-is-using confd[6524]: devel-c get_elem succeeded for callpoint config_cp path /exa:config/product/machine
25-Aug-2019::20:15:12.983 Jessica-is-using confd[6524]: devel-c get_elem request for callpoint config_cp path /exa:config/product/machine
25-Aug-2019::20:15:12.986 Jessica-is-using confd[6524]: devel-c get_elem succeeded for callpoint config_cp path /exa:config/product/machine
25-Aug-2019::20:15:12.988 Jessica-is-using confd[6524]: devel-c get_elem request for callpoint config_cp path /exa:config/product/machine
…
You may try to use ConfD cache for operational data. See ConfD user guide for confd.conf, search for /confdConfig/opcache.
Thanks @mnovak. I tried out this /confdConfig/opcache in confd.conf by reference to ConfD user guide . But it doesn’t work for my case. My case is running configuration data and get-elem() requests are triggered by “when” validation.
Regarding the above discussion, see also for example “tailf:cli-no-key-completion” as an option you may consider to prevent the CLI from querying the data provider for all existing keys.
Regarding “when” and “must” validation, that depends on how (in)efficient your XPath expression is.
Use for example the xpath.trace, see the confd.conf(5) man page under /confdConfig/logs/xpathTraceLog, to debug your xpath expressions. In addition:
- The primary manual for ConfD is the YANG RFC pointing to W3C
standard. The ConfD UG complements the YANG, NETCONF, RESTCONF, etc RFCs. See for example:
https://tools.ietf.org/html/rfc7950#ref-XPATH
https://tools.ietf.org/html/rfc6241#section-8.9.1 - See also ConfD UG under “NETCONF Server” “Capabilities” “:xpath”
- Including the link to the xpath 1.0 standard in those documents (see for example
“4.1 Node Set Functions” for a few useful functions)
https://www.w3.org/TR/1999/REC-xpath-19991116/ - In addition to the xpath 1.0 standard, there are ConfD YANG extensions regarding xpath that are documented in the tailf_yang_extensions man page under section “XPATH FUNCTIONS”
hi @cohult, @mnovak, When I construct a reply array and hit a leaf-list issue. Other type nodes do all work well.
When I tried out to populate the value for a leaf-list node by means of CONFD_SET_TAG_LIST in the response array, and encountered several issues:
#1: If I called CONFD_SET_CBUF macro for all types of leaf-list nodes, the CONFD complained the below error:
- 28-May-2020::14:47:33.502 GPON-8r2D confd[8625]: devel-c get_next_object succeeded for callpoint config_cp path /exa:config/profile/transport-service-profile*
- 28-May-2020::14:47:33.513 GPON-8r2D confd[8625]: devel-c /config/profile/transport-service-profile/vlan-list: expected type uint16, got list.*
- 28-May-2020::14:47:33.516 GPON-8r2D confd[8625]: devel-c /config/profile/transport-service-profile/vlan-list: expected type uint16, got list.*
…
==
code snippet is as below:
CONFD_SET_CBUF(&leaf_list_test[leaf_list_count_test], leaf_list_value[leaf_list_count_test], strlen(leaf_list_value[leaf_list_count_test]));
…
CONFD_SET_TAG_LIST(&confd_elem[return_count], confd_str2hash(sbt_db_get_name(child_hdl)), &leaf_list_test[leaf_list_count_test], 1);
#2: then I simply did hard-code CONFD_SET_UINT16 to return leaf-list in the response array, but CONFD complained another error:
28-May-2020::04:45:03.852 E7-2 confd[8985]: devel-c get_next_object request for callpoint config_cp path /exa:config/profile/transport-service-profile
28-May-2020::04:45:03.853 E7-2 confd[8985]: devel-c get_next_object succeeded for callpoint config_cp path /exa:config/profile/transport-service-profile
28-May-2020::04:45:03.854 E7-2 confd[8985]: devel-c Expected string, got {11,0} for path /exa:config/profile/transport-service-profile{SYSTEM_TSP}/vlan-list
28-May-2020::04:45:03.855 E7-2 confd[8985]: devel-c bad get_next_object() return value: Expected string, got {11,0}
…
code snippet is as below:
CONFD_SET_UINT16 (&leaf_list_test[leaf_list_count_test], leaf_list_count_test);
…
CONFD_SET_TAG_LIST(&confd_elem[return_count], confd_str2hash(sbt_db_get_name(child_hdl)), &leaf_list_test[leaf_list_count_test], 1);
My confd version is CONF-6.7.
The yang is also quite simple as following:
augment /exa:config/exa:profile {
list transport-service-profile{
description “Ethernet services VLAN list”;
key “tsp-name”;
leaf tsp-name{
description “transport-service-profile name”;
type string;
}
leaf-list vlan-list {
description “Range list of vlans (a-b,c,d)”;
type uint16;
tailf:cli-range-list-syntax;
}
leaf description {
type string;
}
}
}
Please give me a direction or share me an example for leaf-list node.
Much appreciated.
So, here leaf_list_test is presumably an array that represents the leaf-list a.k.a. C_LIST, and you set a C_UINT16 value in the leaf_list_count_test’th element of the array (kind of strange that the value is leaf_list_count_test, but I guess this is just a test).
And now you try to set the leaf-list value in the tag-value array - but you aren’t passing leaf_list_test but instead &leaf_list_test[leaf_list_count_test], i.e. the C_UINT16 value you set earlier (while claming that it is a C_LIST). Pass leaf_list_test instead of &leaf_list_test[leaf_list_count_test] and it should work.
PS Note that this use of C_LIST for a leaf-list is a “special” in current (i.e. post-6.4) versions of ConfD. Normally a leaf-list should be handled as a list, but in the interest of getting as much data as possible into a single “bulk reply”, it is allowed. From the confd_lib_dp(3) man page:
Values for leaf-lists may be passed as a single array element with type
C_LIST (as described in the specification) regardless of whether the
the deprecated daemon flag CONFD_DAEMON_FLAG_LEAF_LIST_AS_LEAF is used
or not.
HI @per, There was no context so it didn’t make my code clear to you. I changed the test code to simplify it and hit the same error:
29-May-2020::19:13:05.196 Jessica-is-using confd[8610]: devel-c Expected string, got {11,111} for path /exa:config/profile/transport-service-profile{SYSTEM_TSP}/vlan-list
29-May-2020::19:13:05.197 Jessica-is-using confd[8610]: devel-c bad get_next_object() return value: Expected string, got {11,111}
==code
static confd_value_t leaf_list_test[2];
CONFD_SET_UINT16(&leaf_list_test[0], 111);
CONFD_SET_UINT16(&leaf_list_test[1], 222);
CONFD_SET_TAG_LIST(&confd_elem[return_count], confd_str2hash(sbt_db_get_name(child_hdl)), &leaf_list_test[0], 2); //&leaf_list_test[0] is same with &leaf_list_test
return_count++;
I use the reply function confd_data_reply_next_object_tag_value_arrays().
And our EDP registers the flag CONFD_DAEMON_FLAG_LEAF_LIST_AS_LEAF and use the confd 6.7.
Thanks.
That code seems OK (and actually the previous snippet was OK too, I just got confused by the leaf_list_count_test index). But, it seems you are using the CONFD_DAEMON_FLAG_STRINGSONLY flag, i.e. you should return all values as strings. Which seems to be what you tried in your #1 attempt… There it seems you are setting the individual leaf-list elements as strings - but with CONFD_DAEMON_FLAG_STRINGSONLY (and a leaf-list “as leaf”), ConfD expects that you pass the complete leaf-list as a single string (with space-separated elements) - i.e. the C_LIST representation isn’t used.
Hi @per, yes, we use CONFD_DAEMON_FLAG_STRINGSONLY flag. I changed the code per your suggestion, but it still hits the same errr as #1.
31-May-2020::20:06:14.330 Jessica-is-using confd[8610]: devel-c get_next_object request for callpoint config_cp path /exa:config/profile/transport-service-profile
31-May-2020::20:06:14.332 Jessica-is-using confd[8610]: devel-c get_next_object succeeded for callpoint config_cp path /exa:config/profile/transport-service-profile
31-May-2020::20:06:14.335 Jessica-is-using confd[8610]: devel-c /config/profile/transport-service-profile/vlan-list: expected type uint16, got list.
31-May-2020::20:06:14.339 Jessica-is-using confd[8610]: devel-c /config/profile/transport-service-profile/vlan-list: expected type uint16, got list.
==code
CONFD_SET_CBUF(&leaf_list_test[0], “111 222”, 7);
CONFD_SET_TAG_LIST(&confd_elem[return_count], confd_str2hash(sbt_db_get_name(child_hdl)), &leaf_list_test[0], 1);
And then I deleted this flag, and tested the #2 again, and #2 does work now. But for some history reason, we must use this flag.
So how to address it now ?
Thanks.
No C_LIST! The above is passing the leaf-list as a C_LIST with one string element - you need the leaf-list to be a single string. If you change the above to
CONFD_SET_TAG_CBUF(&confd_elem[return_count], confd_str2hash(sbt_db_get_name(child_hdl)), “111 222”, 7);
it should work.
HI @per, this issue is for get_object() and get_next_object() case, and is not for get_elem().
If CONFD_SET_TAG_LIST is not used and only used CONFD_SET_TAG_CBUF, confd will complain the below error:
31-May-2020::22:59:22.903 Jessica-is-using confd[8610]: devel-c get_next_object request for callpoint config_cp path /exa:config/profile/transport-service-profile
31-May-2020::22:59:22.905 Jessica-is-using confd[8610]: devel-c get_next_object succeeded for callpoint config_cp path /exa:config/profile/transport-service-profile
31-May-2020::22:59:22.906 Jessica-is-using confd[8610]: devel-c bad get_next_object() return value: /exa:config/profile/transport-service-profile{SYSTEM_TSP}/vlan-list: Nested lists or leaf-lists not allowed
The reply function for get_object() is confd_data_reply_tag_value_array().
The reply function for get_next_object() is confd_data_reply_next_object_tag_value_arrays().
The reply to get_elem() does work as what you said.
Thanks.