Slow maapi cursor operations

Hi,

I have a list of maximum 5000 elements - each element consists of a key leaf of type string, several other leaves and one sub-list of maximum 7 elements; each element of the sub-lists consists of 3 keys and some other leaves of type string.

The number of elements of the first list and of each sub-list is variable
I use one maapi cursor on the first list, in order to read the key leaf of type string and a second leaf
I use a second maapi cursor (instantiated and initialized - maapi_vinit_cursor each time) in order to iterate on each of the sub-list’s elements to read the keys + one element of type string (which is not mandatory - see maapi_vexist below)

The running time of reading the whole structure with the maximum number of elements on a PC is about 10 seconds
The running time of reading the same structure on the target embedded system is about 10 times more - 100 seconds
Profiling the code shows that about 70% of the running time is spent in the following 3 maapi functions - maapi_vexist=33% ; maapi_get_next=28%; maapi_vget_elem=9%

Anyway, it seems to me that iterating through these lists seems quite slow - in the end I read a maximum of 5000 * 2 + 5000 * 7 * 4 values (keys included).
Could someone please let me know if the reading algorithm + running time + the code profiling seem OK and if there is some room for improvement for this specific scenario.

Thanks a lot !

Contributions to the ConfD User Community are always welcome. So if you can post a small project that can build and run out of the box on some standard Linux dist, someone may run it to try recreate your findings and take a look at the feasibility of your reading algorithm + running time + the code profiling.

Thanks for the reply - I’ll try to come back with an standalone example.

Meanwhile, is there any bench-marking information on some general (reading) scenarios on tail-f side? At least to see that I am in the right order of magnitude?

Thank you,
Kind regards !

I don’t see how to one can avoid wasting time on a pointless “apples and oranges” comparison unless you provide a project that can be used for recreating your tests.

Sorry, I presumed that the traversal of a list using a cursor was a scenario that might have been bench-marked already - thank you for the reply, I’ll let you know once I have the working example.

Before you spend time on providing a project, have you considered using maapi_get_values()?

Agreed - it is at least an “educated guess” that the processing time in the described case is dominated by round-trips application <-> ConfD. In addition to maapi_get_values(), maapi_get_object() and maapi_get_objects() are likely to be useful. E.g. an entire “sub-list” could be read in a single call to maapi_get_objects() - it might read more than what is wanted if only specific leafs are of interest (maapi_get_values() can avoid that if you know the keys), but it may still be a win.

Thanks for the replies.

I use maapi_get_values in order to read some values(leaves) from the first list of 5000 elements. Using maapi_get_values in order to read 20 values for each of the 5000 elements is one order of magnitude faster that reading them one by one. I could use maapi_get_values because all these elements were mandatory leaves (so not lists) and were not keys.

In the present scenario I want to read an optional list (not leaf-list) element from the first list and mostly key values from this second list, so I am not sure maapi_get_values would speed up the reading, that’s why I am using cursors on these lists.