I have a list of maximum 5000 elements - each element consists of a key leaf of type string, several other leaves and one sub-list of maximum 7 elements; each element of the sub-lists consists of 3 keys and some other leaves of type string.
The number of elements of the first list and of each sub-list is variable
I use one maapi cursor on the first list, in order to read the key leaf of type string and a second leaf
I use a second maapi cursor (instantiated and initialized - maapi_vinit_cursor each time) in order to iterate on each of the sub-list’s elements to read the keys + one element of type string (which is not mandatory - see maapi_vexist below)
The running time of reading the whole structure with the maximum number of elements on a PC is about 10 seconds
The running time of reading the same structure on the target embedded system is about 10 times more - 100 seconds
Profiling the code shows that about 70% of the running time is spent in the following 3 maapi functions - maapi_vexist=33% ; maapi_get_next=28%; maapi_vget_elem=9%
Anyway, it seems to me that iterating through these lists seems quite slow - in the end I read a maximum of 5000 * 2 + 5000 * 7 * 4 values (keys included).
Could someone please let me know if the reading algorithm + running time + the code profiling seem OK and if there is some room for improvement for this specific scenario.
Contributions to the ConfD User Community are always welcome. So if you can post a small project that can build and run out of the box on some standard Linux dist, someone may run it to try recreate your findings and take a look at the feasibility of your reading algorithm + running time + the code profiling.
Thanks for the reply - I’ll try to come back with an standalone example.
Meanwhile, is there any bench-marking information on some general (reading) scenarios on tail-f side? At least to see that I am in the right order of magnitude?
I don’t see how to one can avoid wasting time on a pointless “apples and oranges” comparison unless you provide a project that can be used for recreating your tests.
Sorry, I presumed that the traversal of a list using a cursor was a scenario that might have been bench-marked already - thank you for the reply, I’ll let you know once I have the working example.
Agreed - it is at least an “educated guess” that the processing time in the described case is dominated by round-trips application <-> ConfD. In addition to maapi_get_values(), maapi_get_object() and maapi_get_objects() are likely to be useful. E.g. an entire “sub-list” could be read in a single call to maapi_get_objects() - it might read more than what is wanted if only specific leafs are of interest (maapi_get_values() can avoid that if you know the keys), but it may still be a win.
I use maapi_get_values in order to read some values(leaves) from the first list of 5000 elements. Using maapi_get_values in order to read 20 values for each of the 5000 elements is one order of magnitude faster that reading them one by one. I could use maapi_get_values because all these elements were mandatory leaves (so not lists) and were not keys.
In the present scenario I want to read an optional list (not leaf-list) element from the first list and mostly key values from this second list, so I am not sure maapi_get_values would speed up the reading, that’s why I am using cursors on these lists.