Tailf:transform and create() data callback for a leaf-list

Hello,

I have a yang that looks like this one:

list myList {
    tailf:callpoint myCp {
        tailf:transform true;
    }

    container myContainer {
        leaf-list myLeafList {
            type uint32;
        }
    }
}

My application registers the following calbacks:

confd_trans_cbs trans;
trans.init = MyInit;
trans.finish = MyFinish;

confd_data_cbs data;
data.exists_optional = MyOptional;
data.get_elem = MyGetElem;
data.get_next = MyGetNext;
data.set_elem = MySetElm;
data.create = MyCreate;
data.remove = MyRemove;

Whenever I commit a configuration performed on the leaf-list myLeafList, MyCreate callback is called for every entry on the leaf-list.

My “full” yang model has a much more complicated representation than this one. I’d like to know if there is a way to receive all data on the leaf-list myLeafList at once instead of receiving the entries one by one. This is necessary because, otherwise, my application will have to create and delete entries on the “full” yang model several times during a commit, and this will cause a lot of performance issues.

This is a change few years old - leaf-lists behave pretty much like full lists, in particular create is indeed invoked for every new entry. There’s a flag to make leaf-lists to behave in the old way more like simple leafs, where you would get a single set_elem callback, but it is deprecated, known to cause bugs, and scheduled to be removed in the next ConfD release.

If I recall correctly, this change was actually introduced to improve performance with the reasoning somewhat as follows: with large leaf-lists, it is much more common to add or remove few instances than to create all or almost all instances. So with the new approach, you have many create callbacks invocations at the beginning and few cheap create callbacks later, while with the old approach you had expensive set_elem invocations all the time. Is your use case that different?

Yes, usually the user will once create a large leaf-list with all its elements on it, and will probably never change it. But that is not the problem. The problem here is my full yang model, that gets written through tailf:transform. Since this full yang model is really different from the yang exposed to the user, a lot of conversions are necessary to write the proper values on it. In order to write everything only once, I’d like to handle all entries on the leaf-list at the same time, which means I’d have to know the full configuration instead of receiving one by one.

I was thinking using something like CONFD_ACCUMULATE on create() callback, but for this case it would be necessary to handle all the entries on the prepare() callback. But from my tests, it seems that the prepare() callback is called only after the validation, and I need the full yang model fulfilled before it, so I think I cannot use this.

I could rewrite the whole full yang model, but in that case it would take a lot more time to deliver the feature. It would be better if another solution was available.

I see. What may work for you then is that you accumulate the data yourself. Approach I saw in a real-world use is like this:

  • model a intermediate list capable of holding all accumulated data from the leaf-list
  • modify your create (and remove?) callbacks to just write data to that list
  • declare a transaction hook on this list with invocation mode per-transaction
  • implement the write_all callback for the hook - when invoked, you have all data that you need; I think the hook should wipe the list as its final step.

There may be alternatives (such as - accumulate the data yourself and write it when “done”; there is notorious problem with identifying the “done” moment, per-transaction hooks can again help with that).

Thanks for the tips, mvf. I’ve already tried the hook approach, but maybe I did something wrong. I’d really appreciate your comments on this. The yang with the hook looks like this:

list myList {
    tailf:callpoint myCp {
        tailf:transform true;
    }

    container myContainer {
        tailf:callpoint myCpHook {
            tailf:transaction-hook subtree {
                tailf:invocation-mode per-transaction;
            }
        }

       leaf-list myLeafList {
           type uint32;
       }
    }
}

I have registered the write_all() callback for the myCpHook, but I never get called on it. The only callbacks that are called are the ones registered for the myCp callpoint. Am I doing something wrong here?

From the confD manual: A callpoint is inherited to all child nodes unless another ‘callpoint’ or an ‘cdb-oper’ is defined.

But that does not seem to be the case here, with the transform.

Thanks again

That’s not what I had in mind. I meant something like this:

list myList { // your list (without the hook)
  tailf:callpoint myCp {
    tailf:transform true;
  }
  ...
}
list myIntermediateList {
  tailf:callpoint myCpHook {
    tailf:transaction hook subtree {
      tailf:invocation-mode per-transaction;
    }
  }
  // your data here...
}

So there is one more data structure. The part “your data” would contain what you need to store there so as to be able to process changes in the right myLeafList instance. In the simplest case, only something like leaf-list myIntermediateLeafList needs to be there, and the mode of operation can be like this:

  • your create or remove callback on /myList{x}/myLeafList is invoked; if the corresponding instance /myIntermediateList{x} does not already exist, create it and populate myIntermediateLeafList, otherwise just add/remove an element to the leaf-list
  • when all changes in myList are done, ConfD notes that there have been changes in myIntermediateList; so it invokes your transaction hook callback write_all (for the callpoint myCpHook)
  • this callback goes over all instances of myIntermediateList - every instance there means that the corresponding leaf-list has changed, so you would process the leaf-list there and change your real data in the full model in one shot
  • as a final step in write_all, I think this is necessary to remove complete myIntermediateList to start with a clean slate next time.

Does that make sense?

Hi,

Yes, it makes sense. I couldn’t work at it today, but tomorrow I’ll get into it again. I’ll let you know what was the result.

Thanks again!