ConfD User Community

NETCONF RPC error reply not sent to client for a find_next_object error

Hello,

My data provider application provides access to an external database through the Conf-D Data Provider API with cb_get_object() and cb_find_next_object() callbacks. The application is written in Python and uses the Conf-D Python API.

When the data provider application throws an exception, Conf-D handles NETCONF error reporting differently depending on the callbacks. When an exception is raised in cb_get_object(), the Conf-D NETCONF server returns an . When the same exception is raised in cb_find_next_object(), the Conf-D NETCONF server terminates the connection without sending an . Please see the example provided below.

Conf-D should be sending an for exceptions raised in cb_find_next_object(), correct? Is there something missing from my cb_find_next_object() to trigger Conf-D to send the ?

Response for cb_next_object with exception

$ make query-one
/opt/confd/bin/netconf-console --host localhost --port 830 --get-config -x /my-list/my-entry[name=\'TEST00000002\']
<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="1">
  <rpc-error>
    <error-type>application</error-type>
    <error-tag>operation-failed</error-tag>
    <error-severity>error</error-severity>
    <error-path xmlns:my-test-module="urn:my:yang:my-test-module">
    /my-test-module:my-list/my-test-module:my-entry[my-test-module:name='TEST00000002']
  </error-path>
    <error-message xml:lang="en">/my-list/my-entry[name='TEST00000002']: Python cb_get_object error. ('intt-data',)</error-message>
    <error-info>
      <bad-element>my-entry</bad-element>
    </error-info>
  </rpc-error>
</rpc-reply>

Response for cb_find_next_object with exception

/opt/confd/bin/netconf-console --host localhost --port 830 --get-config -x /my-list/my-entry
No data received in the reply.

my_test_module.py

from __future__ import print_function
import argparse
import select
import socket
import sys
import textwrap
import traceback

import _confd
import _confd.dp as dp
import _confd.maapi as maapi

from my_test_module_ns import ns as my_test_module_ns

def ERROR(*args, **kwargs):
    print("ERROR:", end = '')
    print(*args, **kwargs)

# Test Data
my_list = {
    "TEST00000001": {
        "str-data": "some data (1)",
        "int-data": 1,
    },
    "TEST00000002": {
        "str-data": "some data (2)",
        "int-data": 2,
    },
    "TEST00000003": {
        "str-data": "some data (3)",
        "int-data": 3,
    }
}

my_list_keys = {}
prev_key = None
for next_key in my_list.keys():
    if prev_key:
        my_list_keys[prev_key] = next_key
    prev_key = next_key
my_list_keys[prev_key] = None

def get_first_key():
    global my_list_keys
    if my_list_keys:
        next_key = next(iter(my_list_keys))
    else:
        next_key = None
    return next_key

def get_next_key(key):
    global my_list_keys
    if my_list_keys:
        next_key = my_list_keys[key]
    else:
        next_key = None
    return next_key

def make_tag_value(tag, init, vtype):
    """
    Wrapper to create a _confd.TagValue
    """
    return _confd.TagValue(
        _confd.XmlTag(my_test_module_ns.hash, tag),
        _confd.Value(init, vtype))

def dp_op_to_str(op):
    op_to_str = {
        1: "C_SET_ELEM",
        2: "C_CREATE",
        3: "C_REMOVE",
        4: "C_SET_CASE",
        5: "C_SET_ATTR",
        6: "C_MOVE_AFTER",
    }
    try:
        op_str = op_to_str[op]
    except KeyError:
        op_str = 'UNKNOWN'

    return op_str

V = _confd.Value

# call statistics for each fo the registered data callbacks to keep tabs on
# how many times we access the different cb functions to show in the CLI
K_GET_OBJ = 0
K_FIND_NEXT = 1
K_FIND_NEXT_OBJ = 2
K_NUM_INSTANCES = 3
K_SET_ELEM = 4
K_CREATE = 5
K_REMOVE = 6
calls_keys = [K_GET_OBJ, K_FIND_NEXT, K_FIND_NEXT_OBJ,
              K_NUM_INSTANCES, K_SET_ELEM, K_CREATE, K_REMOVE]
dp_calls = {k: 0 for k in calls_keys}

class TransCbs(object):
    # transaction callbacks
    #
    # The installed init() function gets called every time ConfD
    # wants to establish a new transaction, Each NETCONF
    # command will be a transaction
    #
    # We can choose to create threads here or whatever, we
    # can choose to allocate this transaction to an already existing
    # thread. We must tell ConfD which file descriptor should be
    # Used for all future communication in this transaction
    # this has to be done through the call confd_trans_set_fd();

    def __init__(self, workersocket):
        self._workersocket = workersocket

    def cb_init(self, tctx):
        dp.trans_set_fd(tctx, self._workersocket)
        return _confd.CONFD_OK

    # This callback gets invoked at the end of the transaction
    # when ConfD has accumulated all write operations
    # we're guaranteed that
    # a) no more read ops will occur
    # b) no other transactions will run between here and tr_finish()
    #    for this transaction, i.e ConfD will serialize all transactions
    #  since we need to be prepared for abort(), we may not write
    # our data to the actual database, we can choose to either
    # copy the entire database here and write to the copy in the
    # following write operations _or_ let the write operations
    # accumulate operations create(), set(), delete() instead of actually
    # writing

    # If our db supports transactions (which it doesn't in this
    # silly example, this is the place to do START TRANSACTION

    def cb_write_start(self, tctx):
        return _confd.CONFD_OK

    def cb_prepare(self, tctx):
        return _confd.CONFD_OK

    def cb_commit(self, tctx):
        return _confd.CONFD_OK

    def cb_abort(self, tctx):
        return _confd.CONFD_OK

    def cb_finish(self, tctx):
        return _confd.CONFD_OK

class MyTestModuleDataCbs(object):
    """ Data provider callbacks for the my-test-list list of YANG model. """

    def __init__(self):
        pass

    def cb_get_object(self, tctx, kp):
        dp_calls[K_GET_OBJ] += 1
        print()
        print(f"    cb_get_object '{kp}' {type(kp)}")

        list_filter = dp.data_get_list_filter(tctx)
        print(f"    filter '{list_filter}' {type(list_filter)}")

        key = str(kp[0][0])

        if key in my_list:
            entry = my_list[key]
            vals = [
                make_tag_value(my_test_module_ns.my_test_module_name, key, _confd.C_STR),
                make_tag_value(my_test_module_ns.my_test_module_str_data, entry['str-data'], _confd.C_STR),
                # The following results in a Python KeyError exception
                make_tag_value(my_test_module_ns.my_test_module_int_data, entry['intt-data'], _confd.C_INT32),
            ]
            dp.data_reply_tag_value_array(tctx, vals)
        else:
            dp.data_reply_not_found(tctx)
        return _confd.CONFD_OK


    def cb_find_next(self, tctx, kp, type_, keys):
        dp_calls[K_FIND_NEXT] += 1
        print()
        print(f"    cb_find_next '{kp}' {type(kp)}, {type_}, {keys}")

        list_filter = dp.data_get_list_filter(tctx)
        print(f"    filter '{list_filter}' {type(list_filter)}")

        try:
            key = None
            if keys:
                key = keys[0].as_pyval()
                if key:
                    next_key = get_next_key(key)
            else:
                next_key = get_first_key()

            if next_key:
                next_key_list = [V(next_key, _confd.C_BUF)]  # key of the host is its name
            else:
                next_key_list = None
            dp.data_reply_next_key(tctx, next_key_list, -1)

        except Exception as e:
            ERROR(e)
            ERROR(traceback.format_exc())
            raise

        return _confd.CONFD_OK

    def cb_find_next_object(self, tctx, kp, type_, keys):
        dp_calls[K_FIND_NEXT_OBJ] += 1
        print()
        print(f"    cb_find_next_object '{kp}' {type(kp)}, {type_}, {keys}")

        list_filter = dp.data_get_list_filter(tctx)
        print(f"    filter '{list_filter}' {type(list_filter)}")

        objs = []
        idx = 0
        for key, entry in my_list.items():
            objs.append(([
                make_tag_value(my_test_module_ns.my_test_module_name, key, _confd.C_STR),
                make_tag_value(my_test_module_ns.my_test_module_str_data, entry['str-data'], _confd.C_STR),
                # The following results in a Python KeyError exception
                make_tag_value(my_test_module_ns.my_test_module_int_data, entry['intt-data'], _confd.C_INT32),
            ], idx))
            idx += 1

        if objs:
            objs.append((None, -1))
            dp.data_reply_next_object_tag_value_arrays(tctx, objs, 0)
        else:
            dp.data_reply_next_object_tag_value_arrays(tctx, None, -1)

        return _confd.CONFD_OK

    def cb_num_instances(self, tctx, kp):
        dp_calls[K_NUM_INSTANCES] += 1
        print()
        print(f"    cb_num_instances '{kp}'")

        list_filter = dp.data_get_list_filter(tctx)
        print(f"    filter '{list_filter}' {type(list_filter)}")

        count = len(my_list)
        v_count = V(count, _confd.C_INT32)
        dp.data_reply_value(tctx, v_count)
        return _confd.CONFD_OK

    def cb_set_elem(self, tctx, kp, newval):
        dp_calls[K_SET_ELEM] += 1
        return _confd.ACCUMULATE

    def cb_create(self, tctx, kp):
        dp_calls[K_CREATE] += 1
        return _confd.ACCUMULATE

    def cb_remove(self, tctx, kp):
        dp_calls[K_REMOVE] += 1
        return _confd.ACCUMULATE



def run(debuglevel):

    # In C we use confd_init() which sets the debug-level, but for Python the
    # call to confd_init() is done when we do 'import confd'.
    # Therefore we need to set the debug level here:
    _confd.set_debug(debuglevel, sys.stderr)

    # init library
    daemon_ctx = dp.init_daemon('my_daemon')

    confd_addr = '127.0.0.1'
    confd_port = _confd.PORT
    managed_path = '/'

    maapisock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
    ctlsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
    wrksock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
    try:
        dp.connect(daemon_ctx, ctlsock, dp.CONTROL_SOCKET, confd_addr,
                   confd_port, managed_path)
        dp.connect(daemon_ctx, wrksock, dp.WORKER_SOCKET, confd_addr,
                   confd_port, managed_path)

        maapi.connect(maapisock, confd_addr, confd_port, managed_path)
        maapi.load_schemas(maapisock)


        transaction_cb = TransCbs(wrksock)
        dp.register_trans_cb(daemon_ctx, transaction_cb)

        # database_cb = DatabaseCbs(wrksock, db)
        # dp.register_db_cb(daemon_ctx, database_cb)

        data_cb = MyTestModuleDataCbs()
        dp.register_data_cb(
            daemon_ctx,
            my_test_module_ns.callpoint_my_test_list,
            data_cb,
            flags=dp.DATA_WANT_FILTER)

        dp.register_done(daemon_ctx)

        try:
            _r = [ctlsock, wrksock]
            _w = []
            _e = []

            while True:
                print("Waiting for requests...")
                (r, w, e) = select.select(_r, _w, _e, 1)
                for rs in r:
                    if rs.fileno() == ctlsock.fileno():
                        try:
                            dp.fd_ready(daemon_ctx, ctlsock)
                        except _confd.error.Error as e:
                            if e.confd_errno is not _confd.ERR_EXTERNAL:
                                raise e
                    elif rs.fileno() == wrksock.fileno():
                        try:
                            dp.fd_ready(daemon_ctx, wrksock)
                        except _confd.error.Error as e:
                            if e.confd_errno is not _confd.ERR_EXTERNAL:
                                raise e

        except KeyboardInterrupt:
            print('\nCtrl-C pressed\n')

    finally:
        ctlsock.close()
        wrksock.close()
        dp.release_daemon(daemon_ctx)


if __name__ == '__main__':

    debug_levels = {
        'q': _confd.SILENT,
        'd': _confd.DEBUG,
        't': _confd.TRACE,
        'p': _confd.PROTO_TRACE,
    }

    parser = argparse.ArgumentParser(add_help=False,formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    parser.add_argument(       "--help", action="help", default=argparse.SUPPRESS, help="Show this help message and exit.")
    parser.add_argument("-dl", "--debuglevel", dest="debuglevel", action="store", default='t', required=False, choices=debug_levels.keys(),
                        help=textwrap.dedent(
                            '''\
                            set the debug level:
                                q = quiet (i.e. no) debug
                                d = debug level debug
                                t = trace level debug
                                p = proto level debug
                            '''))
    parser.parse_args()
    args = parser.parse_args()

    confd_debug_level = debug_levels.get(args.debuglevel, _confd.TRACE)

    run(confd_debug_level)

Makefile

######################################################################
# Introduction example 1-2-3-start-query-model
# (C) 2006 Tail-f Systems
#
# See the README files for more information
######################################################################

usage:
	@echo "See README files for more instructions"
	@echo "make all     Build all example files"
	@echo "make clean   Remove all built and intermediary files"
	@echo "make start   Start CONFD daemon and example agent"
	@echo "make stop    Stop any CONFD daemon and example agent"
	@echo "make query-one   Run query for one object, cb_get_object"
	@echo "make query-all   Run query for one object, cb_find_next_object"

######################################################################
# Where is ConfD installed? Make sure CONFD_DIR points it out
CONFD_DIR ?= ../../..

# Include standard ConfD build definitions and rules
include $(CONFD_DIR)/src/confd/build/include.mk

# In case CONFD_DIR is not set (correctly), this rule will trigger
$(CONFD_DIR)/src/confd/build/include.mk:
	@echo 'Where is ConfD installed? Set $$CONFD_DIR to point it out!'
	@echo ''

######################################################################
# Example specific definitions and rules

CONFD_FLAGS = --addloadpath $(CONFD_DIR)/etc/confd
START_FLAGS ?=

all: c-all
	@echo "Build complete"

common-all: $(CDB_DIR) ssh-keydir

c-all: common-all my-test-module.fxs my_test_module_ns.py
	@echo "C build complete"

my-test-module.fxs: my-test-module.yang my-test-module-ann.yang
	$(CONFDC) --fail-on-warnings -a my-test-module-ann.yang -c -o my-test-module.fxs my-test-module.yang

my_test_module_ns.py: my-test-module.fxs
	$(CONFDC) --emit-python my_test_module_ns.py my-test-module.fxs


######################################################################
clean:	iclean
	-rm -rf *log *trace cli-history 2> /dev/null || true
	-rm -rf my_test_module_ns.py *.pyc __init__.py __pycache__ 2> /dev/null || true

######################################################################
#start:  stop start_confd start_subscriber
start:  stop
	$(CONFD)  -c ./confd.conf $(CONFD_FLAGS)
	### * In one terminal window, run: tail -f ./confd.log
	### * In another terminal window, run queries
	###   (try 'make query' for an example)
	### * In this window, the HOSTS confd daemon now starts:
	$(PYTHON) my_test_module.py --debuglevel t

######################################################################
stop:
	### Killing any confd daemon or my-test-module confd agents
	$(CONFD) --stop    || true
	$(KILLALL) `pgrep -f "$(PYTHON) my_test_module.py"` || true

######################################################################
query-one:
	$(CONFD_DIR)/bin/netconf-console --host localhost --port 830 --get-config -x /my-list/my-entry[name=\'TEST00000002\']

######################################################################
query-all:
	$(CONFD_DIR)/bin/netconf-console --host localhost --port 830 --get-config -x /my-list/my-entry

######################################################################

Module

module my-test-module {
    yang-version 1.1;
    namespace "urn:my:yang:my-test-module";
    prefix "my-test-module";
    organization "My Test YANG Module.";
    contact
        "My Test YANG Module.
         ";
    description
        "My Test YANG Module.";
    revision 2022-08-26 {
        reference "My Test YANG Module.";
    }

    container my-list {
        description "An example of a list.";
        list my-entry {
            key name;
            description "An example of a list entry.";
            leaf name {
                type string;
                description "Name identifying a list entry.";
            }
            leaf str-data {
                type string;
                description "String data for this list entry.";
            }
            leaf int-data {
                type int32;
                description "Integer data for this list entry.";
            }
        }
    }
}

If the key does not exist, you need to reply with confd_data_reply_next_key(tctx, None, -1) from your find_next_object() callback as you do from your find_next() callback.

Sorry for the confusion. I think I chose a confusing example, This is not a missing list key, this is Python KeyError Exception related to a Python dictionary lookup. There are lots of different Python exceptions, of which KeyError is one. However, a Python KeyError exception unrelated to a list key that does not exists. These exceptions are raised when the Python code experiences a fault. The Conf-D pyapi raises a number of exceptions itself. This behavior might be unique to the pyapi and perhaps Conf-D APIs for other languages that support exceptions.

The issue is that in some callback sequences where Conf-D NETCONF handles and reports these errors correctly to the client and other callback sequences where Conf-D NETCONF does not.

What does the developer log (with log level “trace”) say?

The errors reported in devel.log look the same to me (see below). Conf-D sends an for the cb_get_object() error, but not for the find_next_object_error().

Just to be clear, I’m not trying to resolve the error as stated above. I’m trying to resolve issue with Conf-D NETCONF error reporting, which I believe outside of the control of my application. I created an error intensionally to test Conf-D’s error reporting. It seems like there is an inconsistency with Conf-D’s error reporting behavior here.

<ERR> 21-Sep-2022::07:26:13.626 hp confd[<0.282.0>]: devel-c get_object error {application, "Python cb_get_object error. ('intt-data',)"} for callpoint my_test_list path /my-test-module:my-list/my-entry{TEST00000002}

<ERR> 21-Sep-2022::07:26:15.767 hp confd[<0.314.0>]: devel-c find_next_object error {application, "Python cb_find_next_object error. ('intt-data',)"} for callpoint my_test_list path /my-test-module:my-list/my-entry

From the confd.conf(5) man page:

/confdConfig/netconf/rpcErrors (close | inline) [close]
If rpcErrors is ‘inline’, and an error occurs during the processing of a or request
when ConfD tries to fetch some data from a data provider, ConfD will generate an rpc-error element
in the faulty element, and continue to process the next element.
If an error occurs and rpcErrors is ‘close’, the NETCONF transport is closed by ConfD.

Check that you have set /confdConfig/netconf/rpcErrors to inline in your confd.conf if you want an error message to be sent.

See also ConfD UG chapter “The NETCONF Server” under “Error Handling”.

That was the issue. Setting rpcError = ‘inline’ resolved the issue. Thank you very much.