ITER_WANT_SCHEMA_ORDER causes cdb_diff_iterate to not report set for default case

k.laskowski · February 12, 2024, 9:07pm

Hello,

I have an issue with cdb_diff_iterate called with flag: ITER_WANT_SCHEMA_ORDER: when I delete non default case from a configuration my iter function is not called with MOP_VALUE_SET for a default case.

My yang file (root.yang):

module root {
  namespace "http://tail-f.com/ns/example/root";
  prefix root;

  container root {
    choice choice-case {
      default "first-case";
      case first-case {
        leaf first {
          type uint8;
          default 12;
        }
      }
      case second-case {
        leaf second {
          type uint8;
          default 44;
        }
      }
    }
  }
}

I will use confd_cli for changing the configuration. Then I will compare output from:

confd_cmd -c 'subwait_iter /root' which uses cdb_diff_iterate with flags set to ITER_WANT_PREV|ITER_WANT_ANCESTOR_DELETE and
reproduction binary which uses cdb_diff_iterate with flags set to ITER_WANT_PREV|ITER_WANT_SCHEMA_ORDER.

In the first commit I set non default case. In the second commit I remove the non default case and I expect that the iter function would be called with set for a default case as it is the active case now.

Set root second

confd_cli

admin@rhel84wsl% set root second 99
admin@rhel84wsl% commit
Commit complete.

subwait_iter output:

COMMIT
/root /root/second set ( -> 99)
/root /root/first deleted
DONE

reproduction output:

*** Config updated 
 /root/first deleted
 /root/second set ( -> 99)

Delete root second

confd_cli

admin@rhel84wsl% delete root second
admin@rhel84wsl% commit
Commit complete.

subwait_iter output:

COMMIT
/root /root/first set ( -> 12)
/root /root/second deleted (99 -> )
DONE

reproduction output:

*** Config updated 
 /root/second deleted (99 -> )

As you can see ITER_WANT_SCHEMA_ORDER flag affects what operations a subscriber receives. I’ve read the documentation but I haven’t found anything that suggests such behaviour.

Please note that I have defaultHandlingMode setting set to report-all.

In our application we rely heavily on the fact that confd informs us about setting a leaf to its default value. Good example is creating list entry with a leaf which has default value but its value is not set explicitly.

Please comment if it is a bug or intended behaviour.

Other files for reproduction:

reproduction.c

#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/poll.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <stdio.h>
#include <signal.h>
#include <errno.h>

#include <confd_lib.h>
#include <confd_cdb.h>
#include "root.h"

static void iter_common(confd_hkeypath_t *kp,
                        enum cdb_iter_op op,
                        confd_value_t *oldv,
                        confd_value_t *newv,
                        void *state)
{
    char tmppath[BUFSIZ];
    char tmpbuf1[BUFSIZ], tmpbuf2[BUFSIZ];
    const char *opstr = "";
    char *subpath = (char *)state;
    struct confd_cs_node *tnode = confd_find_cs_node(kp, kp->len);
    confd_pp_kpath(tmppath, BUFSIZ, kp);

#define PPV(VP, BUF)                                                    \
    {                                                                   \
        if ((tnode == NULL) ||                                          \
            (confd_val2str(tnode->info.type, VP, BUF,                   \
                           sizeof(BUF)/sizeof(*BUF)) == CONFD_ERR)) {   \
            confd_pp_value(BUF, sizeof(BUF)/sizeof(*BUF), VP);          \
        }                                                               \
    }

    switch (op) {
    case MOP_CREATED:      opstr = "created";  break;
    case MOP_DELETED:      opstr = "deleted";  break;
    case MOP_VALUE_SET:    opstr = "set";      break;
    case MOP_MODIFIED:     opstr = "modified"; break;
    case MOP_MOVED_AFTER:  opstr = "moved";    break;
    case MOP_ATTR_SET:     confd_fatal("unexpected MOP_ATTR_SET");
    }

    tmpbuf1[0] = tmpbuf2[0] = '\0';
    if (oldv) { PPV(oldv, tmpbuf1); }
    if (op != MOP_MOVED_AFTER) {
        if (newv) { PPV(newv, tmpbuf2); }
    } else {
        if (newv) {
            char *p = tmpbuf2;
            confd_value_t *vp = newv;
            if (tnode != NULL)
                tnode = tnode->children;
            while (vp->type != C_NOEXISTS && p - tmpbuf2 < BUFSIZ) {
                if (p == tmpbuf2) {
                    p += snprintf(p, BUFSIZ, "after {");
                } else {
                    p += snprintf(p, BUFSIZ - (p - tmpbuf2), " ");
                }
                {
                    int c = 0;
                    int sz = BUFSIZ - (p - tmpbuf2);

                    if ((tnode == NULL) ||
                        ((c = confd_val2str(tnode->info.type, vp, p, sz)) ==
                         CONFD_ERR)) {
                        c = confd_pp_value(p, sz, vp);
                    }
                    p += c;
                }
                if (tnode != NULL)
                    tnode = tnode->next;
                vp++;
            }
            if (p - tmpbuf2 < BUFSIZ)
                snprintf(p, BUFSIZ - (p - tmpbuf2), "}");
        } else {
            snprintf(tmpbuf2, BUFSIZ, "first");
        }
    }

#undef PPV

    if (oldv || newv) {
        printf("%s %s %s (%s -> %s)\n",
               subpath, tmppath, opstr, tmpbuf1, tmpbuf2);
    } else {
        printf("%s %s %s\n", subpath, tmppath, opstr);
    }
}


static enum cdb_iter_ret iter(confd_hkeypath_t *kp,
                              enum cdb_iter_op op,
                              confd_value_t *oldv,
                              confd_value_t *newv,
                              void *state)
{
    iter_common(kp, op, oldv, newv, state);
    return ITER_RECURSE;
}

int main(int argc, char **argv)
{
    struct sockaddr_in addr;
    int c, status, subsock, sock;
    int headpoint;
    enum confd_debug_level dbgl = CONFD_SILENT;
    const char *confd_addr = "127.0.0.1";
    int confd_port = CONFD_PORT;

    while ((c = getopt(argc, argv, "dta:p:")) != EOF) {
        switch (c) {
        case 'd': dbgl = CONFD_DEBUG; break;
        case 't': dbgl = CONFD_TRACE; break;
        case 'a': confd_addr = optarg; break;
        case 'p': confd_port = atoi(optarg); break;
        }
    }

    addr.sin_addr.s_addr = inet_addr(confd_addr);
    addr.sin_family = AF_INET;
    addr.sin_port = htons(confd_port);

    confd_init(argv[0], stderr, dbgl);

    if ((sock = socket(PF_INET, SOCK_STREAM, 0)) < 0)
        confd_fatal("%s: Failed to create socket", argv[0]);

    if (confd_load_schemas((struct sockaddr*)&addr,
                           sizeof (struct sockaddr_in)) != CONFD_OK)
        confd_fatal("%s: Failed to load schemas from confd\n", argv[0]);

    if (cdb_connect(sock, CDB_DATA_SOCKET, (struct sockaddr *)&addr,
                    sizeof(struct sockaddr_in)) != CONFD_OK)
        confd_fatal("%s: Failed to connect to ConfD", argv[0]);

    if ((subsock = socket(PF_INET, SOCK_STREAM, 0)) < 0 )
        confd_fatal("Failed to open socket\n");

    if (cdb_connect(subsock, CDB_SUBSCRIPTION_SOCKET, (struct sockaddr*)&addr,
                      sizeof (struct sockaddr_in)) < 0)
        confd_fatal("Failed to cdb_connect() to confd \n");

    /* setup subscription point */

    if ((status = cdb_subscribe(subsock, 3, root__ns, &headpoint,
                                "/"))
        != CONFD_OK) {
        confd_fatal("Terminate: subscribe %d\n", status);
    }
    if (cdb_subscribe_done(subsock) != CONFD_OK)
        confd_fatal("cdb_subscribe_done() failed");

    while (1) {
        int status;
        struct pollfd set[1];

        set[0].fd = subsock;
        set[0].events = POLLIN;
        set[0].revents = 0;

        if (poll(&set[0], sizeof(set)/sizeof(*set), -1) < 0) {
            if (errno != EINTR) {
                perror("Poll failed:");
                continue;
            }
        }

        if (set[0].revents & POLLIN) {
            int sub_points[1];
            int reslen;

            if ((status = cdb_read_subscription_socket(subsock,
                                                       &sub_points[0],
                                                       &reslen)) != CONFD_OK) {
                confd_fatal("terminate sub_read: %d\n", status);
            }
            if (reslen > 0) {
                fprintf(stderr, "*** Config updated \n");

                if ((status = cdb_start_session(sock,CDB_RUNNING)) != CONFD_OK)
                    confd_fatal("Cannot start session\n");
                if ((status = cdb_set_namespace(sock, root__ns)) != CONFD_OK)
                    confd_fatal("Cannot set namespace\n");

                cdb_diff_iterate(subsock, sub_points[0], iter,
                                 ITER_WANT_PREV|ITER_WANT_SCHEMA_ORDER, (void*)&sock);
                cdb_end_session(sock);
            }

            if ((status = cdb_sync_subscription_socket(subsock,
                                                       CDB_DONE_PRIORITY))
                != CONFD_OK) {
                confd_fatal("failed to sync subscription: %d\n", status);
            }
        }
    }
}

confd.conf

<!-- -*- nxml -*- -->
<!-- This configuration is good for the examples, but are in many ways
     atypical for a production system. It also does not contain all
     possible configuration options.

     Better starting points for a production confd.conf configuration 
     file would be confd.conf.example. For even more information, see 
     the confd.conf man page.
     
     E.g. references to current directory are not good practice in a
     production system, but makes it easier to get started with
     this example. There are many references to the current directory
     in this example configuration.
-->

<confdConfig xmlns="http://tail-f.com/ns/confd_cfg/1.0">
  <!-- The loadPath is searched for .fxs files, javascript files, etc.
       NOTE: if you change the loadPath, the daemon must be restarted,
       or the "In-service Data Model Upgrade" procedure described in
       the User Guide can be used - 'confd - -reload' is not enough.
  -->
  <loadPath>
    <dir>.</dir>
  </loadPath>

  <stateDir>.</stateDir>
  <defaultHandlingMode>report-all</defaultHandlingMode>
  <enableAttributes>true</enableAttributes>

  <cdb>
    <enabled>true</enabled>
    <dbDir>./confd-cdb</dbDir>
    <operational>
      <enabled>true</enabled>
    </operational>
  </cdb> 

  <rollback>
    <enabled>true</enabled>
    <directory>./confd-cdb</directory>
  </rollback>

  <!-- These keys are used to encrypt values adhering to the types
       tailf:des3-cbc-encrypted-string and tailf:aes-cfb-128-encrypted-string
       as defined in the tailf-common YANG module. These types are
       described in confd_types(3). 
  -->
  <encryptedStrings>
    <DES3CBC>
      <key1>0123456789abcdef</key1>
      <key2>0123456789abcdef</key2>
      <key3>0123456789abcdef</key3>
      <initVector>0123456789abcdef</initVector>
    </DES3CBC>
    
    <AESCFB128>
      <key>0123456789abcdef0123456789abcdef</key>
      <initVector>0123456789abcdef0123456789abcdef</initVector>
    </AESCFB128>
  </encryptedStrings>

  <logs>
    <!-- Shared settings for how to log to syslog.
         Each log can be configured to log to file and/or syslog.  If a
         log is configured to log to syslog, the settings below are used.
    -->
    <syslogConfig>
      <!-- facility can be 'daemon', 'local0' ... 'local7' or an integer -->
      <facility>daemon</facility>
      <!-- if udp is not enabled, messages will be sent to local syslog -->
      <udp>
        <enabled>false</enabled>
        <host>syslogsrv.example.com</host>
        <port>514</port>
      </udp>
    </syslogConfig>

    <!-- 'confdlog' is a normal daemon log.  Check this log for
         startup problems of confd itself.
         By default, it logs directly to a local file, but it can be
         configured to send to a local or remote syslog as well.
    -->
    <confdLog>
      <enabled>true</enabled>
      <file>
        <enabled>true</enabled>
        <name>./confd.log</name>
      </file>
      <syslog>
        <enabled>true</enabled>
      </syslog>
    </confdLog>

    <!-- The developer logs are supposed to be used as debug logs
         for troubleshooting user-written javascript and c code.  Enable
         and check these logs for problems with validation code etc.
    -->
    <developerLog>
      <enabled>true</enabled>
      <file>
        <enabled>true</enabled>
        <name>./devel.log</name>
      </file>
      <syslog>
        <enabled>false</enabled>
      </syslog>
    </developerLog>

    <auditLog>
      <enabled>true</enabled>
      <file>
        <enabled>true</enabled>
        <name>./audit.log</name>
      </file>
      <syslog>
        <enabled>true</enabled>
      </syslog>
    </auditLog>

    <errorLog>
      <enabled>true</enabled>
      <filename>./confderr.log</filename>
    </errorLog>

    <!-- The netconf log can be used to troubleshoot NETCONF operations,
         such as checking why e.g. a filter operation didn't return the
         data requested.
    -->
    <netconfLog>
      <enabled>true</enabled>
      <file>
        <enabled>true</enabled>
        <name>./netconf.log</name>
      </file>
      <syslog>
        <enabled>false</enabled>
      </syslog>
    </netconfLog>

    <webuiBrowserLog>
      <enabled>true</enabled>
      <filename>./browser.log</filename>
    </webuiBrowserLog>

    <webuiAccessLog>
      <enabled>true</enabled>
      <dir>./</dir>
    </webuiAccessLog>

    <netconfTraceLog>
      <enabled>false</enabled>
      <filename>./netconf.trace</filename>
      <format>pretty</format>
    </netconfTraceLog>

  </logs>

  <!-- Defines which datastores confd will handle. -->
  <datastores>
    <!-- 'startup' means that the system keeps separate running and
         startup configuration databases.  When the system reboots for
         whatever reason, the running config database is lost, and the
         startup is read.
         Enable this only if your system uses a separate startup and
         running database.
    -->
    <startup>
      <enabled>false</enabled>
    </startup>

    <!-- The 'candidate' is a shared, named alternative configuration
         database which can be modified without impacting the running
         configuration.  Changes in the candidate can be commit to running,
         or discarded.
         Enable this if you want your users to use this feature from
         NETCONF, CLI or WebGUI, or other agents.
    -->
    <candidate>
      <enabled>false</enabled>
      <!-- By default, confd implements the candidate configuration
           without impacting the application.  But if your system
           already implements the candidate itself, set 'implementation' to
           'external'.
      -->
      <!--implementation>external</implementation-->
      <implementation>confd</implementation>
      <storage>auto</storage>
      <filename>./confd_candidate.db</filename>
    </candidate>

    <!-- By default, the running configuration is writable.  This means
         that the application must be prepared to handle changes to
         the configuration dynamically.  If this is not the case, set
         'access' to 'read-only'.  If running is read-only, 'startup'
         must be enabled, and 'candidate' must be disabled.  This means that
         the application reads the configuration at startup, and then
         the box must reboort in order for the application to re-read it's
         configuration.

         NOTE: this is not the same as the NETCONF capability
         :writable-running, which merely controls which NETCONF
         operations are allowed to write to the running configuration.
    -->
    <running>
      <access>read-write</access>
    </running>
  </datastores>

  <aaa>
    <sshServerKeyDir>./ssh-keydir</sshServerKeyDir>
  </aaa>

  <netconf>
    <enabled>true</enabled>
    <transport>
      <ssh>
    <enabled>true</enabled>
    <ip>127.0.0.1</ip>
    <port>2022</port>
      </ssh>

      <!-- NETCONF over TCP is not standardized, but it can be useful
       during development in order to use e.g. netcat for scripting.
      -->
      <tcp>
    <enabled>false</enabled>
    <ip>127.0.0.1</ip>
    <port>2023</port>
      </tcp>
    </transport>

    <capabilities>
      <confirmed-commit>
        <enabled>false</enabled>
      </confirmed-commit>

      <rollback-on-error>
        <enabled>true</enabled>
      </rollback-on-error>

      <actions>
        <enabled>true</enabled>
      </actions>

    </capabilities>
  </netconf>
  <webui>
    <enabled>true</enabled>

    <transport>
      <tcp>
        <enabled>true</enabled>
        <ip>127.0.0.1</ip>
        <port>8008</port>
      </tcp>

      <ssl>
        <enabled>true</enabled>
        <ip>127.0.0.1</ip>
        <port>8888</port>
      </ssl>
    </transport>

    <cgi>
      <enabled>true</enabled>
      <php>
        <enabled>true</enabled>
      </php>
    </cgi>
  </webui>
</confdConfig>

Makefile

######################################################################
# Interface Status example
# (C) 2006-2009 Tail-f Systems
#
# See the README file for more information
######################################################################

usage:
	@echo "See README file for more instructions"
	@echo "make all     Build all example files"
	@echo "make clean   Remove all built and intermediary files"
	@echo "make start   Start CONFD daemon and example agent"
	@echo "make stop    Stop any CONFD daemon and example agent"
	@echo "make cli     Start the CONFD Command Line Interface"

######################################################################
# Where is ConfD installed? Make sure CONFD_DIR points it out
CONFD_DIR ?= ../..

# Include standard ConfD build definitions and rules
include $(CONFD_DIR)/src/confd/build/include.mk

# In case CONFD_DIR is not set (correctly), this rule will trigger
$(CONFD_DIR)/src/confd/build/include.mk:
	@echo 'Where is ConfD installed? Set $$CONFD_DIR to point it out!'
	@echo ''


######################################################################
# Example specific definitions and rules

CONFD_FLAGS ?= --addloadpath $(CONFD_DIR)/etc/confd
START_FLAGS ?=

PROG	= reproduction
SRC	= $(PROG).c
OBJS	= $(SRC:.c=.o)
LIBS	+= -lcrypto 


all:	root.fxs $(PROG) $(CDB_DIR) ssh-keydir
	@echo "Build complete"

$(PROG): $(OBJS)
	g++ -o $@ $(OBJS) $(LIBS) 

$(PROG).o: root.h


######################################################################
clean:	iclean
	rm -rf _tmp* log/*
	-rm -rf root.h root_proto.h $(PROG)  > /dev/null || true

######################################################################

start:  stop
	### Start the confd daemon with our example specific confd-config
	$(CONFD) -c confd.conf $(CONFD_FLAGS) 
	### In another terminal window, start the CLI (make cli)
	### Starting the cdb subscriber 
	./$(PROG) $(START_FLAGS)


######################################################################
stop:
	### Killing any confd daemon and ifstatus confd agents
	$(CONFD) --stop    || true
	killall $(PROG) || true

######################################################################
cli:
	$(CONFD_DIR)/bin/confd_cli --user=admin --groups=admin \
		--interactive || echo Exit

######################################################################
cli-c:
	$(CONFD_DIR)/bin/confd_cli -C --user=admin --groups=admin \
		--interactive || echo Exit

confd version:

> confd --status | head -1
vsn: 7.5.4.3

k.laskowski · February 14, 2024, 8:00pm

@cohult If the description of my case is not clear please let me know.

I’m curious if it is a bug or I did something wrong.
Anyway, we can live with a workaround: setting a leaf explicitly

cohult · February 15, 2024, 9:44am

@k.laskowski, I can’t say if it is a bug; check with ConfD support.