Confirmed commit through netconf deletes nodes

I use CONFD 8.0
I modified slightly the example cdb_subscription/iter_c to have 3 leaves (type of string) in root container. I set initial values for all 3 leaves. I used netconf-console to modify one leaf in candidate db and then commit it to running. What I see is: the leaf has been modified but another 2 leaves get deleted.

Here is console 1 output:

    ypopov@ubuntudsk1:/opt/confd/examples.confd/cdb_subscription/iter_c$ make start
    ### Killing any confd daemon and ifstatus confd agents
    /opt/confd-8.0/bin/confd --stop    || true
    connection refused (stop)
    killall cdbl || true
    cdbl: no process found
    ### Start the confd daemon with our example specific confd-config
    /opt/confd-8.0/bin/confd -c confd.conf --addloadpath /opt/confd-8.0/etc/confd 
    ### In another terminal window, start the CLI (make cli)
    ### Starting the cdb subscriber 
    ./cdbl 
    
    Dumping 
    ---------- 
    *** Config updated 
    Value Set: /root/contact --> (Contact)
    Value Set: /root/hostname --> (MCG-63)
    Value Set: /root/location --> (Toronto)
    Diff match: no
    
    Dumping 
    ---------- 
    *** Config updated 
    Value Set: /root/contact --> (New contact)
    Delete: /root/hostname
    Delete: /root/location
    Diff match: no
    
    Dumping 
    ---------- 
    *** Config updated 
    Value Set: /root/contact --> (Contact)
    Value Set: /root/hostname --> (MCG-63)
    Value Set: /root/location --> (Toronto)
    Diff match: no
    
    Dumping 
    ---------- 
    *** Config updated 
    Value Set: /root/contact --> (New contact)
    Delete: /root/hostname
    Delete: /root/location
    Diff match: no
    
    Dumping 
    ----------

Here is console 2 output:

    ypopov@ubuntudsk1:/opt/confd/examples.confd/cdb_subscription/iter_c$ make cli
    /opt/confd-8.0/bin/confd_cli --user=admin --groups=admin \
    	--interactive || echo Exit
    
    admin connected from 127.0.0.1 using console on ubuntudsk1
    admin@ubuntudsk1> show configuration root
    No entries found.
    [ok][2023-02-21 17:24:49]
    admin@ubuntudsk1> 
    admin@ubuntudsk1> 
    admin@ubuntudsk1> 
    admin@ubuntudsk1> configure 
    Entering configuration mode private
    [ok][2023-02-21 17:24:54]
    
    [edit]
    admin@ubuntudsk1% set root contact Contact location Toronto hostname MCG-63
    [ok][2023-02-21 17:25:30]
    
    [edit]
    admin@ubuntudsk1% commit
    Commit complete.
    [ok][2023-02-21 17:25:34]
    
    [edit]
    admin@ubuntudsk1% exit
    [ok][2023-02-21 17:25:40]
    admin@ubuntudsk1> exit
    ypopov@ubuntudsk1:/opt/confd/examples.confd/cdb_subscription/iter_c$ netconf-console contact_confirm_commit.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="1">
      <ok/>
    </rpc-reply>
    ypopov@ubuntudsk1:/opt/confd/examples.confd/cdb_subscription/iter_c$ make cli
    /opt/confd-8.0/bin/confd_cli --user=admin --groups=admin \
    	--interactive || echo Exit
    
    User admin last logged in 2023-02-21T22:26:45.175566+00:00, to ubuntudsk1, from 127.0.0.1 using netconf-ssh
    admin connected from 127.0.0.1 using console on ubuntudsk1
    admin@ubuntudsk1> show configuration root
    contact "New contact";
    [ok][2023-02-21 17:27:46]
    admin@ubuntudsk1> exit
    ypopov@ubuntudsk1:/opt/confd/examples.confd/cdb_subscription/iter_c$ confd --version
    8.0

Here is yang data model:

    module root {
      namespace "http://tail-f.com/ns/example/root";
      prefix root;
    
      organization "Tail-f Systems (a Cisco company)";
    
      contact "info@tail-f.com";
    
      description
        "YANG datamodel for CDB iteration example
         (C) 2005-2018 Tail-f Systems
    
         Permission to use this code as a starting point hereby granted
         This is ConfD Sample Code.
    
         See the README file for more information";
    
      revision 2019-02-14 {
        description "Normalized YANG identifier names.";
      }
    
      revision 2018-11-28 {
        description "YANG header information updated";
      }
    
      revision 2009-09-30 {
        description "Initial revision.";
      }
    
      container root {
       leaf contact {
          type string;
        }
        leaf hostname {
          type string {
             pattern "MCG-[1-9]{1}[0-9]{0,2}";
          }
        }
        leaf location {
          type string;
        }
        
        container node-b {
          list rf-head {
            key dn;
            max-elements 3;
            leaf dn {
              type int64;
            }
            leaf sector-id {
              type string;
              default "N/A";
            }
            list child {
              key cdn;
              max-elements 2;
              leaf cdn {
                type int64;
              }
              leaf child-attr {
                type string;
                default "N/A";
              }
            }
          }
        }
      }
    }

Here is netconf rpc file contact_confirm_commit.xml:

    <?xml version="1.0" encoding="UTF-8"?>
    <hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
        <capabilities>
            <capability>urn:ietf:params:netconf:base:1.0</capability>
        </capabilities>
    </hello>
    ]]>]]>
    <?xml version="1.0" encoding="UTF-8"?>
    <rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="1">
    <edit-config>
    <target>
        <candidate/>
    </target>
        <default-operation>replace</default-operation>
        <error-option>stop-on-error</error-option>
        <config>
            <root xmlns="http://tail-f.com/ns/example/root">
              <contact>New contact</contact>
            </root>
        </config>
    </edit-config>
    </rpc>
    ]]>]]>
    <?xml version="1.0" encoding="UTF-8"?>
    <rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="2">
    <commit>
        <confirmed/>
    </commit>
    </rpc>
    ]]>]]>
    <?xml version="1.0" encoding="UTF-8"?>
    <rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="1">
    <commit>
    </commit>
    </rpc>
    ]]>]]>
    <?xml version="1.0" encoding="UTF-8"?>
    <rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="3">
        <close-session/>
    </rpc>
    ]]>]]>

Here is confd.conf file:

    <!-- -*- nxml -*- -->
    <!-- This configuration is good for the examples, but are in many ways
         atypical for a production system. It also does not contain all
         possible configuration options.
    
         Better starting points for a production confd.conf configuration 
         file would be confd.conf.example. For even more information, see 
         the confd.conf man page.
         
         E.g. references to current directory are not good practice in a
         production system, but makes it easier to get started with
         this example. There are many references to the current directory
         in this example configuration.
    -->
    
    <confdConfig xmlns="http://tail-f.com/ns/confd_cfg/1.0">
      <!-- The loadPath is searched for .fxs files, javascript files, etc.
           NOTE: if you change the loadPath, the daemon must be restarted,
           or the "In-service Data Model Upgrade" procedure described in
           the User Guide can be used - 'confd - -reload' is not enough.
      -->
      <loadPath>
        <dir>.</dir>
      </loadPath>
    
      <stateDir>.</stateDir>
    
      <enableAttributes>true</enableAttributes>
    
      <cdb>
        <enabled>true</enabled>
        <dbDir>./confd-cdb</dbDir>
        <operational>
          <enabled>true</enabled>
        </operational>
      </cdb> 
    
      <rollback>
        <enabled>true</enabled>
        <directory>./confd-cdb</directory>
      </rollback>
    
      <!-- These keys are used to encrypt values adhering to the types
           tailf:des3-cbc-encrypted-string and tailf:aes-cfb-128-encrypted-string
           as defined in the tailf-common YANG module. These types are
           described in confd_types(3). 
      -->
      <encryptedStrings>
        <DES3CBC>
          <key1>0123456789abcdef</key1>
          <key2>0123456789abcdef</key2>
          <key3>0123456789abcdef</key3>
          <initVector>0123456789abcdef</initVector>
        </DES3CBC>
        
        <AESCFB128>
          <key>0123456789abcdef0123456789abcdef</key>
          <initVector>0123456789abcdef0123456789abcdef</initVector>
        </AESCFB128>
      </encryptedStrings>
    
      <logs>
        <!-- Shared settings for how to log to syslog.
             Each log can be configured to log to file and/or syslog.  If a
             log is configured to log to syslog, the settings below are used.
        -->
        <syslogConfig>
          <!-- facility can be 'daemon', 'local0' ... 'local7' or an integer -->
          <facility>daemon</facility>
          <!-- if udp is not enabled, messages will be sent to local syslog -->
          <udp>
            <enabled>false</enabled>
            <host>syslogsrv.example.com</host>
            <port>514</port>
          </udp>
        </syslogConfig>
    
        <!-- 'confdlog' is a normal daemon log.  Check this log for
             startup problems of confd itself.
             By default, it logs directly to a local file, but it can be
             configured to send to a local or remote syslog as well.
        -->
        <confdLog>
          <enabled>true</enabled>
          <file>
            <enabled>true</enabled>
            <name>./confd.log</name>
          </file>
          <syslog>
            <enabled>true</enabled>
          </syslog>
        </confdLog>
    
        <!-- The developer logs are supposed to be used as debug logs
             for troubleshooting user-written javascript and c code.  Enable
             and check these logs for problems with validation code etc.
        -->
        <developerLog>
          <enabled>true</enabled>
          <file>
            <enabled>true</enabled>
            <name>./devel.log</name>
          </file>
          <syslog>
            <enabled>false</enabled>
          </syslog>
        </developerLog>
    
        <auditLog>
          <enabled>true</enabled>
          <file>
            <enabled>true</enabled>
            <name>./audit.log</name>
          </file>
          <syslog>
            <enabled>true</enabled>
          </syslog>
        </auditLog>
    
        <errorLog>
          <enabled>true</enabled>
          <filename>./confderr.log</filename>
        </errorLog>
    
        <!-- The netconf log can be used to troubleshoot NETCONF operations,
             such as checking why e.g. a filter operation didn't return the
             data requested.
        -->
        <netconfLog>
          <enabled>true</enabled>
          <file>
            <enabled>true</enabled>
            <name>./netconf.log</name>
          </file>
          <syslog>
            <enabled>false</enabled>
          </syslog>
        </netconfLog>
    
        <webuiBrowserLog>
          <enabled>true</enabled>
          <filename>./browser.log</filename>
        </webuiBrowserLog>
    
        <webuiAccessLog>
          <enabled>true</enabled>
          <dir>./</dir>
        </webuiAccessLog>
    
        <netconfTraceLog>
          <enabled>false</enabled>
          <filename>./netconf.trace</filename>
          <format>pretty</format>
        </netconfTraceLog>
    
      </logs>
    
      <!-- Defines which datastores confd will handle. -->
      <datastores>
        <!-- 'startup' means that the system keeps separate running and
             startup configuration databases.  When the system reboots for
             whatever reason, the running config database is lost, and the
             startup is read.
             Enable this only if your system uses a separate startup and
             running database.
        -->
        <startup>
          <enabled>false</enabled>
        </startup>
    
        <!-- The 'candidate' is a shared, named alternative configuration
             database which can be modified without impacting the running
             configuration.  Changes in the candidate can be commit to running,
             or discarded.
             Enable this if you want your users to use this feature from
             NETCONF, CLI or WebGUI, or other agents.
        -->
        <candidate>
          <enabled>true</enabled>
          <!-- By default, confd implements the candidate configuration
               without impacting the application.  But if your system
               already implements the candidate itself, set 'implementation' to
               'external'.
          -->
          <!--implementation>external</implementation-->
          <implementation>confd</implementation>
          <storage>auto</storage>
          <filename>./confd_candidate.db</filename>
        </candidate>
    
        <!-- By default, the running configuration is writable.  This means
             that the application must be prepared to handle changes to
             the configuration dynamically.  If this is not the case, set
             'access' to 'read-only'.  If running is read-only, 'startup'
             must be enabled, and 'candidate' must be disabled.  This means that
             the application reads the configuration at startup, and then
             the box must reboort in order for the application to re-read it's
             configuration.
    
             NOTE: this is not the same as the NETCONF capability
             :writable-running, which merely controls which NETCONF
             operations are allowed to write to the running configuration.
        -->
        <running>
          <access>read-write</access>
        </running>
      </datastores>
    
      <aaa>
        <sshServerKeyDir>./ssh-keydir</sshServerKeyDir>
      </aaa>
    
      <netconf>
        <enabled>true</enabled>
        <transport>
          <ssh>
        <enabled>true</enabled>
        <ip>0.0.0.0</ip>
        <port>2022</port>
          </ssh>
    
          <!-- NETCONF over TCP is not standardized, but it can be useful
           during development in order to use e.g. netcat for scripting.
          -->
          <tcp>
        <enabled>false</enabled>
        <ip>127.0.0.1</ip>
        <port>2023</port>
          </tcp>
        </transport>
    
        <capabilities>
          <!-- enable only if /confdConfig/datastores/candidate is enabled -->
          <candidate>
            <enabled>true</enabled>
          </candidate>
    
          <confirmed-commit>
            <enabled>true</enabled>
          </confirmed-commit>
    
          <!--
               enable only if /confdConfig/datastores/running is read-write
          -->
          <writable-running>
            <enabled>true</enabled>
          </writable-running>
    
          <rollback-on-error>
            <enabled>true</enabled>
          </rollback-on-error>
    
          <actions>
            <enabled>true</enabled>
          </actions>
    
        </capabilities>
      </netconf>
      <webui>
        <enabled>true</enabled>
    
        <transport>
          <tcp>
            <enabled>true</enabled>
            <ip>127.0.0.1</ip>
            <port>8008</port>
          </tcp>
    
          <ssl>
            <enabled>true</enabled>
            <ip>127.0.0.1</ip>
            <port>8888</port>
          </ssl>
        </transport>
    
        <cgi>
          <enabled>true</enabled>
          <php>
            <enabled>true</enabled>
          </php>
        </cgi>
      </webui>
    </confdConfig>

Here is confirmed commit start RPC

All seems to work as expected! :slight_smile:

Your excerpt from contact_confirm_commit.xml includes:

<default-operation>replace</default-operation>

And reading from NETCONF RFC 6241:

replace: The configuration data in the parameter
completely replaces the configuration in the target
datastore. This is useful for loading previously saved
configuration data.

This means not only replacing the items you include into your edit message, but WHOLE configuration → resulting in deletion of items that you did not send as part of message…

Thank you for you response. I removed the line
<default-operation>replace</default-operation>
from contact_confirm_commit.xml file. But the result is the same:
the contact leaf gets updated, but the leaves hostname and location get deleted.

I also tried to lock running and candidate before update and unlock afterward, but the behavior is the same. Leaves get unexpectedly deleted.