Deleted seeded xml data is loaded back on system post upgrade

Installation
1

Hi Team,
We are trying to run upgrade scenario and want to know if this is intentional behaviour from confd.

  1. Bring up application with confd with seeded init xml files, let’s say table1 has 2 entries in it’s seeded xml file, i.e. TableEntry1, TableEntry2.
  2. Once confd is up and running, login to CLI and delete the TableEntry1.
  3. Upgrade application to higher version where table1 seeded data is updated to have 3 entries, like TableEntry1, TableEntry2, TableEntry3.
  4. Post upgrade, TableEntry1, TableEntry2, TableEntry3 are present in confd.
    As per below confd documentation, deleted seeded data should not come back post upgrade. could you confirm if it’s correct behaviour?
8.10. Using initialization files for upgrade
As described earlier, when ConfD starts with an empty CDB database, CDB will load all instantiated XML
documents found in the CDB directory and use these to initialize the the database. We can also use this
mechanism for CDB upgrade, since CDB will again look for files in the CDB directory ending in .xml
when doing an upgrade.
This allows for handling many of the cases that the automatic upgrade can not do by itself, e.g. addition
of mandatory leaves (without default statements), or multiple instances of new dynamic containers. Most
of the time we can probably simply use the XML init file that is appropriate for a fresh install of the new
version also for the upgrade from a previous version.
When using XML files for initialization of CDB, the complete contents of the files is used. On upgrade
however, doing this could lead to modification of the user's existing configuration - e.g. we could end up
resetting data that the user has modified since CDB was first initialized. For this reason two restrictions
are applied when loading the XML files on upgrade:
• Only data for elements that are new as of the upgrade (i.e. elements that did not exist in the previous
schema) will be considered.
• The data will only be loaded if all old (i.e. previously existing) optional/dynamic parent elements and
instances exist in the current configuration.

Thanks,
Usha

2

HI,
How do you load the XML file? By placing it in the CDB directory?
As the documentation describes, if the database is not empty, i.e., there are *.cdb files in the CDB directory, the XML file will not be loaded.
Regards

3

Hi,
we are providing .cdb and seeded xml files in a specific directory(using same for both).
from devel logs created during upgrade time, we could see first .cdb files are loaded. and then each yang module schema update is done, and then we loaded the .xml files.

2024-02-12 06:33:21,725815 EST INFO SBCA confd[12257]: devel-cdb Using delayed CDB journal compaction (automatic mode)
2024-02-12 06:33:21,740849 EST MAJOR SBCA confd[12257]: devel-cdb Loaded schema file: /opt/sonus/sbx/tailf/var/confd/cdb/C.cdb (v5 from ConfD version 7.4.5)
2024-02-12 06:33:21,764997 EST INFO SBCA confd[12257]: devel-cdb do_upgrade:
2024-02-12 06:33:21,863119 EST MAJOR SBCA confd[12257]: devel-cdb loading old namespaces from CDB config file
2024-02-12 06:33:22,141675 EST MAJOR SBCA confd[12257]: devel-cdb Loaded oper data from /opt/sonus/sbx/tailf/var/confd/cdb/O.cdb (19.35 KiB data in 0.015s)
2024-02-12 06:33:23,518795 EST MAJOR SBCA confd[12257]: devel-cdb CDM upgrade not needed
2024-02-12 06:33:23,545632 EST MAJOR SBCA confd[12257]: devel-cdb Loaded configuration from /opt/sonus/sbx/tailf/var/confd/cdb/A.cdb (173.43 KiB data, 5 transactions in 0.026s)
2024-02-12 06:33:23,546022 EST MAJOR SBCA confd[12257]: confd encryptedStrings keys provided from config
2024-02-12 06:33:23,551255 EST MAJOR SBCA confd[12257]: devel-cdb upgrade: processing unchanged namespaces
2024-02-12 06:33:23,560130 EST MAJOR SBCA confd[12257]: devel-cdb upgrade: completed unchanged namespaces
2024-02-12 06:33:23,560215 EST MAJOR SBCA confd[12257]: devel-cdb upgrade: processing data in module sonusIoiToCarrierCodeMappingProfile
2024-02-12 06:33:23,564713 EST MAJOR SBCA confd[12257]: devel-cdb upgrade: completed module sonusIoiToCarrierCodeMappingProfile
2024-02-12 06:33:23,564795 EST MAJOR SBCA confd[12257]: devel-cdb upgrade: processing data in module sonusNodeListATProfile
.
.
.
2024-02-12 06:33:23,730538 EST MAJOR SBCA confd[12257]: devel-cdb upgrade: revision: SNMP-COMMUNITY-MIB@2003-08-06
2024-02-12 06:33:23,730757 EST MAJOR SBCA confd[12257]: devel-cdb upgrade: completed module SNMP-COMMUNITY-MIB
2024-02-12 06:33:23,745911 EST MAJOR SBCA confd[12257]: devel-cdb upgrade: successfully converted /security/eventLogValidation/private{default}/keyValue
2024-02-12 06:33:23,750171 EST MAJOR SBCA confd[12257]: devel-cdb upgrade: successfully converted /security/pki/certificate{defaultSBCCert}/encPkiKeyContent
2024-02-12 06:33:23,750701 EST MAJOR SBCA confd[12257]: devel-cdb upgrade: successfully converted /security/pki/certificate{defaultDtlsSBCCert}/encPkiKeyContent
2024-02-12 06:33:26,195889 EST MAJOR SBCA confd[12257]: devel-cdb init files found in /opt/sonus/sbx/tailf/var/confd/cdb: SNMP-COMMUNITY-MIB.xml, SNMP-NOTIFICATION-MIB.xml, SNMP-TARGET-MIB.xml, SNMP-USER-BASED-SM-MIB.xml, SNMP-VIEW-BASED-ACM-MIB.xml, SNMPv2-MIB.xml, aaa_init.xml, sonusAaa.xml, sonusAccounting.xml, sonusAddressContext.xml, sonusCNFMetaTable.xml, sonusCpcToSipCauseMapProfile.xml, sonusDrmMediaProfile.xml, sonusDrmResourcePad.xml, sonusDtlsProfile.xml, sonusEmaTlsProfile.xml, sonusGen2DtlsSecurity.xml, sonusGen2EventLog.xml, sonusGen2Ipm.xml, sonusGen2NrmAnnouncement.xml, sonusGen2NrmCongestion.xml, sonusGen2NrmCrankback.xml, sonusGen2NrmDiscTreat.xml, sonusGen2NrmDtmftrigger.xml, sonusGen2NrmTone.xml, sonusGen2Ntp.xml, sonusGen2Profiles.xml, sonusGen2Radius.xml, sonusGen2Security.xml, sonusGen2SipFilter.xml, sonusGen2SipJip.xml, sonusGen2SipRph.xml, sonusGenericCodec.xml, sonusHaPort.xml, sonusIpsecProfiles.xml, sonusIsupSignalingProfile.xml, sonusLdapTlsProfile.xml, sonusLicense.xml, sonusLicenseRequired.xml, sonusMetaDataDefaultValues.xml, sonusMetadata.xml, sonusMgmtIpInterface.xml, sonusNetworkSegmentTable.xml, sonusOvldProfile.xml, sonusPacketEncodingType.xml, sonusPolicyServer.xml, sonusPort.xml, sonusSctpProfile.xml, sonusServiceListService.xml, sonusSipPeerOverloadProfile.xml, sonusSipSigControls.xml, sonusSipToCpcCauseMapProfile.xml, sonusSnmp.xml, sonusSystem.xml, sonusSystemSwe.xml, sonusTlsProfile.xml, sonusTrap.xml, sonusXHeaderProfile.xml, sonusXrmMediaPortRange.xml
2024-02-12 06:33:29,417149 EST INFO SBCA confd[12257]: devel-c New daemon connected (name: CPX_App, daemon id: 0)

Post Init we do see confd has the newest entries provided as part of seeded files, so they seem to get loaded.

Thanks,
Usha

4

Yes, sorry for the confusion. During an upgrade, the XML files are loaded by ConfD.

Are you doing this in ConfD start/upgrade phase 0 or after ConfD started?

5

we are deleting the entry prior to upgrade procedure, i.e. we brought up the application for the first time with confd (starting in phases 0,1,2). once confd is up and running we are logging into CLI and deleting the entry. Post this we have initiated upgrade procedure.
During upgrade we are not modifying any entries in phase 0 using upgrade transaction
(but as part of application upgrade, cofd upgrade is triggered new seeded xmls are loaded).

6

So you are:

  1. loading the XML files to CDB which are loaded and committed by the init transaction.
  2. delete TableEntry1 and commit the transaction from the CLI.
  3. upgrade, where the configuration changes in XML files in the CDB directory are again loaded and committed to CDB by the upgrade transaction.

The diff between the config in CDB and the XML files will be applied to CDB by the upgrade transaction.

7

Thanks for prompt reply.
I’m going through section 8.10 Using initialization files for upgrade section of confd user guide (8.0.6, same information is present in early 7.4 versions as well).
It says deleted entries should not come back, it also quotes an example

We can then just use this new init file for the upgrade, and the existing server instances in the user's
configuration will get the new /servers/server/protocol leaf filled in as expected. However
some users may have deleted some of the original servers from their configuration, and in those cases we
obviously do not want those servers to get re-created during the upgrade just because they are present in
the XML file - the above restrictions make sure that this does not happen. Here is what the configuration
looks like after upgrade if the "smtp" server has been deleted before upgrade:

Could you let us know if there is any miss in our understanding or could it be confd bug.

Thanks,
Usha

8

Right, the deleted data that is in the XML file does not come back if the A.cdb file is not deleted. Only data for the new added leafs.

I.e.:

Only data for elements that are new as of the upgrade (i.e. elements that did not exist in the previous schema) will be considered