Internal Error when sending notification to the NetConf client

Dear support,

I managed to implement a notification sending to the NetConf client.
However, when I send the notification I sometimes see the following error in confd_devel.log:
4-May-2017::15:39:10.133 oschwar_Host2 confd[14175]: devel-c Internal error on API request

please advise.

thanks
Inbal

Just as with your previous post Confd_notification_send error "Malformed XML" this looks like an application memory issue

Need more input on the issue, e.g. a libconfd trace with timestamp. If you can have the libconfd trace set to CONFD_PROTO_TRACE debug level, you can match the notification and other API calls timestamps with the entry you see in the developer log on the ConfD side.

For example, here I am sending a NETCONF config change notification over the NETCONF stream,

Application:

static void send_cfg_change(void)
{
    confd_tag_value_t vals[11];
    int i = 0;
    struct in_addr in;
    confd_hkeypath_t *kp;
    int msock;
    struct sockaddr_in addr;

    addr.sin_addr.s_addr = inet_addr("127.0.0.1");
    addr.sin_family = AF_INET;
    addr.sin_port = htons(CONFD_PORT);

    maapi_socket(&msock, &addr);
    maapi_xpath2kpath(msock, "/interfaces/interface", &kp);
    close(msock);

    inet_pton(AF_INET, "160.160.160.160", &in);

    CONFD_SET_TAG_XMLBEGIN(&vals[i],   ncn_netconf_config_change, ncn__ns);  i++;
    CONFD_SET_TAG_XMLBEGIN(&vals[i],   ncn_changed_by,            ncn__ns);      i++;
    CONFD_SET_TAG_STR(&vals[i],        ncn_username,              "admin");  i++;
    CONFD_SET_TAG_UINT32(&vals[i],     ncn_session_id,            0);        i++;
    CONFD_SET_TAG_IPV4(&vals[i],       ncn_source_host,           in);  i++;
    CONFD_SET_TAG_XMLEND(&vals[i],     ncn_changed_by,            ncn__ns);  i++;
    CONFD_SET_TAG_ENUM_VALUE(&vals[i], ncn_datastore,             ncn_running); i++;
    CONFD_SET_TAG_XMLBEGIN(&vals[i],   ncn_edit,                  ncn__ns);  i++;
    CONFD_SET_TAG_OBJECTREF(&vals[i],  ncn_target,                kp);i++;
    CONFD_SET_TAG_XMLEND(&vals[i],     ncn_edit,                  ncn__ns);  i++;
    CONFD_SET_TAG_XMLEND(&vals[i],     ncn_netconf_config_change, ncn__ns);  i++;

    send_notification(vals, i);
    confd_free_hkeypath(kp);
}

libconfd trace with debug level set to CONFD_PROTO_TRACE:

...
sending cfg change
TRACE Connected (maapi) to ConfD
TRACE MAAPI_XPATH2HKP 
 7-May-2017::12:59:09.837 14939/7fffca3bb3c0/5 SEND op=228 isrel=0 th=-1 #Bin</interfaces/interfac(1)....>
 7-May-2017::12:59:09.837 14939/7fffca3bb3c0/5 GOT [221572658,[1945824463|321503962]]
 --> CONFD_OK
TRACE NOTIFICATION_SEND NETCONF

 7-May-2017::12:59:09.837 14939/7fffca3bb3c0/4 SEND op=15 isrel=0 th=-1 {170,'NETCONF',undefined,{19,{2017,5,7,10,59,9,837561,0,0}},0,{hxml,[{[1244620176|2103098691],start},{[1244620176|737643097],start},{870658415,#Bin<admin>},{372565948,{12,0}},{1399987302,{160,160,160,160}},{[1244620176|737643097],stop},{449538578,{28,0}},{[1244620176|1716487530],start},{1914145912,{34,[221572658,[1945824463|321503962]]}},{[1244620176|1716487530],stop},{[1244620176|2103098691],stop}]}}

netconf-console client side printout:

$ netconf-console --create-subscription=NETCONF
<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="1">
  <ok/>
</rpc-reply>
<?xml version="1.0" encoding="UTF-8"?>
<notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
  <eventTime>2017-05-07T10:59:09.837561+00:00</eventTime>
  <netconf-config-change xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-notifications">
    <changed-by>
      <username>admin</username>
      <session-id>0</session-id>
      <source-host>160.160.160.160</source-host>
    </changed-by>
    <datastore>running</datastore>
    <edit>
      <target xmlns:notif="http://tail-f.com/ns/test/notif">/notif:interfaces/notif:interface</target>
    </edit>
  </netconf-config-change>
</notification>

Hi,

Thank you for your reply.
The issue I had in my previous post was fixed when I added to the fxs library the ietf-netconf-notification.fxs.

I opened the traces as you suggested and see the following:
TRACE Connected (maapi) to ConfD
TRACE MAAPI_XPATH2HKP
8-May-2017::11:20:29.679 22974/b2fffb40/25 SEND op=228 isrel=0 th=-1 #Bin</bgp/peer-groups/pee(7)…>
8-May-2017::11:20:29.681 22974/b2fffb40/25 GOT [1033149831,13885182,[1693656965|555337807]]
–> CONFD_OK
netconf-config-change begin
changed-by begin
username “Admin”
session-id 0
source-host 160.160.160.160
changed-by end
datastore enum<0>
edit begin
target /eci-bgp:bgp/peer-groups/peer-group
edit end
netconf-config-change end
TRACE NOTIFICATION_SEND NETCONF

8-May-2017::11:20:29.685 22974/b2fffb40/22 SEND op=15 isrel=0 th=-1 {170,‘NETCONF’,undefined,{19,{2017,5,8,11,20,29,681910,0,0}},2,{hxml,[{[1244620176|2103098691],start},{[1244620176|737643097],start},{870658415,#Bin},{372565948,{12,0}},{1399987302,{160,160,160,160}},{[1244620176|737643097],stop},{449538578,{28,0}},{[1244620176|1716487530],start},{1914145912,{34,[1033149831,13885182,[1693656965|555337807]]}},{[1244620176|1716487530],stop},{[1244620176|2103098691],stop}]}}

the only trace I see in the develop log is:
8-May-2017::11:20:29.702 isim_Host1 confd[22943]: devel-c Internal error on API request

the code is:

CONFD_SET_TAG_XMLBEGIN(&values[Index],   ncn_netconf_config_change, ncn__ns);       Index++;
CONFD_SET_TAG_XMLBEGIN(&values[Index],   ncn_changed_by,            ncn__ns);       Index++;
CONFD_SET_TAG_STR(&values[Index],        ncn_username,              "Admin");       Index++;
CONFD_SET_TAG_UINT32(&values[Index],     ncn_session_id,            0);             Index++;
CONFD_SET_TAG_IPV4(&values[Index],       ncn_source_host,           an_addr);       Index++;
CONFD_SET_TAG_XMLEND(&values[Index],     ncn_changed_by,            ncn__ns);       Index++;
CONFD_SET_TAG_ENUM_VALUE(&values[Index], ncn_datastore,             ncn_running);   Index++;
CONFD_SET_TAG_XMLBEGIN(&values[Index],   ncn_edit,                  ncn__ns);       Index++;
CONFD_SET_TAG_OBJECTREF(&values[Index],  ncn_target,                p_KeyPath);     Index++;
CONFD_SET_TAG_XMLEND(&values[Index],     ncn_edit,                  ncn__ns);       Index++;
CONFD_SET_TAG_XMLEND(&values[Index],     ncn_netconf_config_change, ncn__ns);       Index++;

DEBUG(netConfTraceP, <<"ConfDAdaptorManager::CommitWasDone  CONFD_SET_TAG done ");

struct confd_datetime EventTime;

struct tm Time;
struct timeval TimeValue;

gettimeofday(&TimeValue, NULL);
gmtime_r(&TimeValue.tv_sec, &Time);

memset(&EventTime, 0, sizeof(EventTime));
EventTime.year = 1900 + Time.tm_year;
EventTime.month = Time.tm_mon + 1;
EventTime.day = Time.tm_mday;
EventTime.sec = Time.tm_sec;
EventTime.micro = TimeValue.tv_usec;
EventTime.timezone = 0;
EventTime.timezone_minutes = 0;
EventTime.hour = Time.tm_hour;
EventTime.min = Time.tm_min;

#if 1
print_modifications(values, Index, NULL, 0);
#endif

int status = confd_notification_send(m_LiveContext, &EventTime, values, Index);

can you advise why I receive this error?

Thanks in advanced
Inbal

And did the client receive that notification?
You can check in the ConfD NETCONF trace log if the notification was sent to the client by ConfD.

$ cat confd.conf
  <logs>
  ...
    <netconfTraceLog>
      <enabled>true</enabled>
      <filename>./netconf.trace</filename>
      <format>pretty</format>
    </netconfTraceLog>
  </logs>

$ cat netconf.trace
...
8-May-2017::14:22:51.843 **< sess:28 write:
<notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
  <eventTime>2017-05-08T12:22:51.825977+00:00</eventTime>
  <netconf-config-change xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-notifications">
    <changed-by>
      <username>admin</username>
      <session-id>0</session-id>
      <source-host>160.160.160.160</source-host>
    </changed-by>
    <datastore>running</datastore>
    <edit>
      <target xmlns:notif="http://tail-f.com/ns/test/notif">/notif:interfaces/notif:interface</target>
    </edit>
  </netconf-config-change>
</notification>

Hi,

I checked once again with the traces.
When the “connect” is done from the netconf client, I see in the confd_netconf.trace print outs of the message (which is OK).
When the application sends notification to the confd and the client, I see TRACE that a notification is sends.
However, the clients receives nothing and on the netconf.trace log I see only

22-May-2017::05:23:33.367 npt1800 confd[1026]: devel-c Internal error on API request

Please advise, thanks in advance,
Inbal

Can’t reproduce. What does your NETCONF config in confd.conf look like?

the confd.conf:

netconf>
enabled>true /enabled>

<transport>
  <ssh>
<enabled>true</enabled>
<ip>0.0.0.0</ip>
<!-- Note that the standard port for NETCONF over SSH is 830 -->
<port>2022</port>
  </ssh>

  <!--
  NETCONF over TCP is not standardized, but it can be useful
  during development in order to use e.g. netcat for scripting.
  -->
  <tcp>
<enabled>true</enabled>
<ip>127.0.0.1</ip>
<port>2023</port>
  </tcp>
</transport>

<capabilities>
  <!-- enable only if /confdConfig/datastores/startup is enabled -->
  <startup>
<enabled>false</enabled>
  </startup>

  <!-- enable only if /confdConfig/datastores/candidate is enabled -->
  <candidate>
<enabled>false</enabled>
  </candidate>

  <confirmed-commit>
<enabled>false</enabled>
  </confirmed-commit>

  <!-- 
   enable only if /confdConfig/datastores/running is read-write
  -->
  <writable-running>
<enabled>true</enabled>
  </writable-running>

  <rollback-on-error>
<enabled>true</enabled>
  </rollback-on-error>

  <!-- Turn on the URL capability options you want to support -->
  <url>
<enabled>true</enabled>
<file>
  <enabled>true</enabled>
  <rootDir>/root/kvmpackage/Confd/var/confd/state</rootDir>
</file>
<ftp>
  <enabled>true</enabled>
</ftp>
  </url>

  <xpath>
<enabled>true</enabled>
  </xpath>

  <!--
  Enable this to turn on NETCONF Notifications support.
  -->
  <notification>
<enabled>true</enabled>
<!--
    Enable this to make the agent handle RPCs while sending
    notifications.
-->
<interleave>
  <enabled>false</enabled>
</interleave>
  </notification> 

</capabilities>

<!-- 
 If extendedSessions are enabled, all ConfD sessions can be
 terminated using <kill-session>, i.e. not only can other
 NETCONF session be terminated, but also CLI sessions, WebUI
 sessions etc.  If such a session holds a lock, its session
 id will be returned in the <lock-denied>, instead of "0".

     Strictly speaking, this extension is not covered by the
     NETCONF specification; therefore it's false by default.
-->
<extendedSessions>false</extendedSessions>

How do you allocate memory for your values array?

I allocate it on stack.
please find the code below:

{
...

    confd_tag_value_t values[11];
    int Index = 0;
    confd_hkeypath_t *p_KeyPath;
    struct sockaddr_in SocketAddress;
    struct in_addr an_addr = {0xa0a0a0a0};

    //create socket to translate the keypath
    int MappiSocket;
    SocketAddress.sin_addr.s_addr = inet_addr("127.0.0.1");
    SocketAddress.sin_family = AF_INET;
    SocketAddress.sin_port = htons(CONFD_PORT);

    if ((MappiSocket = socket(PF_INET, SOCK_STREAM, 0)) < 0)
    {
        DEBUG(netConfTraceP, <<"ConfDAdaptorManager::CommitWasDoneByOther socket creation failed " );
    }
    if (maapi_connect(MappiSocket, (struct sockaddr *)&SocketAddress, sizeof (SocketAddress)) < 0)
    {
        DEBUG(netConfTraceP, <<"ConfDAdaptorManager::CommitWasDoneByOther socket connect failed " );
    }
    if (maapi_xpath2kpath(MappiSocket, path, &p_KeyPath) != CONFD_OK) //translate the keypath
    {
        DEBUG(netConfTraceP, << "Cannot translate xpath\n");
    }
    maapi_close(MappiSocket);
    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::CommitWasDoneByOther  maapi_close ");

    //set the values of the notification
    CONFD_SET_TAG_XMLBEGIN(&values[Index],   ncn_netconf_config_change, ncn__ns);
    Index++;
    CONFD_SET_TAG_XMLBEGIN(&values[Index],   ncn_changed_by,            ncn__ns);
    Index++;
    CONFD_SET_TAG_STR(&values[Index],        ncn_username,              "Admin");
    Index++;
    CONFD_SET_TAG_UINT32(&values[Index],     ncn_session_id,            0);
    Index++;
    CONFD_SET_TAG_IPV4(&values[Index],       ncn_source_host,           an_addr);
    Index++;
    CONFD_SET_TAG_XMLEND(&values[Index],     ncn_changed_by,            ncn__ns);
    Index++;
    CONFD_SET_TAG_ENUM_VALUE(&values[Index], ncn_datastore,             ncn_running);
    Index++;
    CONFD_SET_TAG_XMLBEGIN(&values[Index],   ncn_edit,                  ncn__ns);
    Index++;
    CONFD_SET_TAG_OBJECTREF(&values[Index],  ncn_target,                p_KeyPath);
    Index++;
    CONFD_SET_TAG_XMLEND(&values[Index],     ncn_edit,                  ncn__ns);
    Index++;
    CONFD_SET_TAG_XMLEND(&values[Index],     ncn_netconf_config_change, ncn__ns);
    Index++;

    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::CommitWasDoneByOther  CONFD_SET_TAG done ");

    struct confd_datetime EventTime;

    struct tm Time;
    struct timeval TimeValue;

    gettimeofday(&TimeValue, NULL);
    gmtime_r(&TimeValue.tv_sec, &Time);

    memset(&EventTime, 0, sizeof(EventTime));
    EventTime.year = 1900 + Time.tm_year;
    EventTime.month = Time.tm_mon + 1;
    EventTime.day = Time.tm_mday;
    EventTime.sec = Time.tm_sec;
    EventTime.micro = TimeValue.tv_usec;
    EventTime.timezone = 0;
    EventTime.timezone_minutes = 0;
    EventTime.hour = Time.tm_hour;
    EventTime.min = Time.tm_min;

#if 1
    print_modifications(values, Index, NULL, 0);
#endif

    int status = confd_notification_send(m_LiveContext, &EventTime, values, Index);

Tested your code with ConfD 6.4. Cannot recreate the issue. It works fine.

Hi again,

I downloaded version 6.4 and installed it, hoping it will fix this issue.
Unfortunatly it didn’t.

I see the prints that the notification was sent:

TRACE Connected (maapi) to ConfD
TRACE MAAPI_XPATH2HKP
4-Jun-2017::08:57:14.504 27352/b1fffb40/27 SEND op=228 isrel=0 th=-1 #Bin</bgp/peer-groups/pee(7)…>
4-Jun-2017::08:57:14.506 27352/b1fffb40/27 GOT [1033149831,13885182,[1693656965|555337807]]
–> CONFD_OK
netconf-config-change begin
changed-by begin
username “Admin”
session-id 0
source-host 160.160.160.160
changed-by end
datastore enum<0>
edit begin
target /eci-bgp:bgp/peer-groups/peer-group
edit end
netconf-config-change end
TRACE NOTIFICATION_SEND NETCONF

4-Jun-2017::08:57:14.508 27352/b1fffb40/23 SEND op=15 isrel=0 th=-1 {170,‘NETCONF’,undefined,{19,{2017,6,4,8,57,14,506787,0,0}},2,{hxml,[{[1244620176|2103098691],start},{[1244620176|737643097],start},{870658415,#Bin},{372565948,{12,0}},{1399987302,{160,160,160,160}},{[1244620176|737643097],stop},{449538578,{28,0}},{[1244620176|1716487530],start},{1914145912,{34,[1033149831,13885182,[1693656965|555337807]]}},{[1244620176|1716487530],stop},{[1244620176|2103098691],stop}]}}

but on the confd_devel I see error:
4-Jun-2017::08:57:14.544 isim_Host1 confd[27321]: devel-c Internal error on API request

no notification was received.

Please advice what else can I do.

thanks
Inbal

Hi,
Unless you can show this community exactly how to reproduce that error (I sent that same notification without any issues), we can’t help other than recommend that you double check your code, the ConfD documentation and examples.

Thanks for your help so far.

we are reviewing the entire code and the configuration, compare it to the examples and guidelines to see what is missing/wrong.
I have few questions regarding diff that we found, that might give us a good hint…

  1. the registration code is as follows:
    struct confd_notification_stream_cbs NotificationCallBack;
    memset(&NotificationCallBack, 0, sizeof(NotificationCallBack));
    NotificationCallBack.fd = m_WorkerSocket;
    NotificationCallBack.get_log_times = NULL;
    NotificationCallBack.replay = NULL;
    strcpy(NotificationCallBack.streamname, “NETCONF”);
    NotificationCallBack.cb_opaque = NULL;
    if (confd_register_notification_stream(m_DeamonContext, &NotificationCallBack, &m_LiveContext) != CONFD_OK)
    {
    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::Init confd_register_notification_stream FAILED " );
    }

I saw example that the reply wasn’t NULL. as much as I understood from the guideline it is not a callback we need. Can you tell if that can cause such error?

  1. I see the following prints :
    6-Jun-2017::12:08:58.701 22860/b30ffb40/23 SEND op=15 isrel=0 th=-1 {170,‘NETCONF’,undefined,{19,{2017,6,6,12,8,58,701083,0,0}},2,{hxml,[{[1244620176|2103098691],start},{[1244620176|737643097],start},{870658415,#Bin},{372565948,{12,0}},{1399987302,{160,160,160,160}},{[1244620176|737643097],stop},{449538578,{28,0}},{[1244620176|1716487530],start},{1914145912,{34,[1033149831,13885182,[1693656965|555337807]]}},{[1244620176|1716487530],stop},{[1244620176|2103098691],stop}]}}

Does it mean that the message was received OK in the confd code? the path is not printed as text but as numbers. Is it OK?

Thanks again for your guideline.

Inbal

Does it mean that the message was received OK in the confd code?

No, the debug output from libconfd just means that libconfd successfully sent the message. If the message was corrupt when ConfD received it, you may get the “API request” error that you see. Perhaps you should check that using for example wireshark or tcpdump comparing the packets sent by libconfd to the ones received by ConfD.
Looking at your past posts, you seem to have had issues with your socket communication, perhaps that’s something you should look into?

I guess that’s what you suspect too, and therefore you cannot provide an example to enable us to reproduce the issue?

the path is not printed as text but as numbers. Is it OK?

It is printed in binary format, which is what libconfd converts it to, so that’s ok.

Hi,

Is there a way to see print outs from the confD itself?
I am not sure what to suspect… the socket is used for the init registration in addition for the receive callbacks so I guess it is OK, but it worth to double check.
Please see below the init an registration code, and the send notification code, to complete the picture. maybe you can see something that we miss here.

thanks again
Inbal

Init code:

/* Initialize confd daemon context */
confd_init("ConfdAdaptor", stderr , CONFD_PROTO_TRACE);
m_DeamonContext = confd_init_daemon("ConfdAdaptor");
if (m_DeamonContext == NULL)
{
    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::Init confd_init_daemon(confdAdaptor) returned NULL " );
    printf("ConfDAdaptorManager::Init (confd_init_daemon) F A I L E D ! ! !\n");
    return;
}
confd_set_daemon_flags(m_DeamonContext, CONFD_DAEMON_FLAG_STRINGSONLY); //to send and recieve string only

/* Initialize sockets to be used by confd */
struct sockaddr_in socketAddress;
socketAddress.sin_addr.s_addr = inet_addr("127.0.0.1");
socketAddress.sin_family = AF_INET;
socketAddress.sin_port = htons(CONFD_PORT);

m_ControlSocket = socket(PF_INET, SOCK_STREAM, 0);
if (m_ControlSocket < 0)
{
    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::Init m_ControlSocket < 0 " );
}

m_WorkerSocket = socket(PF_INET, SOCK_STREAM, 0);
if (m_WorkerSocket < 0)
{
    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::Init m_WorkerSocket < 0 " );
}

confd_load_schemas((struct sockaddr*)&socketAddress, sizeof (struct sockaddr_in)); //for debug

if (confd_connect(m_DeamonContext, m_ControlSocket, CONTROL_SOCKET, (struct sockaddr*)&socketAddress, sizeof (struct sockaddr_in)) < 0)
{
    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::Init confd_connect(m_ControlSocket) < 0 " );
}

if (confd_connect(m_DeamonContext, m_WorkerSocket, WORKER_SOCKET,(struct sockaddr*)&socketAddress,sizeof (struct sockaddr_in))< 0)
{
    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::Init confd_connect(m_WorkerSocket) < 0 " );
}

//registration to Confd, inside:
//confd_register_trans_cb(m_DeamonP, &TransactionCallbacks);
//confd_register_data_cb(a_pDeamon, &data);
m_pConfDAdaptorCallBackClient->RegisterYourself(m_DeamonContext);
m_pConfDAdaptorObjFactory->RegisterList(m_DeamonContext);

//register the stream for notifications to confd
struct confd_notification_stream_cbs NotificationCallBack;
memset(&NotificationCallBack, 0, sizeof(NotificationCallBack));
NotificationCallBack.fd = m_WorkerSocket;
NotificationCallBack.get_log_times = NULL;
NotificationCallBack.replay = NULL;
strcpy(NotificationCallBack.streamname, "NETCONF");
NotificationCallBack.cb_opaque = NULL;
if (confd_register_notification_stream(m_DeamonContext, &NotificationCallBack, &m_LiveContext) != CONFD_OK)
{
    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::Init confd_register_notification_stream FAILED " );
}

confd_register_done(m_DeamonContext);

Send notification code:

//get a constant string back
char * path = YangMapperP->GetKeyPath(a_TableId);

confd_tag_value_t values[11];
int Index = 0;
confd_hkeypath_t *p_KeyPath ;
struct sockaddr_in SocketAddress;
struct in_addr an_addr = {0xa0a0a0a0};

//create socket to translate the keypath
int MappiSocket;
SocketAddress.sin_addr.s_addr = inet_addr("127.0.0.1");
SocketAddress.sin_family = AF_INET;
SocketAddress.sin_port = htons(CONFD_PORT);

if ((MappiSocket = socket(PF_INET, SOCK_STREAM, 0)) < 0)
{
    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::CommitWasDoneByOther socket creation failed " );
}
if (maapi_connect(MappiSocket, (struct sockaddr *)&SocketAddress, sizeof (SocketAddress)) < 0)
{
    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::CommitWasDoneByOther socket connect failed " );
}
if (maapi_xpath2kpath(MappiSocket, path, &p_KeyPath) != CONFD_OK) //translate the keypath
{
    DEBUG(netConfTraceP, << "Cannot translate xpath\n");
}
maapi_close(MappiSocket);
DEBUG(netConfTraceP, <<"ConfDAdaptorManager::CommitWasDoneByOther  maapi_close ");

//set the values of the notification
CONFD_SET_TAG_XMLBEGIN(&values[Index],   ncn_netconf_config_change, ncn__ns);
Index++;
CONFD_SET_TAG_XMLBEGIN(&values[Index],   ncn_changed_by,            ncn__ns);
Index++;
CONFD_SET_TAG_STR(&values[Index],        ncn_username,              "Admin");
Index++;
CONFD_SET_TAG_UINT32(&values[Index],     ncn_session_id,            0);
Index++;
CONFD_SET_TAG_IPV4(&values[Index],       ncn_source_host,           an_addr);
Index++;
CONFD_SET_TAG_XMLEND(&values[Index],     ncn_changed_by,            ncn__ns);
Index++;
CONFD_SET_TAG_ENUM_VALUE(&values[Index], ncn_datastore,             ncn_running);
Index++;
CONFD_SET_TAG_XMLBEGIN(&values[Index],   ncn_edit,                  ncn__ns);
Index++;
CONFD_SET_TAG_OBJECTREF(&values[Index],  ncn_target,                p_KeyPath);
Index++;
CONFD_SET_TAG_XMLEND(&values[Index],     ncn_edit,                  ncn__ns);
Index++;
CONFD_SET_TAG_XMLEND(&values[Index],     ncn_netconf_config_change, ncn__ns);
Index++;

DEBUG(netConfTraceP, <<"ConfDAdaptorManager::CommitWasDoneByOther  CONFD_SET_TAG done ");

struct confd_datetime EventTime;

struct tm Time;
struct timeval TimeValue;

gettimeofday(&TimeValue, NULL);
gmtime_r(&TimeValue.tv_sec, &Time);

memset(&EventTime, 0, sizeof(EventTime));
EventTime.year = 1900 + Time.tm_year;
EventTime.month = Time.tm_mon + 1;
EventTime.day = Time.tm_mday;
EventTime.sec = Time.tm_sec;
EventTime.micro = TimeValue.tv_usec;
EventTime.timezone = 0;
EventTime.timezone_minutes = 0;
EventTime.hour = Time.tm_hour;
EventTime.min = Time.tm_min;

print_modifications(values, Index, NULL, 0);

int status = confd_notification_send(m_LiveContext, &EventTime, values, Index);
if (status != CONFD_OK)
{
    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::CommitWasDoneByOther confd_notification_send was failed\n");
}
else
{
    DEBUG(netConfTraceP, <<"ConfDAdaptorManager::CommitWasDoneByOther confd_notification_send for table Id = " << a_TableId);
}

confd_free_hkeypath(p_KeyPath);

}

Hi,

using tcpdump on the CONFD_PORT (4565) we see the following packet dump.
please advice if you understand something new from it.

thanks
Inbal

12:41:43.297143 IP (tos 0x0, ttl 64, id 63495, offset 0, flags [DF], proto TCP (6), length 383)
127.0.0.1.47144 > 127.0.0.1.4565: Flags [P.], cksum 0xff73 (incorrect -> 0x4b2f), seq 22:353, ack 10, win 342, options [nop,nop,TS val 2979493033 ecr 2975759089], length 331
0x0000: 0000 0000 0000 0000 0000 0000 0800 4500 …E.
0x0010: 017f f807 4000 4006 436f 7f00 0001 7f00 …@.@.Co…
0x0020: 0001 b828 11d5 021b 82e1 a49b d211 8018 …(…
0x0030: 0156 ff73 0000 0101 080a b197 74a9 b15e .V.s…t…^
0x0040: 7af1 0000 0147 0000 000f 8368 0661 aa64 z…G…h.a.d
0x0050: 0007 4e45 5443 4f4e 4664 0009 756e 6465 …NETCONFd…unde
0x0060: 6669 6e65 6468 0261 1368 0962 0000 07e1 finedh.a.h.b…
0x0070: 6106 610c 610c 6129 612b 6200 0487 8b61 a.a.a.a)a+b…a
0x0080: 0061 0061 0268 0264 0004 6878 6d6c 6c00 .a.a.h.d…hxmll.
0x0090: 0000 0b68 026c 0000 0001 624a 2f65 9062 …h.l…bJ/e.b
0x00a0: 7d5a bd43 6400 0573 7461 7274 6802 6c00 }Z.Cd…starth.l.
0x00b0: 0000 0162 4a2f 6590 622b f78a 5964 0005 …bJ/e.b+…Yd…
0x00c0: 7374 6172 7468 0262 33e5 316f 6d00 0000 starth.b3.1om…
0x00d0: 0561 646d 696e 6802 6216 34e7 bc68 0261 .adminh.b.4…h.a
0x00e0: 0c61 0068 0262 5372 1c66 6804 61a0 61a0 .a.h.bSr.fh.a.a.
0x00f0: 61a0 61a0 6802 6c00 0000 0162 4a2f 6590 a.a.h.l…bJ/e.
0x0100: 622b f78a 5964 0004 7374 6f70 6802 621a b+…Yd…stoph.b.
0x0110: cb6a 1268 0261 1c61 0068 026c 0000 0001 .j.h.a.a.h.l…
0x0120: 624a 2f65 9062 664f 856a 6400 0573 7461 bJ/e.bfO.jd…sta
0x0130: 7274 6802 6272 178c 7868 0261 226c 0000 rth.br…xh.a"l…
0x0140: 0003 623d 949d 8762 00d3 defe 6c00 0000 …b=…b…l…
0x0150: 0162 64f3 2785 6221 19c8 4f6a 6802 6c00 .bd.’.b!..Ojh.l.
0x0160: 0000 0162 4a2f 6590 6266 4f85 6a64 0004 …bJ/e.bfO.jd…
0x0170: 7374 6f70 6802 6c00 0000 0162 4a2f 6590 stoph.l…bJ/e.
0x0180: 627d 5abd 4364 0004 7374 6f70 6a b}Z.Cd…stopj

Hi,

Is the handling of the callbacks and the notifications through the worker socket done from the same or different threads?

A quick look at your TCP dump seems to indicate that you have two notifications/requests/replies being sent on the same socket simultaneously to ConfD.

Hi,

The implementation is that we send the notification from different thread than the thread that receive and answer the confd callbacks.
However, I tried to send the notification from the callback thread, to be sure it is the only message that being sent at the same time. Still got the internal error message…

  • how do you see in the tcp dump I added that 2 message are sent at the same time?
  • Is there a way to see what is being received in the CONFD process? Any other idea of how to proceed?

thanks
Inbal

It was just a guess after comparing your tcpdump with one of a NETCONF notification send that was successful. Have you done that comparison yourself?

tcpdump or similar is what you have.

I suggest you use one socket per thread. No sharing of sockets between threads is best practice. Then compare tcpdumps of NETCONF notifications that failed to send and successful sends.

Hi,

when running the example, there is no Internal Error when sending notification.
However, from technical issues of our servers, I can not operate tcpdump.

Is it possible that you will send me a “good” netconf - notification dump so I can compare?
thanks in advance,
Inbal