Confd crashed after committing the in-service data model upgrade

[hzpfly]

It says in User Guide 15.5 Committing the upgrade: When maapi_perform_upgrade() has completed successfully, we must call maapi_commit_upgrade() to tell ConfD to make the upgrade permanent.

After maapi_commit _upgrade is called, ConfD prints the following log and eventually crashed. Would you please help analyze what causes this error? Does this error cause ConfD crash?

    confd[152]: devel-c close_trans/"cpa" error {external, ""}
     =ERROR REPORT==== 2-Jun-2021::11:04:40.698435 ===
    ** Generic server capi_server terminating
    ** Last message in was {tcp,{socket,<0.30703.0>},
                                <<131,104,4,97,27,98,0,0,222,187,97,1,98,255,255,
                                  255,254>>}
    ** When Server state == {state,normal,4,1910,
                             [{daemon,3,
                               {socket,<0.333.0>},
                               [{worker,14,{socket,<0.334.0>},undefined,[]}],
                               [],0,[],enotifier,[],128,none,false,0,0,done,
                               117702912,
                               [1200,1188]},
                              {daemon,2,
                               {socket,<0.151.0>},
                               [{worker,28,{socket,<0.153.0>},6569016,[]}],
                               [],547,[],cpaValidator,[],0,validate,false,0,0,
                               done,117702912,[]},
                              {daemon,1,
                               {socket,<0.143.0>},
                               [{worker,25,
                                 {socket,<0.30703.0>},
                                 6569016,
                                 [{default,
                                   {27,57019,1,-2,[],1200,0},
                                   undefined,false}]}],
                               [],57020,[],cpa,[],128,read_write,true,255,0,done,
                               117702912,[]},
                              {daemon,0,
                               {socket,<0.136.0>},
                               [{worker,4,{socket,<0.137.0>},undefined,[]}],
                               [],1,[],cm_core_yang_library_ext_dataprovider,[],0,
                               read,false,129,0,done,117702912,
                               [1200,1188,10,9,4]}],
                             [],[],
                             [{usess,1200,<0.30681.0>,
                               #Ref<0.3832024404.1819017217.108786>,system,tcp,
                               <<"yp_internal_user">>,
                               {127,0,0,1},
                               0,
                               {1622,631880,406093},
                               [],undefined,<<>>,<<>>,<<>>,normal,
                               {user_identification,undefined,undefined,
                                undefined,
                                "/root/go/src/adp-gs-cm-nbi/confd/cfuncs.go:777"},
                           undefined,2},
                          {usess,1188,<0.30537.0>,
                           #Ref<0.3832024404.1817706498.130369>,cli,ssh,
                           <<"vduuser">>,
                           {10,209,110,74},
                           16409,
                           {1622,631817,867754},
                           [],undefined,<<>>,<<>>,<<>>,normal,undefined,
                           undefined,0},
                          {usess,10,<0.223.0>,
                           #Ref<0.3832024404.1817444353.208898>,system,system,
                           <<"system">>,
                           {0,0,0,0},
                           0,
                           {1622,540053,727773},
                           [],undefined,<<>>,<<>>,<<>>,normal,undefined,
                           undefined,0},
                          {usess,9,<0.220.0>,
                           #Ref<0.3832024404.1817444353.208879>,system,system,
                           <<"system">>,
                           {0,0,0,0},
                           0,
                           {1622,540053,723344},
                           [],undefined,<<>>,<<>>,<<>>,normal,undefined,
                           undefined,0},
                          {usess,4,<0.168.0>,
                           #Ref<0.3832024404.1817444353.208156>,system,system,
                           <<"system">>,
                           {0,0,0,0},
                           0,
                           {1622,540053,699667},
                           [],undefined,<<>>,<<>>,<<>>,normal,undefined,
                           undefined,0},
                          {usess,1,<0.112.0>,
                           #Ref<0.3832024404.1817444353.207230>,system,system,
                           <<"system">>,
                           {0,0,0,0},
                           0,
                           {1622,539979,149257},
                           [],undefined,<<>>,<<>>,<<>>,normal,undefined,
                           undefined,0}],
                         [<0.30537.0>]}
** Reason for termination ==
** {{badmatch,error},
    [{capi_server,handle_info,2,[{file,"capi_server.erl"},{line,1389}]},
     {gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,653}]},
     {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,727}]},
     {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}

=CRASH REPORT==== 2-Jun-2021::11:04:40.699325 ===
  crasher:
    initial call: capi_server:init/1
    pid: <0.106.0>
    registered_name: capi_server
    exception error: no match of right hand side value error
      in function  capi_server:handle_info/2 (capi_server.erl, line 1389)
      in call from gen_server:try_dispatch/4 (gen_server.erl, line 653)
      in call from gen_server:handle_msg/6 (gen_server.erl, line 727)
    ancestors: [capi_sup,<0.104.0>]
    message_queue_len: 0
    messages: []

BRs
Michael

[cohult]

Take a look at this Markdown Cheatsheet for some guidance on how to format the text in your posts https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet#code

[hzpfly]

And here is the whole log:

2021-06-02T11:04:40.541983712Z <INFO> 2-Jun-2021::11:04:40.541 com-cm-yang-provider-547cf5c97b-c6dh8 confd[152]: - Upgrade performed
2021-06-02T11:04:40.690508815Z <ERR> 2-Jun-2021::11:04:40.690 com-cm-yang-provider-547cf5c97b-c6dh8 confd[152]: devel-c close_trans/"cpa" error {external, ""}
2021-06-02T11:04:40.699331366Z =ERROR REPORT==== 2-Jun-2021::11:04:40.698435 ===
2021-06-02T11:04:40.69935004Z ** Generic server capi_server terminating 
2021-06-02T11:04:40.699353088Z ** Last message in was {tcp,{socket,<0.30703.0>},
2021-06-02T11:04:40.699356126Z                             <<131,104,4,97,27,98,0,0,222,187,97,1,98,255,255,
2021-06-02T11:04:40.699359008Z                               255,254>>}
2021-06-02T11:04:40.699361567Z ** When Server state == {state,normal,4,1910,
2021-06-02T11:04:40.699364057Z                          [{daemon,3,
2021-06-02T11:04:40.699366384Z                            {socket,<0.333.0>},
2021-06-02T11:04:40.699368852Z                            [{worker,14,{socket,<0.334.0>},undefined,[]}],
2021-06-02T11:04:40.699371397Z                            [],0,[],enotifier,[],128,none,false,0,0,done,
2021-06-02T11:04:40.699375821Z                            117702912,
2021-06-02T11:04:40.699388442Z                            [1200,1188]},
2021-06-02T11:04:40.699391035Z                           {daemon,2,
2021-06-02T11:04:40.699393337Z                            {socket,<0.151.0>},
2021-06-02T11:04:40.699395758Z                            [{worker,28,{socket,<0.153.0>},6569016,[]}],
2021-06-02T11:04:40.699398251Z                            [],547,[],cpaValidator,[],0,validate,false,0,0,
2021-06-02T11:04:40.699400827Z                            done,117702912,[]},
2021-06-02T11:04:40.699403151Z                           {daemon,1,
2021-06-02T11:04:40.699405447Z                            {socket,<0.143.0>},
2021-06-02T11:04:40.699407822Z                            [{worker,25,
2021-06-02T11:04:40.699410292Z                              {socket,<0.30703.0>},
2021-06-02T11:04:40.699412833Z                              6569016,
2021-06-02T11:04:40.699415178Z                              [{default,
2021-06-02T11:04:40.699417475Z                                {27,57019,1,-2,[],1200,0},
2021-06-02T11:04:40.699419961Z                                undefined,false}]}],
2021-06-02T11:04:40.699422305Z                            [],57020,[],cpa,[],128,read_write,true,255,0,done,
2021-06-02T11:04:40.699424712Z                            117702912,[]},
2021-06-02T11:04:40.699427013Z                           {daemon,0,
2021-06-02T11:04:40.699429315Z                            {socket,<0.136.0>},
2021-06-02T11:04:40.699431689Z                            [{worker,4,{socket,<0.137.0>},undefined,[]}],
2021-06-02T11:04:40.699434314Z                            [],1,[],cm_core_yang_library_ext_dataprovider,[],0,
2021-06-02T11:04:40.699436753Z                            read,false,129,0,done,117702912,
2021-06-02T11:04:40.699439113Z                            [1200,1188,10,9,4]}],
2021-06-02T11:04:40.69944144Z                          [],[],
2021-06-02T11:04:40.69944379Z                          [{usess,1200,<0.30681.0>,
2021-06-02T11:04:40.699446206Z                            #Ref<0.3832024404.1819017217.108786>,system,tcp,
2021-06-02T11:04:40.699448755Z                            <<"yp_internal_user">>,
2021-06-02T11:04:40.699451352Z                            {127,0,0,1},
2021-06-02T11:04:40.699453747Z                            0,
2021-06-02T11:04:40.699457196Z                            {1622,631880,406093},
2021-06-02T11:04:40.699459554Z                            [],undefined,<<>>,<<>>,<<>>,normal,
2021-06-02T11:04:40.699462335Z                            {user_identification,undefined,undefined,
2021-06-02T11:04:40.699464744Z                             undefined,
2021-06-02T11:04:40.699470509Z                             "/root/go/src/adp-gs-cm-nbi/confd/cfuncs.go:777"},
2021-06-02T11:04:40.69947309Z                            undefined,2},
2021-06-02T11:04:40.699475437Z                           {usess,1188,<0.30537.0>,
2021-06-02T11:04:40.699477868Z                            #Ref<0.3832024404.1817706498.130369>,cli,ssh,
2021-06-02T11:04:40.699480391Z                            <<"vduuser">>,
2021-06-02T11:04:40.69948292Z                            {10,209,110,74},
2021-06-02T11:04:40.699485269Z                            16409,
2021-06-02T11:04:40.699487562Z                            {1622,631817,867754},
2021-06-02T11:04:40.699489908Z                            [],undefined,<<>>,<<>>,<<>>,normal,undefined,
2021-06-02T11:04:40.699492603Z                            undefined,0},
2021-06-02T11:04:40.699494935Z                           {usess,10,<0.223.0>,
2021-06-02T11:04:40.699497353Z                            #Ref<0.3832024404.1817444353.208898>,system,system,
2021-06-02T11:04:40.699499813Z                            <<"system">>,
2021-06-02T11:04:40.699502277Z                            {0,0,0,0},
2021-06-02T11:04:40.699504584Z                            0,
2021-06-02T11:04:40.699506861Z                            {1622,540053,727773},
2021-06-02T11:04:40.699509195Z                            [],undefined,<<>>,<<>>,<<>>,normal,undefined,
2021-06-02T11:04:40.699511824Z                            undefined,0},
2021-06-02T11:04:40.699514136Z                           {usess,9,<0.220.0>,
2021-06-02T11:04:40.699516505Z                            #Ref<0.3832024404.1817444353.208879>,system,system,
2021-06-02T11:04:40.699518932Z                            <<"system">>,
2021-06-02T11:04:40.69952133Z                            {0,0,0,0},
2021-06-02T11:04:40.699523656Z                            0,
2021-06-02T11:04:40.699525921Z                            {1622,540053,723344},
2021-06-02T11:04:40.699528295Z                            [],undefined,<<>>,<<>>,<<>>,normal,undefined,
2021-06-02T11:04:40.699530916Z                            undefined,0},
2021-06-02T11:04:40.699533217Z                           {usess,4,<0.168.0>,
2021-06-02T11:04:40.699535584Z                            #Ref<0.3832024404.1817444353.208156>,system,system,
2021-06-02T11:04:40.699538013Z                            <<"system">>,
2021-06-02T11:04:40.699540496Z                            {0,0,0,0},
2021-06-02T11:04:40.699542788Z                            0,
2021-06-02T11:04:40.699548463Z                            {1622,540053,699667},
2021-06-02T11:04:40.69955085Z                            [],undefined,<<>>,<<>>,<<>>,normal,undefined,
2021-06-02T11:04:40.69955347Z                            undefined,0},
2021-06-02T11:04:40.699555767Z                           {usess,1,<0.112.0>,
2021-06-02T11:04:40.699558141Z                            #Ref<0.3832024404.1817444353.207230>,system,system,
2021-06-02T11:04:40.69956107Z                            <<"system">>,
2021-06-02T11:04:40.699563511Z                            {0,0,0,0},
2021-06-02T11:04:40.699565811Z                            0,
2021-06-02T11:04:40.699568059Z                            {1622,539979,149257},
2021-06-02T11:04:40.699570385Z                            [],undefined,<<>>,<<>>,<<>>,normal,undefined,
2021-06-02T11:04:40.699572987Z                            undefined,0}],
2021-06-02T11:04:40.699575279Z                          [<0.30537.0>]}
2021-06-02T11:04:40.699577669Z ** Reason for termination ==
2021-06-02T11:04:40.699579993Z ** {{badmatch,error},
2021-06-02T11:04:40.699582251Z     [{capi_server,handle_info,2,[{file,"capi_server.erl"},{line,1389}]},
2021-06-02T11:04:40.699584781Z      {gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,653}]},
2021-06-02T11:04:40.69958722Z      {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,727}]},
2021-06-02T11:04:40.699589704Z      {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
2021-06-02T11:04:40.699592176Z 
2021-06-02T11:04:40.705641477Z =CRASH REPORT==== 2-Jun-2021::11:04:40.699325 ===
2021-06-02T11:04:40.70566461Z   crasher:
2021-06-02T11:04:40.705668289Z     initial call: capi_server:init/1
2021-06-02T11:04:40.705671695Z     pid: <0.106.0>
2021-06-02T11:04:40.705675042Z     registered_name: capi_server
2021-06-02T11:04:40.705678624Z     exception error: no match of right hand side value error
2021-06-02T11:04:40.705681971Z       in function  capi_server:handle_info/2 (capi_server.erl, line 1389)
2021-06-02T11:04:40.705685657Z       in call from gen_server:try_dispatch/4 (gen_server.erl, line 653)
2021-06-02T11:04:40.705688864Z       in call from gen_server:handle_msg/6 (gen_server.erl, line 727)
2021-06-02T11:04:40.705692143Z     ancestors: [capi_sup,<0.104.0>]
2021-06-02T11:04:40.705695764Z     message_queue_len: 0
2021-06-02T11:04:40.705698852Z     messages: []
2021-06-02T11:04:40.705701853Z     links: [<0.151.0>,<0.333.0>,<0.30703.0>,<0.334.0>,<0.153.0>,
2021-06-02T11:04:40.705705876Z                   <0.137.0>,<0.143.0>,<0.136.0>,<0.105.0>]
2021-06-02T11:04:40.70571804Z     dictionary: [{{cbtype,trans_write},[1]},
2021-06-02T11:04:40.70572234Z                   {{daemon_cbs,0},[{'yanglibxe-cp',130}]},
2021-06-02T11:04:40.705725928Z                   {{cbtype,any},[2,1,0]},
2021-06-02T11:04:40.705729195Z                   {query_timeout,600},
2021-06-02T11:04:40.705732775Z                   {new_session_timeout,30},
2021-06-02T11:04:40.705736896Z                   {{daemon_cbs,1},
2021-06-02T11:04:40.705740487Z                    [{'cpa-state-data',130},
2021-06-02T11:04:40.705743817Z                     {cpa,130},
2021-06-02T11:04:40.705747144Z                     {'cpa-action',132},
2021-06-02T11:04:40.705750461Z                     {'static-data-cb',130},
2021-06-02T11:04:40.705753423Z                     {'is-system-created-cb',130}]},
2021-06-02T11:04:40.705756854Z                   {connect_timeout,60},
2021-06-02T11:04:40.705760045Z                   {gc_timeout,3000},
2021-06-02T11:04:40.705763232Z                   {{monitor,subscriber,<0.30537.0>},
2021-06-02T11:04:40.705766543Z                    #Ref<0.3832024404.1819017217.107500>},
2021-06-02T11:04:40.70576983Z                   {{cbtype,130},[1,0]},
2021-06-02T11:04:40.705773026Z                   {{daemon_cbs,2},[{cpa_validationpoint,131}]},
2021-06-02T11:04:40.705776071Z                   {{cbtype,131},[2]},
2021-06-02T11:04:40.705779062Z                   {{cbtype,132},[1]}]
2021-06-02T11:04:40.7057824Z     trap_exit: false
2021-06-02T11:04:40.705785368Z     status: running
2021-06-02T11:04:40.705788538Z     heap_size: 10958
2021-06-02T11:04:40.705791769Z     stack_size: 27
2021-06-02T11:04:40.70579469Z     reductions: 18765646
2021-06-02T11:04:40.705798099Z   neighbours:
2021-06-02T11:04:40.705801112Z     neighbour:
2021-06-02T11:04:40.70580411Z       pid: <0.136.0>
2021-06-02T11:04:40.705807255Z       registered_name: []
2021-06-02T11:04:40.705810519Z       initial call: capi_server:socket/2
2021-06-02T11:04:40.70581361Z       current_function: {capi_server,socket_loop,5}
2021-06-02T11:04:40.705816808Z       ancestors: [capi_server,capi_sup,<0.104.0>]
2021-06-02T11:04:40.705819935Z       message_queue_len: 0
2021-06-02T11:04:40.705823023Z       links: [<0.106.0>,#Port<0.16>]
2021-06-02T11:04:40.705826228Z       trap_exit: false
2021-06-02T11:04:40.705835312Z       status: waiting
2021-06-02T11:04:40.705838326Z       heap_size: 233
2021-06-02T11:04:40.705841441Z       stack_size: 10
2021-06-02T11:04:40.705844574Z       reductions: 1094
2021-06-02T11:04:40.70585099Z       current_stacktrace: [{capi_server,socket_loop,5,
2021-06-02T11:04:40.705854405Z                                [{file,"capi_server.erl"},{line,2122}]},
2021-06-02T11:04:40.705857948Z                   {proc_lib,init_p_do_apply,3,
2021-06-02T11:04:40.705860789Z                             [{file,"proc_lib.erl"},{line,249}]}]
2021-06-02T11:04:40.705864083Z     neighbour:
2021-06-02T11:04:40.70586721Z       pid: <0.143.0>
2021-06-02T11:04:40.705871167Z       registered_name: []
2021-06-02T11:04:40.705874422Z       initial call: capi_server:socket/2
2021-06-02T11:04:40.705877315Z       current_function: {capi_server,socket_loop,5}
2021-06-02T11:04:40.70588043Z       ancestors: [capi_server,capi_sup,<0.104.0>]
2021-06-02T11:04:40.705883838Z       message_queue_len: 0
2021-06-02T11:04:40.705886896Z       links: [<0.106.0>,#Port<0.22>]
2021-06-02T11:04:40.705890227Z       trap_exit: false
2021-06-02T11:04:40.705894863Z       status: waiting
2021-06-02T11:04:40.705897917Z       heap_size: 2586
2021-06-02T11:04:40.70590116Z       stack_size: 10
2021-06-02T11:04:40.705904386Z       reductions: 447326
2021-06-02T11:04:40.705907563Z       current_stacktrace: [{capi_server,socket_loop,5,
2021-06-02T11:04:40.705911214Z                                [{file,"capi_server.erl"},{line,2122}]},
2021-06-02T11:04:40.705914968Z                   {proc_lib,init_p_do_apply,3,
2021-06-02T11:04:40.705918122Z                             [{file,"proc_lib.erl"},{line,249}]}]
2021-06-02T11:04:40.705921538Z     neighbour:
2021-06-02T11:04:40.705924897Z       pid: <0.137.0>
2021-06-02T11:04:40.705928393Z       registered_name: []
2021-06-02T11:04:40.705931335Z       initial call: capi_server:socket/2
2021-06-02T11:04:40.705934494Z       current_function: {capi_server,socket_loop,5}
2021-06-02T11:04:40.705939262Z       ancestors: [capi_server,capi_sup,<0.104.0>]
2021-06-02T11:04:40.705942421Z       message_queue_len: 0
2021-06-02T11:04:40.705945614Z       links: [<0.106.0>,#Port<0.17>]
2021-06-02T11:04:40.705948989Z       trap_exit: false
2021-06-02T11:04:40.705952344Z       status: waiting
2021-06-02T11:04:40.705955244Z       heap_size: 233
2021-06-02T11:04:40.705958339Z       stack_size: 10
2021-06-02T11:04:40.705961562Z       reductions: 431
2021-06-02T11:04:40.705964736Z       current_stacktrace: [{capi_server,socket_loop,5,
2021-06-02T11:04:40.705968123Z                                [{file,"capi_server.erl"},{line,2122}]},
2021-06-02T11:04:40.705971545Z                   {proc_lib,init_p_do_apply,3,
2021-06-02T11:04:40.705978382Z                             [{file,"proc_lib.erl"},{line,249}]}]
2021-06-02T11:04:40.70598141Z     neighbour:
2021-06-02T11:04:40.705984886Z       pid: <0.153.0>
2021-06-02T11:04:40.705987923Z       registered_name: []
2021-06-02T11:04:40.70599511Z       initial call: capi_server:socket/2
2021-06-02T11:04:40.705997947Z       current_function: {capi_server,socket_loop,5}
2021-06-02T11:04:40.70600098Z       ancestors: [capi_server,capi_sup,<0.104.0>]
2021-06-02T11:04:40.706003846Z       message_queue_len: 0
2021-06-02T11:04:40.706006667Z       links: [<0.106.0>,#Port<0.27>]
2021-06-02T11:04:40.706011571Z       trap_exit: false
2021-06-02T11:04:40.706015184Z       status: waiting
2021-06-02T11:04:40.70602219Z       heap_size: 610
2021-06-02T11:04:40.706028078Z       stack_size: 10
2021-06-02T11:04:40.706030911Z       reductions: 34776
2021-06-02T11:04:40.706034143Z       current_stacktrace: [{capi_server,socket_loop,5,
2021-06-02T11:04:40.706041891Z                                [{file,"capi_server.erl"},{line,2122}]},
2021-06-02T11:04:40.706047693Z                   {proc_lib,init_p_do_apply,3,
2021-06-02T11:04:40.706055793Z                             [{file,"proc_lib.erl"},{line,249}]}]
2021-06-02T11:04:40.706062702Z     neighbour:
2021-06-02T11:04:40.706066263Z       pid: <0.334.0>
2021-06-02T11:04:40.70606944Z       registered_name: []
2021-06-02T11:04:40.706073269Z       initial call: capi_server:socket/2
2021-06-02T11:04:40.706076437Z       current_function: {capi_server,socket_loop,5}
2021-06-02T11:04:40.706079645Z       ancestors: [capi_server,capi_sup,<0.104.0>]
2021-06-02T11:04:40.706082616Z       message_queue_len: 0
2021-06-02T11:04:40.706086111Z       links: [<0.106.0>,#Port<0.46>]
2021-06-02T11:04:40.706089173Z       trap_exit: false
2021-06-02T11:04:40.706092265Z       status: waiting
2021-06-02T11:04:40.706095472Z       heap_size: 233
2021-06-02T11:04:40.706098586Z       stack_size: 10
2021-06-02T11:04:40.70610092Z       reductions: 20568245
2021-06-02T11:04:40.706103137Z       current_stacktrace: [{capi_server,socket_loop,5,
2021-06-02T11:04:40.706105693Z                                [{file,"capi_server.erl"},{line,2122}]},
2021-06-02T11:04:40.70610807Z                   {proc_lib,init_p_do_apply,3,
2021-06-02T11:04:40.706110442Z                             [{file,"proc_lib.erl"},{line,249}]}]
2021-06-02T11:04:40.706112766Z     neighbour:
2021-06-02T11:04:40.706114924Z       pid: <0.30703.0>
2021-06-02T11:04:40.706117236Z       registered_name: []
2021-06-02T11:04:40.70612237Z       initial call: capi_server:socket/2
2021-06-02T11:04:40.706124634Z       current_function: {capi_server,socket_loop,5}
2021-06-02T11:04:40.706126962Z       ancestors: [capi_server,capi_sup,<0.104.0>]
2021-06-02T11:04:40.70612934Z       message_queue_len: 0
2021-06-02T11:04:40.70613155Z       links: [<0.106.0>,#Port<0.11629>]
2021-06-02T11:04:40.706133875Z       trap_exit: false
2021-06-02T11:04:40.706136106Z       status: waiting
2021-06-02T11:04:40.706140627Z       heap_size: 233
2021-06-02T11:04:40.706142809Z       stack_size: 10
2021-06-02T11:04:40.706144981Z       reductions: 995
2021-06-02T11:04:40.706147162Z       current_stacktrace: [{capi_server,socket_loop,5,
2021-06-02T11:04:40.706149532Z                                [{file,"capi_server.erl"},{line,2122}]},
2021-06-02T11:04:40.70615189Z                   {proc_lib,init_p_do_apply,3,
2021-06-02T11:04:40.706154156Z                             [{file,"proc_lib.erl"},{line,249}]}]
2021-06-02T11:04:40.706156517Z     neighbour:
2021-06-02T11:04:40.70615874Z       pid: <0.333.0>
2021-06-02T11:04:40.706160956Z       registered_name: []
2021-06-02T11:04:40.706163171Z       initial call: capi_server:socket/2
2021-06-02T11:04:40.706165406Z       current_function: {capi_server,socket_loop,5}
2021-06-02T11:04:40.706167912Z       ancestors: [capi_server,capi_sup,<0.104.0>]
2021-06-02T11:04:40.706170198Z       message_queue_len: 0
2021-06-02T11:04:40.706172405Z       links: [<0.106.0>,#Port<0.45>]
2021-06-02T11:04:40.706174731Z       trap_exit: false
2021-06-02T11:04:40.706176983Z       status: waiting
2021-06-02T11:04:40.706179163Z       heap_size: 233
2021-06-02T11:04:40.706181329Z       stack_size: 10
2021-06-02T11:04:40.706183937Z       reductions: 837
2021-06-02T11:04:40.706186128Z       current_stacktrace: [{capi_server,socket_loop,5,
2021-06-02T11:04:40.706188416Z                                [{file,"capi_server.erl"},{line,2122}]},
2021-06-02T11:04:40.70619075Z                   {proc_lib,init_p_do_apply,3,
2021-06-02T11:04:40.706193017Z                             [{file,"proc_lib.erl"},{line,249}]}]
2021-06-02T11:04:40.706195335Z     neighbour:
2021-06-02T11:04:40.706197478Z       pid: <0.151.0>
2021-06-02T11:04:40.706201807Z       registered_name: []
2021-06-02T11:04:40.706204053Z       initial call: capi_server:socket/2
2021-06-02T11:04:40.706206296Z       current_function: {capi_server,socket_loop,5}
2021-06-02T11:04:40.706208542Z       ancestors: [capi_server,capi_sup,<0.104.0>]
2021-06-02T11:04:40.706213097Z       message_queue_len: 0
2021-06-02T11:04:40.706215329Z       links: [<0.106.0>,#Port<0.25>]
2021-06-02T11:04:40.706217653Z       trap_exit: false
2021-06-02T11:04:40.706219843Z       status: waiting
2021-06-02T11:04:40.706222011Z       heap_size: 376
2021-06-02T11:04:40.706224177Z       stack_size: 10
2021-06-02T11:04:40.706226353Z       reductions: 67566
2021-06-02T11:04:40.70622857Z       current_stacktrace: [{capi_server,socket_loop,5,
2021-06-02T11:04:40.706230874Z                                [{file,"capi_server.erl"},{line,2122}]},
2021-06-02T11:04:40.706233208Z                   {proc_lib,init_p_do_apply,3,
2021-06-02T11:04:40.706235464Z                             [{file,"proc_lib.erl"},{line,249}]}]
2021-06-02T11:04:40.70623778Z =SUPERVISOR REPORT==== 2-Jun-2021::11:04:40.705305 ===
2021-06-02T11:04:40.706240035Z     supervisor: {local,capi_sup}
2021-06-02T11:04:40.706242256Z     errorContext: child_terminated
2021-06-02T11:04:40.70624447Z     reason: {{badmatch,error},
2021-06-02T11:04:40.706246695Z              [{capi_server,handle_info,2,
2021-06-02T11:04:40.706248957Z                            [{file,"capi_server.erl"},{line,1389}]},
2021-06-02T11:04:40.706251288Z               {gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,653}]},
2021-06-02T11:04:40.706253691Z               {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,727}]},
2021-06-02T11:04:40.706258139Z               {proc_lib,init_p_do_apply,3,
2021-06-02T11:04:40.70626354Z                         [{file,"proc_lib.erl"},{line,249}]}]}
2021-06-02T11:04:40.706265973Z     offender: [{pid,<0.106.0>},
2021-06-02T11:04:40.706268283Z                {id,capi_server},
2021-06-02T11:04:40.706270501Z                {mfargs,{capi_server,start_link,[]}},
2021-06-02T11:04:40.706272758Z                {restart_type,permanent},
2021-06-02T11:04:40.706274976Z                {shutdown,2000},
2021-06-02T11:04:40.706277192Z                {child_type,worker}]
2021-06-02T11:04:40.706425155Z =ERROR REPORT==== 2-Jun-2021::11:04:40.705406 ===
2021-06-02T11:04:40.706434513Z cs_trans:496: internal error: {{{badmatch,error},
2021-06-02T11:04:40.706438212Z                                 [{capi_server,handle_info,2,
2021-06-02T11:04:40.706441948Z                                   [{file,"capi_server.erl"},{line,1389}]},
2021-06-02T11:04:40.706446074Z                                  {gen_server,try_dispatch,4,
2021-06-02T11:04:40.706461989Z                                   [{file,"gen_server.erl"},{line,653}]},
2021-06-02T11:04:40.706465244Z                                  {gen_server,handle_msg,6,
2021-06-02T11:04:40.706468514Z                                   [{file,"gen_server.erl"},{line,727}]},
2021-06-02T11:04:40.706479518Z                                  {proc_lib,init_p_do_apply,3,
2021-06-02T11:04:40.706482786Z                                   [{file,"proc_lib.erl"},{line,249}]}]},
2021-06-02T11:04:40.706486215Z                                {gen_server,call,
2021-06-02T11:04:40.706489402Z                                 [capi_server,
2021-06-02T11:04:40.706492697Z                                  {new_trans,running,1188,cli,read,2488,
2021-06-02T11:04:40.706497621Z                                   unhide_inactive},
2021-06-02T11:04:40.706500975Z                                  infinity]}}
2021-06-02T11:04:40.706504203Z [{cs_trans,'-fmt_error/3-fun-0-',0,[{file,"cs_trans.erl"},{line,496}]},
2021-06-02T11:04:40.706507553Z  {cs_trans,fmt_error,3,[{file,"cs_trans.erl"},{line,496}]},
2021-06-02T11:04:40.70651092Z  {cli_backend_server,do_loop,4,[{file,"cli_backend_server.erl"},{line,803}]},
2021-06-02T11:04:40.706514202Z  {proc_lib,init_p,3,[{file,"proc_lib.erl"},{line,234}]}]
2021-06-02T11:04:40.706517372Z 
2021-06-02T11:04:40.706674136Z =SUPERVISOR REPORT==== 2-Jun-2021::11:04:40.705581 ===
2021-06-02T11:04:40.706679101Z     supervisor: {local,capi_sup}
2021-06-02T11:04:40.706682321Z     errorContext: shutdown
2021-06-02T11:04:40.706685342Z     reason: reached_max_restart_intensity
2021-06-02T11:04:40.706688356Z     offender: [{pid,<0.106.0>},
2021-06-02T11:04:40.706691987Z                {id,capi_server},
2021-06-02T11:04:40.706695168Z                {mfargs,{capi_server,start_link,[]}},
2021-06-02T11:04:40.706698223Z                {restart_type,permanent},
2021-06-02T11:04:40.706701266Z                {shutdown,2000},
2021-06-02T11:04:40.706704253Z                {child_type,worker}]
2021-06-02T11:04:40.706707374Z =ERROR REPORT==== 2-Jun-2021::11:04:40.705399 ===
2021-06-02T11:04:40.706710504Z ** Generic server kicker_server terminating 
2021-06-02T11:04:40.706713547Z ** Last message in was commit_upgrade
2021-06-02T11:04:40.706716571Z ** When Server state == {state,
2021-06-02T11:04:40.706719709Z                             {xds_ram,
2021-06-02T11:04:40.706722737Z                                 {otts,#Ref<0.3832024404.1817575425.208187>,0,
2021-06-02T11:04:40.706726092Z                                     #Ref<0.3832024404.1817444353.208188>},
2021-06-02T11:04:40.706732068Z                                 140143580589404,0,[],[],[],[],140143580589404,
2021-06-02T11:04:40.706735421Z                                 undefined,undefined,undefined},
2021-06-02T11:04:40.70673852Z                             {pre,{undefined,0}},
2021-06-02T11:04:40.706741591Z                             [nonode@nohost],
2021-06-02T11:04:40.70674474Z                             [{data_kicker,undefined,
2021-06-02T11:04:40.706751985Z                                  [[['http://tail-f.com/ns/kicker'|kickers]]],
2021-06-02T11:04:40.706755215Z                                  [[[1780965560|1612823674]]],
2021-06-02T11:04:40.706758317Z                                  undefined,[],undefined,undefined,
2021-06-02T11:04:40.706761554Z                                  {kicker_server,kicker_data_changed,[]},
2021-06-02T11:04:40.706764695Z                                  [],undefined,undefined,once_for_each}],
2021-06-02T11:04:40.706767779Z                             [],[],false,undefined,4,[],4,
2021-06-02T11:04:40.706770846Z                             [{0,<0.169.0>},
2021-06-02T11:04:40.706774144Z                              {1,<0.170.0>},
2021-06-02T11:04:40.706777246Z                              {2,<0.171.0>},
2021-06-02T11:04:40.706780338Z                              {3,<0.172.0>},
2021-06-02T11:04:40.706783406Z                              {4,<0.173.0>},
2021-06-02T11:04:40.706786613Z                              {5,<0.174.0>},
2021-06-02T11:04:40.706789685Z                              {6,<0.175.0>},
2021-06-02T11:04:40.706792738Z                              {7,<0.176.0>},
2021-06-02T11:04:40.70679581Z                              {8,<0.177.0>},
2021-06-02T11:04:40.706798868Z                              {9,<0.178.0>},
2021-06-02T11:04:40.706802051Z                              {10,<0.179.0>},
2021-06-02T11:04:40.706805185Z                              {11,<0.180.0>},
2021-06-02T11:04:40.70681089Z                              {12,<0.181.0>},
2021-06-02T11:04:40.706814043Z                              {13,<0.182.0>},
2021-06-02T11:04:40.706817147Z                              {14,<0.183.0>},
2021-06-02T11:04:40.706820223Z                              {15,<0.184.0>}]}
2021-06-02T11:04:40.706823325Z ** Reason for termination ==
2021-06-02T11:04:40.706826374Z ** {function_clause,[{cs_trans,stop,
2021-06-02T11:04:40.706829383Z                                [undefined],
2021-06-02T11:04:40.706832447Z                                [{file,"cs_trans.erl"},{line,688}]},
2021-06-02T11:04:40.70683563Z                      {kicker_server,terminate,2,
2021-06-02T11:04:40.706838742Z                                     [{file,"kicker_server.erl"},{line,528}]},
2021-06-02T11:04:40.706842Z                      {gen_server,try_terminate,3,
2021-06-02T11:04:40.706845166Z                                  [{file,"gen_server.erl"},{line,689}]},
2021-06-02T11:04:40.70684833Z                      {gen_server,terminate,10,
2021-06-02T11:04:40.706851406Z                                  [{file,"gen_server.erl"},{line,874}]},
2021-06-02T11:04:40.706864414Z                      {proc_lib,init_p_do_apply,3,
2021-06-02T11:04:40.70686759Z                                [{file,"proc_lib.erl"},{line,249}]}]}
2021-06-02T11:04:40.70687079Z ** Client confd_upgrade stacktrace
2021-06-02T11:04:40.706873803Z ** [{gen,do_call,4,[{file,"gen.erl"},{line,169}]},
2021-06-02T11:04:40.706876973Z     {gen_server,call,3,[{file,"gen_server.erl"},{line,230}]},
2021-06-02T11:04:40.706880217Z     {lists,foreach,2,[{file,"lists.erl"},{line,1338}]},
2021-06-02T11:04:40.70688346Z     {confd_upgrade,terminate,2,[{file,"confd_upgrade.erl"},{line,348}]},
2021-06-02T11:04:40.706886676Z     {gen_server,try_terminate,3,[{file,"gen_server.erl"},{line,689}]},
2021-06-02T11:04:40.706889871Z     {gen_server,terminate,10,[{file,"gen_server.erl"},{line,874}]},
2021-06-02T11:04:40.706893018Z     {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,720}]},
2021-06-02T11:04:40.70689617Z     {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]
2021-06-02T11:04:40.706900145Z 
2021-06-02T11:04:40.706903665Z =ERROR REPORT==== 2-Jun-2021::11:04:40.705876 ===
2021-06-02T11:04:40.706907395Z failed to allocate running th when exiting upgrade {{{badmatch,error},
2021-06-02T11:04:40.706911161Z                                                      [{capi_server,
2021-06-02T11:04:40.706914864Z                                                        handle_info,2,
2021-06-02T11:04:40.706918012Z                                                        [{file,
2021-06-02T11:04:40.706922142Z                                                          "capi_server.erl"},
2021-06-02T11:04:40.706925464Z                                                         {line,1389}]},
2021-06-02T11:04:40.706929398Z                                                       {gen_server,
2021-06-02T11:04:40.70693262Z                                                        try_dispatch,4,
2021-06-02T11:04:40.706937143Z                                                        [{file,
2021-06-02T11:04:40.706940402Z                                                          "gen_server.erl"},
2021-06-02T11:04:40.706943663Z                                                         {line,653}]},
2021-06-02T11:04:40.706946771Z                                                       {gen_server,handle_msg,
2021-06-02T11:04:40.706949928Z                                                        6,
2021-06-02T11:04:40.706953237Z                                                        [{file,
2021-06-02T11:04:40.706956372Z                                                          "gen_server.erl"},
2021-06-02T11:04:40.706959547Z                                                         {line,727}]},
2021-06-02T11:04:40.706962662Z                                                       {proc_lib,
2021-06-02T11:04:40.706969868Z                                                        init_p_do_apply,3,
2021-06-02T11:04:40.706973245Z                                                        [{file,"proc_lib.erl"},
2021-06-02T11:04:40.706976441Z                                                         {line,249}]}]},
2021-06-02T11:04:40.706979557Z                                                     {gen_server,call,
2021-06-02T11:04:40.706982645Z                                                      [capi_server,
2021-06-02T11:04:40.706985717Z                                                       {new_trans,running,
2021-06-02T11:04:40.706988833Z                                                        1188,cli,read,2488,
2021-06-02T11:04:40.706991952Z                                                        unhide_inactive},
2021-06-02T11:04:40.706995066Z                                                       infinity]}}
2021-06-02T11:04:40.706998221Z <ERR> 2-Jun-2021::11:04:40.706 com-cm-yang-provider-547cf5c97b-c6dh8 confd[152]: confd Socket connection from "capi" rejected - probably due to ConfD being in phase0 or service not enabled in config file

... ...

2021-06-02T11:04:41.715855679Z =ERROR REPORT==== 2-Jun-2021::11:04:41.715275 ===
2021-06-02T11:04:41.715877217Z init:boot_msg: "Internal error: Supervision terminated\n"
2021-06-02T11:04:43.222261404Z "Internal error: Component terminated (application_controller) ({application_terminated,capi,shutdown})\n"
2021-06-02T11:04:43.735729471Z Ended with exit code 19.

[cohult]

Seems like a dataprovider issue.

See my reply to a similar issue that you posted in March:

Erlang ERROR REPORT "case_clause" when committing a in-service upgrade

Hi, From the confd_lib_lib(3) man page: CONFD_ERR_EXTERNAL (19) All errors that originate in user code. Often due to a data provider (DP API) application that returns CONFD_ERR from one of the callbacks it serves when ConfD call them or the data provider application crash or has crashed. Regards

[hzpfly]

Thank you for the quick response.
Any idea on what kind of dataprovider issue?

[cohult]

2021-06-02T11:04:40.690508815Z <ERR> 2-Jun-2021::11:04:40.690 com-cm-yang-provider-547cf5c97b-c6dh8 confd[152]: devel-c close_trans/"cpa" error {external, ""}

likely means that the data provider application trans.finish callback failed for some reason. For example by returning CONFD_ERR

Data provider application errors are best debugged using the developer log at trace level (that you show a snippet from) + the libconfd data provider application log at CONFD_PROTO_TRACE level. Or with the CONFD_TRACE level where you provide the timestamps yourself.

[hzpfly]

In order to find the callback which returns CONFD_ERROR, I compared the two logs of data provider. And found the maapi_commit_upgrade succeeded and its return code is CONFD_OK. But data provider didn’t receive the upgrade event of type 4.
I am sure about this because in the following code snippet, the last log is printed:

	log.Info("Commit upgrade")
	if err := conn.CommitUpgrade(); err != nil {
		log.WithError(err).Error("Failed to commit upgrade")
		return err
	}

	log.Info("Succeeded")

Would you please explain why this happened? And another question: Does the close_trans error cause ConfD crash?

Here is the log of data provider when upgrade is ok:

{"context":"loadYangSchemas","message":"Notify Confd about schema changes","service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T08:54:13.493Z","version":"0.2.0"}
{"message":"Waiting for phase 2","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.493Z","version":"0.2.0"}
{"message":"Load schemas","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.493Z","version":"0.2.0"}
{"message":"Start user session","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.519Z","version":"0.2.0"}
{"message":"partial lock lockId 216","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.52Z","version":"0.2.0"}
{"message":"Initializing upgrade","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.52Z","version":"0.2.0"}
{"message":"Upgrade event type: 1","notif_fd":23,"service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T08:54:13.521Z","version":"0.2.0"}
{"message":"Upgrade event type: 2","notif_fd":23,"service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T08:54:13.522Z","version":"0.2.0"}
{"message":"Perform upgrade","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.522Z","version":"0.2.0"}
{"message":"Upgrade event type: 3","notif_fd":23,"service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T08:54:13.655Z","version":"0.2.0"}
{"message":"Commit upgrade","service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T08:54:13.655Z","version":"0.2.0"}
{"NewValidatorSession":"tmp log","message":"","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.699Z","version":"0.2.0"}
{"message":"Validator.txInit","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.7Z","version":"0.2.0"}
{"db":"running","fd":587,"message":"lockEarly:false","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.7Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"message":"CANDIDATE_ENABLED: false","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.7Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"message":"tx_init Tx worker successfully created/connected","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.7Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"message":"Validator.validate","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.7Z","version":"0.2.0"}
{"db":"running","fd":587,"func":"tx_lock","message":"enter write_lock","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.701Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"func":"tx_lock","message":"Lock","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.701Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"func":"tx_lock","message":"enter checkBeginDbSession","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.701Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","error":"no etag header found","fd":587,"func":"tx_lock","message":"Failed to find expected etag","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.708Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"body":"{\"result\":\"created\",\"id\":\"64450f2b354abdfcf8787557d0803703\"}\n","db":"running","etag":"","fd":587,"func":"tx_lock","message":"POST","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","status":201,"timestamp":"2021-06-02T08:54:13.708Z","tx":-2,"tx_mode":"READ_WRITE","url":"https://com-cm-backend:443/v1/tx","version":"0.2.0"}
{"NewValidatorSession":"tmp log","message":"","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.708Z","version":"0.2.0"}
{"db":"running","fd":587,"func":"tx_lock","message":"exit checkBeginDbSession, cleared accessors/validators","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.708Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"func":"tx_lock","message":"exit write_lock","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.708Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"message":"enter go_confd_validation_init","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.709Z","version":"0.2.0"}
{"message":"exit go_confd_validation_init","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.709Z","version":"0.2.0"}
{"message":"go_confd_validation_execute","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.709Z","version":"0.2.0"}
{"db":"running","fd":587,"func":"go_confd_validation_execute","message":"enter","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.709Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"message":"TxChangeIterator.CanApply","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.709Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"message":"TxChangeIterator.CanApply userContext: system, dbname: running","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.709Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"message":"TxChangeIterator.Apply TX not for validation","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.709Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"func":"go_confd_validation_execute","message":"exit","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.71Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"message":"go_confd_validation_execute","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.71Z","version":"0.2.0"}
{"db":"running","fd":587,"func":"go_confd_validation_execute","message":"enter","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.71Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"message":"TxChangeIterator.CanApply","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.71Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"message":"TxChangeIterator.CanApply userContext: system, dbname: running","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.71Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"message":"TxChangeIterator.Apply TX not for validation","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.71Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"func":"go_confd_validation_execute","message":"exit","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.71Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"func":"tx_write_start","message":"enter write_start","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.71Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"func":"tx_write_start","message":"exit write_start","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.71Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"message":"enter-exit go_confd_validation_stop","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.711Z","version":"0.2.0"}
{"db":"running","fd":587,"func":"tx_commit","message":"enter tx_commit","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.713Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"from":2,"func":"tx_commit","message":"Invalid state change","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.713Z","to":6,"tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","error":"invalid state change","fd":587,"func":"tx_commit","message":"Error in empty transaction commit","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"warning","timestamp":"2021-06-02T08:54:13.713Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"func":"tx_commit","message":"exit tx_commit","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.713Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":587,"func":"tx_finish","message":"Tx Finish","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.714Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"NewValidatorSession":"tmp log","message":"","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.714Z","version":"0.2.0"}
{"db":"running","error":"no etag header found","fd":587,"func":"tx_lock","message":"Failed to find expected etag","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.72Z","tx":"64450f2b354abdfcf8787557d0803703","tx_mode":"READ_WRITE","version":"0.2.0"}
{"body":"{\"result\":\"deleted\",\"id\":\"64450f2b354abdfcf8787557d0803703\"}\n","db":"running","etag":"","fd":587,"func":"tx_lock","message":"POST","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","status":200,"timestamp":"2021-06-02T08:54:13.72Z","tx":"64450f2b354abdfcf8787557d0803703","tx_mode":"READ_WRITE","url":"https://com-cm-backend:443/v1/tx/64450f2b354abdfcf8787557d0803703/abort","version":"0.2.0"}
{"db":"running","fd":587,"message":"Closing Tx worker","service_id":"com-cm-yang-provider","session_id":1150,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.72Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"NewValidatorSession":"tmp log","message":"","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.732Z","version":"0.2.0"}
{"message":"Validator.txInit","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.732Z","version":"0.2.0"}
{"db":"running","fd":677,"message":"lockEarly:false","service_id":"com-cm-yang-provider","session_id":1151,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.732Z","tx":2419,"tx_mode":"READ","version":"0.2.0"}
{"db":"running","fd":677,"message":"CANDIDATE_ENABLED: false","service_id":"com-cm-yang-provider","session_id":1151,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.732Z","tx":2419,"tx_mode":"READ","version":"0.2.0"}
{"db":"running","fd":677,"message":"tx_init Tx worker successfully created/connected","service_id":"com-cm-yang-provider","session_id":1151,"session_type":"system","severity":"debug","timestamp":"2021-06-02T08:54:13.732Z","tx":2419,"tx_mode":"READ","version":"0.2.0"}
{"message":"Upgrade event type: 4","notif_fd":23,"service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T08:54:13.733Z","version":"0.2.0"}
{"message":"Succeeded","service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T08:54:13.733Z","version":"0.2.0"}
{"message":"partal unlock lockId 216","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.733Z","version":"0.2.0"}
{"message":"Reload AAA","service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T08:54:13.734Z","version":"0.2.0"}
{"message":"partial lock lockId 217","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.735Z","version":"0.2.0"}
{"message":"partal unlock lockId 217","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T08:54:13.735Z","version":"0.2.0"}
{"message":"Reload AAA Succeeded","service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T08:54:13.735Z","version":"0.2.0"}

And here is the log of data provider when ConfD crashed:

{"context":"loadYangSchemas","message":"Notify Confd about schema changes","service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T11:04:40.335Z","version":"0.2.0"}
{"message":"Waiting for phase 2","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.336Z","version":"0.2.0"}
{"message":"Load schemas","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.336Z","version":"0.2.0"}
{"message":"Start user session","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.405Z","version":"0.2.0"}
{"message":"partial lock lockId 224","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.407Z","version":"0.2.0"}
{"message":"Initializing upgrade","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.407Z","version":"0.2.0"}
{"message":"Upgrade event type: 1","notif_fd":23,"service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T11:04:40.408Z","version":"0.2.0"}
{"db":"running","fd":25,"func":"tx_finish","message":"Tx Finish","service_id":"com-cm-yang-provider","session_id":1188,"session_type":"cli","severity":"debug","timestamp":"2021-06-02T11:04:40.408Z","tx":2473,"tx_mode":"READ","version":"0.2.0"}
{"NewValidatorSession":"tmp log","message":"","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.408Z","version":"0.2.0"}
{"db":"running","fd":25,"message":"Closing Tx worker","service_id":"com-cm-yang-provider","session_id":1188,"session_type":"cli","severity":"debug","timestamp":"2021-06-02T11:04:40.408Z","tx":2473,"tx_mode":"READ","version":"0.2.0"}
{"message":"Upgrade event type: 2","notif_fd":23,"service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T11:04:40.409Z","version":"0.2.0"}
{"message":"Perform upgrade","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.409Z","version":"0.2.0"}
{"message":"Upgrade event type: 3","notif_fd":23,"service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T11:04:40.541Z","version":"0.2.0"}
{"message":"Commit upgrade","service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T11:04:40.541Z","version":"0.2.0"}
{"cmd":"/opt/confd/bin/confdc --yangpath /opt/adp/yang -c /tmp/yangArchive952954455/com-brm-ext-adp.yang --use-description -o /tmp/yangArchive952954455/com-brm-ext-adp.fxs --yangpath /tmp/yangArchive952954455","message":"Command exec succeeded!","service_id":"com-cm-yang-provider","severity":"debug","stderr":"","stdout":"","timestamp":"2021-06-02T11:04:40.602Z","version":"0.2.0"}
{"file":"/tmp/yangArchive952954455/com-brm-ext-adp.yang","message":"Compilation of YANG file successful","service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T11:04:40.602Z","version":"0.2.0"}
{"NewValidatorSession":"tmp log","message":"","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.614Z","version":"0.2.0"}
{"message":"Validator.txInit","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.614Z","version":"0.2.0"}
{"db":"running","fd":25,"message":"lockEarly:false","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.614Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"message":"CANDIDATE_ENABLED: false","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.614Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"message":"tx_init Tx worker successfully created/connected","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.614Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"message":"Validator.validate","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.614Z","version":"0.2.0"}
{"db":"running","fd":25,"func":"tx_lock","message":"enter write_lock","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.615Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"func":"tx_lock","message":"Lock","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.615Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"func":"tx_lock","message":"enter checkBeginDbSession","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.615Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","error":"no etag header found","fd":25,"func":"tx_lock","message":"Failed to find expected etag","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.622Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"body":"{\"result\":\"created\",\"id\":\"db10724e06daee7f8304e7085fbe1a23\"}\n","db":"running","etag":"","fd":25,"func":"tx_lock","message":"POST","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","status":201,"timestamp":"2021-06-02T11:04:40.622Z","tx":-2,"tx_mode":"READ_WRITE","url":"https://com-cm-backend:443/v1/tx","version":"0.2.0"}
{"NewValidatorSession":"tmp log","message":"","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.622Z","version":"0.2.0"}
{"db":"running","fd":25,"func":"tx_lock","message":"exit checkBeginDbSession, cleared accessors/validators","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.622Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"func":"tx_lock","message":"exit write_lock","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.622Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"message":"enter go_confd_validation_init","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.624Z","version":"0.2.0"}
{"message":"exit go_confd_validation_init","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.624Z","version":"0.2.0"}
{"message":"go_confd_validation_execute","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.624Z","version":"0.2.0"}
{"db":"running","fd":25,"func":"go_confd_validation_execute","message":"enter","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.624Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"message":"TxChangeIterator.CanApply","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.624Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"message":"TxChangeIterator.CanApply userContext: system, dbname: running","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.624Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"message":"TxChangeIterator.Apply TX not for validation","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.624Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"func":"go_confd_validation_execute","message":"exit","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.624Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"message":"go_confd_validation_execute","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.625Z","version":"0.2.0"}
{"db":"running","fd":25,"func":"go_confd_validation_execute","message":"enter","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.625Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"message":"TxChangeIterator.CanApply","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.625Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"message":"TxChangeIterator.CanApply userContext: system, dbname: running","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.625Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"message":"TxChangeIterator.Apply TX not for validation","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.625Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"func":"go_confd_validation_execute","message":"exit","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.625Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"func":"tx_write_start","message":"enter write_start","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.625Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"func":"tx_write_start","message":"exit write_start","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.625Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"message":"enter-exit go_confd_validation_stop","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.626Z","version":"0.2.0"}
{"db":"running","fd":25,"func":"tx_commit","message":"enter tx_commit","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.63Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"from":2,"func":"tx_commit","message":"Invalid state change","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.63Z","to":6,"tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","error":"invalid state change","fd":25,"func":"tx_commit","message":"Error in empty transaction commit","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"warning","timestamp":"2021-06-02T11:04:40.63Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"func":"tx_commit","message":"exit tx_commit","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.63Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"db":"running","fd":25,"func":"tx_finish","message":"Tx Finish","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.681Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
{"NewValidatorSession":"tmp log","message":"","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.681Z","version":"0.2.0"}
{"cmd":"/opt/confd/bin/confdc --yangpath /opt/adp/yang -c /tmp/yangArchive952954455/com-brm.yang --use-description -o /tmp/yangArchive952954455/com-brm.fxs --yangpath /tmp/yangArchive952954455 --deviation /tmp/yangArchive952954455/com-brm-ext-adp.yang -F com-brm:housekeeping,scheduler","message":"Command exec succeeded!","service_id":"com-cm-yang-provider","severity":"debug","stderr":"","stdout":"","timestamp":"2021-06-02T11:04:40.69Z","version":"0.2.0"}
{"file":"/tmp/yangArchive952954455/com-brm.yang","message":"Compilation of YANG file successful","service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T11:04:40.69Z","version":"0.2.0"}
{"message":"Compilation successful","name":"com-brm","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.691Z","tx":"afb2ba9a639dbbcda3eed48b2457dce9","url":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/v1/yang","RawPath":"","ForceQuery":false,"RawQuery":"tx=afb2ba9a639dbbcda3eed48b2457dce9","Fragment":"","RawFragment":""},"version":"0.2.0"}
{"db":"running","error":"no etag header found","fd":25,"func":"tx_lock","message":"Failed to find expected etag","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.693Z","tx":"db10724e06daee7f8304e7085fbe1a23","tx_mode":"READ_WRITE","version":"0.2.0"}
{"body":"{\"result\":\"deleted\",\"id\":\"db10724e06daee7f8304e7085fbe1a23\"}\n","db":"running","etag":"","fd":25,"func":"tx_lock","message":"POST","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","status":200,"timestamp":"2021-06-02T11:04:40.693Z","tx":"db10724e06daee7f8304e7085fbe1a23","tx_mode":"READ_WRITE","url":"https://com-cm-backend:443/v1/tx/db10724e06daee7f8304e7085fbe1a23/abort","version":"0.2.0"}
{"db":"running","fd":25,"message":"Closing Tx worker","service_id":"com-cm-yang-provider","session_id":1200,"session_type":"system","severity":"debug","timestamp":"2021-06-02T11:04:40.693Z","tx":-2,"tx_mode":"READ_WRITE","version":"0.2.0"}
DEBUG EOF on socket to ConfD
DEBUG EOF on socket to ConfD
{"message":"Socket closed","ret":-2,"service_id":"com-cm-yang-provider","severity":"error","timestamp":"2021-06-02T11:04:40.705Z","version":"0.2.0"}
{"message":"Trying to connect to Confdvalidation control","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.705Z","version":"0.2.0"}
{"message":"Succeeded","service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T11:04:40.707Z","version":"0.2.0"}
{"message":"partal unlock lockId 224","service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.707Z","version":"0.2.0"}
DEBUG internal error - 
{"confd_errno":18,"confd_errno_str":"internal error","confd_lasterror":0,"message":"maapi partial unlock failed","ret":-1,"service_id":"com-cm-yang-provider","severity":"debug","timestamp":"2021-06-02T11:04:40.709Z","version":"0.2.0"}
{"message":"Reload AAA","service_id":"com-cm-yang-provider","severity":"info","timestamp":"2021-06-02T11:04:40.709Z","version":"0.2.0"}
DEBUG internal error - 
{"error":"failed to start user session","message":"Failed to reload NACM cache","service_id":"com-cm-yang-provider","severity":"error","timestamp":"2021-06-02T11:04:40.71Z","version":"0.2.0"}
{"error":"failed to start user session","message":"Failed to reload AAA","service_id":"com-cm-yang-provider","severity":"fatal","timestamp":"2021-06-02T11:04:40.71Z","version":"0.2.0"}
Ended with exit code 1.