05f1a5a31e
Add watchdog mechanism to swss service and generate alert when swss have issue. **Work item tracking** Microsoft ADO (number only): 16578912 **What I did** Add orchagent watchdog to monitor and alert orchagent stuck issue. **Why I did it** Currently SONiC monit system only monit orchagent process exist or not. If orchagent process stuck and stop processing, current monit can't find and report it. **How I verified it** Pass all UT. Manually test process_monitoring/test_critical_process_monitoring.py can pass. Add new UT https://github.com/sonic-net/sonic-mgmt/pull/8306 to check watchdog works correctly. Manually test, after pause orchagent with 'kill -STOP <pid>', check there are warning message exist in log: Apr 28 23:36:41.504923 vlab-01 ERR swss#supervisor-proc-watchdog-listener: Process 'orchagent' is stuck in namespace 'host' (1.0 minutes). **Details if related** Heartbeat message PR: https://github.com/sonic-net/sonic-swss/pull/2737 UT PR: https://github.com/sonic-net/sonic-mgmt/pull/8306 |
||
---|---|---|
.. | ||
arp_update | ||
asic_status.py | ||
asic_status.sh | ||
bgp.sh | ||
configdb-load.sh | ||
core_cleanup.py | ||
database.sh | ||
gbsyncd-platform.sh | ||
gbsyncd.sh | ||
lldp.sh | ||
mark_dhcp_packet.py | ||
mgmt-framework.sh | ||
radv.sh | ||
service_mgmt.sh | ||
snmp.sh | ||
sonic-netns-exec | ||
supervisor-proc-exit-listener | ||
swss.sh | ||
syncd_common.sh | ||
syncd.sh | ||
teamd.sh | ||
telemetry.sh | ||
update_chassisdb_config | ||
write_standby.py |