Commit Graph

19 Commits

Author SHA1 Message Date
Joe LeVeque
72b32a96fc
[201911][dockers][supervisor] Increase event buffer size for process exit listener (#7106)
Backport of https://github.com/Azure/sonic-buildimage/pull/7083 to the 201911 branch.

#### Why I did it

To prevent error [messages](https://dev.azure.com/mssonic/build/_build/results?buildId=2254&view=logs&j=9a13fbcd-e92d-583c-2f89-d81f90cac1fd&t=739db6ba-1b35-5485-5697-de102068d650&l=802) like the following from being logged:

```
Mar 17 02:33:48.523153 vlab-01 INFO swss#supervisord 2021-03-17 02:33:48,518 ERRO pool supervisor-proc-exit-listener event buffer overflowed, discarding event 46
```

This is basically an addendum to https://github.com/Azure/sonic-buildimage/pull/5247, which increased the event buffer size for dependent-startup. While supervisor-proc-exit-listener doesn't subscribe to as many events as dependent-startup, there is still a chance some containers (like swss, as in the example above) have enough processes running to cause an overflow of the default buffer size of 10.

This is especially important for preventing erroneous log_analyzer failures in the sonic-mgmt repo regression tests, which have started occasionally causing PR check builds to fail. Example [here](https://dev.azure.com/mssonic/build/_build/results?buildId=2254&view=logs&j=9a13fbcd-e92d-583c-2f89-d81f90cac1fd&t=739db6ba-1b35-5485-5697-de102068d650&l=802).

I set all supervisor-proc-exit-listener event buffer sizes to 1024, and also updated all dependent-startup event buffer sizes to 1024, as well, to keep things simple, unified, and allow headroom so that we will not need to adjust these values frequently, if at all.
2021-03-29 10:07:43 -07:00
Joe LeVeque
b70c6f72b2 [dockers][supervisor] Increase event buffer size for dependent-startup (#5247)
When stopping the swss, pmon or bgp containers, log messages like the following can be seen:

```
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,061 ERRO pool dependent-startup event buffer overflowed, discarding event 34
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,063 ERRO pool dependent-startup event buffer overflowed, discarding event 35
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,064 ERRO pool dependent-startup event buffer overflowed, discarding event 36
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,066 ERRO pool dependent-startup event buffer overflowed, discarding event 37
```

This is due to the number of programs in the container managed by supervisor, all generating events at the same time. The default event queue buffer size in supervisor is 10. This patch increases that value in all containers in order to eliminate these errors. As more programs are added to the containers, we may need to further adjust these values. I increased all buffer sizes to 25 except for containers with more programs or templated supervisor.conf files which allow for a variable number of programs. In these cases I increased the buffer size to 50. One final exception is the swss container, where the buffer fills up to ~50, so I increased this buffer to 100.

Resolves https://github.com/Azure/sonic-buildimage/issues/5241
2020-09-28 16:12:53 +00:00
Guohan Lu
569766f698 [docker-snmp-sv2]: use service dependency in supervisord to start services
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-08-15 22:32:19 -07:00
yozhao101
71225ea4cc [Service] Enable/disable container auto-restart based on configuration. (#4073) 2020-02-13 16:20:21 -08:00
yozhao101
a0fbeeaca5 [Services] Restart SNMP service upon unexpected critical process exit. (#3650)
Signed-off-by: Yong Zhao <yozhao@microsoft.com>
2019-10-22 14:41:12 -07:00
Joe LeVeque
1e1add90f9
Remove Arista-specific service ACL solution; All platforms now use caclmgrd (#2202) 2018-10-29 10:25:18 -07:00
Qi Luo
5817e99520 Revert "Remove snmpd hw_fsys module, remove disk monitoring which is not in use" (#1665)
* Revert "[swss]: update swss submodule (#1664)"

This reverts commit c74be37ae0.

* Revert "Remove snmpd hw_fsys module, remove disk monitoring which are not in use (#1663)"

This reverts commit 3d61df9d75.
2018-05-01 18:32:49 -07:00
Qi Luo
3d61df9d75 Remove snmpd hw_fsys module, remove disk monitoring which are not in use (#1663)
Signed-off-by: Qi Luo <qiluo-msft@users.noreply.github.com>
2018-04-30 23:06:51 -07:00
Joe LeVeque
6ccd1601b8
Modify Arista service ACL solution to listen to ACL changes in ConfigDB (#1385) 2018-02-12 11:10:01 -08:00
byu343
ecf5c8d311 ssh and snmp allow list (#1363)
- Service ACL framework for Arista platforms
2018-02-08 17:43:52 -08:00
Qi Luo
bec99f257a Disable snmpd module disk_hw, so no syslog messages about unavailable disks (#977)
* Revert "snmpd ignores unavailable disks (#607)"

This reverts commit d70e973496.

* Disable snmpd module disk_hw, so no syslog messages about unavailable disks

Signed-off-by: Qi Luo <qiluo-msft@users.noreply.github.com>

* Strip white spaces

Signed-off-by: Qi Luo <qiluo-msft@users.noreply.github.com>
2017-09-21 17:31:39 -07:00
Joe LeVeque
f49cac086f Remove extra trailing newlines at EOF (#804)
Files now end with a single newline
2017-07-12 20:54:37 -07:00
Qi Luo
ec273c8892 Disable snmpd native ipCidrRouteTable implementation (#675)
The SNMP subagent implements ipCidrRouteTable. If there is no nexthops
for default route, ie. switch isolated, the SNMP subagent will response
'no instance'. Then snmpd will response kernel routes, which include
non-front panel interfaces. To remove snmpd native implementation, we
only expose SNMP subagent's implementation.
2017-06-06 22:58:58 -07:00
Joe LeVeque
d5c13c0a83 [dockers]: Disable autorestart on all supervisor processes inside containers (#580) 2017-05-09 17:37:08 -07:00
Joe LeVeque
8f348399f5 [Dockers]: Manage all Docker containers with Supervisord (#573)
- Consolidate config.sh and start.sh scripts into one script (start.sh)
 - Solve issue #435 - All dockers now run supervisord as their ENTRYPOINT
 - All stdout/stderr output from processes managed by supervisord is now sent to syslog instead of their own files
 - Supervisord log messages are now also sent to syslog
 - Removed unused smartmontools package from docker-platform-monitor
2017-05-08 15:43:31 -07:00
Qi Luo
7a2a374917 [docker-snmp]: Disable snmpd IP module and use snmp subagent implementation (#447)
* Disable snmpd ip module, will use snmp subagent implementation
* Update git submodule
2017-03-28 15:48:29 -07:00
Qi Luo
815bf53de7 snmpd disables inetCidrRouteTable module (#286) 2017-02-10 10:40:09 -08:00
Qi Luo
822680d21b Build python3.6 deb packages and install them into docker-snmp-sv2 (#245) 2017-02-08 22:02:21 -08:00
thomasbo
135ba232ca SNMP/LLDP Containers: Sonic V2 Support (#41)
* Adding support for V2 in SNMP/LLDP (-sv2 postfix)
* Fixes for V1 containers: logging
* Fixes for V1 LLDP: limit LLDP to Front-panel or MGMT interfaces.
2016-10-28 15:19:29 -07:00