Commit Graph

12 Commits

Author SHA1 Message Date
Joe LeVeque
5b3b4804ad
[dockers][supervisor] Increase event buffer size for dependent-startup (#5247)
When stopping the swss, pmon or bgp containers, log messages like the following can be seen:

```
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,061 ERRO pool dependent-startup event buffer overflowed, discarding event 34
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,063 ERRO pool dependent-startup event buffer overflowed, discarding event 35
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,064 ERRO pool dependent-startup event buffer overflowed, discarding event 36
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,066 ERRO pool dependent-startup event buffer overflowed, discarding event 37
```

This is due to the number of programs in the container managed by supervisor, all generating events at the same time. The default event queue buffer size in supervisor is 10. This patch increases that value in all containers in order to eliminate these errors. As more programs are added to the containers, we may need to further adjust these values. I increased all buffer sizes to 25 except for containers with more programs or templated supervisor.conf files which allow for a variable number of programs. In these cases I increased the buffer size to 50. One final exception is the swss container, where the buffer fills up to ~50, so I increased this buffer to 100.

Resolves https://github.com/Azure/sonic-buildimage/issues/5241
2020-09-08 23:36:38 -07:00
pavel-shirshov
1eb3dfe541
[docker-teamd]: Introducing tlm_teamd: telemetry for teamd (#4824)
**- What I did**
1. Updated submodule sonic-swss to bring tlm_teamd to the buildimage.
2. Updated supervisord for the teamd
3. Updated critical process list (not sure that tlm_teamd is critical for now)

**- How to verify it**
Build an image and run. Check that tlm_teamd is running and STATE_DB has information in the LAG_INTERFACE, and :LAG_MEMBER_INTERFACE
```
admin@sonic:~$ redis-cli -n 6 hgetall 'LAG_TABLE|PortChannel16'
 1) "state"
 2) "ok"
 3) "team_device.ifinfo.dev_addr"
 4) "4c:76:25:f5:48:80"
 5) "setup.kernel_team_mode_name"
 6) "loadbalance"
 7) "team_device.ifinfo.ifindex"
 8) "6"
 9) "runner.fast_rate"
10) "false"
11) "runner.active"
12) "true"
13) "setup.pid"
14) "35"
15) "runner.fallback"
16) "false"
```

```
admin@sonic:~$ redis-cli -n 6 hgetall 'LAG_MEMBER_TABLE|PortChannel16|Ethernet16'
 1) "runner.selected"
 2) "true"
 3) "runner.aggregator.selected"
 4) "true"
 5) "runner.aggregator.id"
 6) "26"
 7) "runner.actor_lacpdu_info.state"
 8) "61"
 9) "runner.state"
10) "current"
11) "runner.actor_lacpdu_info.system"
12) "4c:76:25:f5:48:80"
13) "runner.partner_lacpdu_info.state"
14) "61"
15) "link.up"
16) "true"
17) "ifinfo.dev_addr"
18) "4c:76:25:f5:48:80"
19) "ifinfo.ifindex"
20) "26"
21) "link_watches.list.link_watch_0.up"
22) "true"
23) "runner.actor_lacpdu_info.port"
24) "17"
25) "runner.partner_lacpdu_info.port"
26) "1"
27) "runner.partner_lacpdu_info.system"
28) "52:54:00:ff:34:1b"
```
2020-06-27 01:22:23 -07:00
Guohan Lu
448d1cdd49 [docker-teamd]: use service dependency in supervisord to start services 2020-05-22 11:01:28 -07:00
yozhao101
91e5fb5602
[Service] Enable/disable container auto-restart based on configuration. (#4073) 2020-02-07 12:34:07 -08:00
judyjoseph
c95d8e4d47
[teamd]: increase startsecs to 5 seconds for teamsyncd (#4083)
Updating the startsecs=5sec for teamsyncd to make the time for which the process needs to stay up before declaring the startup successfull.
2020-01-30 00:46:55 -08:00
yozhao101
4c31ef3cd2 [Services] Restart Teamd service upon unexpected critical process exit. (#3703)
Signed-off-by: Yong Zhao <yozhao@microsoft.com>
2019-11-04 17:45:41 -08:00
Shuotian Cheng
7313e7d9bc [teamd]: Add teammgrd in docker-teamd (#2064)
Remove the teamd.j2 templates used for starting the teamd. Add
teammgrd instead to manage all port channel related configuration
changes. Remove front panel port related configurations in
interfaces.j2 templates as well.

Remove teamd.sh script and use teammgrd to start all the teamd
processes. Remove all the logics in the start.sh script as well.

Update the sonic-swss submodule.

Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
2018-10-19 03:41:53 -07:00
Shuotian Cheng
c93c008bae
Revert "[docker-teamd]: Manage teamd and teamsyncd processes with supervisor (#1137)" (#1156)
This reverts commit a6edef2fa5.

The reason to revert this commit is that it breaks the current nightly test as
no port channel interfaces are get created after boot. teamd failed to start and
complained about 'Cannot allocate memory' possibly due to nlmsg_alloc function
failure.

Will revert this commit to investigate it further before moving to supervisor.

Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
2017-11-16 10:30:41 -08:00
Joe LeVeque
a6edef2fa5
[docker-teamd]: Manage teamd and teamsyncd processes with supervisor (#1137) 2017-11-13 14:16:19 -08:00
Joe LeVeque
f49cac086f Remove extra trailing newlines at EOF (#804)
Files now end with a single newline
2017-07-12 20:54:37 -07:00
Joe LeVeque
d5c13c0a83 [dockers]: Disable autorestart on all supervisor processes inside containers (#580) 2017-05-09 17:37:08 -07:00
Joe LeVeque
8f348399f5 [Dockers]: Manage all Docker containers with Supervisord (#573)
- Consolidate config.sh and start.sh scripts into one script (start.sh)
 - Solve issue #435 - All dockers now run supervisord as their ENTRYPOINT
 - All stdout/stderr output from processes managed by supervisord is now sent to syslog instead of their own files
 - Supervisord log messages are now also sent to syslog
 - Removed unused smartmontools package from docker-platform-monitor
2017-05-08 15:43:31 -07:00