Signed-off-by: Yong Zhao yozhao@microsoft.com
Why I did it
This PR aims to monitor the critical processes in PMon container by Monit in 201911 branch.
How I did it
I created a template configuration file of Monit and it will be rendered to generate Monit configuration file of PMon container
by a service generate_monit_config.service.
How to verify it
I verified this on a Mellanox device str-msn2700-03 and an Arista device str-a7050-acs-1.
Which release branch to backport (provide reason below if selected)
201811
[x ] 201911
202006
202012
Increase startretires value from default of 10 to 50 to prevent supervisor from placing thermalctld in FATAL state during regression testing. Also ensures supervisord tries hard to get thermalctld running in production, as thermalctld is critical to prevent device from overheating.
When stopping the swss, pmon or bgp containers, log messages like the following can be seen:
```
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,061 ERRO pool dependent-startup event buffer overflowed, discarding event 34
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,063 ERRO pool dependent-startup event buffer overflowed, discarding event 35
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,064 ERRO pool dependent-startup event buffer overflowed, discarding event 36
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,066 ERRO pool dependent-startup event buffer overflowed, discarding event 37
```
This is due to the number of programs in the container managed by supervisor, all generating events at the same time. The default event queue buffer size in supervisor is 10. This patch increases that value in all containers in order to eliminate these errors. As more programs are added to the containers, we may need to further adjust these values. I increased all buffer sizes to 25 except for containers with more programs or templated supervisor.conf files which allow for a variable number of programs. In these cases I increased the buffer size to 50. One final exception is the swss container, where the buffer fills up to ~50, so I increased this buffer to 100.
Resolves https://github.com/Azure/sonic-buildimage/issues/5241
Copy proper fancontrol config file to the proper destination. Also some minor refactoring for code reuse to help prevent issues like this in the future.
Fixes a bug introduced by #4599
Extend the PMON daemon start control to lm-sensors and fancontrol.
change template docker-pmon.supervisord.conf.j2 and start.sh.j2 to have lm-sensors and fancontrol start scripts and supervisord config file controlled by pmon_daemon_control.json.
the intention is to avoid wrong daemon status in "supervisorctl status" output. For example, on some platform, if there is no fancontrol config file, and it is not ruled out from supervisord conf file and start.sh, we'll see fancontrol in "STOPPED" status from "supervisorctl status" output, which will violate some check in the platform test(check daemon status as expected)
- create a dockerfile-marcros.j2 file with all common operations
written as j2 macro
- use single dockerfile instruction for COPY and RUN commands
when possible to improve build time
- reorganize dockerfile instructions to make more cache friendly
(in case someday we will remove --no-cache to build docker images)
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
* Fix showing systemd shutdown sequence when verbose is set
* Fix creation of kernel-cmdline file
Sometimes boot0 prints error
"mv: can't preserve ownership of '/mnt/flash/image-arsonic.xxxx/kernel-cmdline': Operation not permitted"
* Improve flash space usage during installation
Some older systems only have 2GB of flash available. Installing a second
image on these can prove to be challenging.
The new installation process moves the installer swi to memory in order
to avoid free up space from the flash before uncompressing it there.
It removes all the flash space usage spike and also improves the IO
since the installation is no more reading and writting to the flash at
the same time.
* Add support of 7060CX-32S-SSD
* 7260CX3: use inventory powerCycle procedures
* 7050QX-32S: use inventory powerCycle procedures
* 7050QX-32: use inventory powerCycle procedures
* platform: arista: add common platform_reboot
Replace platform_reboot by a link to new common for devices already
using a similar script.
* 7060CX-32S: use inventory powerCycle procedures
* Install python smbus in pmon
Some platform plugin need the python smbus library to perform some actions.
This installs the dependency.
* [build]: put stretch debian packages under target/debs/stretch/
* in stretch build phase, all debian packages built in that stage are placed under target/debs/stretch directory.
* for python-based debian packages, since they are really the same for jessie and stretch, they are placed under target/python-debs directory.
Signed-off-by: Guohan Lu <gulv@microsoft.com>
* Add psud daemon to pmon container
* Update submodule sonic-platform-daemons
Submodule update sonic-platform-daemons:
e5d8155 - [sonic-psud] add a new daemon sonic-psud to platform monitor (#20)
Signed-off-by: Kevin Wang <kevinw@mellanox.com>
Previously use / to separate container name and program name.
However, in rsyslogd:
Precisely, the programname is terminated by either (whichever occurs first):
end of tag
nonprintable character
‘:’
‘[‘
‘/’
The above definition has been taken from the FreeBSD syslogd sources.
Signed-off-by: Guohan Lu <gulv@microsoft.com>
* Added new platform support DellEMC - Z92264f - 64x100
* Includes changes with Makefiles, sfputil, eeprom and default minigraph
* Led support for Z9264f platform
* Includes changes on default minigraph
* ipmitool implementation in pmon docker. platform_sensors script is inclued in pmon startup
* [docker-platform-monitor] make file and supervisord conf change for new xcvrd deamon
* make file change for the new daemon
* supervisord conf change for the new daemon
signed-off-by Liu Kebo kebol@mellanox.com
* make xcvrd start sequence aligned with the supervisord conf
* update submodules to include xcvrd modification
- Consolidate config.sh and start.sh scripts into one script (start.sh)
- Solve issue #435 - All dockers now run supervisord as their ENTRYPOINT
- All stdout/stderr output from processes managed by supervisord is now sent to syslog instead of their own files
- Supervisord log messages are now also sent to syslog
- Removed unused smartmontools package from docker-platform-monitor
