[S6100] Improve S6100 serial-getty monitor, wait and re-check when getty not running to avoid false alert.
#### Why I did it
On S6100, the serial-getty service some time can't auto-restart by systemd. So there is a monit unit to check serial-getty service status and restart it.
However, this monit will report false alert, because in most case when serial-getty not running, systemd can restart it successfully.
To avoid the false alert, improve the monitor to wait and re-check.
Steps to reproduce this issue:
1. User login to device via console, and keep the connection.
2. User login to device via SSH, check the serial-getty@ttyS1.service service, it's running.
3. Run 'monit reload' from SSH connection.
4. Check syslog 1 minutes later, there will be false alert: ' 'serial-getty' process is not running'
#### How I did it
Add check-getty.sh script to recheck again later when getty service not running.
And update monit unit to check serial-getty service status with this script to avoid false alert.
#### How to verify it
Pass all UT.
Manually check fixed code work correctly:
```
admin@***:~$ sudo systemctl stop serial-getty@ttyS1.service
admin@***:~$ sudo /usr/local/bin/check-getty.sh
admin@***:~$ echo $?
1
admin@***:~$ sudo systemctl status serial-getty@ttyS1.service
● serial-getty@ttyS1.service - Serial Getty on ttyS1
Loaded: loaded (/lib/systemd/system/serial-getty@.service; enabled-runtime; vendor preset: enabled)
Active: inactive (dead) since Tue 2023-03-28 07:15:21 UTC; 1min 13s ago
admin@***:~$ sudo /usr/local/bin/check-getty.sh
admin@***:~$ echo $?
0
admin@***:~$ sudo systemctl status serial-getty@ttyS1.service
● serial-getty@ttyS1.service - Serial Getty on ttyS1
Loaded: loaded (/lib/systemd/system/serial-getty@.service; enabled-runtime; vendor preset: enabled)
```
syslog:
```
Mar 28 07:10:37.597458 *** INFO systemd[1]: serial-getty@ttyS1.service: Succeeded.
Mar 28 07:12:43.010550 *** ERR monit[593]: 'serial-getty' status failed (1) -- no output
Mar 28 07:12:43.010744 *** INFO monit[593]: 'serial-getty' trying to restart
Mar 28 07:12:43.010846 *** INFO monit[593]: 'serial-getty' stop: '/bin/systemctl stop serial-getty@ttyS1.service'
Mar 28 07:12:43.132172 *** INFO monit[593]: 'serial-getty' start: '/bin/systemctl start serial-getty@ttyS1.service'
Mar 28 07:13:43.286276 *** INFO monit[593]: 'serial-getty' status succeeded (0) -- no output
```
#### Description for the changelog
[S6100] Improve S6100 serial-getty monitor.
#### Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.
Why I did it
smartctl tool is available only in PMON docker. Hence, the tool may be not accessible incase PMON docker goes down.
Using iSMART_64 tool to fetch the SSD firmware version and device model information.
How I did it
Replacing smartctl with iSMART_64.
To reduce rc.local script execution time. Porting changes from [DellEMC] S6100 Platform Service optimization #10989
Changes:
Moving platform-modules-s6100.service and s6100-lpc-monitor.service asynchronous to rc.local script.
Why I did it
To include ONIE version in show platform firmware status command output in DellEMC S6100 and Z9332f platforms.
How I did it
Include ‘ONIE’ in the list of components provided by platform APIs in DellEMC S6100 and Z9332f.
Unmount ONIE-BOOT if mounted using fast/soft/warm-reboot plugins in DellEMC S6100.
Why I did it
To reduce the processing time of rc.local, refactoring s6100 platform initialization.
Porting changes from 202012 branch [202012] Refactoring DELL platform init to reduce rc.local processing time #10171
Why I did it
To incorporate the below changes in DellEMC S6100, S6000 platforms.
S6100, S6000:
Implement 'get_revision' method for Chassis
Implement 'get_maximum_consumed_power' method for FanDrawer
Implement 'get_revision', 'get_maximum_supplied_power' methods for PSU
Implement 'get_error_description' method for SFP.
S6100:
Implement 'get_module_index' method for Chassis
Implement 'get_description', 'get_maximum_consumed_power', 'get_oper_status', 'get_slot' methods for Module
Update component names in platform.json
How I did it
Implement the platform API methods in the respective device files
How to verify it
Verified that the respective sonic-mgmt platform API test cases report success.
Why I did it
Adding SSD as part of platform components list.
Introducing platform_fw_au_reboot_handle to use auto-update functionality in fwutil
How I did it
Changes to be committed:
(use "git restore --staged <file>..." to unstage)
modified: platform/broadcom/sonic-platform-modules-dell/debian/platform-modules-s6100.install
new file: platform/broadcom/sonic-platform-modules-dell/s6100/scripts/platform_fw_au_reboot_handle
modified: platform/broadcom/sonic-platform-modules-dell/s6100/sonic_platform/chassis.py
modified: platform/broadcom/sonic-platform-modules-dell/s6100/sonic_platform/component.py
How to verify it
By running fwutil command.
Warning: fwupdate_fwimage_dir: /var/platform/fwpackage/.
Chassis Module Component Firmware Version (Current/Available) Status
--------- -------- ----------- ------------------------------- ----------------------------- ------------------
S6100-ON BIOS S6100-BIOS-3.25.0.2-9-noRP2.bin 3.25.0.2-8 / 3.25.0.2-9 update is required
FPGA smf_firmware_upgrade.tar 2.4 / 2.4 up-to-date
CPLD cpld_firmware_upgrade.tar 4 / 4 up-to-date
SSD ssd_firmware_upgrade.tar S16425cG / S16425cG up-to-date
root@sonic:~#
Why I did it
To support iTCO watchdog using watchdog APIs.
How I did it
Implemented a new watchdog class WatchdogTCO for interfacing with iTCO watchdog.
Updated reboot cause determination logic.
How to verify it
Verified that the watchdog APIs' return values are as expected.
Logs: UT_logs.txt
Why I did it
To monitor the SSD health condition in DellEMC S6100 platform post upgrade.
A daemon is introduced to monitor the SSD every one hour.
To check for SSD status at boot time and at the time of cold-reboot.
All these changes are supported only for newer SSD firmware.
Porting changes from 201911 branch
Added a platform_reboot_pre_check script to prevent cold-reboot based on SSD status.
Depends on Azure/sonic-utilities#1556
DO NOT MERGE UNTIL ABOVE PR IS MERGED
Why I did it
Added a check in determining CPU reset in fast-/warm-reboot in their respective platform plugin.
Introducing reboot plugin for "reboot" command to handle its own platform plugin.
How I did it
On branch s6100_fast_warm_check
Changes to be committed:
(use "git reset HEAD ..." to unstage)
modified: ../../debian/platform-modules-s6100.install
modified: ../scripts/fast-reboot_plugin
modified: ../scripts/platform_reboot_override
new file: ../scripts/reboot_plugin
modified: ../scripts/track_reboot_reason.sh
modified: chassis.py
How to verify it
Triggered cold-reset inside fast-reboot to test out the reboot-cause 2.0 API.
Why I did it
serial-getty service exited in Dell S6100 device randomly.
How I did it
Added serial-getty to monit services.
How to verify it
Stop serial-getty in ssh session and check whether the service restarts or not.
Why I did it
To determine the revision of the pcie.yaml to be used based on BIOS version in DellEMC S6100 platform.
Depends on: Azure/sonic-platform-common#195
How I did it
Added two revisions of pcie.yaml pcie_1.yaml and pcie_2.yaml
Included a platform-specific Pcie class to provide the revision of the pcie.yaml to be used by pcieutil/pcied.
How to verify it
Execute pcieutil check (Azure/sonic-utilities#1672) command and verify the list of PCIe devices displayed.
Logs: UT_logs.txt
#### Why I did it
- xcvrd crash was seen in latest 201811 images.
- For Dell S6100,API 2.0 uses poll mode while 1.0 was still using interrupt mode.
#### How I did it
- Modified get_transceiver_change_event in 1.0 to poll mode.
- Why I did it
To collect platform based logs along with "show techsupport" on S6000 and S6100 plaforms.
- How I did it
On branch dell_techsupport_dump
Changes to be committed:
(use "git reset HEAD ..." to unstage)
new file: platform/broadcom/sonic-platform-modules-dell/common/actions.sh
modified: platform/broadcom/sonic-platform-modules-dell/debian/platform-modules-s6000.install
modified: platform/broadcom/sonic-platform-modules-dell/debian/platform-modules-s6100.install
new file: platform/broadcom/sonic-platform-modules-dell/s6000/scripts/hw-management-generate-dump.sh
new file: platform/broadcom/sonic-platform-modules-dell/s6100/scripts/hw-management-generate-dump.sh
- How to verify it
hw-mgmt-dump.tar.gz will be found in sonic_dump__< YYYYMMDD_HHMMSS>.tar.gz.
In preparation for the merging of Azure/sonic-platform-common#173, which properly defines class and instance members in the Platform API base classes.
It is proper object-oriented methodology to call the base class initializer, even if it is only the default initializer. This also future-proofs the potential addition of custom initializers in the base classes down the road.
**- Why I did it**
To incorporate the below changes in DellEMC S6100, S6000 platforms.
- S6100, S6000:
- Enable 'thermalctld'
- Implement DeviceBase methods (presence, status, model, serial) for Fantray and Component
- Implement ‘get_position_in_parent’, ‘is_replaceable’ methods for all device types
- Implement ‘get_status’ method for Fantray
- Implement ‘get_temperature’, ‘get_temperature_high_threshold’, ‘get_voltage_high_threshold’, ‘get_voltage_low_threshold’ methods for PSU
- Implement ‘get_status_led’, ‘set_status_led’ methods for Chassis
- SFP:
- Make EEPROM read both Python2 and Python3 compatible
- Fix ‘get_tx_disable_channel’ method’s return type
- Implement ‘tx_disable’, ‘tx_disable_channel’ and ‘set_power_override’ methods
- S6000:
- Move PSU thermal sensors from Chassis to respective PSU
- Make available the data of both Fans present in each Fantray
**- How I did it**
- Remove 'skip_thermalctld:true' in pmon_daemon_control.json
- Implement the platform API methods in the respective device files
- Use `bytearray` for data read from transceiver EEPROM
- Change return type of 'get_tx_disable_channel' to match specification in sonic_platform_common/sfp_base.py
Add a systemd dependency to make platform-modules service as a prerequisite for determine-reboot-cause service to ensure platform initialization is complete before determine-reboot-cause.service executes.
- Why I did it
For determining reboot-cause while running newer BIOS, SMF firmware.
- How I did it
Made changes in reboot-cause determination script to add support for behavior of newer firmware.
- How to verify it
Performed different type of resets and verified "show reboot-cause" provides the correct reason.
Logs: UT_logs.txt
- Description for the changelog
DellEMC S6100: Update reboot-cause determination to support new firmware
- Change return type of SFP methods to match specification in sonic_platform_common/sfp_base.py
- Use init methods of base classes to initialize common instance variables
- Handle negative timeout values in S6100's watchdog ‘arm’ method
- Return appropriate values for 'get_target_speed', 'set_status_led' to avoid false warnings
- Implement FanDrawer and get_high_critical_threshold Platform API for S6000, S6100, Z9100 and Z9264F.
- Fix incorrect fan direction values in S6100, Z9100
- Make DellEMC platform modules Python3 compliant.
- Change return type of PSU Platform APIs in DellEMC Z9264, S5232 and Thermal Platform APIs in S5232 to 'float'.
- Remove multiple copies of pcisysfs.py.
- PEP8 style changes for utility scripts.
- Build and install Python3 version of sonic_platform package.
- Fix minor Platform API issues.
- Why I did it
For fixing PCA MUX attachment issue in Dell S6100 platform.
- How I did it
Wait till IOM MUX powered up properly and start I2C enumeration.
Align SFP key names with new standard defined in https://github.com/Azure/sonic-platform-common/pull/97
- hardwarerev -> hardware_rev
- serialnum -> serial
- manufacturename -> manufacturer
- modelname -> model
- Connector -> connector
- Xilinx/pericom peripherals are not actively used in DellEMC S6100 switch.
- These peripherals are throwing PCIE corrected messages in some of the units and filling syslog.
- Since it is not usable disabling it at startup.
- Skip thermalctld in DellEMC S6000, S6100, Z9100 and Z9264 platforms.
- Change the return type of thermal Platform APIs in DellEMC S6000, S6100, Z9100 and Z9264 platforms to 'float'.
For detecting transceiver change events through xcvrd in DellEMC S6000, S6100 and Z9100 platforms.
- In S6000, rename 'get_transceiver_change_event' in chassis.py to 'get_change_event' and return appropriate values.
- In S6100, implement 'get_change_event' through polling method (poll interval = 1 second) in chassis.py (Transceiver insertion/removal does not generate interrupts due to a CPLD bug)
- In Z9100, implement 'get_change_event' through interrupt method using select.epoll().
- optoe driver truncates invalid pages(ff) but sff driver doesn't truncate.so,the DOM related calculation made by sff8436 driver will show incorrect data.
- Few optics doesn't support DOM.
- SFP plugins currently returns None for unreadable pages and this'd throw the below mentioned error in sfpshow eeprom --dom.
Implement Watchdog platform2.0 API for DellEMC S6100 platform.
- Added new file watchdog.py in sonic_platform directory.
- Enabled API support to Enable/disable watchdog.
