Backport #15461
#### Why I did it
* To fix `hiredis` compilation
#### How I did it
* Changed package version: `0.14.0-3~bpo9+1` -> `0.14.1-1`
#### How to verify it
1. make configure PLATFORM=mellanox
2. make target/sonic-mellanox.bin
#### Tested branch (Please provide the tested image version)
- [X] 202012 <!-- image version 1 -->
Why I did it
Refine PR test template format.
How I did it
Refine PR test template format.
How to verify it
PR test executed normally.
Signed-off-by: Chun'ang Li <chunangli@microsoft.com>
Cherry pick PR for https://github.com/sonic-net/sonic-host-services/pull/62
#### Why I did it
Fix the issue https://github.com/sonic-net/sonic-buildimage/issues/10883.
##### Work item tracking
- Microsoft ADO **(17795594)**:
#### How I did it
For performance reason, libswsscommon is not thread safe by design.
caclmgrd share config DB connection cross thread, so change to use new db connector in child thread.
#### How to verify it
Load scale ipv4/ipv6 rules and verify if caclmgrd is crashed
#### Why I did it
Aikido FPD update, power CPLD version display under fwutil CLI and fix for cpu lockup causing telemetry container crash or system reboot
##### Work item tracking
- Microsoft ADO **(number only)**: 24174212
#### How I did it
update cisco module to 202012.3.1.1
BACKPORT OF https://github.com/sonic-net/sonic-buildimage/pull/14925
#### Why I did it
ISSU version check fails due to inability to mount squashfs from 202211 on 201911
#### How I did it
Put ISSU version file under platform directory
#### How to verify it
202012 (with [202012][mlnx-ffb.sh] Update issu-version location #14927) to master
This reverts commit 02b17839c3.
Reverts #14933
The earlier commit caused a race condition that particularly broke cross branch warm upgrade.
Issue happens when db_migrator is still migrating the DB and finalizer is checking DB for list of components to reconcile.
If migration is not complete, finalizer get an empty list to wait for. Due to this, finalizer concludes warmboot (deletes system wide warmboot flag) and cause all the services to do cold restart.
ADO: 24274591
#### Why I did it
fix possible cpld race read issue between watchdog and reboot cause process
##### Work item tracking
- Microsoft ADO **(number only)**:
#### How I did it
Use flock to limit parallel access to cpld sys file
#### How to verify it
It can be simulate and verified with following python script
```python3
import signal
import subprocess
import threading
exit_flag = False
def run_command(cmd):
status = True
result = ""
try:
p = subprocess.Popen(
cmd, shell=True, universal_newlines=True,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
raw_data, err = p.communicate()
if err == '':
result = raw_data.strip()
except:
status = False
return status, result
def get_cpld_reg_value(getreg_path, register):
#cmd = "echo {1} > {0}; cat {0}".format(getreg_path, register)
cmd = "flock {0} -c 'echo {1} > {0}; cat {0}'".format(getreg_path,
register)
status, result = run_command(cmd)
return result if status else None
def cpld_read(thread_num, cpld_reg):
while not exit_flag:
val
= get_cpld_reg_value("/sys/devices/platform/dx010_cpld/getreg",
cpld_reg)
print(f"Thread {thread_num}: get cpld reg {cpld_reg}, value
{val}")
def signal_handler(sig, frame):
global exit_flag
print("Ctrl+C detected. Quitting...")
exit_flag = True
if __name__ == '__main__':
# Register the signal handler for Ctrl+C
signal.signal(signal.SIGINT, signal_handler)
t1 = threading.Thread(target=cpld_read, args=(1, '0x103',))
t2 = threading.Thread(target=cpld_read, args=(2, '0x141',))
t1.start()
t2.start()
t1.join()
t2.join()
```
Why I did it
Fix all mirror is commented out in sources.list in slave image issue. It will have an issue when installing more packages in the slave container.
It will add additional space character after running add-apt-repository command.
For example:
The original config in /etc/apt/sources.list
#deb [arch=amd64] http://deb.debian.org/debian/ bullseye main contrib non-free
Run the following command:
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable"
Then the setting changed to: (added a new space character after #)
# deb [arch=amd64] http://deb.debian.org/debian/ bullseye main contrib non-free
How I did it
Fix the regex string to add the space pattern. After fixed, whether there is a space character or not, it will not be an issue.
How to verify it
Co-authored-by: xumia <59720581+xumia@users.noreply.github.com>
Why I did it
Set build options in pipeline UI.
Support setting reproducible build options to py2,py3 in release branch and none in master branch.
Work item tracking
Microsoft ADO (number only): 22335854
How I did it
How to verify it
Why I did it
Cherry-pick of #11827
This is to fix issue: [201811->202012] During warm recovery, TOR did not announce Loopback, VLAN route after upgrade
Suspected cause: 202012 does not have system dependency for bgp service to start after interfaces-config.service.
This opens a window for race condition: bgp service completing before interfaces are initialized.
BGP will miss announcing some routes if the interfaces are not ready.
Why I did it
Fix the issue where db_migrator is called before DB is loaded w/ config. This leads to db_migrator:
Not finding anything, and resumes to incorrectly migrate every missing config
This is not expected. migration should happen after the old config is loaded and only new schema changes need migration.
Since DB does not have anything when migrator is called, db_migrator fails when some APIs return None.
The reason for incorrect call is that:
database service starts db_migrator as part of startup sequence.
config-setup service loads data from old-config/minigraph. However, since it has Requires=database.service.
Hence, config-setup starts only when database service is started. And database service is started when db_migrator is completed.
Fixed by:
Check if this is first time boot by checking pending_config_migration flag.
If pending_config_migration is enabled, then do not call db_migrator as part of database service startup.
Let database service start which triggers config-setup service to start.
Now call db_migrator after when config-setup service loads old-config/minigraph
Why I did it
Release Notes for Cisco 8102-64H:
Updated mtd-utils.mk and pyudev.mk for addressing build failures
How I did it
Update platform version to 202012.3.0.1
Why I did it
Run kvmtest when update package versions to avoid test break.
Work item tracking
Microsoft ADO (number only): 22335854
How I did it
How to verify it
#### Why I did it
Fix endless build log issue.
Cherry pick [PR#11846](https://github.com/sonic-net/sonic-buildimage/pull/11846)
##### Work item tracking
- Microsoft ADO **(number only)**: 19299131
#### How I did it
The current error handling code for when a deb package fails to be installed currently has a chain of commands linked together by && and ends with `exit 1`. The assumption is that the commands would succeed, and the last `exit 1` would end it with a non-zero return code, thus fully failing the target and causing the build to stop because of bash's -e flag.
However, if one of the commands prior to `exit 1` returns a non-zero return code, then bash won't actually treat it as a terminating error. From bash's man page:
```
-e Exit immediately if a pipeline (which may consist of a single simple
command), a list, or a compound command (see SHELL GRAMMAR above),
exits with a non-zero status. The shell does not exit if the
command that fails is part of the command list immediately
following a while or until keyword, part of the test following the
if or elif reserved words, part of any command executed in a && or
|| list except the command following the final && or ||, any
command in a pipeline but the last, or if the command's return
value is being inverted with !. If a compound command other than a
subshell returns a non-zero status because a command failed while
-e was being ignored, the shell does not exit.
```
The part `part of any command executed in a && or || list except the command following the final && or ||` says that if the failing command is not the `exit 1` that we have at the end, then bash doesn't treat it as an error and exit immediately. Additionally, since this is a compound command, but isn't in a subshell (subshell are marked by `(` and `)`, whereas `{` and `}` just tells bash to run the commands in the current environment), bash doesn't exist. The result of this is that in the deb-install target, if a package installation fails, it may be infinitely stuck in that while-loop.
There are two fixes for this: change to using a subshell, or use `;` instead of `&&`. Using a subshell would, I think, require exporting any shell variables used in the subshell, so I chose to change the `&&` to `;`. In addition, at the start of the subshell, `set +e` is added in, which removes the exit-on-error handling of bash. This makes sure that all commands are run (the output of which may help for debugging) and that it still exits with 1, which will then fully fail the target.
#### How to verify it
Why I did it
Advance sonic-utilities submodule head
Added below commits:
878be48e kellyyeh Wed May 10 15:21:52 2023 -0700 Revert "[warm-reboot] Use kexec_file_load instead of kexec_load when available
094513f8 Vaibhav Hemant Dixit Tue May 9 13:03:52 2023 -0700 [202012] LAG keepalive script to reduce lacp session wait during warm-reboot
Work item tracking
Microsoft ADO (number only): 23687678
Why I did it
Remove 'kvmtest-t0' and 'kvmtest-t1-lag' test jobs since all the test jobs are required (continueOnError: false) already, and will only enable one of classical and testbedV2 tests, no need to do an unnecessary 'or' compute test job.
Change agent pool to reduce cost and avoid congestion
* To resolve NEIGH table entries present in CONFIG_DB. Without this change arp/ndp entries which we wish to resolve, and configured via CONFIG_DB are not resolved.
Why I did it
src/sonic-py-swsssdk
* d44e0d8 - (HEAD -> 202012, origin/202012) [Security] Fix the redis security issue CVE-2023-28858 and CVE-2023-28859 (#135) (3 days ago) [xumia]
#### Why I did it
[Build] Upgrade the python docker version to fix bgp not up issue
##### Work item tracking
- Microsoft ADO **(number only)**: 22236397
Using timer-override.conf, we modify the fstrim.timer service.
For armhf, Nokia-7215 platform, we modify fstrim.timer to run daily
instead of weekly. This is required because the size of the SSD on
this platform is 16GB, which on average is nearly 10 times smaller than
most other sonic platforms. With smaller disk and the ever increasing
level of logging done by sonic, this change is required to prevent
the SSD from entering a read-only state due to inadequate free blocks.
#### Why I did it
sonic-utilities submodule update for 202012
```
* d20fc3c8 2023-04-07 | [202012][DBMigrator] Update db_migrator to support EdgeZoneAggregator Buffer Config for T0s (#2768) (HEAD, origin/202012) [Dev Ojha]
* 322a74dd 2023-03-27 | Resolved rc!=0 problem by replacing fgrep with awk. Added ipv4 filtering to get only v4 peers in case of show ip bgp neighbors (#2743) [saurabhab]
```
##### Work item tracking
- Microsoft ADO **(number only)**: 20782336
Why I did it
Fix similar issue seen on #13739 but only for DCS-7050CX3-32S
How I did it
Add a kernel parameter to tell libata to disable NCQ
How to verify it
The message ata2.00: FORCE: horkage modified (noncq) should appear on the dmesg.
Test results using: fio --direct=1 --rw=randrw --bs=64k --ioengine=libaio --iodepth=64 --runtime=120 --numjobs=4
with NCQ
READ: bw=26.1MiB/s (27.4MB/s), 26.1MiB/s-26.1MiB/s (27.4MB/s-27.4MB/s), io=3136MiB (3288MB), run=120053-120053msec
WRITE: bw=26.3MiB/s (27.6MB/s), 26.3MiB/s-26.3MiB/s (27.6MB/s-27.6MB/s), io=3161MiB (3315MB), run=120053-120053msec
without NCQ
READ: bw=22.0MiB/s (23.1MB/s), 22.0MiB/s-22.0MiB/s (23.1MB/s-23.1MB/s), io=2647MiB (2775MB), run=120069-120069msec
WRITE: bw=22.2MiB/s (23.3MB/s), 22.2MiB/s-22.2MiB/s (23.3MB/s-23.3MB/s), io=2665MiB (2795MB), run=120069-120069msec
Why I did it
Innovium platform has build issue.
Remove it from upgrade version pipeline.
Work item tracking
Microsoft ADO (number only):
How I did it
How to verify it
[S6100] Improve S6100 serial-getty monitor, wait and re-check when getty not running to avoid false alert.
This is cherry-pick PR for: https://github.com/sonic-net/sonic-buildimage/pull/14402
#### Why I did it
On S6100, the serial-getty service some time can't auto-restart by systemd. So there is a monit unit to check serial-getty service status and restart it.
However, this monit will report false alert, because in most case when serial-getty not running, systemd can restart it successfully.
To avoid the false alert, improve the monitor to wait and re-check.
Steps to reproduce this issue:
1. User login to device via console, and keep the connection.
2. User login to device via SSH, check the serial-getty@ttyS1.service service, it's running.
3. Run 'monit reload' from SSH connection.
4. Check syslog 1 minutes later, there will be false alert: ' 'serial-getty' process is not running'
##### Work item tracking
- Microsoft ADO :17424426
#### How I did it
Add check-getty.sh script to recheck again later when getty service not running.
And update monit unit to check serial-getty service status with this script to avoid false alert.
#### How to verify it
Pass all UT.
Manually check fixed code work correctly:
```
admin@***:~$ sudo systemctl stop serial-getty@ttyS1.service
admin@***:~$ sudo /usr/local/bin/check-getty.sh
admin@***:~$ echo $?
1
admin@***:~$ sudo systemctl status serial-getty@ttyS1.service
● serial-getty@ttyS1.service - Serial Getty on ttyS1
Loaded: loaded (/lib/systemd/system/serial-getty@.service; enabled-runtime; vendor preset: enabled)
Active: inactive (dead) since Tue 2023-03-28 07:15:21 UTC; 1min 13s ago
admin@***:~$ sudo /usr/local/bin/check-getty.sh
admin@***:~$ echo $?
0
admin@***:~$ sudo systemctl status serial-getty@ttyS1.service
● serial-getty@ttyS1.service - Serial Getty on ttyS1
Loaded: loaded (/lib/systemd/system/serial-getty@.service; enabled-runtime; vendor preset: enabled)
```
syslog:
```
Mar 28 07:10:37.597458 *** INFO systemd[1]: serial-getty@ttyS1.service: Succeeded.
Mar 28 07:12:43.010550 *** ERR monit[593]: 'serial-getty' status failed (1) -- no output
Mar 28 07:12:43.010744 *** INFO monit[593]: 'serial-getty' trying to restart
Mar 28 07:12:43.010846 *** INFO monit[593]: 'serial-getty' stop: '/bin/systemctl stop serial-getty@ttyS1.service'
Mar 28 07:12:43.132172 *** INFO monit[593]: 'serial-getty' start: '/bin/systemctl start serial-getty@ttyS1.service'
Mar 28 07:13:43.286276 *** INFO monit[593]: 'serial-getty' status succeeded (0) -- no output
```
#### Tested branch (Please provide the tested image version)
- [x] 20201231.77
#### Description for the changelog
[S6100] Improve S6100 serial-getty monitor.
#### Why I did it
Cherry-pick commits from master to support the snapshot based mirror, and fix the code conflicts. And add the last commit to fix the build broken issue according to the mirror change.
ad162ae0e [Build] Optimize the version control for Debian packages (https://github.com/sonic-net/sonic-buildimage/pull/14557)
38c5d7fce [Build] Support j2 template for debian sources for docker ptf (https://github.com/sonic-net/sonic-buildimage/pull/13198)
5e4826ebf [Ci] Support to use the same snapshot for all platform builds (#13913)
820692563 [Build] Change the default mirror version config file (#13786)
5e4a866e3 [Build] Support Debian snapshot mirror to improve build stability (#13097)
ac5d89c6a [Build] Support j2 template for debian sources (#12557)
Update golang version for telemetry build in sonic-slave-jessie to fix CVE-2021-33195, this PR will be merged into 201911 branch finally.
#### Why I did it
Go before 1.15.13 and 1.16.x before 1.16.5 has functions for DNS lookups that do not validate replies from DNS servers, and thus a return value may contain an unsafe injection (e.g., XSS) that does not conform to the RFC1035 format. Now in 201911 and 202012 branch we're using 1.14.2
##### Work item tracking
- Microsoft ADO **(number only)**:17727291
#### How I did it
Bump golang version into 1.15.15 which contains corresponding fix.
#### How to verify it
unit test to do sanity check.
