* [Build] Fix the PyYang python package installation issue (#15890)
Why I did it
Fix the armhf build failure.
How to reproduce the issue:
docker run -it debain:bullseye bash
apt-get update && apt-get install -y python3-pip
pip3 install PyYAML==5.4.1
Error message:
Collecting PyYAML==5.4.1
Installing build dependencies ... done
Getting requirements to build wheel ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 /tmp/tmp6xabslgb_in_process.py get_requires_for_build_wheel /tmp/tmp_er01ztl
....
raise AttributeError(attr)
AttributeError: cython_sources
----------------------------------------
WARNING: Discarding d63f2d7597/PyYAML-5.4.1.tar.gz (sha256)=607774cbba28732bfa802b54baa7484215f530991055bb562efbed5b2f20a45e (from https://pypi.org/simple/pyyaml/) (requires-python:>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*). Command errored out with exit status 1: /usr/bin/python3 /tmp/tmp6xabslgb_in_process.py get_requires_for_build_wheel /tmp/tmp_er01ztl Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement PyYAML==5.4.1
ERROR: No matching distribution found for PyYAML==5.4.1
root@fa2fa92edcfd:/#
But if adding the option --no-build-isolation, then it is good, see fix.
install "PyYAML==5.4.1" --no-build-isolation
The same error can be found in the multiple builds.
Work item tracking
Microsoft ADO (number only): 24567457
How I did it
Add a build option --no-build-isolation.
* Fix docker-platform-monitor python2 issue
* Fix wheel dependency issue
* Fix wheel dependency issue
* Fix the pip2 PyYAML in host image
#### Why I did it
cherry pick, #15535
Graceful restart is a key event for bgpd, related log print is debug level. To change it to info level to get more visibilities when this kind of event is triggered.
##### Work item tracking
- Microsoft ADO **(13875291)**:
#### How I did it
To create patch file to change from debug level to info level.
#### How to verify it
To run PR test and capture the print.
Fix libtacsupport.so can't parse tacplus_nss.conf issue and not reset server list before parse config file issue.
##### Work item tracking
- Microsoft ADO **(number only)**: 24433713
#### Why I did it
1. Fix libtacsupport.so can't parse tacplus_nss.conf correctly issue:
Support debug=on setting.
Support put server address and secret in same row.
2. Fix the parse_config_file method not reset server list before parse config file issue.
#### How I did it
Fix libtacsupport.so can't parse tacplus_nss.conf issue and not reset server list before parse config file issue.
#### How to verify it
UT with CUnit cover all code in this plugin.
Also pass all current UT.
#### Which release branch to backport (provide reason below if selected)
N/A
#### Tested branch (Please provide the tested image version)
Extract tacacs support functions into library, this will share TACACS config file parse code with other project.
Also fix memory leak issue in parse config code.
- [ ] SONiC.202012-15723.312602-e230e2d3e
#### Description for the changelog
Fix libtacsupport.so can't parse tacplus_nss.conf issue and not reset server list before parse config file issue.
This pull request integrate audisp-tacplus to SONiC for per-command accounting.
##### Work item tracking
- Microsoft ADO **(number only)**: 24433713
#### Why I did it
To support TACACS per-command accounting, we integrate audisp-tacplus project to sonic.
#### How I did it
1. Add auditd service to SONiC
2. Port and patch audisp-tacplus to SONiC
#### How to verify it
UT with CUnit to cover all new code in usersecret-filter.c
Also pass all current UT.
#### Tested branch (Please provide the tested image version)
Extract tacacs support functions into library, this will share TACACS config file parse code with other project.
Also fix memory leak issue in parse config code.
- [ ] SONiC.202012-15723.312602-e230e2d3e
#### Description for the changelog
Add audisp-tacplus for per-command accounting.
sonic-utilities submodule update
#### Why I did it
sonic-utilities submodule update:
```
399b1e3 2023-07-06 [202012][Show][BGP] Show BGP Change for no neighbor scenario (#2886)
7b47641 2023-07-10 [[202012] [TACACS+] Add config command for AAA authorization and accounting. (#1889)
```
##### Work item tracking
- Microsoft ADO **(number only)**:24433713
#### How I did it
Update sonic-utilities submodule.
#### How to verify it
Pass all test case.
#### Tested branch (Please provide the tested image version)
- [ ] SONiC.202012-15703.306864-1ef589c19
This pull request add Config DB schema and HostCfg Enforcer plugin to support TACACS+ per-command authorization&accounting.
##### Work item tracking
- Microsoft ADO **(number only)**: 24433713
#### Why I did it
Support TACACS per-command authorization&accounting.
#### How I did it
Change ConfigDB schema and HostCfg enforcer.
Add UT to cover changed code.
#### How to verify it
Build following project and pass all UTs:
make target/python-wheels/sonic_host_services-1.0-py3-none-any.whl
#### Which release branch to backport (provide reason below if selected)
N/A
#### Tested branch (Please provide the tested image version)
Extract tacacs support functions into library, this will share TACACS config file parse code with other project.
Also fix memory leak issue in parse config code.
- [ ] SONiC.202012-15723.309781-38d8852cd
#### Description for the changelog
Add Config DB schema and HostCfg Enforcer plugin to support TACACS+ per-command authorization&accounting.
This pull request extract tacacs support functions into library to share TACACS config file parse code with other project. Also fix memory leak issue in parse config code.
#### Why I did it
To support TACACS per command authorization, we need reuse the TACACS config file parse code in bash plugin project.
##### Work item tracking
- Microsoft ADO **(number only)**: 24433713
#### How I did it
Add libtacsupport.pc.in to extract tacacs support functions into library.
Fix memory leak issue in TACACS config parse code by convert the dynamic memory allocation memory to static memory allocation.
#### How to verify it
Pass all current UT.
Check shared library generated manually.
#### Tested branch (Please provide the tested image version)
Extract tacacs support functions into library, this will share TACACS config file parse code with other project.
Also fix memory leak issue in parse config code.
- [ ] SONiC.202012-15703.306864-1ef589c19
#### Description for the changelog
Extract tacacs support functions into library, this will share TACACS config file parse code with other project.
Also fix memory leak issue in parse config code.
Backport #15461
#### Why I did it
* To fix `hiredis` compilation
#### How I did it
* Changed package version: `0.14.0-3~bpo9+1` -> `0.14.1-1`
#### How to verify it
1. make configure PLATFORM=mellanox
2. make target/sonic-mellanox.bin
#### Tested branch (Please provide the tested image version)
- [X] 202012 <!-- image version 1 -->
Why I did it
Refine PR test template format.
How I did it
Refine PR test template format.
How to verify it
PR test executed normally.
Signed-off-by: Chun'ang Li <chunangli@microsoft.com>
Cherry pick PR for https://github.com/sonic-net/sonic-host-services/pull/62
#### Why I did it
Fix the issue https://github.com/sonic-net/sonic-buildimage/issues/10883.
##### Work item tracking
- Microsoft ADO **(17795594)**:
#### How I did it
For performance reason, libswsscommon is not thread safe by design.
caclmgrd share config DB connection cross thread, so change to use new db connector in child thread.
#### How to verify it
Load scale ipv4/ipv6 rules and verify if caclmgrd is crashed
#### Why I did it
Aikido FPD update, power CPLD version display under fwutil CLI and fix for cpu lockup causing telemetry container crash or system reboot
##### Work item tracking
- Microsoft ADO **(number only)**: 24174212
#### How I did it
update cisco module to 202012.3.1.1
BACKPORT OF https://github.com/sonic-net/sonic-buildimage/pull/14925
#### Why I did it
ISSU version check fails due to inability to mount squashfs from 202211 on 201911
#### How I did it
Put ISSU version file under platform directory
#### How to verify it
202012 (with [202012][mlnx-ffb.sh] Update issu-version location #14927) to master
This reverts commit 02b17839c3.
Reverts #14933
The earlier commit caused a race condition that particularly broke cross branch warm upgrade.
Issue happens when db_migrator is still migrating the DB and finalizer is checking DB for list of components to reconcile.
If migration is not complete, finalizer get an empty list to wait for. Due to this, finalizer concludes warmboot (deletes system wide warmboot flag) and cause all the services to do cold restart.
ADO: 24274591
#### Why I did it
fix possible cpld race read issue between watchdog and reboot cause process
##### Work item tracking
- Microsoft ADO **(number only)**:
#### How I did it
Use flock to limit parallel access to cpld sys file
#### How to verify it
It can be simulate and verified with following python script
```python3
import signal
import subprocess
import threading
exit_flag = False
def run_command(cmd):
status = True
result = ""
try:
p = subprocess.Popen(
cmd, shell=True, universal_newlines=True,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
raw_data, err = p.communicate()
if err == '':
result = raw_data.strip()
except:
status = False
return status, result
def get_cpld_reg_value(getreg_path, register):
#cmd = "echo {1} > {0}; cat {0}".format(getreg_path, register)
cmd = "flock {0} -c 'echo {1} > {0}; cat {0}'".format(getreg_path,
register)
status, result = run_command(cmd)
return result if status else None
def cpld_read(thread_num, cpld_reg):
while not exit_flag:
val
= get_cpld_reg_value("/sys/devices/platform/dx010_cpld/getreg",
cpld_reg)
print(f"Thread {thread_num}: get cpld reg {cpld_reg}, value
{val}")
def signal_handler(sig, frame):
global exit_flag
print("Ctrl+C detected. Quitting...")
exit_flag = True
if __name__ == '__main__':
# Register the signal handler for Ctrl+C
signal.signal(signal.SIGINT, signal_handler)
t1 = threading.Thread(target=cpld_read, args=(1, '0x103',))
t2 = threading.Thread(target=cpld_read, args=(2, '0x141',))
t1.start()
t2.start()
t1.join()
t2.join()
```
Why I did it
Fix all mirror is commented out in sources.list in slave image issue. It will have an issue when installing more packages in the slave container.
It will add additional space character after running add-apt-repository command.
For example:
The original config in /etc/apt/sources.list
#deb [arch=amd64] http://deb.debian.org/debian/ bullseye main contrib non-free
Run the following command:
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable"
Then the setting changed to: (added a new space character after #)
# deb [arch=amd64] http://deb.debian.org/debian/ bullseye main contrib non-free
How I did it
Fix the regex string to add the space pattern. After fixed, whether there is a space character or not, it will not be an issue.
How to verify it
Co-authored-by: xumia <59720581+xumia@users.noreply.github.com>
Why I did it
Set build options in pipeline UI.
Support setting reproducible build options to py2,py3 in release branch and none in master branch.
Work item tracking
Microsoft ADO (number only): 22335854
How I did it
How to verify it
Why I did it
Cherry-pick of #11827
This is to fix issue: [201811->202012] During warm recovery, TOR did not announce Loopback, VLAN route after upgrade
Suspected cause: 202012 does not have system dependency for bgp service to start after interfaces-config.service.
This opens a window for race condition: bgp service completing before interfaces are initialized.
BGP will miss announcing some routes if the interfaces are not ready.
Why I did it
Fix the issue where db_migrator is called before DB is loaded w/ config. This leads to db_migrator:
Not finding anything, and resumes to incorrectly migrate every missing config
This is not expected. migration should happen after the old config is loaded and only new schema changes need migration.
Since DB does not have anything when migrator is called, db_migrator fails when some APIs return None.
The reason for incorrect call is that:
database service starts db_migrator as part of startup sequence.
config-setup service loads data from old-config/minigraph. However, since it has Requires=database.service.
Hence, config-setup starts only when database service is started. And database service is started when db_migrator is completed.
Fixed by:
Check if this is first time boot by checking pending_config_migration flag.
If pending_config_migration is enabled, then do not call db_migrator as part of database service startup.
Let database service start which triggers config-setup service to start.
Now call db_migrator after when config-setup service loads old-config/minigraph
Why I did it
Release Notes for Cisco 8102-64H:
Updated mtd-utils.mk and pyudev.mk for addressing build failures
How I did it
Update platform version to 202012.3.0.1
Why I did it
Run kvmtest when update package versions to avoid test break.
Work item tracking
Microsoft ADO (number only): 22335854
How I did it
How to verify it
#### Why I did it
Fix endless build log issue.
Cherry pick [PR#11846](https://github.com/sonic-net/sonic-buildimage/pull/11846)
##### Work item tracking
- Microsoft ADO **(number only)**: 19299131
#### How I did it
The current error handling code for when a deb package fails to be installed currently has a chain of commands linked together by && and ends with `exit 1`. The assumption is that the commands would succeed, and the last `exit 1` would end it with a non-zero return code, thus fully failing the target and causing the build to stop because of bash's -e flag.
However, if one of the commands prior to `exit 1` returns a non-zero return code, then bash won't actually treat it as a terminating error. From bash's man page:
```
-e Exit immediately if a pipeline (which may consist of a single simple
command), a list, or a compound command (see SHELL GRAMMAR above),
exits with a non-zero status. The shell does not exit if the
command that fails is part of the command list immediately
following a while or until keyword, part of the test following the
if or elif reserved words, part of any command executed in a && or
|| list except the command following the final && or ||, any
command in a pipeline but the last, or if the command's return
value is being inverted with !. If a compound command other than a
subshell returns a non-zero status because a command failed while
-e was being ignored, the shell does not exit.
```
The part `part of any command executed in a && or || list except the command following the final && or ||` says that if the failing command is not the `exit 1` that we have at the end, then bash doesn't treat it as an error and exit immediately. Additionally, since this is a compound command, but isn't in a subshell (subshell are marked by `(` and `)`, whereas `{` and `}` just tells bash to run the commands in the current environment), bash doesn't exist. The result of this is that in the deb-install target, if a package installation fails, it may be infinitely stuck in that while-loop.
There are two fixes for this: change to using a subshell, or use `;` instead of `&&`. Using a subshell would, I think, require exporting any shell variables used in the subshell, so I chose to change the `&&` to `;`. In addition, at the start of the subshell, `set +e` is added in, which removes the exit-on-error handling of bash. This makes sure that all commands are run (the output of which may help for debugging) and that it still exits with 1, which will then fully fail the target.
#### How to verify it
Why I did it
Advance sonic-utilities submodule head
Added below commits:
878be48e kellyyeh Wed May 10 15:21:52 2023 -0700 Revert "[warm-reboot] Use kexec_file_load instead of kexec_load when available
094513f8 Vaibhav Hemant Dixit Tue May 9 13:03:52 2023 -0700 [202012] LAG keepalive script to reduce lacp session wait during warm-reboot
Work item tracking
Microsoft ADO (number only): 23687678
Why I did it
Remove 'kvmtest-t0' and 'kvmtest-t1-lag' test jobs since all the test jobs are required (continueOnError: false) already, and will only enable one of classical and testbedV2 tests, no need to do an unnecessary 'or' compute test job.
Change agent pool to reduce cost and avoid congestion
* To resolve NEIGH table entries present in CONFIG_DB. Without this change arp/ndp entries which we wish to resolve, and configured via CONFIG_DB are not resolved.
Why I did it
src/sonic-py-swsssdk
* d44e0d8 - (HEAD -> 202012, origin/202012) [Security] Fix the redis security issue CVE-2023-28858 and CVE-2023-28859 (#135) (3 days ago) [xumia]
#### Why I did it
[Build] Upgrade the python docker version to fix bgp not up issue
##### Work item tracking
- Microsoft ADO **(number only)**: 22236397
Using timer-override.conf, we modify the fstrim.timer service.
For armhf, Nokia-7215 platform, we modify fstrim.timer to run daily
instead of weekly. This is required because the size of the SSD on
this platform is 16GB, which on average is nearly 10 times smaller than
most other sonic platforms. With smaller disk and the ever increasing
level of logging done by sonic, this change is required to prevent
the SSD from entering a read-only state due to inadequate free blocks.
#### Why I did it
sonic-utilities submodule update for 202012
```
* d20fc3c8 2023-04-07 | [202012][DBMigrator] Update db_migrator to support EdgeZoneAggregator Buffer Config for T0s (#2768) (HEAD, origin/202012) [Dev Ojha]
* 322a74dd 2023-03-27 | Resolved rc!=0 problem by replacing fgrep with awk. Added ipv4 filtering to get only v4 peers in case of show ip bgp neighbors (#2743) [saurabhab]
```
##### Work item tracking
- Microsoft ADO **(number only)**: 20782336
