- Why I did it
Added ECMP calculator tool.
- How I did it
New files were added.
- How to verify it
Manual tests performed according to tests chapter in HLD
Automated tests will be added by verification.
pre-compiled bazel is not work in arm64 docker container
shil@2f910d8d37b2:/sonic/src/sonic-p4rt/sonic-pins$ uname -a
Linux 2f910d8d37b2 5.4.0-132-generic #148-Ubuntu SMP Mon Oct 17 16:02:06 UTC 2022 aarch64 GNU/Linux
shil@2f910d8d37b2:/sonic/src/sonic-p4rt/sonic-pins$ bazel
Opening zip "/proc/self/exe": lseek(): Bad file descriptor
FATAL: Failed to open '/proc/self/exe' as a zip file: (error: 9): Bad file descriptor
shil@2f910d8d37b2:/sonic/src/sonic-p4rt/sonic-pins$
Why I did it
arm64 bullseye docker image source is set to jessie for VERSION_CODENAME is miss.
shil@localhost:~/sonic-buildimage$ docker run -it multiarch/debian-debootstrap:arm64-bullseye bash
root@b2e2fea86e2d:/# cat /etc/os-release | grep VERSION_CODENAME | cut -d= -f2
root@b2e2fea86e2d:/# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux bullseye/sid"
NAME="Debian GNU/Linux"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
How I did it
if DISTRO is NULL, find it in /etc/apt/sources.list
root@b2e2fea86e2d:/# cat /etc/apt/sources.list | grep deb.debian.org | awk '{print $3}'
bullseye
root@b2e2fea86e2d:/# cat /etc/apt/sources.list
deb http://deb.debian.org/debian bullseye main
root@b2e2fea86e2d:/#
Why I did it
support multi-platform device tree for default dtb may not suitable on all vender hardware designs.
How I did it
use onie_platform variable to load device tree blob
Previously, we set timeout in each step such as Lock testbed, Prepare testbed, Run test and KVM dump. When some issue suck like retry happens in one step, it will cause timeout error, but actually, it only needs more time to success. In this pr, we remove the timeout limit in each step and control the timeout outside in each job. When the job runs more than four hours, it will be cancelled.
Why I did it
Previously, we set timeout in each step such as Lock testbed, Prepare testbed, Run test and KVM dump. When some issue suck like retry happens in one step, it will cause timeout error, but actually, it only needs more time to success. In this pr, we remove the timeout limit in each step and control the timeout outside in each job. When the job runs more than four hours, it will be cancelled.
How I did it
Remove the timeout parameter in each step, and control the timeout outside in each job.
How to verify it
Set the timeout of one job to 4 hours, and when timeout happens, azure pipeline will cancel this job.
Why I did it
smartctl tool is available only in PMON docker. Hence, the tool may be not accessible incase PMON docker goes down.
Using iSMART_64 tool to fetch the SSD firmware version and device model information.
How I did it
Replacing smartctl with iSMART_64.
During docker build, host files can be passed to the docker build through
docker context files. But there is no straightforward way to transfer
the files from docker build to host.
This feature provides a tricky way to pass the cache contents from docker
build to host. It tar's the cached content and encodes them as base64 format
and passes it through a log file with a special tag as 'VCSTART and VCENT'.
Slave.mk in the host, it extracts the cache contents from the log and stores them
in the cache folder. Cache contents are encoded as base64 format for
easy passing.
<!--
Please make sure you've read and understood our contributing guidelines:
https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md
** Make sure all your commits include a signature generated with `git commit -s` **
If this is a bug fix, make sure your description includes "fixes #xxxx", or
"closes #xxxx" or "resolves #xxxx"
Please provide the following information:
-->
#### Why I did it
#### How I did it
#### How to verify it
Update submodule for sonic-pins to be aligned to following swss PRs
*New P4Orch development. sonic-swss#2425
*Upstream new development on p4orch sonic-swss#2237
- Why I did it
Fixed sflow yang model to include collector_vrf field.
- How I did it
Added leaf for collector_vrf under sflow_collector. Additionally aligned the configuration guide
- How to verify it
Added UT to verify.
The only platforms that currently need the stretch slave container are
innovium and nephos, and both are not building with the current code due
to other issues. All other platforms only need buster and bullseye slave
containers.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Signed-off-by: maipbui <maibui@microsoft.com>
Dependency: [https://github.com/sonic-net/sonic-buildimage/pull/12065](https://github.com/sonic-net/sonic-buildimage/pull/12065)
#### Why I did it
`subprocess.Popen()` and `subprocess.run()` is used with `shell=True`, which is very dangerous for shell injection.
`os` - not secure against maliciously constructed input and dangerous if used to evaluate dynamic content
`getstatusoutput` is dangerous because it contains `shell=True` in the implementation
#### How I did it
Replace `os` by `subprocess`, use with `shell=False`
Remove unused functions
4a2ef99 Avoid printing message in error level when DEVICE_METADATA|localhost updates (sonic-net/sonic-host-services#25)
6c131c4 Use github code scanning instead of LGTM(sonic-net/sonic-host-services#26)
c55f5d1 Use github code scanning instead of LGTM
Signed-off-by: Stephen Sun <stephens@nvidia.com>
Signed-off-by: vaibhav-dahiya vdahiya@microsoft.com
e474335 (HEAD -> master, origin/master, origin/HEAD) [ycabled] fix minor appl_db retrieving logic for update (#319)
9b84b58 Use github code scanning instead of LGTM (#316)
f784ad7 Pass grid parameter while calling set_laser_freq (#317)
ed818f8 [PSU daemon] Support PSU power threshold checking (#288)
707a720 (origin/202211) [chassisd] update chassisd to write fabric and lc asics on sep erate table (#311)
e8c5657 [ycabled] fix exception-handling logic for ycabled (#306)
905874d [ycabled] move swsscommon API's from subroutines to call them exactly once per task_worker/thread (#303)
510d330 Fix typo in xcvrd (#313)
9ae551f [ycabled] add support for detach mode in 'active-active' topology (#309)
The above commits are added to sonic-platform-daemons
Fixes: #11521
- Why I did it
When build SONiC dockers, SONiC build system tags all of them with latest tag. This is Ok for all built-in dockers because we will also tag them with image version tag in sonic_debian_extension.j2 script. On the other hand, some of these dockers are SONiC packages and they are installed by sonic-package-manager which creates a only one tag whcih is recorded in the corresponding .gz file. This leads to having these dockers tagged only with latest tag. This change saves the tag as an image version string in .gz file, so that these dockers have version identification in their tag.
- How I did it
I modified slave.mk to save the version tag instead of latest tag.
- How to verify it
I verified this change by running show version
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
Previously, we hard code the min and max numbers of instance in a plan. In this pr, we support passing the instance numbers of a testplan.
Why I did it
Previously, we hard code the min and max numbers of instance in a plan. In this pr, we support passing the instance numbers of a testplan.
How I did it
Use a variable to set the instance number.
Why I did it
There is an issue on the Arista PikeZ platform (using T3.X2: BCM56274) while running SONiC. If the 'syncd' container in SONiC is restarted, the expected behaviour is that syncd will automatically restart/recover; however it does not and always fails at create_switch due to BCM SDK kmod DMA operation cancellation getting stuck.
Sep 16 22:19:44.855125 pkz208 ERR syncd#syncd: [none] SAI_API_SWITCH:platform_process_command:428 Platform command "init soc" failed, rc = -1. Sep 16 22:19:44.855206 pkz208 INFO syncd#supervisord: syncd CMIC_CMC0_PKTDMA_CH4_DESC_COUNT_REQ:0x33#015 Sep 16 22:19:44.855264 pkz208 CRIT syncd#syncd: [none] SAI_API_SWITCH:platformInit:1909 initialization command "init soc" failed, rc = -1 (Internal error). Sep 16 22:19:44.855403 pkz208 CRIT syncd#syncd: [none] SAI_API_SWITCH:sai_driver_init:642 Error initializing driver, rc = -1. ... Sep 16 22:19:44.855891 pkz208 CRIT syncd#syncd: [none] SAI_API_SWITCH:brcm_sai_create_switch:1173 initializing SDK failed with error Operation failed (0xfffffff5).
Reloading the BCM SDK kmods allows the switch init to continue properly.
How I did it
If BCM SDK kmods are loaded, unload and load them again on syncd docker start script.
How to verify it
Steps to reproduce:
In SONiC, run 'docker ps' to see current running containers; 'syncd' should be present.
Run 'docker stop syncd'
Wait ~1 minute.
Run 'docker ps' to see that syncd is missing.
Check logs to see messages similar to the above.
Signed-off-by: Michael Li <michael.li@broadcom.com>
#### Why I did it
`os` and `commands` modules are not secure against maliciously constructed input
`getstatusoutput` is detected without a static string, uses `shell=True`
#### How I did it
Eliminate the use of `os` and `commands`
Use `subprocess` instead
Signed-off-by: Neetha John <nejo@microsoft.com>
Why I did it
ECN parameters need to be updated for storage backend
How I did it
Included the check for storage backend devices to update qos configs
How to verify it
Verified that the new ecn settings are applied on storage backend device.
Verified that the old ecn settings are applied for storage frontend, non storage frontend/backend devices
Signed-off-by: maipbui <maibui@microsoft.com>
#### Why I did it
The [xml.etree.ElementTree](https://docs.python.org/3/library/xml.etree.elementtree.html#module-xml.etree.ElementTree) module is not secure against maliciously constructed data.
`os` - not secure against maliciously constructed input and dangerous if used to evaluate dynamic content
`subprocess.getstatusoutput` is dangerous because include shell=True in the implementation
#### How I did it
Remove xml. Use [lxml](https://pypi.org/project/lxml/) XML parsers package that prevent potentially malicious operation.
Replace `os` by `subprocess`
Use command as an array instead of string
Use `getstatusoutput_noshell` in `sonic_py_common` lib
- Why I did it
Change YANG model to support syslog rate limit configuration feature
- How I did it
modified sonic-syslog.yang and sonic-feature.yang to support the new added configuration schema
- How to verify it
Unit test
- Why I did it
Add support for compiling Spectrum-4 ASIC firmware to the SONiC image
Add support for Spectrum-4 ASIC firmware upgrade
- How I did it
Update Mellanox fw make files to include Spectrum-4 ASIC firmware binaries.
Update firmware upgrade scripts to be able to detect Spectrum-4 ASIC.
- How to verify it
Run regression tests
Signed-off-by: Kebo Liu <kebol@nvidia.com>
Why I did it
Provide GNMI native write interface for configuration.
How I did it
Add configuration parameters for GNMI native write.
How to verify it
Check build pipeline.
Why I did it
Submodule update for sonic-gnmi
Incorporates:
8226e46 Upgrade pipeline to use bullseye. (sonic-net/sonic-gnmi#58)
ae72767 Add gnmi_dump tool for debug and unit test (sonic-net/sonic-gnmi#60)
6b0253a Add conditional check for split (sonic-net/sonic-gnmi#55)
99bfa8f Remove LOGLEVEL DB since is no longer used (sonic-net/sonic-gnmi#56)
54806a8 Support new gnmi config interface in telemetry container. (sonic-net/sonic-gnmi#7)
How I did it
Move submodule
How to verify it
Check build pipeline.
Why I did it
Added to allow test_crm_route to pass; the test tries to add a /126 ipv6 route and this change is required in order for the count of available routes to be updated correctly.
Why I did it
Submodule update for sonic-swss-common
Incorporates:
5d481da Install swsscommon.i with libswsscommon-dev (#717)
How I did it
I have updated sonic-swss-common repo, this PR is used to update submodule.
How to verify it
Build image, install libswsscommon-dev, and check /usr/share/swss.
- Why I did it
Upgrade the app-extension developer environments (sonic-sdk & sonic-sdk-bullseye) to bullseye
- How to verify it
Built an app-extension using these images and verified if it is up and running.
Signed-off-by: Vivek Reddy <vkarri@nvidia.com>
Why I did it
The variable is not initialized in src/systemd-sonic-generator/systemd-sonic-generator.c. I fixed this one.
How I did it
Assure pointer is initialized to NULL when defined and before use.
- Why I did it
Add SDK hash calculator Debian and update SDK makefile to compile it.
- How I did it
SDK hash calculator Debian will be used by ECMP calculator (PR #12482)
- How to verify it
Compile sonic-buildimage and verify SDK hash calculator Debian exist in target folder.
* [sonic-platform-daemons] submodule update
git log --oneline 7c0a326..master
e8c5657 (HEAD -> master, origin/master, origin/HEAD) [ycabled] fix
exception-handling logic for ycabled (#306)
905874d [ycabled] move swsscommon API's from subroutines to call them
exactly once per task_worker/thread (#303)
510d330 Fix typo in xcvrd (#313)
9ae551f [ycabled] add support for detach mode in 'active-active'
topology (#309)
82fc7a6 Added filtering logic to send filtered fields from DB event
(#307)
8a2dad9 [ycabled] fix no port/state returned by grpc server (#308)
4ea12cf Fix xcvrd to support 400G ZR optic (#293)
23b6970 [ycabled] fix naming error for error condition for CLI handling
(#302)
501abb2 [ycabled] add some exception catching logic to some vendor
specific API's (#301)
534f839 [ycabled] add support for getting grpc secerts via shared file
(#298)
3622aac Remove shell=True (#300)
143422b add support to execute new ycable API's, add datetime field to
mux_info (#297)
6522c46 [ycabled] add notification for gRPC connection state transitions
to IDLE/TRANSIENT_FAILURE (#295)
4b3b238 Install libyang to azure pipeline (#292)
8ff5f37 Use get() to fetch default value from dictionary for port
admin_status #286
b03cc74 [Xcvrd] Soak duplicate events and process only updated
interested events (#285)
3acb171 [ycable] cleanup logic for creating grpc future ready (#289)
ce3b6db [ycabled] fix insert events from xcvrd;cleanup some mux toggle
logic (#287)
Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
Why I did it
sonic-swss-common needs to support SWIG wrapper for go.
Submodule update for sonic-swss-common
Incorporates:
d63130c [[CI]Install libyang from common-lib when build bullseye (#710)
bfe123e Add decorator for Yang default value. (#713)
1d66080 Use SWIGPYTHON to improve SWIG for GO wrapper. (#714)
How I did it
I have updated sonic-swss-common repo, this PR is used to update submodule.
How to verify it
Build image, and sonic-gnmi pipeline will verify SWIG wrapper.
#### Why I did it
When build the sonic-slave-bash target, it cannot go to the shell failed in the step to build sonic-build-hooks, the error logs as below. It may have impact on some of the users, it may be relative to different version of the make.
```
$ QUIET=n BLDENV=bullseye make NOJESSIE=1 NOSTRETCH=1 sonic-slave-bash
+++ Making sonic-slave-bash +++
BLDENV=buster make -f Makefile.work sonic-slave-bash
make[1]: Entering directory `/builds2/stephens/wip/update-submodule/sonic-buildimage'
echo -n ""
pushd src/sonic-build-hooks; TRUSTED_GPG_URLS=https://packages.trafficmanager.net/debian/public_key.gpg,https://packages.microsoft.com/keys/microsoft.asc make all; popd
/builds2/stephens/wip/update-submodule/sonic-buildimage/src/sonic-build-hooks /builds2/stephens/wip/update-submodule/sonic-buildimage
make[2]: Entering directory `/builds2/stephens/wip/update-submodule/sonic-buildimage/src/sonic-build-hooks'
dpkg-deb: building package 'sonic-build-hooks' in 'buildinfo/sonic-build-hooks_1.0_all.deb'.
make[2]: Leaving directory `/builds2/stephens/wip/update-submodule/sonic-buildimage/src/sonic-build-hooks'
/builds2/stephens/wip/update-submodule/sonic-buildimage
mkdir -p sonic-slave-buster/buildinfo
cp src/sonic-build-hooks/buildinfo/sonic-build-hooks* sonic-slave-buster/buildinfo
[ "n" == y ] && scripts/build_mirror_config.sh sonic-slave-buster amd64 buster
make[1]: *** [sonic-build-hooks] Error 1
make[1]: Leaving directory `/builds2/stephens/wip/update-submodule/sonic-buildimage'
make: *** [sonic-slave-bash] Error 2
```
#### How I did it
Change the format as below:
```
[ xxx = yyy ] && do something
```
To
```
if [ xxx = yyy ]; then do something; if
```
#### How to verify it
Verified by who found the issue, the issue gone when the patch applied.
* Update submodule sonic-utilities
72b9cf9 [psushow & psuutil] Support PSU power threshold checking (#2326)
dfdc92e Revert "Support the bandit check by GitHub Action (#2358)" (#2509)
85638b2 YANG Validation for ConfigDB Updates: DEVICE_METADATA, SNMP, SNMP_COMMUNITY tables (#2481)
da44d71 YANG Validation for ConfigDB Updates: PORT_STORM_CONTROL, PORT_QOS_MAP, BUFFER_PROFILE, BUFFER_PG, BUFFER_QUEUE, BUFFER_POOL, FEATURE, DEFAULT_LOSSLESS_BUFFER_PARAMETER tables (#
ba9b628 [sfp] Fix issue: Application Advertisement is not well formatted (#2491)
Signed-off-by: Stephen Sun <stephens@nvidia.com>
* Update submodule sonic-platform-daemons
ed818f8 [PSU daemon] Support PSU power threshold checking (#288)
707a720 [chassisd] update chassisd to write fabric and lc asics on sep erate table (#311)
e8c5657 [ycabled] fix exception-handling logic for ycabled (#306)
905874d [ycabled] move swsscommon API's from subroutines to call them exactly once per task_worker/thread (#303)
510d330 Fix typo in xcvrd (#313)
9ae551f [ycabled] add support for detach mode in 'active-active' topology (#309)
82fc7a6 Added filtering logic to send filtered fields from DB event (#307)
8a2dad9 [ycabled] fix no port/state returned by grpc server (#308)
4ea12cf Fix xcvrd to support 400G ZR optic (#293)
23b6970 [ycabled] fix naming error for error condition for CLI handling (#302)
501abb2 [ycabled] add some exception catching logic to some vendor specific API's (#301)
534f839 [ycabled] add support for getting grpc secerts via shared file (#298)
3622aac Remove shell=True (#300)
143422b add support to execute new ycable API's, add datetime field to mux_info (#297)
6522c46 [ycabled] add notification for gRPC connection state transitions to IDLE/TRANSIENT_FAILURE (#295)
4b3b238 Install libyang to azure pipeline (#292)
8ff5f37 Use get() to fetch default value from dictionary for port admin_status #286
b03cc74 [Xcvrd] Soak duplicate events and process only updated interested events (#285)
3acb171 [ycable] cleanup logic for creating grpc future ready (#289)
ce3b6db [ycabled] fix insert events from xcvrd;cleanup some mux toggle logic (#287)
Signed-off-by: Stephen Sun <stephens@nvidia.com>
* Update submodule sonic-platform-common
7c48be2 Add warning/critical thresholds for PSU power (#304)
f5ba0d0 Add missing PM and VDM related EEPROM read (#326)
02dd597 implement CMIS set_laser_freq to support 100GHz grid space (#294)
Signed-off-by: Stephen Sun <stephens@nvidia.com>
* Revert "Update submodule sonic-platform-daemons"
This reverts commit aea648b157.
Signed-off-by: Stephen Sun <stephens@nvidia.com>
