This feature caches all the deb files during docker build and stores them
into version cache.
It loads the cache file if already exists in the version cache and copies the extracted
deb file from cache file into Debian cache path( /var/cache/apt/archives).
The apt-install always installs the deb file from the cache if exists, this
avoid unnecessary package download from the repo and speeds up the overall build.
The cache file is selected based on the SHA value of version dependency
files.
Why I did it
How I did it
How to verify it
* 03.Version-cache - framework environment settings
It defines and passes the necessary version cache environment variables
to the caching framework.
It adds the utils script for shared cache file access.
It also adds the post-cleanup logic for cleaning the unwanted files from
the docker/image after the version cache creation.
* 04.Version cache - debug framework
Added DBGOPT Make variable to enable the cache framework
scripts in trace mode. This option takes the part name of the script to
enable the particular shell script in trace mode.
Multiple shell script names can also be given.
Eg: make DBGOPT="image|docker"
Added verbose mode to dump the version merge details during
build/dry-run mode.
Eg: scripts/versions_manager.py freeze -v \
'dryrun|cmod=docker-swss|cfile=versions-deb|cname=all|stage=sub|stage=add'
* 05.Version cache - docker dpkg caching support
This feature caches all the deb files during docker build and stores them
into version cache.
It loads the cache file if already exists in the version cache and copies the extracted
deb file from cache file into Debian cache path( /var/cache/apt/archives).
The apt-install always installs the deb file from the cache if exists, this
avoid unnecessary package download from the repo and speeds up the overall build.
The cache file is selected based on the SHA value of version dependency
files.
Signed-off-by: maipbui <maibui@microsoft.com>
Dependency: [PR (#12065)](https://github.com/sonic-net/sonic-buildimage/pull/12065) needs to merge first.
#### Why I did it
1. `eval()` - not secure against maliciously constructed input, can be dangerous if used to evaluate dynamic content. This may be a code injection vulnerability.
2. `subprocess()` - when using with `shell=True` is dangerous. Using subprocess function without a static string can lead to command injection.
3. `os` - not secure against maliciously constructed input and dangerous if used to evaluate dynamic content.
4. `is` operator - string comparison should not be used with reference equality.
5. `globals()` - extremely dangerous because it may allow an attacker to execute arbitrary code on the system
#### How I did it
1. `eval()` - use `literal_eval()`
2. `subprocess()` - use `shell=False` instead. use an array string. Ref: [https://semgrep.dev/docs/cheat-sheets/python-command-injection/#mitigation](https://semgrep.dev/docs/cheat-sheets/python-command-injection/#mitigation)
3. `os` - use with `subprocess`
4. `is` - replace by `==` operator for value equality
5. `globals()` - avoid the use of globals()
Debian is shipping a systemd timer unit for logrotate, but we're also
packaging in a cron job, which means both of them will run, potentially
at the same time. Remove our cron file, and add an override to the
shipped timer file to have it be run every 10 minutes.
Fixes#12392.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Why I did it
As described in detail in #12753, the current FRR patch 0009-ignore-route-from-default-table.patch is causing unwanted FRR/zebra error logs. This change gets rid of the error messages for routes from kernel default table while these routes are ignored in prefix encoding.
How I did it
This fix updates the original 0009 patch by checking if the routes are from table default before printing the error logs. The original patch checks the same condition and ignores the routes from table default in prefix encoding.
How to verify it
Follow the steps to repro as described in #12753.
Also verify the test case ipfwd/test_nhop_count.py no longer fails due to the error messages.
Signed-off-by: Stephen Xu <stexu@linkedin.com>
Why I did it
Initial implementation of Watchdog platform plugin for BMC-based boards
How I did it
How to verify it
Run platform_tests/test_reload_config.py
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan arlakshm@microsoft.com
Why I did it
Fixes#12575 and #12575
How I did it
In the PR sonic-net/sonic-platform-daemons#311 chassisd updates to CHASSIS_FABRIC_ASIC_INFO with the fabric asic info.
Updating the asic_status.py to read from the correct table.
How to verify it
test on chassis
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
*Replaced BRCM SDK's psample support flag(PSAMPLE_SUPPORT) with linux kernel psample module support config flag(CONFIG_PSAMPLE) in saibcm-modules.
*Replaced BUILD_PSAMPLE conditioanl check with CONFIG_PSAMPLE to build psample callback library(psample-cb.o), only if psample config is enabled in linux kernel.
*Cleaned up PSAMPLE_SUPPORT related commented code.
Signed-off-by: haris@celestica.com
Signed-off-by: haris@celestica.com
This change is to disable the pcie firmware check done by Broadcom SAI. The change is needed for the Arista platform x86_64-arista_7050cx3_32s; otherwise, the check will fail, blocking the initialization.
There was a pcie firmware check added in brcm SDK and certain Arista hardwares do not compliant with the check, so we added the disable_pcie_firmware_check originally for x86_64-arista_7060dx4_32. For x86_64-arista_7050cx3_32s, it was able to pass the check but some firmware change done in August made it fail.
Why I did it
SIGTERM takes more than 10 seconds to be processed, so psud is stopped by SIGKILL, this causes unexpected behavior since data base is not cleared
How I did it
Decorate get_presence api to cancel it on SIGTERM signal in order to avoid long processing.
How to verify it
test_pmon_psud_stop_and_start_status
test_pmon_psud_term_and_start_status
PR #12829 modified the docker tagging scheme such that optional docker
containers would be tagged with the SONiC image version. However, the
docker-image-load macro wasn't updated for these changes. Update it
here.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Why I did it
Limiting #12804 changes to PikeZ platform only (Arista-720DT-48S). Note that this is a short term workaround for this platform until SDK investigation on SDK init failure on docker syncd restart due to DMA issues is resolved.
How I did it
Retrieve platform name from /host/machine.conf and only reload SDK kmods on Arista-720DT-48S platform.
Signed-off-by: Michael Li <michael.li@broadcom.com>
- Why I did it
There's a slowdown in bootup related to the execution of a show command during startup of swss service. show is a pretty heavy command and takes long time to execute ~2 sec.
- How I did it
I replaced show with sonic-db-cli which takes a ms to run.
- How to verify it
Boot the switch and verify swss is active.
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
# Why I did it
Update sairedis submodule to include following changes:
1. Use github code scanning instead of LGTM sonic-sairedis#1160
2. enable cisco8000 SAI bulk API feature sonic-sairedis#1153
3. [submodule] Advance SAI header sonic-sairedis#1168
# How I did it
Advance sairedis header to keep up with master.
Signed-off-by: zitingguo-ms <zitingguo@microsoft.com>
Recently, the job of t0-sonic, multi-asic and wan run stably in master branch, so in this pr, I set them mandatory in azure pipeline.
Why I did it
Recently, the job of t0-sonic, multi-asic and wan run stably in master branch, so in this pr, I set them mandatory in azure pipeline.
How I did it
Modify the value of continueOnError in each job from `true` to `false`.
Signed-off-by: Yutong Zhang <yutongzhang@microsoft.com>
1d53bf4 Skip platform NDK health check two times in watchdog.sh
d68297c Added code to shutdown the channel after the grpc call also fixed the show fp-status command
0769efe Impelemented the module API to return the correct eeprom info for fabric card.
171569c Remove explicit logger identifier for transceiver module operations; use inherited id
6c4d651 Corrected the log messages for firmware install
Signed-off-by: mlok <marty.lok@nokia.com>
Why I did it:
Fix multiple seastone platform issues caused by sonic kernel upgrade.
How I did it:
Get gpio base id with new label path in gpio sys fs.
How to verify it:
After the change, show platform fan/psustatus/temperature works well.
- Why I did it
Added ECMP calculator tool.
- How I did it
New files were added.
- How to verify it
Manual tests performed according to tests chapter in HLD
Automated tests will be added by verification.
pre-compiled bazel is not work in arm64 docker container
shil@2f910d8d37b2:/sonic/src/sonic-p4rt/sonic-pins$ uname -a
Linux 2f910d8d37b2 5.4.0-132-generic #148-Ubuntu SMP Mon Oct 17 16:02:06 UTC 2022 aarch64 GNU/Linux
shil@2f910d8d37b2:/sonic/src/sonic-p4rt/sonic-pins$ bazel
Opening zip "/proc/self/exe": lseek(): Bad file descriptor
FATAL: Failed to open '/proc/self/exe' as a zip file: (error: 9): Bad file descriptor
shil@2f910d8d37b2:/sonic/src/sonic-p4rt/sonic-pins$
Why I did it
arm64 bullseye docker image source is set to jessie for VERSION_CODENAME is miss.
shil@localhost:~/sonic-buildimage$ docker run -it multiarch/debian-debootstrap:arm64-bullseye bash
root@b2e2fea86e2d:/# cat /etc/os-release | grep VERSION_CODENAME | cut -d= -f2
root@b2e2fea86e2d:/# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux bullseye/sid"
NAME="Debian GNU/Linux"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
How I did it
if DISTRO is NULL, find it in /etc/apt/sources.list
root@b2e2fea86e2d:/# cat /etc/apt/sources.list | grep deb.debian.org | awk '{print $3}'
bullseye
root@b2e2fea86e2d:/# cat /etc/apt/sources.list
deb http://deb.debian.org/debian bullseye main
root@b2e2fea86e2d:/#
Why I did it
support multi-platform device tree for default dtb may not suitable on all vender hardware designs.
How I did it
use onie_platform variable to load device tree blob
Previously, we set timeout in each step such as Lock testbed, Prepare testbed, Run test and KVM dump. When some issue suck like retry happens in one step, it will cause timeout error, but actually, it only needs more time to success. In this pr, we remove the timeout limit in each step and control the timeout outside in each job. When the job runs more than four hours, it will be cancelled.
Why I did it
Previously, we set timeout in each step such as Lock testbed, Prepare testbed, Run test and KVM dump. When some issue suck like retry happens in one step, it will cause timeout error, but actually, it only needs more time to success. In this pr, we remove the timeout limit in each step and control the timeout outside in each job. When the job runs more than four hours, it will be cancelled.
How I did it
Remove the timeout parameter in each step, and control the timeout outside in each job.
How to verify it
Set the timeout of one job to 4 hours, and when timeout happens, azure pipeline will cancel this job.
Why I did it
smartctl tool is available only in PMON docker. Hence, the tool may be not accessible incase PMON docker goes down.
Using iSMART_64 tool to fetch the SSD firmware version and device model information.
How I did it
Replacing smartctl with iSMART_64.
During docker build, host files can be passed to the docker build through
docker context files. But there is no straightforward way to transfer
the files from docker build to host.
This feature provides a tricky way to pass the cache contents from docker
build to host. It tar's the cached content and encodes them as base64 format
and passes it through a log file with a special tag as 'VCSTART and VCENT'.
Slave.mk in the host, it extracts the cache contents from the log and stores them
in the cache folder. Cache contents are encoded as base64 format for
easy passing.
<!--
Please make sure you've read and understood our contributing guidelines:
https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md
** Make sure all your commits include a signature generated with `git commit -s` **
If this is a bug fix, make sure your description includes "fixes #xxxx", or
"closes #xxxx" or "resolves #xxxx"
Please provide the following information:
-->
#### Why I did it
#### How I did it
#### How to verify it
Update submodule for sonic-pins to be aligned to following swss PRs
*New P4Orch development. sonic-swss#2425
*Upstream new development on p4orch sonic-swss#2237
- Why I did it
Fixed sflow yang model to include collector_vrf field.
- How I did it
Added leaf for collector_vrf under sflow_collector. Additionally aligned the configuration guide
- How to verify it
Added UT to verify.
The only platforms that currently need the stretch slave container are
innovium and nephos, and both are not building with the current code due
to other issues. All other platforms only need buster and bullseye slave
containers.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Signed-off-by: maipbui <maibui@microsoft.com>
Dependency: [https://github.com/sonic-net/sonic-buildimage/pull/12065](https://github.com/sonic-net/sonic-buildimage/pull/12065)
#### Why I did it
`subprocess.Popen()` and `subprocess.run()` is used with `shell=True`, which is very dangerous for shell injection.
`os` - not secure against maliciously constructed input and dangerous if used to evaluate dynamic content
`getstatusoutput` is dangerous because it contains `shell=True` in the implementation
#### How I did it
Replace `os` by `subprocess`, use with `shell=False`
Remove unused functions
4a2ef99 Avoid printing message in error level when DEVICE_METADATA|localhost updates (sonic-net/sonic-host-services#25)
6c131c4 Use github code scanning instead of LGTM(sonic-net/sonic-host-services#26)
c55f5d1 Use github code scanning instead of LGTM
Signed-off-by: Stephen Sun <stephens@nvidia.com>
Signed-off-by: vaibhav-dahiya vdahiya@microsoft.com
e474335 (HEAD -> master, origin/master, origin/HEAD) [ycabled] fix minor appl_db retrieving logic for update (#319)
9b84b58 Use github code scanning instead of LGTM (#316)
f784ad7 Pass grid parameter while calling set_laser_freq (#317)
ed818f8 [PSU daemon] Support PSU power threshold checking (#288)
707a720 (origin/202211) [chassisd] update chassisd to write fabric and lc asics on sep erate table (#311)
e8c5657 [ycabled] fix exception-handling logic for ycabled (#306)
905874d [ycabled] move swsscommon API's from subroutines to call them exactly once per task_worker/thread (#303)
510d330 Fix typo in xcvrd (#313)
9ae551f [ycabled] add support for detach mode in 'active-active' topology (#309)
The above commits are added to sonic-platform-daemons
Fixes: #11521
- Why I did it
When build SONiC dockers, SONiC build system tags all of them with latest tag. This is Ok for all built-in dockers because we will also tag them with image version tag in sonic_debian_extension.j2 script. On the other hand, some of these dockers are SONiC packages and they are installed by sonic-package-manager which creates a only one tag whcih is recorded in the corresponding .gz file. This leads to having these dockers tagged only with latest tag. This change saves the tag as an image version string in .gz file, so that these dockers have version identification in their tag.
- How I did it
I modified slave.mk to save the version tag instead of latest tag.
- How to verify it
I verified this change by running show version
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
