* sonic-platform-modules-cel: broadcom: adapt for kernel 6.1 and bookworm
The i2c_driver->remove API declaration has been updated to return void instead
of int, as part of cleanup patches in 6.1. More details can be referred from
here: [1]. Update the remove API definition in the modules accordingly and
cleanup variables that go unused from the remove API.
Update python build commands for bookworm. The packaging based on calling
setup.py is deprecated and using build module/pip utility is the recommended
method for python packaging/installation. Further details can be referred to
from here: [2], [3]. The build module is picky about the package information file,
which needs to be either setup.py or pyproject.toml.
Additionally, fix formatting inconsistencies in debian/changelog reported by
`dh_installchangelogs` during the build.
Tested the changes by compiling the changes as below:
make sonic-slave-bash NOBUSTER=1 NOBULLSEYE=1
sudo dpkg -i target/debs/bookworm/linux-headers-6.1.0-11-2-*.deb
cd platform/broadcom/sonic-platform-modules-cel
KVERSION=6.1.0-11-2-amd64 dpkg-buildpackage
Also verified the python scripts under the sonic-platform-modules-cel with
pyflakes to ensure no new errors are flagged (with exception of unused modules).
References:
[1] - https://github.com/torvalds/linux/commit/ed5c2f5f
[2] - https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.htm
[3] - 0b20a4863 (Update Python build commands for Bookworm, 2023-09-07)
Signed-off-by: Ramasamy Chandramouli <rachandr@celestica.com>
* platform/pddf: i2c: adapt for kernel 6.1 and bookworm
* Fixup i2c_driver->remove API due to changes in the function
prototype (ref: [1]).
* Cleanup `MODULE_SUPPORTED_DEVICE` macros that were cleaned up in
the upstream (ref: [2]).
* Sanitize python packaging and installation using the `build` module
instead of calling the setup.py directly (ref: [3]. [4]).
Tested the changes by compiling pddf module as below:
make sonic-slave-bash NOBUSTER=1 NOBULLSEYE=1
sudo dpkg -i target/debs/bookworm/linux-headers-6.1.0-11-2-*.deb
cd platform/pddf/i2c
KVERSION=6.1.0-11-2-amd64 dpkg-buildpackage
References:
[1] - https://github.com/torvalds/linux/commit/ed5c2f5f
[2] - https://github.com/torvalds/linux/commit/6417f031
[2] - https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.htm
[3] - 0b20a4863 (Update Python build commands for Bookworm, 2023-09-07)
Signed-off-by: Ramasamy Chandramouli <rachandr@celestica.com>
* platform/broadcom: include platform-modules-cel in builds
With pddf modules patched for 6.1, platform-modules-cel can be compiled
and included in the final image.
Testing by building sonic-broadcom.bin/sonic-broadcom-dnx.bin.
Signed-off-by: Ramasamy Chandramouli <rachandr@celestica.com>
* pddf/i2c: revert correct rootdir for pip install
The pip install directory has been set to test-pkg1/ for testing the build and
incorrectly retained as is. Revert this to the correct path $(PACKAGE_PRE_NAME).
Signed-off-by: Ramasamy Chandramouli <rachandr@celestica.com>
* platform/broadcom: include pddf/modules-cel in the base package
Without this change, the modules were built but not packaged in the final .bin.
The final sonic-broadcom.bin has been tested for bootup on Celestica's
Silverstone platform.
admin@sonic:~$ uname -a
Linux sonic 6.1.0-11-2-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.38-4 (2023-08-08) x86_64 GNU/Linux
admin@sonic:~$ show platform summary
Platform: x86_64-cel_silverstone-r0
HwSKU: Silverstone
ASIC: broadcom
ASIC Count: 1
Serial Number: R4009B2F062504LK200024
Model Number: N/A
Hardware Revision: N/A
admin@sonic:~$ show version | head
SONiC Software Version: SONiC.g0aad6c67c-rachandr
SONiC OS Version: 12
Distribution: Debian 12.2
Kernel: 6.1.0-11-2-amd64
Build commit: 0aad6c67c
Build date: Thu Oct 26 07:13:47 UTC 2023
Built by: rachandr@AZUHPS14
Platform: x86_64-cel_silverstone-r0
Signed-off-by: Ramasamy Chandramouli <rachandr@celestica.com>
---------
Signed-off-by: Ramasamy Chandramouli <rachandr@celestica.com>
Starting with Bookworm, Debian moved the non-free Linux firmware blobs
into a new non-free-firmware component, since they are frequently needed
by users and since they need to be updated frequently. Since the only
thing we currently install from the non-free component (that I can think
of) is the Linux firmware, have Bookworm use non-free-firmware instead
of non-free.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Notable changes:
* Use j2cli from Debian repos instead of pip
* Use setuptools from Debian repos instead of pip
* Use wheel from Debian repos instead of pip
* Update grpcio and grpcio-tools python packages to match version in
Bookworm
* Use m2crypto from Debian repos instead of pip
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
This fixes 3 issues:
* Specify test dependencies under extra_requires
* Update the PAM configuration for Bookworm
* Break a cyclical dependency between sonic-host-services and
sonic-buildimage by moving the contents of
src/sonic-host-services-data into sonic-host-services submodule
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
This fixes 4 issues:
* Update tabulate to 0.9.0 and deepdiff to 6.2.2
* Specify test dependencies under extra_requires
* Add check_output parameter to the setup function due to the patch
* Fix error about having a mutable default for field headers in
dataclass
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
With the new test command, test_cfggen_from_yang.py is now being run,
whereas previously, it was never run. This results in new failures
appearing from changes that have occurred some time back.
Therefore, for now, disable tests for sonic-config-engine when building
for Bookworm.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
The help text printed for sonic-yang-mgmt has slight differences
depending on the package versions. Loosen this check to only check the
options themselves, rather than the surrounding text.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
This ordering dependency causes FRR to get built for Bookworm, which we
don't need currently. Skip this by having it apply only to Bookworm.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Newer versions of pip/setuptools don't support test_requires, and the
current standard is to specify any extra dependencies (such as those
required for testing) under extra_requires.
Therefore, specify the testing dependencies under extra_requires. These
can be installed via pip using `pip install '.[testing]'`.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
In Bookworm's version of setuptools, direct calls to setup.py are
deprecated and no longer guaranteed to work. One of the recommended
commands is to use the `build` python package to build packages, and
call it with `python -m build`. This, by default, builds the packages in
a virtualenv to ensure that only the specified dependencies in setup.py
are needed to build the package. This also extends to running tests,
where directly calling `setup.py test` may not work, and the recommended
alternatives are to either call `pytest` directly, or call `tox` or
`nox.` More details are available at [1].
For SONiC's use case, for building python packages, we cannot build all
Python packages in a virtualenv since there are dependencies that we
would have built earlier, and these packages are not pushed to pypi or
any package registry. (There may be a cleaner approach to this, though,
but I'm not aware of it.) For this reason, the `-n` flag is added to not
build the package in a virtualenv.
For testing, `pytest` is now called instead of `setup.py test`.
[1] https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Changes from Bullseye slave container:
* Python 2 is no longer available at all
* Python 3.11 (instead of Python 3.9)
* GCC 12 (instead of GCC 10)
* Python ipaddr package is no longer available
* OpenJDK 17 (instead of OpenJDK 11)
* Remove doxygen armhf manual compilation (no longer needed)
* Disable FIPS, as the FIPS binaries are currently not yet available
* Install Python setuptools through Debian instead of pip
* Install Python wheel through Debian instead of pip
* Install Python nose through Debian instead of pip
* Install Python j2cli through Debian instead of pip
* Install Python pexpect through Debian instead of pip
* Install Python parameterized through Debian instead of pip
* Install Python pyyaml through Debian instead of pip
* Install Python pyfakefs through Debian instead of pip
* Install Python m2crypto through Debian instead of pip
* Python pympler 1.0 (instead of 0.8)
* Install Python build (as a replacement to setup.py)
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
This repo has been deprecated, and will not have Bookworm packages
available. Remove it from the build mirror config script.
Snapshot packages are still available from this repo.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
#### Why I did it
src/sonic-host-services
```
* beb8bbe - (HEAD -> master, origin/master, origin/HEAD) [DualToR][caclmgrd] Fix IPtables rules for multiple vlan interfaces for DualToR config (#82) (3 hours ago) [vdahiya12]
```
#### How I did it
#### How to verify it
#### Description for the changelog
fixes#16011
Why I did it
seeing below warning ,essage:
libyang[1]: Default value "" in the list key "port" is ignored. (/sonic-snmp:sonic-snmp/SNMP_AGENT_ADDRESS_CONFIG/SNMP_AGENT_ADDRESS_LIST)
libyang[1]: Default value "" in the list key "vrf_name" is ignored. (/sonic-snmp:sonic-snmp/SNMP_AGENT_ADDRESS_CONFIG/SNMP_AGENT_ADDRESS_LIST)
name of list is not <model_name>_LIST.
Work item tracking
Microsoft ADO 25646016:
How I did it
Remove default value provided to key in yang model to avoid seeing below error:
libyang[1]: Default value "" in the list key "port" is ignored. (/sonic-snmp:sonic-snmp/SNMP_AGENT_ADDRESS_CONFIG/SNMP_AGENT_ADDRESS_LIST)
libyang[1]: Default value "" in the list key "vrf_name" is ignored. (/sonic-snmp:sonic-snmp/SNMP_AGENT_ADDRESS_CONFIG/SNMP_AGENT_ADDRESS_LIST)
Modify the LIST name to have <model_name>_LIST as this was failing yang validation during unit-tests.
How to verify it
unit-tests passing.
Before fix
admin@vlab-01:~$ sudo sonic-package-manager list
libyang[1]: Default value "" in the list key "port" is ignored. (/sonic-snmp:sonic-snmp/SNMP_AGENT_ADDRESS_CONFIG/SNMP_AGENT_ADDRESS_LIST)
libyang[1]: Default value "" in the list key "vrf_name" is ignored. (/sonic-snmp:sonic-snmp/SNMP_AGENT_ADDRESS_CONFIG/SNMP_AGENT_ADDRESS_LIST)
Name Repository Description Version Status
-------------- --------------------------- ---------------------------- --------- ---------
database docker-database SONiC database package 1.0.0 Built-In
dhcp-relay docker-dhcp-relay N/A 1.0.0 Installed
eventd docker-eventd SONiC eventd package 1.0.0 Built-In
fpm-frr docker-fpm-frr SONiC fpm-frr package 1.0.0 Built-In
gbsyncd docker-gbsyncd-vs SONiC gbsyncd package 1.0.0 Built-In
lldp docker-lldp SONiC lldp package 1.0.0 Built-In
macsec docker-macsec N/A 1.0.0 Installed
mgmt-framework docker-sonic-mgmt-framework SONiC mgmt-framework package 1.0.0 Built-In
mux docker-mux SONiC mux package 1.0.0 Built-In
nat docker-nat SONiC nat package 1.0.0 Built-In
pmon docker-platform-monitor SONiC pmon package 1.0.0 Built-In
radv docker-router-advertiser SONiC radv package 1.0.0 Built-In
sflow docker-sflow SONiC sflow package 1.0.0 Built-In
snmp docker-snmp SONiC snmp package 1.0.0 Built-In
swss docker-orchagent SONiC swss package 1.0.0 Built-In
syncd docker-syncd-vs SONiC syncd package 1.0.0 Built-In
teamd docker-teamd SONiC teamd package 1.0.0 Built-In
telemetry docker-sonic-telemetry SONiC telemetry package 1.0.0 Built-In
After fix:
admin@vlab-01:~$ sudo sonic-package-manager list
Name Repository Description Version Status
-------------- --------------------------- ---------------------------- --------- ---------
database docker-database SONiC database package 1.0.0 Built-In
dhcp-relay docker-dhcp-relay N/A 1.0.0 Installed
eventd docker-eventd SONiC eventd package 1.0.0 Built-In
fpm-frr docker-fpm-frr SONiC fpm-frr package 1.0.0 Built-In
gbsyncd docker-gbsyncd-vs SONiC gbsyncd package 1.0.0 Built-In
lldp docker-lldp SONiC lldp package 1.0.0 Built-In
macsec docker-macsec N/A 1.0.0 Installed
mgmt-framework docker-sonic-mgmt-framework SONiC mgmt-framework package 1.0.0 Built-In
mux docker-mux SONiC mux package 1.0.0 Built-In
nat docker-nat SONiC nat package 1.0.0 Built-In
pmon docker-platform-monitor SONiC pmon package 1.0.0 Built-In
radv docker-router-advertiser SONiC radv package 1.0.0 Built-In
sflow docker-sflow SONiC sflow package 1.0.0 Built-In
snmp docker-snmp SONiC snmp package 1.0.0 Built-In
swss docker-orchagent SONiC swss package 1.0.0 Built-In
syncd docker-syncd-vs SONiC syncd package 1.0.0 Built-In
teamd docker-teamd SONiC teamd package 1.0.0 Built-In
telemetry docker-sonic-telemetry SONiC telemetry package 1.0.0 Built-In
Why I did it
Enable the suppress fib feature by default.
Work item tracking
Microsoft ADO (25564723):
How I did it
In minigraph.py, to add the field suppress-fib-pending, and enable it for leafrouter.
How to verify it
Build / load image and check the config_db by show CLI.
admin@str-7260cx3-acs-2:~$ show suppress-fib-pending
Enabled
Need to modify the tests/bgp/test_bgp_suppress_fib.py in sonic-mgmt repo, to check the config before restore. Otherwise, after this test, it will turn off the suppress-fib-pending.
sonic-net/sonic-mgmt#10612
Why I did it
This is part of Python3 migration project. This PR will add a new makefile flag: LEGACY_SONIC_MGMT_DOCKER
Now by default: LEGACY_SONIC_MGMT_DOCKER = y will build sonic-mgmt-docker with Python2 and Python3
If LEGACY_SONIC_MGMT_DOCKER = n will will sonic-mgmt-docker with Python3 only
Work item tracking
Microsoft ADO (number only): 25254349
How I did it
Add makefile flag: LEGACY_SONIC_MGMT_DOCKER
How to verify it
By default will build sonic-mgmt-docker with Python2 and Python3. No change compared to before.
Set LEGACY_SONIC_MGMT_DOCKER=n will build sonic-mgmt-docker with Python3 only
This is CSP CS00012280996.
The issue to fix is that the checksum was incorrect for all TCP packets leaving the system so that the BGP connection cannot be established. We found the issue on BCM56993, and it is possible to affect all platforms using linux_ngknet.
#### Why I did it
src/sonic-swss-common
```
* a57cf9e - (HEAD -> master, origin/master, origin/HEAD) Add batch support in ZmqProducerStateTable. (#803) (10 hours ago) [mint570]
```
#### How I did it
#### How to verify it
#### Description for the changelog
#### Why I did it
src/sonic-linux-kernel
```
* a75a3df - (HEAD -> master, origin/master, origin/HEAD) arm64: Kconfig inclusions to fix PCI hang and MTD detection (#350) (3 hours ago) [Pavan Naregundi]
```
#### How I did it
#### How to verify it
#### Description for the changelog
#### Why I did it
src/sonic-swss
```
* 917c21e0 - (HEAD -> master, origin/master, origin/HEAD) Add more debug information when PFC WD is triggered (#2858) (10 hours ago) [Stephen Sun]
```
#### How I did it
#### How to verify it
#### Description for the changelog
Why I did it
This PR is part of sonic-mgmt-docker Python3 migration project.
Work item tracking
Microsoft ADO (number only): 24397943
How I did it
Upgrade Ansible to 6.7.0
Make Python3 as the default interpreter. python is a soft link to python3. If you want to use python2, use the command python2 explicitly.
Upgrade some pip packages to higher version in order to meet security requirement.
How to verify it
Build a private sonic-mgmt-docker successfully.
Verify python is python3.
Verify python2 is working with 202012 and 202205 branch.
Verify python3 is working with master branch.
Verify with github PR test.
### Why I did it
We use `EdgeZoneAggregator` in `db_migrator`, but we don't support this pattern in sonic yang models. Hence, we update this in the sonic-yang model.
##### Work item tracking
- Microsoft ADO **(number only)**: 25574132
#### How I did it
Update the device pattern list.
#### Why I did it
HLD implementation: Container Hardening (https://github.com/sonic-net/SONiC/pull/1364)
##### Work item tracking
- Microsoft ADO **(number only)**: 14807420
#### How I did it
Reduce linux capabilities in privileged flag
#### How to verify it
Run sflow sonic-mgmt tests
Check container's settings: Privileged is false and container only has default Linux caps, does not have extended caps.
```
admin@vlab-01:~$ docker inspect sflow | grep Privi
"Privileged": false,
admin@vlab-01:~$ docker exec -it sflow bash
root@vlab-01:/# capsh --print
Current: cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_raw,cap_sys_chroot,cap_mknod,cap_audit_write,cap_setfcap=ep
Bounding set =cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_raw,cap_sys_chroot,cap_mknod,cap_audit_write,cap_setfcap
```
#### Why I did it
This header file comes from an external package, and a very old version of the header file has been checked into swss-common. This will cause problems for the upcoming Bookworm upgrade.
##### Work item tracking
- Microsoft ADO **(number only)**: 25411155
#### How I did it
Change references to the header file to use the Debian package nlohmann-json-dev, instead of from swss-common.
### Tested branch (Please provide the tested image version)
- [ ] <!-- image version 1 -->
- [ ] VS image from pipeline build
Verified that eventd was running
Why I did it
To avoid orchagent crash issue like sonic-net/sonic-swss#2935, disable unsupported counters on SONiC management devices.
Work item tracking
Microsoft ADO (number only): 25437720
How I did it
Update the minigraph parser to disable unsupported counters on management devices.
How to verify it
Verified by unittest.
Manually apply patch to DUT and do config load_minigraph
