Why I did it
This is a fix for PR [kernel] Change grub cmdline to set c-states to 0 for "Intel" CPUs by shlomibitton · Pull Request #6051 · sonic-net/sonic-buildimage (github.com)
The original PR will disable intel idle driver but it cannot limit the max c-state to 1 due to system will fall back to acpi idle driver.
Currently intel_idle.max_cstate=0 is already present, which will disable intel idle driver. With the added option, common idle driver will be disabled as well, so there will not be idle management. This is to prevent a bug that can be triggered by idle instruction on intel platform.
How I did it
Add the option to installer file beside intel_idle.max_cstate=0
- What I did
Added support for secure upgrade.
- How I did it
During sonic_installer install, added secure upgrade image verification.
HLD can be found in the following PR: sonic-net/SONiC#1024
- Why I did it
Feature is used to allow image was not modified since built from vendor. During installation, image can be verified with a signature attached to it.
- How I did it
Feature includes image signing during build (in sonic buildimage repo) and verification during image install (in sonic-utilities).
- How to verify it
In order for image verification - image must be signed - need to provide signing key and certificate (paths in SECURE_UPGRADE_DEV_SIGNING_KEY and SECURE_UPGRADE_DEV_SIGNING_CERT in rules/config) during build , and during image install, need to enable secure boot flag in bios, and signing_certificate should be available in bios.
- Feature dependencies
In order for this feature to work smoothly, need to have secure boot feature implemented as well.
The Secure boot feature will be merged in the near future.
- Why I did it
Add Secure Boot support to SONiC OS.
Secure Boot (SB) is a verification mechanism for ensuring that code launched by a computer's UEFI firmware is trusted. It is designed to protect a system against malicious code being loaded and executed early in the boot process before the operating system has been loaded.
- How I did it
Added a signing process to sign the following components:
shim, grub, Linux kernel, and kernel modules when doing the build, and when feature is enabled in build time according to the HLD explanations (the feature is disabled by default).
- How to verify it
There are self-verifications of each boot component when building the image, in addition, there is an existing end-to-end test in sonic-mgmt repo that checks that the boot succeeds when loading a secure system (details below).
How to build a sonic image with secure boot feature: (more description in HLD)
Required to use the following build flags from rules/config:
SECURE_UPGRADE_MODE="dev"
SECURE_UPGRADE_DEV_SIGNING_KEY="/path/to/private/key.pem"
SECURE_UPGRADE_DEV_SIGNING_CERT="/path/to/cert/key.pem"
After setting those flags should build the sonic-buildimage.
Before installing the image, should prepared the setup (switch device) with the follow:
check that the device support UEFI
stored pub keys in UEFI DB
enabled Secure Boot flag in UEFI
How to run a test that verify the Secure Boot flow:
The existing test "test_upgrade_path" under "sonic-mgmt/tests/upgrade_path/test_upgrade_path", is enough to validate proper boot
You need to specify the following arguments:
Base_image_list your_secure_image
Taget_image_list your_second_secure_image
Upgrade_type cold
And run the test, basically the test will install the base image given in the parameter and then upgrade to target image by doing cold reboot and validates all the services are up and working correctly
Co-authored-by: davidpil2002 <91657985+davidpil2002@users.noreply.github.com>
#### Why I did it
The %%EXTRA_CMDLINE_LINUX%% is not replaced to the real value, it has impact on the kernel parameter settings.
See the log sonic-vs.img.gz.log in the latest master build. In the grub.cfg, the %%EXTRA_CMDLINE_LINUX%% is set in the linux command line.
```
Installing for i386-pc platform.
Installation finished. No error reported.
Switch CPU vendor is: GenuineIntel
Switch CPU cstates are: disabled
EXTRA_CMDLINE_LINUX=%%EXTRA_CMDLINE_LINUX%%
Installed SONiC base image SONiC-OS successfully
ONIE: NOS install successful: file://dev/vdb/onie-installer.bin
```
Refactors the SONiC Installer to support greater flexibility in building for a given architecture and bootloader.
#### Why I did it
Currently the SONiC installer assumes that if a platform is ARM based that it uses the `uboot` bootloader and uses the `grub` bootloader otherwise. This is not a correct assumption to make as ARM is not strictly tied to uboot and x86 is not strictly tied to grub.
#### How I did it
To implement this I introduce the following changes:
* Remove the different arch folders from the `installer/` directory
* Merge the generic components of the ARM and x86 installer into `installer/installer.sh`
* Refactor x86 + grub specific functions into `installer/default_platform.conf`
* Modify installer to call `default_platform.conf` file and also call `platform/[platform]/patform.conf` file as well to override as needed
* Update references to the installer in the `build_image.sh` script
* Add `TARGET_BOOTLOADER` variable that is by default `uboot` for ARM devices and `grub` for x86 unless overridden in `platform/[platform]/rules.mk`
* Update bootloader logic in `build_debian.sh` to be based on `TARGET_BOOTLOADER` instead of `TARGET_ARCH` and to reference the grub package in a generic manner
#### How to verify it
This has been tested on a ARM test platform as well as on Mellanox amd64 switches as well to ensure there was no impact.
#### Description for the changelog
[arm] Refactor installer and build to allow arm builds targeted at grub platforms
#### Link to config_db schema for YANG module changes
N/A
Why I did it
Support to use symbol links in platform folder to reduce the image size.
The current solution is to copy each lazy installation targets (xxx.deb files) to each of the folders in the platform folder. The size will keep growing when more and more packages added in the platform folder. For cisco-8000 as an example, the size will be up to 2G, while most of them are duplicate packages in the platform folder.
How I did it
Create a new folder in platform/common, all the deb packages are copied to the folder, any other folders where use the packages are the symbol links to the common folder.
Why platform.tar?
We have implemented a patch for it, see #10775, but the problem is the the onie use really old unzip version, cannot support the symbol links.
The current solution is similar to the PR 10775, but make the platform folder into a tar package, which can be supported by onie. During the installation, the package.tar will be extracted to the original folder and removed.
If it is run during image install, it's not guaranteed that the
installation environment will have tune2fs available. Therefore, run it
during initramfs instead.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
# Why I did it
Reduce the disk space taken up during bootup and runtime.
# How I did it
1. Remove python package cache from the base image and from the containers.
2. During bootup, if logs are to be stored in memory, then don't create the `var-log.ext4` file just to delete it later during bootup.
3. For the partition containing `/host`, don't reserve any blocks for just the root user. This just makes sure all disk space is available for all users, if needed during upgrades (for example).
* Remove pip2 and pip3 caches from some containers
Only containers which appeared to have a significant pip cache size are
included here.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
* Don't create var-log.ext4 if we're storing logs in memory
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
* Run tune2fs on the device containing /host to not reserve any blocks for just the root user
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
#### Why I did it
The test plan described in the `How to verify it` section caused an issue when 3 images (instead of 2) were present when executing `show boot` or `sonic-installer list` commands:
```
root@sonic:/home/admin# show boot
Current: SONiC-OS-master.0-dirty-20220118.165941
Next: SONiC-OS-master.0-dirty-20220118.165941
Available:
SONiC-OS-master.0-dirty-20220118.165941
SONiC-OS-202012.201-a0376a6e5_Internal
SONiC-OS-202012.201-a0376a6e5_Internal_RPC
```
#### How I did it
Fixed the `sed` pattern to match the current image revision in the `install.sh` script.
#### How to verify it
Test plan:
1. Install the `imageA` by using ONIE
2. Install the `imageA-rpc` by using `sonic-installer`
3. Reboot the switch
4. Swap to the `imageA` - `sonic-installer set-default imageA`
5. Reboot the switch
6. Install the `imageB` by using `sonic-installer`
7. Check an installed images - `show boot`
8. Reboot the switch
9. Check an installed images - `show boot`
Why I did it
Provide onie image install verification
How I did it
Add devices folder based on platform_asic and check if current platform inside the list
How to verify it
In onie env, install a bin image which differs the running platform's ASIC. For example;
install sonic-broadcom.bin to mellanox ASIC platform
install sonic-vs.bin to broadcom ASIC platform
#### Why I did it
while sonic upgrade, Image will be extracted to tmpfs for installation so tmpfs size should be larger than image size. Image installation will fail if image size is larger than tmpfs size.
we are facing below error while installing debug image with size greater than tmpfs which is 1.5g in marvell armhf platform.
sonic-installer install <url>
New image will be installed, continue? [y/N]: y
Downloading image...
...99%, 1744 MB, 708 KB/s, 0 seconds left...
Installing image SONiC-OS-202012.0-dirty-20210311.224845 and setting it as default...
Command: bash /tmp/sonic_image
tar: installer/fs.zip: Wrote only 7680 of 10240 bytes
tar: installer/onie-image-arm64.conf: Cannot write: No space left on device
tar: Exiting with failure status due to previous errors
Verifying image checksum ... OK.
Preparing image archive ...
#### How I did it
compare downloaded image size with tmpfs size, if size less than image size update the tmpfs size according to image size.
#### How to verify it
Install an Image with size larger than tmpfs. we verified by installing debug image with size 1.9gb which is larger than tmpfs size 1.5gb.
This improvement reads current SONiC version directly from `/proc/cmdline`.
it supports `grub/aboot/uboot` bootloaders.
**Code snippet**:
```bash
cat /proc/cmdline | sed -n 's/^.*loop=\/*image-\(\S\+\)\/.*$/\1/p'
```
**Description**:
```
-n don't print lines
s substitute
^.* matches anything before the <image_version>
loop= matches <loop> kernel parameter
\/*image- matches <image_version> prefix
\(\S\+\) matches <image_version> group and assigns it to \1
\/.*$ matches anything after the <image_version>
\1 replace everything with <image_version>
p print it
```
closes#6267
#### Why I did it
* To fix#6267
#### How I did it
* Fixed installer scripts
#### How to verify it
1. Write invalid SONiC version to sonic_version.yml
2. Run SONiC-To-SONiC update
Fix#7180
Update systemd to v247 in order to pick the fix for "core: coldplug possible nop_job" systemd/systemd#13124
Install systemd, systemd-sysv from buster-backports. Pass "systemd.unified_cgroup_hierarchy=0" as kernel argument to force systemd to not use unified cgroup hierarchy, otherwise dockerd won't start moby/moby#16238.
Also, chown $FILSYSTEM_ROOT for root, otherwise apt systemd installation complains, see similar https://unix.stackexchange.com/questions/593529/can-not-configure-systemd-inside-a-chrooted-environment
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
The motivation of these changes is to fix (#6051):
- Why I did it
To fix CPU cstates configuration
- How I did it
Updated code to be POSIX compatible
- How to verify it
root@sonic:/home/admin# sonic_installer install sonic-mellanox.bin
Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>
Encounter error when install SONiC image if there are some onie_discovery variables assigned with multiple values inside machine.conf
- How I did it
Replace original ". /machine.conf" method and add another function to do the same thing.
- How to verify it
Add a item inside /host/machine.conf like onie_disco_ntpsrv=10.254.141.10 10.254.141.131
Do sonic_installer install to check if any error occurs
In order to install a SONiC image on top of a NVMe SSD disc properly with ONIE we must configure it properly on the installer.sh script.
Signed-off-by: Shlomi Bitton <shlomibi@nvidia.com>
Usually for a use case like networking - should not be configured to reach c6, the maximum used is c1e – due to the added latency getting in & out of states (bad for networking).
Following a recommendation by Intel, networking system should avoid getting in & out of states which introduce latency. The recommended state is c1e and no state change enabling.
In addition, c-state sole purpose is to save power and when inside a networking switch its really negligent being such a tiny consumer vs. the whole cluster.
Signed-off-by: Shlomi Bitton <shlomibi@nvidia.com>
[baseimage]: upgrade base image to debian buster
bring up the base image to debian buster 4.19 kernel.
using the merge commits to preserve the individual commits to better track the history.
console means the vga port. this is mainly used for virtual environment such as hyper-v or virtualbox where you have the virtual vga port to display the boot up process.
Signed-off-by: Guohan Lu <lguohan@gmail.com>
- build SONIC_STRETCH_DOCKERS in sonic-slave-stretch docker
- build image related module in sonic-slave-buster docker.
This includes all kernels modules and some packages
Signed-off-by: Guohan Lu <lguohan@gmail.com>
This patch upgrade the kernel from version
4.9.0-9-2 (4.9.168-1+deb9u3) to 4.9.0-11-2 (4.9.189-3+deb9u2)
Co-authored-by: rajendra-dendukuri <47423477+rajendra-dendukuri@users.noreply.github.com>
* [initramfs] Updated reuired tools for initramfs
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [initramfs] Updated required tools for initramfs
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [Platform] [Marvell] Platform specific debian package for et6448m device
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* Removed auto-generated files
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [initramfs] Added mtd and uboot firmware tools package required for arm arch
Its been enabled to all arch including amd64
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [initramfs] Added mtd and uboot firmware tools package required for arm arch
Its been enabled to all arch including amd64
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [initramfs] Marvell arm modules update and platform config update
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [iniramfs] add initramfs uboot-utils hook script only for ARM
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
when device disk is small, do not unzip dockerfs.tar.gz on disk.
keep the tar file on the disk, unzip to tmpfs in the initrd phase.
enabled this for 7050-qx32
Signed-off-by: Guohan Lu <gulv@microsoft.com>
* [Makefile/slave docker] ARM arch doesn't support few packages
iproute2 is missing for ARM sonic-slave docker
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
ARM Architecture support in SONIC
make configure platform=[ASIC_VENDOR_ARCH] PLATFORM_ARCH=[ARM_ARCH]
SONIC_ARCH: default amd64
armhf - arm32bit
arm64 - arm64bit
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [submodule] update sonic-linux-kernel
* update linux kernel version
* Fix many version strings
* update mellanox components (built with new kernel)
* [mlnx] add make files for SDK WJH libs
* Update arista driver submodule (#8)
Make the debian packaging point to a newer kernel version.
* [security kernel] Upgrade kernel from 4.9.110-3+deb9u2 to 4.9.110-3+deb9u6
short version: 4.9.0-7 to 4.9.0-8
See changelogs for security fixes:
https://tracker.debian.org/media/packages/l/linux/changelog-4.9.110-3deb9u6
Signed-off-by: Zhenggen Xu <zxu@linkedin.com>
* Update sonic-linux-kernel submodule after it was merged
Signed-off-by: Zhenggen Xu <zxu@linkedin.com>
