- Why I did it
To fix BIOS firmware update after fresh image installation from ONiE
- How I did it
Initialized empty GRUB environment file after ONiE installation
- How to verify it
1. Install image from ONiE
2. Run BIOS firmware upgrade
Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>
Why I did it
fixes#15949
Problem 1: Setting ONIE_IMAGE_PART_SIZE using env variable or using "make ONIE_IMAGE_PART_SIZE=65536 USERNAME=test PASSWORD=test all" did not work.
Problem 2: The platform specific file for example "device/x86_64-8201_32fh_o-r0/installer.conf" cannot override it by setting value of ONIE_IMAGE_PART_SIZE in the file. change 2 adds support to do that.
How I did it
Change 1: when ONIE_IMAGE_PART_SIZE, the files Makefile.work and slave.mk should pass that setting along all the way to build_image.sh. Please see commit 1.
Change 2: In installer/install.sh, save the value set during build time string replace into a value and then let this value be overridden later when installer.conf get read which is platform specific. If platform does not override it, the original value will continue to work. Please see commit 2.
How to verify it
1: The below command works now
make ONIE_IMAGE_PART_SIZE=65536 USERNAME=test PASSWORD=test all"
The image properly was installed using ONIE and the partition size reflects what was passed in the above build command.
If the above value is not set, the default from "onie-image.conf" takes effect and still works.
2: Set ONIE_IMAGE_PART_SIZE in platform specific file like below example
--------------Diff----
device/x86_64-8201_32fh_o-r0/installer.conf
@@ -1 +1,2 @@
ONIE_PLATFORM_EXTRA_CMDLINE_LINUX=" intel_iommu=off"
+ONIE_IMAGE_PART_SIZE=128000
and built the image using "make USERNAME=test PASSWORD=test all" and verified that the final installation properly partitioned the disk to the requested value from installer.conf file.
Why I did it
This is a fix for PR [kernel] Change grub cmdline to set c-states to 0 for "Intel" CPUs by shlomibitton · Pull Request #6051 · sonic-net/sonic-buildimage (github.com)
The original PR will disable intel idle driver but it cannot limit the max c-state to 1 due to system will fall back to acpi idle driver.
Currently intel_idle.max_cstate=0 is already present, which will disable intel idle driver. With the added option, common idle driver will be disabled as well, so there will not be idle management. This is to prevent a bug that can be triggered by idle instruction on intel platform.
How I did it
Add the option to installer file beside intel_idle.max_cstate=0
Why I did it
Support FIPS DB configuration
Design Doc: sonic-net/SONiC#1372
Work item tracking
Microsoft ADO (number only): 24411148
How I did it
Add the FIPS Yang model to make FIPS configurable in ConfigDB.
How to verify it
See TestPlan: sonic-net/sonic-mgmt#9092
Build the image and run the tests: sonic-net/sonic-mgmt#9091
- What I did
Added support for secure upgrade.
- How I did it
During sonic_installer install, added secure upgrade image verification.
HLD can be found in the following PR: sonic-net/SONiC#1024
- Why I did it
Feature is used to allow image was not modified since built from vendor. During installation, image can be verified with a signature attached to it.
- How I did it
Feature includes image signing during build (in sonic buildimage repo) and verification during image install (in sonic-utilities).
- How to verify it
In order for image verification - image must be signed - need to provide signing key and certificate (paths in SECURE_UPGRADE_DEV_SIGNING_KEY and SECURE_UPGRADE_DEV_SIGNING_CERT in rules/config) during build , and during image install, need to enable secure boot flag in bios, and signing_certificate should be available in bios.
- Feature dependencies
In order for this feature to work smoothly, need to have secure boot feature implemented as well.
The Secure boot feature will be merged in the near future.
…1.11 by using efivar tool instead
#### Why I did it
solution to BUG below/
https://github.com/sonic-net/sonic-buildimage/issues/14316
bug report also in this issue:
backport: secureboot support #14246
#### How I did it
When installing an image secure boot is checking if the UEFI have the secure boot flag enabled or disabled using a tool name `mokutil` this tool its not exist in ONIE version older than 2021.11 so its crasshing the install.
To fix that we add a coded that checking secure boot enabled/disabled by using efivar tool that should exist in any UEFI system
#### How to verify it
Install the image in a device with ONIE version older than 2021.11 and check that the installation and boot succeed (all docker up).
Why I did it
The demo_part_size should be initialized before creating partition.
How I did it
Move the initializing setting to the line before using it.
How to verify it
- Why I did it
Add Secure Boot support to SONiC OS.
Secure Boot (SB) is a verification mechanism for ensuring that code launched by a computer's UEFI firmware is trusted. It is designed to protect a system against malicious code being loaded and executed early in the boot process before the operating system has been loaded.
- How I did it
Added a signing process to sign the following components:
shim, grub, Linux kernel, and kernel modules when doing the build, and when feature is enabled in build time according to the HLD explanations (the feature is disabled by default).
- How to verify it
There are self-verifications of each boot component when building the image, in addition, there is an existing end-to-end test in sonic-mgmt repo that checks that the boot succeeds when loading a secure system (details below).
How to build a sonic image with secure boot feature: (more description in HLD)
Required to use the following build flags from rules/config:
SECURE_UPGRADE_MODE="dev"
SECURE_UPGRADE_DEV_SIGNING_KEY="/path/to/private/key.pem"
SECURE_UPGRADE_DEV_SIGNING_CERT="/path/to/cert/key.pem"
After setting those flags should build the sonic-buildimage.
Before installing the image, should prepared the setup (switch device) with the follow:
check that the device support UEFI
stored pub keys in UEFI DB
enabled Secure Boot flag in UEFI
How to run a test that verify the Secure Boot flow:
The existing test "test_upgrade_path" under "sonic-mgmt/tests/upgrade_path/test_upgrade_path", is enough to validate proper boot
You need to specify the following arguments:
Base_image_list your_secure_image
Taget_image_list your_second_secure_image
Upgrade_type cold
And run the test, basically the test will install the base image given in the parameter and then upgrade to target image by doing cold reboot and validates all the services are up and working correctly
#### Why I did it
The %%EXTRA_CMDLINE_LINUX%% is not replaced to the real value, it has impact on the kernel parameter settings.
See the log sonic-vs.img.gz.log in the latest master build. In the grub.cfg, the %%EXTRA_CMDLINE_LINUX%% is set in the linux command line.
```
Installing for i386-pc platform.
Installation finished. No error reported.
Switch CPU vendor is: GenuineIntel
Switch CPU cstates are: disabled
EXTRA_CMDLINE_LINUX=%%EXTRA_CMDLINE_LINUX%%
Installed SONiC base image SONiC-OS successfully
ONIE: NOS install successful: file://dev/vdb/onie-installer.bin
```
Refactors the SONiC Installer to support greater flexibility in building for a given architecture and bootloader.
#### Why I did it
Currently the SONiC installer assumes that if a platform is ARM based that it uses the `uboot` bootloader and uses the `grub` bootloader otherwise. This is not a correct assumption to make as ARM is not strictly tied to uboot and x86 is not strictly tied to grub.
#### How I did it
To implement this I introduce the following changes:
* Remove the different arch folders from the `installer/` directory
* Merge the generic components of the ARM and x86 installer into `installer/installer.sh`
* Refactor x86 + grub specific functions into `installer/default_platform.conf`
* Modify installer to call `default_platform.conf` file and also call `platform/[platform]/patform.conf` file as well to override as needed
* Update references to the installer in the `build_image.sh` script
* Add `TARGET_BOOTLOADER` variable that is by default `uboot` for ARM devices and `grub` for x86 unless overridden in `platform/[platform]/rules.mk`
* Update bootloader logic in `build_debian.sh` to be based on `TARGET_BOOTLOADER` instead of `TARGET_ARCH` and to reference the grub package in a generic manner
#### How to verify it
This has been tested on a ARM test platform as well as on Mellanox amd64 switches as well to ensure there was no impact.
#### Description for the changelog
[arm] Refactor installer and build to allow arm builds targeted at grub platforms
#### Link to config_db schema for YANG module changes
N/A
Why I did it
Support to use symbol links in platform folder to reduce the image size.
The current solution is to copy each lazy installation targets (xxx.deb files) to each of the folders in the platform folder. The size will keep growing when more and more packages added in the platform folder. For cisco-8000 as an example, the size will be up to 2G, while most of them are duplicate packages in the platform folder.
How I did it
Create a new folder in platform/common, all the deb packages are copied to the folder, any other folders where use the packages are the symbol links to the common folder.
Why platform.tar?
We have implemented a patch for it, see #10775, but the problem is the the onie use really old unzip version, cannot support the symbol links.
The current solution is similar to the PR 10775, but make the platform folder into a tar package, which can be supported by onie. During the installation, the package.tar will be extracted to the original folder and removed.
If it is run during image install, it's not guaranteed that the
installation environment will have tune2fs available. Therefore, run it
during initramfs instead.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
# Why I did it
Reduce the disk space taken up during bootup and runtime.
# How I did it
1. Remove python package cache from the base image and from the containers.
2. During bootup, if logs are to be stored in memory, then don't create the `var-log.ext4` file just to delete it later during bootup.
3. For the partition containing `/host`, don't reserve any blocks for just the root user. This just makes sure all disk space is available for all users, if needed during upgrades (for example).
* Remove pip2 and pip3 caches from some containers
Only containers which appeared to have a significant pip cache size are
included here.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
* Don't create var-log.ext4 if we're storing logs in memory
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
* Run tune2fs on the device containing /host to not reserve any blocks for just the root user
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
#### Why I did it
The test plan described in the `How to verify it` section caused an issue when 3 images (instead of 2) were present when executing `show boot` or `sonic-installer list` commands:
```
root@sonic:/home/admin# show boot
Current: SONiC-OS-master.0-dirty-20220118.165941
Next: SONiC-OS-master.0-dirty-20220118.165941
Available:
SONiC-OS-master.0-dirty-20220118.165941
SONiC-OS-202012.201-a0376a6e5_Internal
SONiC-OS-202012.201-a0376a6e5_Internal_RPC
```
#### How I did it
Fixed the `sed` pattern to match the current image revision in the `install.sh` script.
#### How to verify it
Test plan:
1. Install the `imageA` by using ONIE
2. Install the `imageA-rpc` by using `sonic-installer`
3. Reboot the switch
4. Swap to the `imageA` - `sonic-installer set-default imageA`
5. Reboot the switch
6. Install the `imageB` by using `sonic-installer`
7. Check an installed images - `show boot`
8. Reboot the switch
9. Check an installed images - `show boot`
Why I did it
Provide onie image install verification
How I did it
Add devices folder based on platform_asic and check if current platform inside the list
How to verify it
In onie env, install a bin image which differs the running platform's ASIC. For example;
install sonic-broadcom.bin to mellanox ASIC platform
install sonic-vs.bin to broadcom ASIC platform
#### Why I did it
while sonic upgrade, Image will be extracted to tmpfs for installation so tmpfs size should be larger than image size. Image installation will fail if image size is larger than tmpfs size.
we are facing below error while installing debug image with size greater than tmpfs which is 1.5g in marvell armhf platform.
sonic-installer install <url>
New image will be installed, continue? [y/N]: y
Downloading image...
...99%, 1744 MB, 708 KB/s, 0 seconds left...
Installing image SONiC-OS-202012.0-dirty-20210311.224845 and setting it as default...
Command: bash /tmp/sonic_image
tar: installer/fs.zip: Wrote only 7680 of 10240 bytes
tar: installer/onie-image-arm64.conf: Cannot write: No space left on device
tar: Exiting with failure status due to previous errors
Verifying image checksum ... OK.
Preparing image archive ...
#### How I did it
compare downloaded image size with tmpfs size, if size less than image size update the tmpfs size according to image size.
#### How to verify it
Install an Image with size larger than tmpfs. we verified by installing debug image with size 1.9gb which is larger than tmpfs size 1.5gb.
This improvement reads current SONiC version directly from `/proc/cmdline`.
it supports `grub/aboot/uboot` bootloaders.
**Code snippet**:
```bash
cat /proc/cmdline | sed -n 's/^.*loop=\/*image-\(\S\+\)\/.*$/\1/p'
```
**Description**:
```
-n don't print lines
s substitute
^.* matches anything before the <image_version>
loop= matches <loop> kernel parameter
\/*image- matches <image_version> prefix
\(\S\+\) matches <image_version> group and assigns it to \1
\/.*$ matches anything after the <image_version>
\1 replace everything with <image_version>
p print it
```
closes#6267
#### Why I did it
* To fix#6267
#### How I did it
* Fixed installer scripts
#### How to verify it
1. Write invalid SONiC version to sonic_version.yml
2. Run SONiC-To-SONiC update
Fix#7180
Update systemd to v247 in order to pick the fix for "core: coldplug possible nop_job" systemd/systemd#13124
Install systemd, systemd-sysv from buster-backports. Pass "systemd.unified_cgroup_hierarchy=0" as kernel argument to force systemd to not use unified cgroup hierarchy, otherwise dockerd won't start moby/moby#16238.
Also, chown $FILSYSTEM_ROOT for root, otherwise apt systemd installation complains, see similar https://unix.stackexchange.com/questions/593529/can-not-configure-systemd-inside-a-chrooted-environment
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
The motivation of these changes is to fix (#6051):
- Why I did it
To fix CPU cstates configuration
- How I did it
Updated code to be POSIX compatible
- How to verify it
root@sonic:/home/admin# sonic_installer install sonic-mellanox.bin
Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>
Encounter error when install SONiC image if there are some onie_discovery variables assigned with multiple values inside machine.conf
- How I did it
Replace original ". /machine.conf" method and add another function to do the same thing.
- How to verify it
Add a item inside /host/machine.conf like onie_disco_ntpsrv=10.254.141.10 10.254.141.131
Do sonic_installer install to check if any error occurs
In order to install a SONiC image on top of a NVMe SSD disc properly with ONIE we must configure it properly on the installer.sh script.
Signed-off-by: Shlomi Bitton <shlomibi@nvidia.com>
Usually for a use case like networking - should not be configured to reach c6, the maximum used is c1e – due to the added latency getting in & out of states (bad for networking).
Following a recommendation by Intel, networking system should avoid getting in & out of states which introduce latency. The recommended state is c1e and no state change enabling.
In addition, c-state sole purpose is to save power and when inside a networking switch its really negligent being such a tiny consumer vs. the whole cluster.
Signed-off-by: Shlomi Bitton <shlomibi@nvidia.com>
[baseimage]: upgrade base image to debian buster
bring up the base image to debian buster 4.19 kernel.
using the merge commits to preserve the individual commits to better track the history.
console means the vga port. this is mainly used for virtual environment such as hyper-v or virtualbox where you have the virtual vga port to display the boot up process.
Signed-off-by: Guohan Lu <lguohan@gmail.com>
- build SONIC_STRETCH_DOCKERS in sonic-slave-stretch docker
- build image related module in sonic-slave-buster docker.
This includes all kernels modules and some packages
Signed-off-by: Guohan Lu <lguohan@gmail.com>
This patch upgrade the kernel from version
4.9.0-9-2 (4.9.168-1+deb9u3) to 4.9.0-11-2 (4.9.189-3+deb9u2)
Co-authored-by: rajendra-dendukuri <47423477+rajendra-dendukuri@users.noreply.github.com>
* [initramfs] Updated reuired tools for initramfs
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [initramfs] Updated required tools for initramfs
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [Platform] [Marvell] Platform specific debian package for et6448m device
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* Removed auto-generated files
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [initramfs] Added mtd and uboot firmware tools package required for arm arch
Its been enabled to all arch including amd64
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [initramfs] Added mtd and uboot firmware tools package required for arm arch
Its been enabled to all arch including amd64
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [initramfs] Marvell arm modules update and platform config update
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [iniramfs] add initramfs uboot-utils hook script only for ARM
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
when device disk is small, do not unzip dockerfs.tar.gz on disk.
keep the tar file on the disk, unzip to tmpfs in the initrd phase.
enabled this for 7050-qx32
Signed-off-by: Guohan Lu <gulv@microsoft.com>
* [Makefile/slave docker] ARM arch doesn't support few packages
iproute2 is missing for ARM sonic-slave docker
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
ARM Architecture support in SONIC
make configure platform=[ASIC_VENDOR_ARCH] PLATFORM_ARCH=[ARM_ARCH]
SONIC_ARCH: default amd64
armhf - arm32bit
arm64 - arm64bit
Signed-off-by: Antony Rheneus <arheneus@marvell.com>
* [submodule] update sonic-linux-kernel
* update linux kernel version
* Fix many version strings
* update mellanox components (built with new kernel)
* [mlnx] add make files for SDK WJH libs
* Update arista driver submodule (#8)
Make the debian packaging point to a newer kernel version.
* [security kernel] Upgrade kernel from 4.9.110-3+deb9u2 to 4.9.110-3+deb9u6
short version: 4.9.0-7 to 4.9.0-8
See changelogs for security fixes:
https://tracker.debian.org/media/packages/l/linux/changelog-4.9.110-3deb9u6
Signed-off-by: Zhenggen Xu <zxu@linkedin.com>
* Update sonic-linux-kernel submodule after it was merged
Signed-off-by: Zhenggen Xu <zxu@linkedin.com>
