Scripts which perform an installable binary image build for SONiC
This repository has been archived on 2025-03-20. You can view files and clone it, but cannot push or open issues or pull requests.
Go to file
Taras Keryk bfe5835650
[BFN] Fix exception when fwutil run without sudo (#10335)
* [BFN] Fix for run fwutil without sudo
            SONiC has a concept of "platform components"
            this may include - CPLD, FPGA, BIOS, BMC, etc.

            These changes are needed to read the version of the BIOS and BMC component.

            What I did
                    The previous implementaion of component.py expect fwutil run with sudo.
	       When fwutil run without sudo, there are an exception:
```
Traceback (most recent call last):
  File "/usr/local/bin/fwutil", line 5, in <module>
    from fwutil.main import cli
  File "/usr/local/lib/python3.9/dist-packages/fwutil/__init__.py", line 3, in <module>
    from . import main
  File "/usr/local/lib/python3.9/dist-packages/fwutil/main.py", line 40, in <module>
    pdp = PlatformDataProvider()
  File "/usr/local/lib/python3.9/dist-packages/fwutil/lib.py", line 159, in __init__
    self.__platform = Platform()
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/platform.py", line 21, in __init__
    self._chassis = Chassis()
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/chassis.py", line 48, in __init__
    self.__initialize_components()
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/chassis.py", line 136, in __initialize_components
    component = Components(index)
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/component.py", line 184, in __init__
    self.version = get_bios_version()
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/component.py", line 19, in get_bios_version
    return subprocess.check_output(['dmidecode', '-s', 'bios-version']).strip().decode()
  File "/usr/lib/python3.9/subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.9/subprocess.py", line 505, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.9/subprocess.py", line 1823, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'dmidecode'

```
           How I did it
                    Modification of dmidecode command

           How to verify it
                    Run manually 'fwutil' (without sudo)

	   Previous command output had exception

	   New command output:
		Root privileges are required

Signed-off-by: Taras Keryk <tarasx.keryk@intel.com>

* Why I did it
The previous implementaion of component.py expect fwutil run with sudo.
When fwutil run without sudo, there are an exception:

Traceback (most recent call last):
  File "/usr/local/bin/fwutil", line 5, in <module>
    from fwutil.main import cli
  File "/usr/local/lib/python3.9/dist-packages/fwutil/__init__.py", line 3, in <module>
    from . import main
  File "/usr/local/lib/python3.9/dist-packages/fwutil/main.py", line 40, in <module>
    pdp = PlatformDataProvider()
  File "/usr/local/lib/python3.9/dist-packages/fwutil/lib.py", line 159, in __init__
    self.__platform = Platform()
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/platform.py", line 21, in __init__
    self._chassis = Chassis()
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/chassis.py", line 48, in __init__
    self.__initialize_components()
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/chassis.py", line 136, in __initialize_components
    component = Components(index)
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/component.py", line 184, in __init__
    self.version = get_bios_version()
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/component.py", line 19, in get_bios_version
    return subprocess.check_output(['dmidecode', '-s', 'bios-version']).strip().decode()
  File "/usr/lib/python3.9/subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.9/subprocess.py", line 505, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.9/subprocess.py", line 1823, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'dmidecode'

How I did it
Modification of dmidecode command

How to verify it
Run manually 'fwutil' (without sudo)
Previous command output had exception

New command output:
Root privileges are required

Signed-off-by: Taras Keryk tarasx.keryk@intel.com

Signed-off-by: Taras Keryk <tarasx.keryk@intel.com>

* rewrite a call of dmidecode, when run without sudo

Signed-off-by: Taras Keryk <tarasx.keryk@intel.com>

* Why I did it
The previous implementaion of component.py expect fwutil run with sudo.
When fwutil run without sudo, there are an exception:

Traceback (most recent call last):
  File "/usr/local/bin/fwutil", line 5, in <module>
    from fwutil.main import cli
  File "/usr/local/lib/python3.9/dist-packages/fwutil/__init__.py", line 3, in <module>
    from . import main
  File "/usr/local/lib/python3.9/dist-packages/fwutil/main.py", line 40, in <module>
    pdp = PlatformDataProvider()
  File "/usr/local/lib/python3.9/dist-packages/fwutil/lib.py", line 159, in __init__
    self.__platform = Platform()
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/platform.py", line 21, in __init__
    self._chassis = Chassis()
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/chassis.py", line 48, in __init__
    self.__initialize_components()
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/chassis.py", line 136, in __initialize_components
    component = Components(index)
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/component.py", line 184, in __init__
    self.version = get_bios_version()
  File "/usr/local/lib/python3.9/dist-packages/sonic_platform/component.py", line 19, in get_bios_version
    return subprocess.check_output(['dmidecode', '-s', 'bios-version']).strip().decode()
  File "/usr/lib/python3.9/subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.9/subprocess.py", line 505, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.9/subprocess.py", line 1823, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'dmidecode'

The previous implementaion of eeprom.py expect fwutil run with sudo.
When fwutil run without sudo, there are an exception:

Traceback (most recent call last):
File "/usr/lib/python3.9/logging/config.py", line 564, in configure
handler = self.configure_handler(handlers[name])
File "/usr/lib/python3.9/logging/config.py", line 745, in configure_handler
result = factory(**kwargs)
File "/usr/lib/python3.9/logging/handlers.py", line 153, in init
BaseRotatingHandler.init(self, filename, mode, encoding=encoding,
File "/usr/lib/python3.9/logging/handlers.py", line 58, in init
logging.FileHandler.init(self, filename, mode=mode,
File "/usr/lib/python3.9/logging/init.py", line 1142, in init
StreamHandler.init(self, self._open())
File "/usr/lib/python3.9/logging/init.py", line 1171, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding,
PermissionError: [Errno 13] Permission denied: '/var/log/platform.log'
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/bin/fwutil", line 5, in
from fwutil.main import cli
File "/usr/local/lib/python3.9/dist-packages/fwutil/init.py", line 3, in
from . import main
File "/usr/local/lib/python3.9/dist-packages/fwutil/main.py", line 41, in
pdp = PlatformDataProvider()
File "/usr/local/lib/python3.9/dist-packages/fwutil/lib.py", line 162, in init
self.chassis_component_map = self.__get_chassis_component_map()
File "/usr/local/lib/python3.9/dist-packages/fwutil/lib.py", line 168, in __get_chassis_component_map
chassis_name = self.__chassis.get_name()
File "/usr/local/lib/python3.9/dist-packages/sonic_platform/chassis.py", line 146, in get_name
return self._eeprom.modelstr()
File "/usr/local/lib/python3.9/dist-packages/sonic_platform/chassis.py", line 54, in _eeprom
self.__eeprom = Eeprom()
File "/usr/local/lib/python3.9/dist-packages/sonic_platform/eeprom.py", line 50, in init
logging.config.dictConfig(config_dict)
File "/usr/lib/python3.9/logging/config.py", line 809, in dictConfig
dictConfigClass(config).configure()
File "/usr/lib/python3.9/logging/config.py", line 571, in configure
raise ValueError('Unable to configure handler '
ValueError: Unable to configure handler 'file'

How I did it
Modification call of dmidecode command.
Added modification of log files access attributes before file open operations.

How to verify it
Run manually 'fwutil' (without sudo)

New command output have no exception.

Signed-off-by: Taras Keryk <tarasx.keryk@intel.com>

* Added file_check for checking access to log files for eeprom.py

Signed-off-by: Taras Keryk <tarasx.keryk@intel.com>

* Removed unused import

* Added logfile_create to eeprom.py and chassis.py

Signed-off-by: Taras Keryk <tarasx.keryk@intel.com>

* Created platform_utils.py

Signed-off-by: Taras Keryk <tarasx.keryk@intel.com>

* Added interpreter  string to  platform_utils.py

Signed-off-by: Taras Keryk <tarasx.keryk@intel.com>
2022-04-05 11:33:51 -07:00
.azure-pipelines [Ci]: Fix an azp template syntax error (#10446) 2022-04-04 07:06:44 +08:00
.github Update template pull_request_template.md (#9888) 2022-02-14 13:55:40 -08:00
device [arista] Update serdes tuning values for 7280cr3 (#9966) 2022-04-05 09:49:26 -07:00
dockers Upgrade router-advertiser container to Bullseye (#10374) 2022-04-01 16:12:43 -07:00
files Update qos config to clear queues for bounced back traffic (#10176) 2022-04-05 22:32:25 +08:00
installer Image disk space reduction (#10172) 2022-03-15 18:12:49 -07:00
platform [BFN] Fix exception when fwutil run without sudo (#10335) 2022-04-05 11:33:51 -07:00
rules Upgrade router-advertiser container to Bullseye (#10374) 2022-04-01 16:12:43 -07:00
scripts [build] Fix issues found in reproducible build. (#10407) 2022-03-31 13:07:46 +08:00
sonic-slave-bullseye Add a config variable to override default container registry instead of dockerhub. (#10166) 2022-03-14 18:09:20 +08:00
sonic-slave-buster Add a config variable to override default container registry instead of dockerhub. (#10166) 2022-03-14 18:09:20 +08:00
sonic-slave-jessie Add a config variable to override default container registry instead of dockerhub. (#10166) 2022-03-14 18:09:20 +08:00
sonic-slave-stretch Add a config variable to override default container registry instead of dockerhub. (#10166) 2022-03-14 18:09:20 +08:00
src Update qos config to clear queues for bounced back traffic (#10176) 2022-04-05 22:32:25 +08:00
.artifactignore [ci] Archive compiled Debian packages and Python wheels (#6650) 2021-02-02 23:42:03 -08:00
.gitignore [Mellanox] Refactor Mellanox platform API to support dynamic port configuration (#8422) 2021-10-25 07:59:06 +03:00
.gitmodules Bump Thrift version from 0.13.0 to 0.14.1 (#9881) 2022-02-15 16:39:47 -08:00
azure-pipelines.yml [ci] Set default ACR in UpgrateVersion/PR/official pipeline. (#10341) 2022-04-02 07:48:44 +08:00
build_debian.sh Introduce the asic_subtype field for adding the sub platform variants. (#10235) 2022-03-28 11:22:32 -07:00
build_debug_docker_j2.sh [sonic-buildimage] Fix build issue for docker-dhcp-relay-dbg.gz. Issue (#4136) 2020-02-10 17:16:42 -08:00
build_docker.sh Split script: push_docker.sh (#89) 2016-12-01 02:18:59 -08:00
build_image.sh [Build]: Fix the bin image generated from raw image issue (#10083) 2022-03-08 21:15:43 +08:00
check_install.py Remove the rw folder from the image after installing in KVM (#8746) 2021-12-10 13:13:35 -08:00
functions.sh [build] When generating image version, handle case where current commit has no reachable tags (#2506) 2019-01-31 14:48:48 -08:00
get_docker-base.sh Add mkdir if the target dir does not exist (#130) 2016-12-16 02:19:15 +00:00
install_sonic.py Remove the rw folder from the image after installing in KVM (#8746) 2021-12-10 13:13:35 -08:00
LICENSE updating readme, formatting in license 2016-03-09 17:39:34 +00:00
MAINTAINERS Adding license and maintainers 2016-03-08 19:10:18 -08:00
Makefile Add infra to support building Bullseye base image with Buster containers 2021-11-10 15:27:22 -08:00
Makefile.cache [Build]: Improve the linux kernel build cache hit rate (#9604) 2022-01-15 18:15:00 +08:00
Makefile.work Add a config variable to override default container registry instead of dockerhub. (#10166) 2022-03-14 18:09:20 +08:00
onie-image-arm64.conf New sonic-buildimage images for Broadcom DNX ASIC family. (#7598) 2021-06-22 11:12:22 -07:00
onie-image-armhf.conf New sonic-buildimage images for Broadcom DNX ASIC family. (#7598) 2021-06-22 11:12:22 -07:00
onie-image.conf [vs][multi-asic]: Add support to build multi-asic vs images (#8270) 2021-09-01 17:28:13 -07:00
onie-mk-demo.sh New sonic-buildimage images for Broadcom DNX ASIC family. (#7598) 2021-06-22 11:12:22 -07:00
push_docker.sh Update dockers with platform & SONiC version as part of name/tag. (#4337) 2020-04-09 21:18:14 -07:00
README.buildsystem.md [doc]: Fix typos in documents (#7296) 2021-04-14 18:27:24 -07:00
README.md [doc]: Update build instructions (#8470) 2021-08-25 04:35:31 -07:00
slave.mk [Security]: Enable hardening build options (#10369) 2022-04-02 23:07:10 +08:00
ThirdPartyLicenses.txt [TACACS+] Add Bash TACACS+ plugin for per-command authorization. (#8715) 2021-11-13 09:57:30 +08:00
update_screen.sh [build]: Added support for cache status on the build output (#5564) 2020-10-09 02:49:20 -07:00

static analysis:

Total alerts Language grade: Python

master builds:

Barefoot Broadcom Centec Centec(arm64) Innovium Mellanox Marvell(armhf) Nephos P4 VS

202012 builds:

Barefoot Broadcom Centec Centec(arm64) Innovium Marvell(armhf) Mellanox Nephos VS

201911 builds:

Barefoot Broadcom Innovium Mellanox Nephos VS

201811 builds:

Broadcom Mellanox Innovium Nephos VS

201807 builds:

Broadcom Barefoot

201803 builds:

Broadcom Nephos Marvell Mellanox

sonic-buildimage

Build SONiC Switch Images

Description

Following is the instruction on how to build an (ONIE) compatible network operating system (NOS) installer image for network switches, and also how to build docker images running inside the NOS. Note that SONiC image are build per ASIC platform. Switches using the same ASIC platform share a common image. For a list of supported switches and ASIC, please refer to this list

Hardware

Any server can be a build image server as long as it has:

  • Multiple cores to increase build speed
  • Plenty of RAM (less than 8 GiB is likely to cause issues)
  • 300G of free disk space

A good choice of OS for building SONiC is currently Ubuntu 20.04.

Prerequisites

  • Install pip and jinja in host build machine, execute below commands if j2/j2cli is not available:
sudo apt install -y python3-pip
sudo pip3 install j2cli
  • Install Docker and configure your system to allow running the 'docker' command without 'sudo':
    • Add current user to the docker group: sudo gpasswd -a ${USER} docker
    • Log out and log back in so that your group membership is re-evaluated

Clone or fetch the code repository with all git submodules

To clone the code repository recursively, assuming git version 1.9 or newer:

git clone https://github.com/Azure/sonic-buildimage.git

Usage

To build SONiC installer image and docker images, run the following commands:

# Ensure the 'overlay' module is loaded on your development system
sudo modprobe overlay

# Enter the source directory
cd sonic-buildimage

# (Optional) Checkout a specific branch. By default, it uses master branch. For example, to checkout the branch 201911, use "git checkout 201911"
git checkout [branch_name]

# Execute make init once after cloning the repo, or after fetching remote repo with submodule updates
make init

# Execute make configure once to configure ASIC
make configure PLATFORM=[ASIC_VENDOR]

# Build SONiC image with 4 jobs in parallel.
# Note: You can set this higher, but 4 is a good number for most cases
# and is well-tested.
make SONIC_BUILD_JOBS=4 all

The supported ASIC vendors are:

  • PLATFORM=broadcom
  • PLATFORM=marvell
  • PLATFORM=mellanox
  • PLATFORM=cavium
  • PLATFORM=centec
  • PLATFORM=nephos
  • PLATFORM=innovium
  • PLATFORM=p4
  • PLATFORM=vs

Usage for ARM Architecture

To build Arm32 bit for (ARMHF) platform ARM build has dependency in docker version 18, if docker version is 19, downgrade to 18 as below sudo apt-get install --allow-downgrades -y docker-ce=5:18.09.0~3-0~ubuntu-xenial sudo apt-get install --allow-downgrades -y docker-ce-cli=5:18.09.0~3-0~ubuntu-xenial

# Execute make configure once to configure ASIC and ARCH

make configure PLATFORM=[ASIC_VENDOR] PLATFORM_ARCH=armhf

make target/sonic-[ASIC_VENDER]-armhf.bin

# example:

make configure PLATFORM=marvell-armhf PLATFORM_ARCH=armhf

make target/sonic-marvell-armhf.bin

To build Arm64 bit for platform

# Execute make configure once to configure ASIC and ARCH

make configure PLATFORM=[ASIC_VENDOR] PLATFORM_ARCH=arm64

# example:

make configure PLATFORM=marvell-arm64 PLATFORM_ARCH=arm64

NOTE:

  • Recommend reserving at least 100G free space to build one platform with a single job. The build process will use more disk if you are setting SONIC_BUILD_JOBS to more than 1.

  • If Docker's workspace folder, /var/lib/docker, resides on a partition without sufficient free space, you may encounter an error like the following during a Docker container build job:

    /usr/bin/tar: /path/to/sonic-buildimage/<some_file>: Cannot write: No space left on device

    The solution is to move the directory to a partition with more free space.

  • Use http_proxy=[your_proxy] https_proxy=[your_proxy] no_proxy=[your_no_proxy] make to enable http(s) proxy in the build process.

  • Add your user account to docker group and use your user account to make. root or sudo are not supported.

The SONiC installer contains all docker images needed. SONiC uses one image for all devices of a same ASIC vendor.

For Broadcom ASIC, we build ONIE and EOS image. EOS image is used for Arista devices, ONIE image is used for all other Broadcom ASIC based devices.

make configure PLATFORM=broadcom
# build debian stretch required targets
BLDENV=stretch make stretch
# build ONIE image
make target/sonic-broadcom.bin
# build EOS image
make target/sonic-aboot-broadcom.swi

You may find the rules/config file useful. It contains configuration options for the build process, like adding more verbosity or showing dependencies, username and password for base image etc.

Every docker image is built and saved to target/ directory. So, for instance, to build only docker-database, execute:

make target/docker-database.gz

Same goes for debian packages, which are under target/debs/:

make target/debs/swss_1.0.0_amd64.deb

Every target has a clean target, so in order to clean swss, execute:

make target/debs/swss_1.0.0_amd64.deb-clean

It is recommended to use clean targets to clean all packages that are built together, like dev packages for instance. In order to be more familiar with build process and make some changes to it, it is recommended to read this short Documentation.

Build debug dockers and debug SONiC installer image:

SONiC build system supports building dockers and ONIE-image with debug tools and debug symbols, to help with live & core debugging. For details refer to (SONiC Buildimage Guide).

SAI Version

Please refer to SONiC roadmap on the SAI version for each SONiC release.

Notes:

  • If you are running make for the first time, a sonic-slave-${USER} docker image will be built automatically. This may take a while, but it is a one-time action, so please be patient.

  • The root user account is disabled. However, the created user can sudo.

  • The target directory is ./target, containing the NOS installer image and docker images.

    • sonic-generic.bin: SONiC switch installer image (ONIE compatible)
    • sonic-aboot.bin: SONiC switch installer image (Aboot compatible)
    • docker-base.gz: base docker image where other docker images are built from, only used in build process (gzip tar archive)
    • docker-database.gz: docker image for in-memory key-value store, used as inter-process communication (gzip tar archive)
    • docker-fpm.gz: docker image for quagga with fpm module enabled (gzip tar archive)
    • docker-orchagent.gz: docker image for SWitch State Service (SWSS) (gzip tar archive)
    • docker-syncd-brcm.gz: docker image for the daemon to sync database and Broadcom switch ASIC (gzip tar archive)
    • docker-syncd-cavm.gz: docker image for the daemon to sync database and Cavium switch ASIC (gzip tar archive)
    • docker-syncd-mlnx.gz: docker image for the daemon to sync database and Mellanox switch ASIC (gzip tar archive)
    • docker-syncd-nephos.gz: docker image for the daemon to sync database and Nephos switch ASIC (gzip tar archive)
    • docker-syncd-invm.gz: docker image for the daemon to sync database and Innovium switch ASIC (gzip tar archive)
    • docker-sonic-p4.gz: docker image for all-in-one for p4 software switch (gzip tar archive)
    • docker-sonic-vs.gz: docker image for all-in-one for software virtual switch (gzip tar archive)
    • docker-sonic-mgmt.gz: docker image for managing, configuring and monitoring SONiC (gzip tar archive)

Contribution Guide

All contributors must sign a contribution license agreement before contributions can be accepted. Contact sonic-cla-agreements@microsoft.com.

GitHub Workflow

We're following basic GitHub Flow. If you have no idea what we're talking about, check out GitHub's official guide. Note that merge is only performed by the repository maintainer.

Guide for performing commits:

  • Isolate each commit to one component/bugfix/issue/feature
  • Use a standard commit message format:
[component/folder touched]: Description intent of your changes

[List of changes]

Signed-off-by: Your Name your@email.com

For example:

swss-common: Stabilize the ConsumerTable

* Fixing autoreconf
* Fixing unit-tests by adding checkers and initialize the DB before start
* Adding the ability to select from multiple channels
* Health-Monitor - The idea of the patch is that if something went wrong with the notification channel,
  we will have the option to know about it (Query the LLEN table length).

  Signed-off-by: user@dev.null
  • Each developer should fork this repository and add the team as a Contributor
  • Push your changes to your private fork and do "pull-request" to this repository
  • Use a pull request to do code review
  • Use issues to keep track of what is going on

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.