Scripts which perform an installable binary image build for SONiC
Go to file
Ze Gan 3a01ee9be8
[submodule][master] Update submodule sonic-swss/sonic-sairedis to the latest HEAD (#16599)
sonic-swss:
```
* 13ef25bf - (HEAD -> master, origin/master, origin/HEAD) [teamd]: Clean teamd process if LAG creation fails (#2888) (5 days ago) [Lawrence Lee]
* ae010bfa - Support type7 encoded CAK key for macsec in config_db (#2892) (13 days ago) [judyjoseph]
* e6f134fb - [orchagent]: admin-disable port before setPortSerdesAttribute() (#2831) (4 weeks ago) [Aman Singhal]
* a67d4a77 - Change default branch to build_branch (#2885) (4 weeks ago) [Ze Gan]
* d44761cc - Make sure new binaries replace existing binaries in docker-sonic-vs (#2870) (4 weeks ago) [Saikrishna Arcot]
* 7102220a - [Fixbug]: Fix vnet attribute miss if route action is vnet_direct and vnet test cases (#2873) (#2877) (4 weeks ago) [Ze Gan]
*   873455b7 - Merge pull request #2878 from Pterosaur/enable_dash_vstest (4 weeks ago) [Guohan Lu]
|\
| * 44457c86 - Simplify test task and remove duplicated artifacts (4 weeks ago) [Ze Gan]
| * 8bca4ed2 - Trigger Azp (4 weeks ago) [Ze Gan]
| *   6de56ee1 - Merge branch 'master' into enable_dash_vstest (4 weeks ago) [Ze Gan]
| |\
| * | aad88a36 - Disable test_dash_crm (5 weeks ago) [Ze Gan]
| * | 61126eb0 - Enable Dash test in Pipeline (5 weeks ago) [Ze Gan]
* | | ecd88108 - update portStatIds for cisco (#2876) (4 weeks ago) [Zhixin Zhu]
* | | f1294999 - [ppi] Relax port attributes validation (#2872) (4 weeks ago) [Nazarii Hnydyn]
| |/
|/|
* | b4fcfc9f - Remove fabric queue counters. (#2862) (5 weeks ago) [jfeng-arista]
|/
*   bb99f418 - Merge pull request #2856 from theasianpianist/master-dash-merge (5 weeks ago) [Guohan Lu]
|\
| *   84b32af2 - Merge branch 'master' into master-dash-merge (5 weeks ago) [Lawrence Lee]
| |\
| |/
|/|
* | ca728200 - [FEC] Adding support of override based on attribute query of SAI_PORT_ATTR_AUTO_NEG_FEC_MODE_OVERRIDE  (#2874) (5 weeks ago) [Sudharsan Dhamal Gopalarathnam]
 /
* 3bb71809 - Merge branch 'master' into master-dash-merge (5 weeks ago) [Lawrence Lee]
* 574940dd - Merge branch 'master' into master-dash-merge (6 weeks ago) [Lawrence Lee]
* 660e5e4c - Merge branch 'master' into master-dash-merge (6 weeks ago) [Lawrence Lee]
* 6d941746 - Merge branch 'master' into master-dash-merge (6 weeks ago) [Lawrence Lee]
* c87c86e6 - [dash]: Refactor DASH orch by protobuf format (#2722) (8 weeks ago) [Ze Gan]
* c999ea32 - [tests]: Change DVS ENV HWKSU to DPU-2P for DASH vstest (#2847) (8 weeks ago) [prabhataravind]
* b2c25dcd - (conflict)[dash] Improve dash orchagent ZMQ code. (#2836) (8 weeks ago) [Hua Liu]
* 409b3833 - [tests]: Set HWSKU to NPU-2P for dash vstests (#2833) (8 weeks ago) [prabhataravind]
* f2365af9 - (conflict)Enable/disable Zmq by parameter (#2828) (8 weeks ago) [Hua Liu]
* 3ade5fc3 - (conflict)[dash] Change dash orchagent from Redis consumer state table to ZMQ consumer state table. (#2779) (8 weeks ago) [Hua Liu]
* eaf1bb85 - [crm]: Remove NOT_IMPLEMENTED checks (8 weeks ago) [Lawrence Lee]
* 916d2f10 - [azp]: Don't run DASH tests for regular test runs (8 weeks ago) [Lawrence Lee]
* 208e80bd - update azp to use public pipeline artifacts (8 weeks ago) [Lawrence Lee]
* 2168554d - [dash][ci] fix build pipeline (8 weeks ago) [Yakiv Huryk]
* b6036635 - [dash][ci] update build pipeline to build with bullseye (8 weeks ago) [Yakiv Huryk]
* 37a61ddb - (conflict)[azp] Add DASH to PR trigger for non-DASH VS tests (#2813) (8 weeks ago) [Lawrence Lee]
* 37d27b01 - Fix Dash orchagent build issue. (#2788) (8 weeks ago) [Hua Liu]
* 07cce313 - (conflict)[CRM][DASH] Extend CrmOrch to support DASH resources. (#2739) (8 weeks ago) [Oleksandr Ivantsiv]
* 7c435d1e - [DASH]: Miscellaneous bug fixes and adding vstests (#2745) (8 weeks ago) [prabhataravind]
* 6613dd4f - [dash]: Check if overlay IP is specified (#2741) (8 weeks ago) [Lawrence Lee]
* c863d48b - [dash] Do not use an action drop with the inbound routing table. (#2710) (8 weeks ago) [Oleksandr Ivantsiv]
* 89ce4e0c - [dash]: Don't attempt to bind empty ACL groups (#2613) (8 weeks ago) [Lawrence Lee]
* 8ec36a6f - (conflict)[dash]: ACL orchagent (#2470) (8 weeks ago) [Ze Gan]
* 29c23b12 - [DASH] Fix compilation issue caused by merge from the master branch. (#2594) (8 weeks ago) [Oleksandr Ivantsiv]
* fd3539e5 - [DASH] Add retry logic for VNET mapping table (#2583) (8 weeks ago) [Lawrence Lee]
* 9b179c07 - [dash] add USE_DST_VNET_VNI attribute to CA-to-PA entry (#2533) (8 weeks ago) [Yakiv Huryk]
* f7fe55fa - Add SAI_ENI_ATTR_VM_UNDERLAY_DIP and SAI_ENI_ATTR_VM_VNI attributes to ENI entry (#2514) (8 weeks ago) [prabhataravind]
* aa2a02c5 - (conflict)[Azp]: Add Azp for DASH (#2501) (8 weeks ago) [Ze Gan]
* 2d1972f2 - (conflict)orchagent: DASH changes (#2459) (8 weeks ago) [prabhataravind]
* acf0fe42 - [DPU] Fix unit tests compilation after merge from master branch. (#2478) (8 weeks ago) [Oleksandr Ivantsiv]
* 22c62f63 - (conflict)[DPU] Simplify SWSS initialization to meet DPU requirements. (#2440) (8 weeks ago) [Oleksandr Ivantsiv]
```

sonic-sairedis
```
* cfa8da4 - (HEAD -> master, origin/master, origin/HEAD) Add extra parameter to pass vendor LDFLAGS for libsai.so (#1291) (3 days ago) [Kamil Cudnik]
* 8046908 - [CRM][DASH] Add the possibility of querying availability for OIDs. (#1245) (5 days ago) [Oleksandr Ivantsiv]
* 9547060 - Install nlohmann-json3-dev package for codeql (#1290) (10 days ago) [Saikrishna Arcot]
* f3b4dd5 - Use json.hpp from nlohmann-json-dev instead of swss-common (#1289) (11 days ago) [Saikrishna Arcot]
* 40c9d13 - [azp] Update az pipeline for swss docker to add syslog (#1287) (2 weeks ago) [Kamil Cudnik]
* 4c2527f - port counter support on sonic-vs (#1275) (3 weeks ago) [Vishnu Shetty]
* 92c58cf - [Azp]: Change default branch to build_branch (#1279) (4 weeks ago) [Ze Gan]
* 7178fb6 - [submodule] Update SAI to latest v1.12 branch (#1284) (4 weeks ago) [Oleksandr Ivantsiv]
* 52247b9 - [syncd] Fix missing comma (#1278) (4 weeks ago) [Kamil Cudnik]
* 44cd8c4 - [azp] Attempt to fix swss missing libs (#1277) (4 weeks ago) [Ze Gan]
* ee308bb - [submodule] Update SAI to latest v1.12 branch (#1272) (5 weeks ago) [Kamil Cudnik]
```
2023-09-25 19:05:01 -07:00
.azure-pipelines [ci] Disable building broadcom raw image because of S6100 device disk space limit. (#16516) 2023-09-11 15:10:03 -07:00
.github [action] Update workflow to parse & monitor pending automation PRs. (#16446) 2023-09-07 13:34:34 +08:00
device [Arista] Add new hwskus to x86_64-arista_7060dx5_32 (#16077) 2023-09-23 01:42:31 -07:00
dockers Improve remote fetch (#12795) 2023-09-23 18:07:04 -07:00
files Improve remote fetch (#12795) 2023-09-23 18:07:04 -07:00
installer Update to Linux 5.10.179 (#15926) 2023-09-20 15:24:39 -07:00
platform [Nokia][sonic-platform] Update Nokia sonic-platform submodule - SFP support for CMIS CDB operations (#16572) 2023-09-23 14:09:02 -07:00
rules Update to Linux 5.10.179 (#15926) 2023-09-20 15:24:39 -07:00
scripts Improve remote fetch (#12795) 2023-09-23 18:07:04 -07:00
sonic-slave-bullseye Improve remote fetch (#12795) 2023-09-23 18:07:04 -07:00
sonic-slave-buster Improve remote fetch (#12795) 2023-09-23 18:07:04 -07:00
sonic-slave-jessie Improve remote fetch (#12795) 2023-09-23 18:07:04 -07:00
sonic-slave-stretch Improve remote fetch (#12795) 2023-09-23 18:07:04 -07:00
src [submodule][master] Update submodule sonic-swss/sonic-sairedis to the latest HEAD (#16599) 2023-09-25 19:05:01 -07:00
.artifactignore [ci] Archive compiled Debian packages and Python wheels (#6650) 2021-02-02 23:42:03 -08:00
.gitignore Improve remote fetch (#12795) 2023-09-23 18:07:04 -07:00
.gitmodules [frr]: FRR 8.5.1 integration changes (#15965) 2023-08-07 09:45:13 -07:00
azure-pipelines.yml [ci]: Install swig and nlohmann-json3-dev for vstest (#16609) 2023-09-23 00:03:36 -07:00
build_debian.sh Improve remote fetch (#12795) 2023-09-23 18:07:04 -07:00
build_debug_docker_j2.sh [sonic-buildimage] Fix build issue for docker-dhcp-relay-dbg.gz. Issue (#4136) 2020-02-10 17:16:42 -08:00
build_docker.sh [Build] use pigz to speed up a build (#12825) 2022-12-17 14:38:31 -08:00
build_image.sh [build] Remove compression of raw image (#16462) 2023-09-07 09:19:17 -07:00
check_install.py Add California-SB237 feature. Requires to change default user password (#12678) 2023-02-23 15:36:37 -08:00
functions.sh [build] fix CI warnings issued by "git describe" (#13098) 2023-01-03 10:04:31 -08:00
get_docker-base.sh Add mkdir if the target dir does not exist (#130) 2016-12-16 02:19:15 +00:00
install_sonic.py [build] Increase timeout value when installing SONiC image on kvm (#11191) 2022-07-20 08:13:28 +08:00
LICENSE Change build-image licenses to Apache (#15377) 2023-07-27 08:30:12 -07:00
MAINTAINERS Adding license and maintainers 2016-03-08 19:10:18 -08:00
Makefile [build] Add retry when make SONiC image to improve success rate. (#12325) 2022-12-19 12:18:36 +08:00
Makefile.cache [Build] Update SLAVE_BASE_TAG and DPKG cache if Debian mirrors were changed (#12702) 2022-11-15 13:02:34 +08:00
Makefile.work [dhcp_server] Add dhcp_server container (#14031) 2023-09-11 09:15:56 -07:00
onie-image-arm64.conf update DOCKER_RAMFS_SIZE (#16305) 2023-08-31 16:49:03 +08:00
onie-image-armhf.conf update DOCKER_RAMFS_SIZE (#16305) 2023-08-31 16:49:03 +08:00
onie-image.conf [build] Change raw image disk size to 1700MB (#16463) 2023-09-07 09:19:54 -07:00
onie-mk-demo.sh Add support for secure upgrade (#11862) 2023-06-26 12:04:40 +03:00
push_docker.sh [ci] Support multi tags when pushing docker image (#10771) 2022-05-09 16:43:21 +08:00
README.buildsystem.md [docs] Correct clone instructions & typos (#12733) 2022-11-18 15:00:16 +08:00
README.md [doc] add pipeline link into README (#16347) 2023-09-18 17:53:58 -07:00
slave.mk [dhcp_server] Add dhcp_server container (#14031) 2023-09-11 09:15:56 -07:00
ThirdPartyLicenses.txt [TACACS+] Add Bash TACACS+ plugin for per-command authorization. (#8715) 2021-11-13 09:57:30 +08:00
update_screen.sh [build]: Added support for cache status on the build output (#5564) 2020-10-09 02:49:20 -07:00

master builds:

Barefoot Broadcom Centec Centec(arm64) Innovium Mellanox Marvell(armhf) Marvell(arm64) Nephos VS

202305 builds:

Barefoot Broadcom Centec Centec(arm64) Innovium Mellanox Marvell(armhf) Nephos VS

202211 builds:

Barefoot Broadcom Centec Centec(arm64) Innovium Mellanox Marvell(armhf) Nephos VS

202205 builds:

Barefoot Broadcom Centec Centec(arm64) Innovium Mellanox Marvell(armhf) Nephos VS

202111 builds:

Barefoot Broadcom Centec Centec(arm64) Innovium Mellanox Marvell(armhf) Nephos VS

202012 builds:

Barefoot Broadcom Centec Centec(arm64) Innovium Marvell(armhf) Mellanox Nephos VS

201911 builds:

Barefoot Broadcom Innovium Mellanox Nephos VS

201811 builds:

Broadcom Mellanox Innovium Nephos VS

sonic-buildimage

Build SONiC Switch Images

Description

Following are the instructions on how to build an (ONIE) compatible network operating system (NOS) installer image for network switches, and also how to build docker images running inside the NOS. Note that SONiC images are build per ASIC platform. Switches using the same ASIC platform share a common image. For a list of supported switches and ASIC, please refer to this list

Hardware

Any server can be a build image server as long as it has:

  • Multiple cores to increase build speed
  • Plenty of RAM (less than 8 GiB is likely to cause issues)
  • 300G of free disk space
  • KVM Virtualization Support.

Note: If you are in a VM, make sure you have support for nested virtualization. Some cases (e.g. building OVS image) also requires extra configuration options to expose the full KVM interface to the VM (e.g. the KVM paravirtualization support on VirtualBox).

A good choice of OS for building SONiC is currently Ubuntu 20.04.

Prerequisites

  • Install pip and jinja in host build machine, execute below commands if j2/j2cli is not available:
sudo apt install -y python3-pip
pip3 install --user j2cli
  • Install Docker and configure your system to allow running the 'docker' command without 'sudo':
    • Add current user to the docker group: sudo gpasswd -a ${USER} docker
    • Log out and log back in so that your group membership is re-evaluated

Note: If a previous installation of Docker using snap was present on the system, remove it and also remove docker from snap before reinstallating docker. This will avoid known bugs that falsely report read-only filesystems issues during the build process.

Clone the repository with all the git submodules

To clone the code repository recursively:

git clone --recurse-submodules https://github.com/sonic-net/sonic-buildimage.git

Usage

To build SONiC installer image and docker images, run the following commands:

# Ensure the 'overlay' module is loaded on your development system
sudo modprobe overlay

# Enter the source directory
cd sonic-buildimage

# (Optional) Checkout a specific branch. By default, it uses master branch.
# For example, to checkout the branch 201911, use "git checkout 201911"
git checkout [branch_name]

# Execute make init once after cloning the repo,
# or after fetching remote repo with submodule updates
make init

# Execute make configure once to configure ASIC
make configure PLATFORM=[ASIC_VENDOR]

# Build SONiC image with 4 jobs in parallel.
# Note: You can set this higher, but 4 is a good number for most cases
#       and is well-tested.
make SONIC_BUILD_JOBS=4 all

The supported ASIC vendors are:

  • PLATFORM=barefoot
  • PLATFORM=broadcom
  • PLATFORM=marvell
  • PLATFORM=mellanox
  • PLATFORM=cavium
  • PLATFORM=centec
  • PLATFORM=nephos
  • PLATFORM=innovium
  • PLATFORM=vs

Usage for ARM Architecture

ARM build has dependency in docker version 18. If docker version is 19, downgrade to 18 with:

sudo apt-get install --allow-downgrades -y docker-ce=5:18.09.0~3-0~ubuntu-xenial
sudo apt-get install --allow-downgrades -y docker-ce-cli=5:18.09.0~3-0~ubuntu-xenial

To build Arm32 bit for (ARMHF) platform

# Execute make configure once to configure ASIC and ARCH
make configure PLATFORM=[ASIC_VENDOR] PLATFORM_ARCH=armhf
make target/sonic-[ASIC_VENDER]-armhf.bin

example:

make configure PLATFORM=marvell-armhf PLATFORM_ARCH=armhf
make target/sonic-marvell-armhf.bin

To build Arm32 bit for (ARMHF) Marvell platform on amd64 host for debian buster using cross-compilation, run the following commands:

# Execute make configure once to configure ASIC and ARCH for cross-compilation build

NOJESSIE=1 NOSTRETCH=1 BLDENV=buster CROSS_BLDENV=1 \
make configure PLATFORM=marvell-armhf PLATFORM_ARCH=armhf

# Execute Arm32 build using cross-compilation environment

NOJESSIE=1 NOSTRETCH=1 BLDENV=buster CROSS_BLDENV=1 make target/sonic-marvell-armhf.bin

Running the above Arm32 build using cross-compilation instead of qemu emulator drastically reduces the build time.

To build Arm64 bit for platform

# Execute make configure once to configure ASIC and ARCH

make configure PLATFORM=[ASIC_VENDOR] PLATFORM_ARCH=arm64

# example:

make configure PLATFORM=marvell-arm64 PLATFORM_ARCH=arm64

NOTE:

  • Recommend reserving at least 100G free space to build one platform with a single job. The build process will use more disk if you are setting SONIC_BUILD_JOBS to more than 1.

  • If Docker's workspace folder, /var/lib/docker, resides on a partition without sufficient free space, you may encounter an error like the following during a Docker container build job:

    /usr/bin/tar: /path/to/sonic-buildimage/<some_file>: Cannot write: No space left on device

    The solution is to move the directory to a partition with more free space.

  • Use http_proxy=[your_proxy] https_proxy=[your_proxy] no_proxy=[your_no_proxy] make to enable http(s) proxy in the build process.

  • Add your user account to docker group and use your user account to make. root or sudo are not supported.

The SONiC installer contains all docker images needed. SONiC uses one image for all devices of a same ASIC vendor.

For Broadcom ASIC, we build ONIE and EOS image. EOS image is used for Arista devices, ONIE image is used for all other Broadcom ASIC based devices.

make configure PLATFORM=broadcom
# build debian stretch required targets
BLDENV=stretch make stretch
# build ONIE image
make target/sonic-broadcom.bin
# build EOS image
make target/sonic-aboot-broadcom.swi

You may find the rules/config file useful. It contains configuration options for the build process, like adding more verbosity or showing dependencies, username and password for base image etc.

Every docker image is built and saved to target/ directory. So, for instance, to build only docker-database, execute:

make target/docker-database.gz

Same goes for debian packages, which are under target/debs/:

make target/debs/swss_1.0.0_amd64.deb

Every target has a clean target, so in order to clean swss, execute:

make target/debs/swss_1.0.0_amd64.deb-clean

It is recommended to use clean targets to clean all packages that are built together, like dev packages for instance. In order to be more familiar with build process and make some changes to it, it is recommended to read this short Documentation.

Build debug dockers and debug SONiC installer image

SONiC build system supports building dockers and ONIE-image with debug tools and debug symbols, to help with live & core debugging. For details refer to SONiC Buildimage Guide.

SAI Version

Please refer to SONiC roadmap on the SAI version for each SONiC release.

Notes

  • If you are running make for the first time, a sonic-slave-${USER} docker image will be built automatically. This may take a while, but it is a one-time action, so please be patient.
  • The root user account is disabled. However, the created user can sudo.
  • The target directory is ./target, containing the NOS installer image and docker images.
    • sonic-generic.bin: SONiC switch installer image (ONIE compatible)
    • sonic-aboot.bin: SONiC switch installer image (Aboot compatible)
    • docker-base.gz: base docker image where other docker images are built from, only used in build process (gzip tar archive)
    • docker-database.gz: docker image for in-memory key-value store, used as inter-process communication (gzip tar archive)
    • docker-fpm.gz: docker image for quagga with fpm module enabled (gzip tar archive)
    • docker-orchagent.gz: docker image for SWitch State Service (SWSS) (gzip tar archive)
    • docker-syncd-brcm.gz: docker image for the daemon to sync database and Broadcom switch ASIC (gzip tar archive)
    • docker-syncd-cavm.gz: docker image for the daemon to sync database and Cavium switch ASIC (gzip tar archive)
    • docker-syncd-mlnx.gz: docker image for the daemon to sync database and Mellanox switch ASIC (gzip tar archive)
    • docker-syncd-nephos.gz: docker image for the daemon to sync database and Nephos switch ASIC (gzip tar archive)
    • docker-syncd-invm.gz: docker image for the daemon to sync database and Innovium switch ASIC (gzip tar archive)
    • docker-sonic-p4.gz: docker image for all-in-one for p4 software switch (gzip tar archive)
    • docker-sonic-vs.gz: docker image for all-in-one for software virtual switch (gzip tar archive)
    • docker-sonic-mgmt.gz: docker image for managing, configuring and monitoring SONiC (gzip tar archive)

SONiC Image Azure Pipelines

All SONiC project build pipeline could be found at Download Portal for SONiC Images

Contribution Guide

All contributors must sign a contribution license agreement before contributions can be accepted. Visit EasyCLA - Linux Foundation.

GitHub Workflow

We're following basic GitHub Flow. If you have no idea what we're talking about, check out GitHub's official guide. Note that merge is only performed by the repository maintainer.

Guide for performing commits:

  • Isolate each commit to one component/bugfix/issue/feature
  • Use a standard commit message format:

[component/folder touched]: Description intent of your changes

[List of changes]

Signed-off-by: Your Name your@email.com

For example:

swss-common: Stabilize the ConsumerTable

  • Fixing autoreconf
  • Fixing unit-tests by adding checkers and initialize the DB before start
  • Adding the ability to select from multiple channels
  • Health-Monitor - The idea of the patch is that if something went wrong with the notification channel, we will have the option to know about it (Query the LLEN table length).

Signed-off-by: user@dev.null

  • Each developer should fork this repository and add the team as a Contributor
  • Push your changes to your private fork and do "pull-request" to this repository
  • Use a pull request to do code review
  • Use issues to keep track of what is going on

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.