#### Why I did it Fixes https://github.com/Azure/sonic-buildimage/issues/8619 #### How I did it 1) Listening to CFG_DB notifications was migrated from ConfigDBConnector to SubscriberStateTable & Select 2) This change in design helped me to remove `update_all_features_config` which was roughly taking a 5-10 sec time to execute and thus the reason for blackout 3) Edited FeatureHandler, Feature & NtpCfgd classes to suit this design 4) Added corresponding mocks and UT's **Changes made to classes other than HostConfigDaemon:** With the previous design, the initially read data from the config db was applied by using hardcoded methods even before the config_db.listen() was called. For Eg: `update_all_features_config` for FeatureHandler and load() named methods for NtpCfgd etc But with this design, since the existing data is read and given out as a notification by SubscriberStateTable, i've pretty much removed these hardcoded methods. Thus changes made to these class will be around adapting them to the new design and no change in the actual functionality . #### How to verify it UT's: ``` tests/determine-reboot-cause_test.py ......... [ 29%] tests/procdockerstatsd_test.py . [ 32%] tests/caclmgrd/caclmgrd_dhcp_test.py ...... [ 51%] tests/hostcfgd/hostcfgd_radius_test.py .. [ 58%] tests/hostcfgd/hostcfgd_test.py ............. [100%] ``` Verified manually, ``` Sep 10 22:53:25.662621 sonic INFO systemd[1]: hostcfgd.service: Succeeded. Sep 10 22:55:04.127719 sonic INFO /hostcfgd: ConfigDB connect success Sep 10 22:55:04.128108 sonic INFO /hostcfgd: KdumpCfg init ... Sep 10 22:55:04.148819 sonic INFO /hostcfgd: Waiting for systemctl to finish initialization Sep 10 22:55:04.163452 sonic INFO /hostcfgd: systemctl has finished initialization -- proceeding ... Sep 10 22:55:04.163834 sonic INFO /hostcfgd: Kdump handler... Sep 10 22:55:04.164019 sonic INFO /hostcfgd: Kdump global configuration update Sep 10 22:55:04.758784 sonic INFO hostcfgd[184471]: kdump is already disabled Sep 10 22:55:04.758876 sonic INFO hostcfgd[184471]: Kdump is already disabled Sep 10 22:55:05.182021 sonic INFO hostcfgd[184511]: Kdump configuration has been updated in the startup configuration Sep 10 22:55:05.596919 sonic INFO hostcfgd[184528]: Kdump configuration has been updated in the startup configuration Sep 10 22:55:06.140627 sonic INFO /hostcfgd: Feature nat is stopped and disabled Sep 10 22:55:06.642629 sonic INFO /hostcfgd: Feature telemetry is enabled and started Sep 10 22:55:07.101297 sonic INFO /hostcfgd: Feature pmon is enabled and started Sep 10 22:55:07.554366 sonic INFO /hostcfgd: Feature database is enabled and started Sep 10 22:55:08.009329 sonic INFO /hostcfgd: Feature mgmt-framework is enabled and started Sep 10 22:55:08.394952 sonic INFO /hostcfgd: Feature macsec is stopped and disabled Sep 10 22:55:08.782853 sonic INFO /hostcfgd: Feature snmp is enabled and started Sep 10 22:55:09.205381 sonic INFO /hostcfgd: Feature teamd is enabled and started Sep 10 22:55:09.224877 sonic INFO /hostcfgd: Feature what-just-happened is enabled and started Sep 10 22:55:09.627929 sonic INFO /hostcfgd: Feature lldp is enabled and started Sep 10 22:55:10.086993 sonic INFO /hostcfgd: Feature swss is enabled and started Sep 10 22:55:10.170312 sonic INFO /hostcfgd: cmd - service aaastatsd stop Sep 10 22:55:11.012236 sonic INFO /hostcfgd: cmd - service aaastatsd stop Sep 10 22:55:12.225946 sonic INFO /hostcfgd: Feature bgp is enabled and started Sep 10 22:55:12.712792 sonic INFO /hostcfgd: Feature dhcp_relay is enabled and started Sep 10 22:55:13.166656 sonic INFO /hostcfgd: Feature sflow is stopped and disabled Sep 10 22:55:13.593639 sonic INFO /hostcfgd: Feature radv is enabled and started Sep 10 22:55:14.034106 sonic INFO /hostcfgd: Feature syncd is enabled and started Sep 10 22:55:14.113064 sonic INFO /hostcfgd: cmd - service aaastatsd stop Sep 10 22:55:14.863601 sonic INFO /hostcfgd: RADIUS_SERVER update: key: 10.10.10.1, op: SET, data: {'auth_type': 'pap', 'passkey': 'p*****', 'retransmit': '1', 'timeout': '1'} Sep 10 22:55:14.938605 sonic INFO /hostcfgd: cmd - service aaastatsd stop Sep 10 22:55:15.667545 sonic INFO /hostcfgd: RADIUS_SERVER update: key: 10.10.10.3, op: SET, data: {'auth_type': 'chap', 'passkey': 'p*****', 'retransmit': '2', 'timeout': '2'} Sep 10 22:55:15.667801 sonic INFO /hostcfgd: RADIUS (NAS) IP change - key:eth0, current global info {} Sep 10 22:55:15.746531 sonic INFO /hostcfgd: cmd - service aaastatsd stop Sep 10 23:04:47.435340 sonic INFO /hostcfgd: ntp server update key 0.debian.pool.ntp.org Sep 10 23:04:47.435661 sonic INFO /hostcfgd: ntp server update, restarting ntp-config, ntp servers configured {'0.debian.pool.ntp.org'} Sep 10 23:04:47.866394 sonic INFO /hostcfgd: NTP GLOBAL Update Sep 10 23:04:47.866557 sonic INFO /hostcfgd: ntp global update for source intf old {''} new {'eth0', 'Loopback0'}, restarting ntp-config Sep 10 23:16:25.157600 sonic INFO /hostcfgd: Running cmd: 'sudo systemctl unmask sflow.service' Sep 10 23:16:25.178472 sonic INFO hostcfgd[192106]: Removed /etc/systemd/system/sflow.service. Sep 10 23:16:25.582018 sonic INFO /hostcfgd: Running cmd: 'sudo systemctl enable sflow.service' Sep 10 23:16:25.604534 sonic INFO hostcfgd[192123]: Created symlink /etc/systemd/system/sonic.target.wants/sflow.service → /lib/systemd/system/sflow.service. Sep 10 23:16:26.029416 sonic INFO /hostcfgd: Running cmd: 'sudo systemctl start sflow.service' Sep 10 23:16:26.691927 sonic INFO /hostcfgd: Feature sflow is enabled and started ``` |
||
---|---|---|
.azure-pipelines | ||
.github | ||
device | ||
dockers | ||
files | ||
installer | ||
platform | ||
rules | ||
scripts | ||
sonic-slave-bullseye | ||
sonic-slave-buster | ||
sonic-slave-jessie | ||
sonic-slave-stretch | ||
src | ||
.artifactignore | ||
.gitignore | ||
.gitmodules | ||
azure-pipelines.yml | ||
build_debian.sh | ||
build_debug_docker_j2.sh | ||
build_docker.sh | ||
build_image.sh | ||
check_install.py | ||
functions.sh | ||
get_docker-base.sh | ||
LICENSE | ||
MAINTAINERS | ||
Makefile | ||
Makefile.cache | ||
Makefile.work | ||
onie-image-arm64.conf | ||
onie-image-armhf.conf | ||
onie-image.conf | ||
onie-mk-demo.sh | ||
push_docker.sh | ||
README.buildsystem.md | ||
README.md | ||
slave.mk | ||
ThirdPartyLicenses.txt | ||
update_screen.sh |
static analysis:
master builds:
202012 builds:
201911 builds:
201811 builds:
201807 builds:
201803 builds:
sonic-buildimage
Build SONiC Switch Images
Description
Following is the instruction on how to build an (ONIE) compatible network operating system (NOS) installer image for network switches, and also how to build docker images running inside the NOS. Note that SONiC image are build per ASIC platform. Switches using the same ASIC platform share a common image. For a list of supported switches and ASIC, please refer to this list
Hardware
Any server can be a build image server as long as it has:
- Multiple cores to increase build speed
- Plenty of RAM (less than 8 GiB is likely to cause issues)
- 300G of free disk space
A good choice of OS for building SONiC is currently Ubuntu 20.04.
Prerequisites
- Install pip and jinja in host build machine, execute below commands if j2/j2cli is not available:
sudo apt install -y python3-pip
sudo pip3 install j2cli
- Install Docker and configure your system to allow running the 'docker' command without 'sudo':
- Add current user to the docker group:
sudo gpasswd -a ${USER} docker
- Log out and log back in so that your group membership is re-evaluated
- Add current user to the docker group:
Clone or fetch the code repository with all git submodules
To clone the code repository recursively, assuming git version 1.9 or newer:
git clone https://github.com/Azure/sonic-buildimage.git
Usage
To build SONiC installer image and docker images, run the following commands:
# Ensure the 'overlay' module is loaded on your development system
sudo modprobe overlay
# Enter the source directory
cd sonic-buildimage
# (Optional) Checkout a specific branch. By default, it uses master branch. For example, to checkout the branch 201911, use "git checkout 201911"
git checkout [branch_name]
# Execute make init once after cloning the repo, or after fetching remote repo with submodule updates
make init
# Execute make configure once to configure ASIC
make configure PLATFORM=[ASIC_VENDOR]
# Build SONiC image with 4 jobs in parallel.
# Note: You can set this higher, but 4 is a good number for most cases
# and is well-tested.
make SONIC_BUILD_JOBS=4 all
The supported ASIC vendors are:
- PLATFORM=broadcom
- PLATFORM=marvell
- PLATFORM=mellanox
- PLATFORM=cavium
- PLATFORM=centec
- PLATFORM=nephos
- PLATFORM=innovium
- PLATFORM=p4
- PLATFORM=vs
Usage for ARM Architecture
To build Arm32 bit for (ARMHF) platform ARM build has dependency in docker version 18, if docker version is 19, downgrade to 18 as below sudo apt-get install --allow-downgrades -y docker-ce=5:18.09.0~3-0~ubuntu-xenial sudo apt-get install --allow-downgrades -y docker-ce-cli=5:18.09.0~3-0~ubuntu-xenial
# Execute make configure once to configure ASIC and ARCH
make configure PLATFORM=[ASIC_VENDOR] PLATFORM_ARCH=armhf
make target/sonic-[ASIC_VENDER]-armhf.bin
# example:
make configure PLATFORM=marvell-armhf PLATFORM_ARCH=armhf
make target/sonic-marvell-armhf.bin
To build Arm64 bit for platform
# Execute make configure once to configure ASIC and ARCH
make configure PLATFORM=[ASIC_VENDOR] PLATFORM_ARCH=arm64
# example:
make configure PLATFORM=marvell-arm64 PLATFORM_ARCH=arm64
NOTE:
-
Recommend reserving at least 100G free space to build one platform with a single job. The build process will use more disk if you are setting
SONIC_BUILD_JOBS
to more than 1. -
If Docker's workspace folder,
/var/lib/docker
, resides on a partition without sufficient free space, you may encounter an error like the following during a Docker container build job:/usr/bin/tar: /path/to/sonic-buildimage/<some_file>: Cannot write: No space left on device
The solution is to move the directory to a partition with more free space.
-
Use
http_proxy=[your_proxy] https_proxy=[your_proxy] no_proxy=[your_no_proxy] make
to enable http(s) proxy in the build process. -
Add your user account to
docker
group and use your user account to make.root
orsudo
are not supported.
The SONiC installer contains all docker images needed. SONiC uses one image for all devices of a same ASIC vendor.
For Broadcom ASIC, we build ONIE and EOS image. EOS image is used for Arista devices, ONIE image is used for all other Broadcom ASIC based devices.
make configure PLATFORM=broadcom
# build debian stretch required targets
BLDENV=stretch make stretch
# build ONIE image
make target/sonic-broadcom.bin
# build EOS image
make target/sonic-aboot-broadcom.swi
You may find the rules/config file useful. It contains configuration options for the build process, like adding more verbosity or showing dependencies, username and password for base image etc.
Every docker image is built and saved to target/ directory. So, for instance, to build only docker-database, execute:
make target/docker-database.gz
Same goes for debian packages, which are under target/debs/:
make target/debs/swss_1.0.0_amd64.deb
Every target has a clean target, so in order to clean swss, execute:
make target/debs/swss_1.0.0_amd64.deb-clean
It is recommended to use clean targets to clean all packages that are built together, like dev packages for instance. In order to be more familiar with build process and make some changes to it, it is recommended to read this short Documentation.
Build debug dockers and debug SONiC installer image:
SONiC build system supports building dockers and ONIE-image with debug tools and debug symbols, to help with live & core debugging. For details refer to (SONiC Buildimage Guide).
SAI Version
Please refer to SONiC roadmap on the SAI version for each SONiC release.
Notes:
-
If you are running make for the first time, a sonic-slave-${USER} docker image will be built automatically. This may take a while, but it is a one-time action, so please be patient.
-
The root user account is disabled. However, the created user can
sudo
. -
The target directory is
./target
, containing the NOS installer image and docker images.- sonic-generic.bin: SONiC switch installer image (ONIE compatible)
- sonic-aboot.bin: SONiC switch installer image (Aboot compatible)
- docker-base.gz: base docker image where other docker images are built from, only used in build process (gzip tar archive)
- docker-database.gz: docker image for in-memory key-value store, used as inter-process communication (gzip tar archive)
- docker-fpm.gz: docker image for quagga with fpm module enabled (gzip tar archive)
- docker-orchagent.gz: docker image for SWitch State Service (SWSS) (gzip tar archive)
- docker-syncd-brcm.gz: docker image for the daemon to sync database and Broadcom switch ASIC (gzip tar archive)
- docker-syncd-cavm.gz: docker image for the daemon to sync database and Cavium switch ASIC (gzip tar archive)
- docker-syncd-mlnx.gz: docker image for the daemon to sync database and Mellanox switch ASIC (gzip tar archive)
- docker-syncd-nephos.gz: docker image for the daemon to sync database and Nephos switch ASIC (gzip tar archive)
- docker-syncd-invm.gz: docker image for the daemon to sync database and Innovium switch ASIC (gzip tar archive)
- docker-sonic-p4.gz: docker image for all-in-one for p4 software switch (gzip tar archive)
- docker-sonic-vs.gz: docker image for all-in-one for software virtual switch (gzip tar archive)
- docker-sonic-mgmt.gz: docker image for managing, configuring and monitoring SONiC (gzip tar archive)
Contribution Guide
All contributors must sign a contribution license agreement before contributions can be accepted. Contact sonic-cla-agreements@microsoft.com.
GitHub Workflow
We're following basic GitHub Flow. If you have no idea what we're talking about, check out GitHub's official guide. Note that merge is only performed by the repository maintainer.
Guide for performing commits:
- Isolate each commit to one component/bugfix/issue/feature
- Use a standard commit message format:
[component/folder touched]: Description intent of your changes [List of changes] Signed-off-by: Your Name your@email.com
For example:
swss-common: Stabilize the ConsumerTable * Fixing autoreconf * Fixing unit-tests by adding checkers and initialize the DB before start * Adding the ability to select from multiple channels * Health-Monitor - The idea of the patch is that if something went wrong with the notification channel, we will have the option to know about it (Query the LLEN table length). Signed-off-by: user@dev.null
- Each developer should fork this repository and add the team as a Contributor
- Push your changes to your private fork and do "pull-request" to this repository
- Use a pull request to do code review
- Use issues to keep track of what is going on
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.