Commit Graph

293 Commits

Author SHA1 Message Date
rajendra-dendukuri
9c7105b5f3
Install swsssdk-py3 in the base Debian image for python3 based apps (#4542)
Signed-off-by: Rajendra Dendukuri <rajendra.dendukuri@broadcom.com>
2020-05-19 11:15:05 -07:00
lguohan
087e5fbf78
[build]: build missing stretch dockers when debug build is enabled (#4494)
$DOCKER_IMAGES_FOR_INSTALLERS contains all dockers to be included
in the image. If INSTALL_DEBUG_TOOLS=y, then it contains debug docker
images as well non-debug docker images when there is no debug version
of that docker image.

$STRETCH_DOCKER_IMAGES will have the non-debug docker images needed
by the image. $STRETCH_DBG_DOCKER_IMAGES should contain the debug
version of the docker images.

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-04-29 11:07:08 -07:00
joyas-joseph
fc55329b22
[mgmt-framework]: convert mgmt-framework to use buster docker (#4480)
This change adds support to build dockers using buster as base.
- Define docker-base-buster using docker-base-stretch as starting point
- Define docker-config-engine-buster using docker-config-engine-stretch as starting point.
- sonic-mgmt-framework docker is updated to build using buster as base

Signed-off-by: Joyas Joseph <joyas_joseph@dell.com>
2020-04-28 15:57:13 -07:00
Guohan Lu
67cc385e36 [baseimage]: remove ixgbe.ko as 4.19 has new ixgbe driver
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-04-17 04:51:51 +00:00
Guohan Lu
fc0d4d1bab [build]: setup correct debs files dependency for docker targets
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-04-17 04:51:51 +00:00
Guohan Lu
01cb7934b0 [build]: add buster docker as the last step of the build proces
- build SONIC_STRETCH_DOCKERS in sonic-slave-stretch docker
- build image related module in sonic-slave-buster docker.
  This includes all kernels modules and some packages

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-04-16 10:26:18 +00:00
Praveen Chaudhary
a02255e2f4
[sonic-yang-models]: First version of yang models for Port, VLan, Interface, PortChannel, loopback and ACL. (#3730)
[sonic-yang-models]: First version of yang models for Port, VLan, Interface, PortChannel, loopback and ACL.

YANG models as per Guidelines.

Guideline doc: https://github.com/Azure/SONiC/blob/master/doc/mgmt/SONiC_YANG_Model_Guidelines.md

[sonic-yang-models/tests]: YANG model test code and JSON input for testing.

[sonic-yang-models/setup.py]: Build infra for yang models.

**- What I did**
Created Yang model for Sonic.
Tables:  PORT, VLAN, VLAN_INTERFACE, VLAN_MEMBER, ACL_RULE, ACL_TABLE, INTERFACE.

Created build infra files using which a new package (sonic-yang-models) can be build and can be deployed on sonic switches. Yang models will be part of this new package.

**- How I did it**
Wrote yang models based on Guideline doc: 
https://github.com/Azure/SONiC/blob/master/doc/mgmt/SONiC_YANG_Model_Guidelines.md
and 
https://github.com/Azure/SONiC/wiki/Configuration.

Wrote python wheel Package infra which runs test for these Yang models using a json files which consists configuration as per yang models. These configs are for negative tests, which means we want to test that most must condition, pattern and when condition works as expected.

**- How to verify it**
Build Logs and testing:
———————————————————————————————————

```
/sonic/src/sonic-yang-models /sonic
running test
running egg_info
writing top-level names to sonic_yang_models.egg-info/top_level.txt
writing dependency_links to sonic_yang_models.egg-info/dependency_links.txt
writing sonic_yang_models.egg-info/PKG-INFO
reading manifest file 'sonic_yang_models.egg-info/SOURCES.txt'
writing manifest file 'sonic_yang_models.egg-info/SOURCES.txt'
running build_ext

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
running bdist_wheel
running build
running build_py
(Reading database ... 155852 files and directories currently installed.)
Preparing to unpack .../libyang_1.0.73_amd64.deb ...
Unpacking libyang (1.0.73) over (1.0.73) ...
Setting up libyang (1.0.73) ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
Processing triggers for man-db (2.7.6.1-2) ...
(Reading database ... 155852 files and directories currently installed.)
Preparing to unpack .../libyang-cpp_1.0.73_amd64.deb ...
Unpacking libyang-cpp (1.0.73) over (1.0.73) ...
Setting up libyang-cpp (1.0.73) ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
(Reading database ... 155852 files and directories currently installed.)
Preparing to unpack .../python3-yang_1.0.73_amd64.deb ...
Unpacking python3-yang (1.0.73) over (1.0.73) ...
Setting up python3-yang (1.0.73) ...
INFO:YANG-TEST:module: sonic-vlan is loaded successfully
ERROR:YANG-TEST:Could not get module: sonic-head
INFO:YANG-TEST:module: sonic-portchannel is loaded successfully
INFO:YANG-TEST:module: sonic-acl is loaded successfully
INFO:YANG-TEST:module: sonic-loopback-interface is loaded successfully
ERROR:YANG-TEST:Could not get module: sonic-port
INFO:YANG-TEST:module: sonic-interface is loaded successfully
INFO:YANG-TEST:
------------------- Test 1: Configure a member port in VLAN_MEMBER table which does not exist.---------------------
libyang[0]: Leafref "/sonic-port:sonic-port/sonic-port:PORT/sonic-port:PORT_LIST/sonic-port:port_name" of value "Ethernet156" points to a non
-existing leaf. (path: /sonic-vlan:sonic-vlan/VLAN_MEMBER/VLAN_MEMBER_LIST[vlan_name='Vlan100'][port='Ethernet156']/port)
INFO:YANG-TEST:Configure a member port in VLAN_MEMBER table which does not exist. Passed

INFO:YANG-TEST:
------------------- Test 2: Configure non-existing ACL_TABLE in ACL_RULE.---------------------
libyang[0]: Leafref "/sonic-acl:sonic-acl/sonic-acl:ACL_TABLE/sonic-acl:ACL_TABLE_LIST/sonic-acl:ACL_TABLE_NAME" of value "NOT-EXIST" points
to a non-existing leaf. (path: /sonic-acl:sonic-acl/ACL_RULE/ACL_RULE_LIST[ACL_TABLE_NAME='NOT-EXIST'][RULE_NAME='Rule_20']/ACL_TABLE_NAME)
INFO:YANG-TEST:Configure non-existing ACL_TABLE in ACL_RULE. Passed

INFO:YANG-TEST:
------------------- Test 3: Configure IP_TYPE as ARP and ICMPV6_CODE in ACL_RULE.---------------------
libyang[0]: When condition "boolean(IP_TYPE[.='ANY' or .='IP' or .='IPV6' or .='IPv6ANY'])" not satisfied. (path: /sonic-acl:sonic-acl/ACL_RU
LE/ACL_RULE_LIST[ACL_TABLE_NAME='NO-NSW-PACL-V4'][RULE_NAME='Rule_40']/ICMPV6_CODE)
INFO:YANG-TEST:Configure IP_TYPE as ARP and ICMPV6_CODE in ACL_RULE. Passed
INFO:YANG-TEST:

INFO:YANG-TEST:
------------------- Test 4: Configure IP_TYPE as ipv4any and SRC_IPV6 in ACL_RULE.---------------------
libyang[0]: When condition "boolean(IP_TYPE[.='ANY' or .='IP' or .='IPV6' or .='IPv6ANY'])" not satisfied. (path: /sonic-acl:sonic-acl/ACL_RU
LE/ACL_RULE_LIST[ACL_TABLE_NAME='NO-NSW-PACL-V4'][RULE_NAME='Rule_20']/SRC_IPV6)
INFO:YANG-TEST:Configure IP_TYPE as ipv4any and SRC_IPV6 in ACL_RULE. Passed

------------------- Test 5: Configure l4_src_port_range as 99999-99999 in ACL_RULE---------------------
libyang[0]: Value "99999-99999" does not satisfy the constraint "([0-9]{1,4}|[0-5][0-9]{4}|[6][0-4][0-9]{3}|[6][5][0-2][0-9]{2}|[6][5][3][0-5]{2}|[6][5][3][6][0-5])-([0-9]{1,4}|[0-5][0-9]{4}|[6][0-4][0-9]{3}|[6][5][0-2][0-9]{2}|[6][5][3][0-5]{2}|[6][5][3][6][0-5])" (range, length, or pattern). (path: /sonic-acl:sonic-acl/ACL_RULE/ACL_RULE_LIST[ACL_TABLE_NAME='NO-NSW-PACL-V6'][RULE_NAME='Rule_20']/L4_SRC_PORT_RANGE)
INFO:YANG-TEST:Configure l4_src_port_range as 99999-99999 in ACL_RULE Passed

INFO:YANG-TEST:
------------------- Test 6: Configure empty string as ip-prefix in INTERFACE table.---------------------
libyang[0]: Invalid value "" in "ip-prefix" element. (path: /sonic-interface:sonic-interface/INTERFACE/INTERFACE_LIST[interface='Ethernet8'][ip-prefix='']/ip-prefix)
INFO:YANG-TEST:Configure empty string as ip-prefix in INTERFACE table. Passed

INFO:YANG-TEST:
------------------- Test 7: Configure Wrong family with ip-prefix for VLAN_Interface Table---------------------
libyang[0]: Must condition "(contains(../ip-prefix, ':') and current()='IPv6') or                               (contains(../ip-prefix, '.') and current()='IPv4')" not satisfied. (path: /sonic-vlan:sonic-vlan/VLAN_INTERFACE/VLAN_INTERFACE_LIST[vlanid='100'][ip-prefix='2a04:5555:66:7777::1/64']/family)
INFO:YANG-TEST:Configure Wrong family with ip-prefix for VLAN_Interface Table Passed

INFO:YANG-TEST:
------------------- Test 8: Configure IP_TYPE as ARP and DST_IPV6 in ACL_RULE.---------------------
libyang[0]: When condition "boolean(IP_TYPE[.='ANY' or .='IP' or .='IPV6' or .='IPV6ANY'])" not satisfied. (path: /sonic-acl:sonic-acl/ACL_RULE/ACL_RULE_LIST[ACL_TABLE_NAME='NO-NS
W-PACL-V6'][RULE_NAME='Rule_20']/DST_IPV6)
INFO:YANG-TEST:Configure IP_TYPE as ARP and DST_IPV6 in ACL_RULE. Passed

INFO:YANG-TEST:
------------------- Test 9: Configure INNER_ETHER_TYPE as 0x080C in ACL_RULE.---------------------
libyang[0]: Value "0x080C" does not satisfy the constraint "(0x88CC|0x8100|0x8915|0x0806|0x0800|0x86DD|0x8847)" (range, length, or pattern). (path: /sonic-acl:sonic-acl/ACL_RULE/ACL_RULE_LIST[ACL_TABLE_NAME='NO-NSW-PACL-V4'][RULE_NAME='Rule_40']/INNER_ETHER_TYPE)
INFO:YANG-TEST:Configure INNER_ETHER_TYPE as 0x080C in ACL_RULE. Passed

INFO:YANG-TEST:
------------------- Test 10: Add dhcp_server which is not in correct ip-prefix format.---------------------
libyang[0]: Invalid value "10.186.72.566" in "dhcp_servers" element. (path: /sonic-vlan:sonic-vlan/VLAN/VLAN_LIST/dhcp_servers[.='10.186.72.566'])
INFO:YANG-TEST:Add dhcp_server which is not in correct ip-prefix format. Passed

INFO:YANG-TEST:
------------------- Test 11: Configure undefined acl_table_type in ACL_TABLE table.---------------------
libyang[0]: Invalid value "LAYER3V4" in "type" element. (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='NO-NSW-PACL-V6']/type)
INFO:YANG-TEST:Configure undefined acl_table_type in ACL_TABLE table. Passed

INFO:YANG-TEST:
------------------- Test 12: Configure undefined packet_action in ACL_RULE table.---------------------
libyang[0]: Invalid value "SEND" in "PACKET_ACTION" element. (path: /sonic-acl:sonic-acl/ACL_RULE/ACL_RULE_LIST/PACKET_ACTION)
INFO:YANG-TEST:Configure undefined packet_action in ACL_RULE table. Passed

INFO:YANG-TEST:
------------------- Test 13: Configure wrong value for tagging_mode.---------------------
libyang[0]: Invalid value "non-tagged" in "tagging_mode" element. (path: /sonic-vlan:sonic-vlan/VLAN_MEMBER/VLAN_MEMBER_LIST/tagging_mode)
INFO:YANG-TEST:Configure wrong value for tagging_mode. Passed

INFO:YANG-TEST:
------------------- Test 14: Configure vlan-id in VLAN_MEMBER table which does not exist in VLAN  table.---------------------
libyang[0]: Leafref "../../../VLAN/VLAN_LIST/vlanid" of value "200" points to a non-existing leaf. (path: /sonic-vlan:sonic-vlan/VLAN_MEMBER/VLAN_MEMBER_LIST[vlanid='200'][port='Ethernet0']/vlanid)
libyang[0]: Leafref "../../../VLAN/VLAN_LIST/vlanid" of value "200" points to a non-existing leaf. (path: /sonic-vlan:sonic-vlan/VLAN_MEMBER/VLAN_MEMBER_LIST[vlanid='200'][port='Ethernet0']/vlanid)
INFO:YANG-TEST:Configure vlan-id in VLAN_MEMBER table which does not exist in VLAN  table. Passed

INFO:YANG-TEST:All Test Passed
../../target/debs/stretch/libyang0.16_0.16.105-1_amd64.deb installtion failed
../../target/debs/stretch/libyang-cpp0.16_0.16.105-1_amd64.deb installtion failed
../../target/debs/stretch/python2-yang_0.16.105-1_amd64.deb installtion failed
YANG Tests passed
Passed: pyang -f tree ./yang-models/*.yang > ./yang-models/sonic_yang_tree
copying tests/yangModelTesting.py -> build/lib/tests
copying tests/test_sonic_yang_models.py -> build/lib/tests
copying tests/__init__.py -> build/lib/tests
running egg_info
writing top-level names to sonic_yang_models.egg-info/top_level.txt
writing dependency_links to sonic_yang_models.egg-info/dependency_links.txt
writing sonic_yang_models.egg-info/PKG-INFO
reading manifest file 'sonic_yang_models.egg-info/SOURCES.txt'
writing manifest file 'sonic_yang_models.egg-info/SOURCES.txt'
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/tests
copying build/lib/tests/yangModelTesting.py -> build/bdist.linux-x86_64/wheel/tests
copying build/lib/tests/test_sonic_yang_models.py -> build/bdist.linux-x86_64/wheel/tests
copying build/lib/tests/__init__.py -> build/bdist.linux-x86_64/wheel/tests
running install_data
creating build/bdist.linux-x86_64/wheel/sonic_yang_models-1.0.data
creating build/bdist.linux-x86_64/wheel/sonic_yang_models-1.0.data/data
creating build/bdist.linux-x86_64/wheel/sonic_yang_models-1.0.data/data/yang-models
copying ./yang-models/sonic-head.yang -> build/bdist.linux-x86_64/wheel/sonic_yang_models-1.0.data/data/yang-models
copying ./yang-models/sonic-acl.yang -> build/bdist.linux-x86_64/wheel/sonic_yang_models-1.0.data/data/yang-models
copying ./yang-models/sonic-interface.yang -> build/bdist.linux-x86_64/wheel/sonic_yang_models-1.0.data/data/yang-models
copying ./yang-models/sonic-loopback-interface.yang -> build/bdist.linux-x86_64/wheel/sonic_yang_models-1.0.data/data/yang-models
copying ./yang-models/sonic-port.yang -> build/bdist.linux-x86_64/wheel/sonic_yang_models-1.0.data/data/yang-models
copying ./yang-models/sonic-portchannel.yang -> build/bdist.linux-x86_64/wheel/sonic_yang_models-1.0.data/data/yang-models
copying ./yang-models/sonic-vlan.yang -> build/bdist.linux-x86_64/wheel/sonic_yang_models-1.0.data/data/yang-models
```
2020-04-14 15:36:02 -07:00
Renuka Manavalan
f128153706
[baseimage]: Install Kubernetes packages if enabled in image (#4374)
* Install kubernetes worker node packages, if enabled.

* Minor updates

* Added some comments

* Updates per review comments.
Built a private image to test to work fine.

* Remove the removed file.

* Update per comments
Make a fix, as kubeadm no demands a higher version of kubelet & kubectl.
As kubeadm auto install kubectl & kubelet, removing explicit install is an easier/robust fix.

* Changes per review comments.

* Updates per comments.
1) Dropped helper & pod scripts
2) Made install verbose

* Drop creation of pods subdir, as this PR does not use them.

* From comments to 'n' per review comments.

* 1) kubeadm.conf is created as part of kubeadm package install. Hence dropped explicit copy.
2020-04-13 08:41:18 -07:00
Renuka Manavalan
768d18d45b
Pass image_type to build_debian.sh, so it can do any required customizations. (#4401) 2020-04-08 14:24:08 -07:00
SuvarnaMeenakshi
4b8067e913
Multi-ASIC implementation (#3888)
Changes made to support multi-asic platform. Added multi-instance support for swss, syncd, database, bgp, teamd and lldp.
2020-03-31 10:06:19 -07:00
lguohan
760e763935
[build]: allow to specify timestamp format in the build log (#4311)
only simple/none are supported currently

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-03-23 09:45:43 -07:00
lguohan
20260ceb1d
[build]: add SONIC_CONFIG_BUILD_LOG_TIMESTAMP to add timestamp in build log (#4269)
add timestamp in each job build log

example:

   [01:39:21] dh clean  --with autotools-dev
   [01:39:22]    dh_auto_clean
   [01:39:27]      make -j16 distclean

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-03-21 14:21:26 -07:00
Kalimuthu-Velappan
7d2ebf8116
[build]: support for DPKG local caching (#4117)
DPKG caching framework provides the infrastructure to cache the sonic module/target .deb files into a local cache by tracking the target dependency files.SONIC build infrastructure is designed as a plugin framework where any new source code can be easily integrated into sonic as a module and that generates output as a .deb file. The source code compilation of a module is completely independent of other modules compilation. Inter module dependency is resolved through build artifacts like header files, libraries, and binaries in the form of Debian packages. For example module A depends on module B. While module A is being built, it uses B's .deb file to install it in the build docker.

The DPKG caching framework provides an infrastructure that caches a module's deb package and restores it back to the build directory if its dependency files are not modified. When a module is compiled for the first time, the generated deb package is stored at the DPKG cache location. On the subsequent build, first, it checks the module dependency file modification. If none of the dependent files is changed, it copies the deb package from the cache location, otherwise, it goes for local compilation and generates the deb package. The modified files should be checked-in to get the newer cache deb package.

This provides a huge improvement in build time and also supports the true incremental build by tracking the dependency files.

- How I did it
It takes two global arguments to enable the DPKG caching, the first one indicates the caching method and the second one describes the location of the cache.
SONIC_DPKG_CACHE_METHOD=cache
SONIC_DPKG_CACHE_SOURCE=

    where  SONIC_DPKG_CACHE_METHOD - Default method is 'cache' for deb package caching
                            none:     no caching
                            cache:    cache from local directory
Dependency file tracking:
Dependency files are tracked for each target in two levels.
1. Common make infrastructure files - rules/config, rules/functions, slave.mk etc.
2. Per module files - files which are specific to modules, Makefile, debian/rules, patch files, etc.

    For example: dependency files for Linux Kernel - src/sonic-linux-kernel,

            SPATH       := $($(LINUX_HEADERS_COMMON)_SRC_PATH)
            DEP_FILES   := $(SONIC_COMMON_FILES_LIST) rules/linux-kernel.mk rules/linux-kernel.dep
            DEP_FILES   += $(SONIC_COMMON_BASE_FILES_LIST)
            SMDEP_FILES := $(addprefix $(SPATH)/,$(shell cd $(SPATH) && git ls-files))

            DEP_FLAGS := $(SONIC_COMMON_FLAGS_LIST) \
                         $(KERNEL_PROCURE_METHOD) $(KERNEL_CACHE_PATH)

            $(LINUX_HEADERS_COMMON)_CACHE_MODE  := GIT_CONTENT_SHA
            $(LINUX_HEADERS_COMMON)_DEP_FLAGS   := $(DEP_FLAGS)
            $(LINUX_HEADERS_COMMON)_DEP_FILES   := $(DEP_FILES)
            $(LINUX_HEADERS_COMMON)_SMDEP_FILES := $(SMDEP_FILES)
            $(LINUX_HEADERS_COMMON)_SMDEP_PATHS := $(SPATH)
Cache file tracking:
The Cache file is a compressed TAR ball of a module's target DEB file and its derived-target DEB files.
The cache filename is formed with the following format

    FORMAT:
            <module deb filename>.<24 byte of DEP SHA hash >-<24 byte of MOD SHA hash>.tgz
            Eg:
              linux-headers-4.9.0-9-2-common_4.9.168-1+deb9u3_all.deb-23658712fd21bb776fa16f47-c0b63ef593d4a32643bca228.tgz

            < 24-byte DEP SHA value > - the SHA value is derived from all the dependent packages.
            < 24-byte MOD SHA value > - the SHA value is derived from either of the following.
                    GIT_COMMIT_SHA  - SHA value of the last git commit ID if it is a submodule
                    GIT_CONTENT_SHA - SHA value is generated from the content of the target dependency files.
Target Specific rules:
Caching can be enabled/disabled on a global level and also on the per-target level.

            $(addprefix $(DEBS_PATH)/, $(SONIC_DPKG_DEBS)) : $(DEBS_PATH)/% : .platform $$(addsuffix -install,$$(addprefix $(DEBS_PATH)/,$$($$*_DEPENDS))) \
                    $(call dpkg_depend,$(DEBS_PATH)/%.dep )
            $(HEADER)


            # Load the target deb from DPKG cache
            $(call LOAD_CACHE,$*,$@)


            # Skip building the target if it is already loaded from cache
            if [ -z '$($*_CACHE_LOADED)' ] ; then

                  .....
                 # Rules for Generating the target DEB file.
                  .....

                  # Save the target deb into DPKG cache
                  $(call SAVE_CACHE,$*,$@)
            fi


            $(FOOTER)


    The make rule-'$(call dpkg_depend,$(DEBS_PATH)/%.dep )' checks for target dependency file modification. If it is newer than the target, it will go for re-generation of that target.

    Two main macros 'LOAD_CACHE' and 'SAVE_CACHE' are used for loading and storing the cache contents.
    The 'LOAD_CACHE' macro is used to load the cache file from cache storage and extracts them into the target folder. It is done only if target dependency files are not modified by checking the GIT file status, otherwise, cache loading is skipped and full compilation is performed.
    It also updates the target-specific variable to indicate the cache is loaded or not.
    The 'SAVE_CACHE' macro generates the compressed tarball of the cache file and saves them into cache storage. Saving into the cache storage is protected with a lock.
- How to verify it

    The caching functionality is verified by enabling it in Linux kernel submodule.
    It uses the cache directory as 'target/cache' where Linux cache file gets stored on the first-time build and it is picked from the cache location during the subsequent clean build.
- Description for the changelog
The DPKG caching framework provides the infrastructure to save the module-specific deb file to be cached by tracking the module's dependency files.
If the module's dependency files are not changed, it restores the module deb files from the cache storage.

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

DOCUMENT PR:

           https://github.com/Azure/SONiC/pull/559
2020-03-11 20:04:52 -07:00
abdosi
f5ab24d8f1
Made Change to have Configurable option to enable/disable docker image (#4171)
* Made Change to have Configurable option to enable/disable docker image

* [Submodule-Update] sonic-utilities to latest master
2020-02-21 10:20:07 -08:00
SuvarnaMeenakshi
c9483796dc [baseimage]: support building multi-asic component (#3856)
- move single instance services into their own folder
- generate Systemd templates for any multi-instance service files in slave.mk
- detect single or multi-instance platform in systemd-sonic-generator based on asic.conf platform specific file.
- update container hostname after creation instead of during creation (docker_image_ctl)
- run Docker containers in a network namespace if specified
- add a service to create a simulated multi-ASIC topology on the virtual switch platform

Signed-off-by: Lawrence Lee <t-lale@microsoft.com>
Signed-off-by: Suvarna Meenakshi <Suvarna.Meenaksh@microsoft.com>
2020-01-26 13:56:42 -08:00
Sumukha Tumkur Vani
ebb07d9942 [docker-restapi]: Avoid building RestAPI docker by default (#4018)
Provide build option for RESTAPI docker
2020-01-14 17:13:45 -08:00
Stepan Blyshchak
e45ff4edb4 [build] make debian package install noninteractive (#3980)
Otherwise, build process gets stuck because isc-dhcp-relay gives promt:

```
$ make SONIC_PROFILING_ON=y INSTALL_DEBUG_TOOLS=y SONIC_BUILD_JOBS=12
target/debs/stretch/isc-dhcp-relay_4.3.5-2_amd64.deb-install
+++ --- Making
target/debs/stretch/isc-dhcp-relay_4.3.5-2_amd64.deb-install --- +++
EXTRA_JESSIE_TARGETS=isc-dhcp-relay_4.3.5-2_amd64.deb-install make -f
Makefile.work jessie
make[1]: Entering directory `/build2/stepanb/hitless/sonic-buildimage'
SONiC Build System

Build Configuration
"CONFIGURED_PLATFORM"             : "mellanox"
"CONFIGURED_ARCH"                 : "amd64"
"SONIC_CONFIG_PRINT_DEPENDENCIES" : ""
"SONIC_BUILD_JOBS"                : "12"
"SONIC_CONFIG_MAKE_JOBS"          : "12"
"SONIC_USE_DOCKER_BUILDKIT"       : ""
"USERNAME"                        : "admin"
"PASSWORD"                        : "YourPaSsWoRd"
"ENABLE_DHCP_GRAPH_SERVICE"       : ""
"SHUTDOWN_BGP_ON_START"           : ""
"ENABLE_PFCWD_ON_START"           : ""
"INSTALL_DEBUG_TOOLS"             : ""
"ROUTING_STACK"                   : "frr"
"FRR_USER_UID"                    : "300"
"FRR_USER_GID"                    : "300"
"ENABLE_SYNCD_RPC"                : ""
"ENABLE_ORGANIZATION_EXTENSIONS"  : "y"
"HTTP_PROXY"                      : ""
"HTTPS_PROXY"                     : ""
"ENABLE_SYSTEM_TELEMETRY"         : "y"
"ENABLE_ZTP"                      : ""
"SONIC_DEBUGGING_ON"              : ""
"SONIC_PROFILING_ON"              : ""
"KERNEL_PROCURE_METHOD"           : "build"
"BUILD_TIMESTAMP"                 : "20200106.124851"
"BLDENV"                          : ""
"VS_PREPARE_MEM"                  : "yes"
"ENABLE_SFLOW"                    : "y"

make: Nothing to be done for 'jessie'.
make[1]: Leaving directory `/build2/stepanb/hitless/sonic-buildimage'
BLDENV=stretch make -f Makefile.work
target/debs/stretch/isc-dhcp-relay_4.3.5-2_amd64.deb-install
make[1]: Entering directory `/build2/stepanb/hitless/sonic-buildimage'
SONiC Build System

Build Configuration
"CONFIGURED_PLATFORM"             : "mellanox"
"CONFIGURED_ARCH"                 : "amd64"
"SONIC_CONFIG_PRINT_DEPENDENCIES" : ""
"SONIC_BUILD_JOBS"                : "12"
"SONIC_CONFIG_MAKE_JOBS"          : "12"
"SONIC_USE_DOCKER_BUILDKIT"       : ""
"USERNAME"                        : "admin"
"PASSWORD"                        : "YourPaSsWoRd"
"ENABLE_DHCP_GRAPH_SERVICE"       : ""
"SHUTDOWN_BGP_ON_START"           : ""
"ENABLE_PFCWD_ON_START"           : ""
"INSTALL_DEBUG_TOOLS"             : ""
"ROUTING_STACK"                   : "frr"
"FRR_USER_UID"                    : "300"
"FRR_USER_GID"                    : "300"
"ENABLE_SYNCD_RPC"                : ""
"ENABLE_ORGANIZATION_EXTENSIONS"  : "y"
"HTTP_PROXY"                      : ""
"HTTPS_PROXY"                     : ""
"ENABLE_SYSTEM_TELEMETRY"         : "y"
"ENABLE_ZTP"                      : ""
"SONIC_DEBUGGING_ON"              : ""
"SONIC_PROFILING_ON"              : ""
"KERNEL_PROCURE_METHOD"           : "build"
"BUILD_TIMESTAMP"                 : "20200106.124855"
"BLDENV"                          : "stretch"
"VS_PREPARE_MEM"                  : "yes"
"ENABLE_SFLOW"                    : "y"

[ 01 ] [ target/debs/stretch/isc-dhcp-relay_4.3.5-2_amd64.deb-install ]
Servers the DHCP relay should forward requests to:
Interfaces the DHCP relay should listen on:
```

Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2020-01-10 12:55:29 -08:00
Joe LeVeque
24a0c46464
[monit] Build from source and patch to use MemAvailable value if available on system (#3875) 2019-12-30 18:25:57 -08:00
Stepan Blyshchak
baea7e2a24 [slave.mk] make curl follow HTTP redirects (#3947)
Fix an issue with SONIC_ONLINE_DEBS with curl by instructing curl to follow http redirects

Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2019-12-27 09:26:51 -08:00
Stepan Blyshchak
183c945ab1 [slave.mk] use curl instead of wget (#3939)
wget creates empty file on failure which makes subsequent make runs
think that file is already there and won't try to download it again.

e.g.:

$ make target/files/stretch/fw-SPC-rel-13_2000_2602-EVB.mfa
...
Fails to download
...
$ ls target/files/stretch/fw-SPC-rel-13_2000_2602-EVB.mfa
target/files/stretch/fw-SPC-rel-13_2000_2602-EVB.mfa
$ make target/files/stretch/fw-SPC-rel-13_2000_2602-EVB.mfa
make: `target/files/stretch/fw-SPC-rel-13_2000_2602-EVB.mfa' is up to
date.

Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2019-12-23 10:34:22 -08:00
Stepan Blyshchak
4ba0ff25d2 [services] make snmp.timer work again and delay telemetry.service (#3742)
Delay CPU intensive services at boot

- How I did it
Made snmp.timer work and add telemetry.timer.
But this is not enough because it breaks the existing snmp dependency on swss.
So, in this solution snmp timer is a wanted by swss service, but since OnBootSec timer expires only once it will not trigger snmp service, so I added line "OnUnitActiveSec=0 sec" which will start snmp service based on the last time it was active. On boot only OnBootSec will expire, on swss start/restarts only second timer will expire immediately and trigger snmp service.
However, snmp service will not stop after "systemctl stop snmp" because of the second timer which will always expire when snmp service because unavailable.
So there is a conflict which will be handled by systemd if we add "Conflicts=" line to both snmp.service and snmp.timer.

So during boot:

snmp does not start by default
swss starts and starts snmp timer
OnUnitActiveSec=0 does not expire since there is no snmp active
OnBootSec expires and starts snmp service and snmp timer gets stopped
During "systemctl restart swss"

snmp stops because of Requisite on swss
snmp unblocks snmp timer from running
swss starts and starts snmp timer
OnUnitActiveSec=0 expires imidiately and start snmp which stops snmp timer
During "systemctl stop snmp"

stop of snmp service unblocks snmp timer but no one starts the timer so it is not started by "OnUnitActiveSec=0"
2019-12-16 09:07:05 -08:00
rajendra-dendukuri
eec594adf2 [sonic-ztp]: Build sonic-ztp package (#3299)
* Build sonic-ztp package

- Add changes in make rules to conditionally include sonic-ztp package

Signed-off-by: Rajendra Dendukuri <rajendra.dendukuri@broadcom.com>
2019-12-04 04:50:56 -08:00
Olivier Singla
c70d8bca9f [baseimage]: kdump support (#3722)
* In the event of a kernel crash, we need to gather as much information
as possible to understand and identify the root cause of the crash.
Currently, the kernel does not provide much information, which make
kernel crash investigation difficult and time consuming.

Fortunately, there is a way in the kernel to provide more information
in the case of a kernel crash. kdump is a feature of the Linux kernel
that creates crash dumps in the event of a kernel crash. This PR
will add kermel kdump support.

An extension to the CLI utilities config and show is provided to
configure and manage kdump:
 - enable / disable kdump functionality
 - configure kdump (how many kernel crash logs can be saved, memory
   allocated for capture kernel)
 - view kernel crash logs
2019-11-08 23:08:42 -08:00
Stephen Sun
c34a4783e0 [build] install new platform api on host (#3282)
slave.mk: add SONIC_PLATFORM_API_PY2 as dependency of host
sonic_debian_extension.j2: install sonic_daemon_base and Mellanox-specific sonic_platform on host
mlnx-platform-api.mk: export mlnx_platform_api_py2_wheel_path for sonic_debian_extension.j2
sonic-daemon-base.mk: export daemon_base_py2_wheel_path for sonic_debian_extension.j2
daemon_base.py: hind unnecessary dependency of swss_common on host
2019-09-25 11:00:24 -07:00
padmanarayana
75104bb35d [sflow]: Build infrastructure changes to support sflow docker and utilities (#3251)
Introduce a new "sflow" container (if ENABLE_SFLOW is set). The new docker will include:
hsflowd : host-sflow based daemon is the sFlow agent
psample : Built from libpsample repository. Useful in debugging sampled packets/groups.
sflowtool : Locally dump sflow samples (e.g. with a in-unit collector)

In case of SONiC-VS, enable psample & act_sample kernel modules.

VS' syncd needs iproute2=4.20.0-2~bpo9+1 & libcap2-bin=1:2.25-1 to support tc-sample

tc-syncd is provided as a convenience tool for debugging (e.g. tc-syncd filter show ...)
2019-09-14 20:27:09 -07:00
Renuka Manavalan
14458b79ba
Create the src archive in target/ when INSTALL_DEBUG_TOOLS=y (#3323)
* Create the src archive in target/ when INSTALL_DEBUG_TOOLS=y

* Dropped -v (verbose flag) from tar command
2019-08-28 09:29:48 -07:00
Greg Paussa
f65c7969b6 [build] Fix cleaning SONIC_PYTHON_STDEB_DEBS (#3343)
* slave.mk
* .gitignore

The SONIC_PYTHON_STDEB_DEBS group is part of SONIC_CLEAN_DEBS
for cleaning with 'make clean', but this group is overlooked
since they reside in PYTHON_DEBS_PATH, not DEBS_PATH. This
fix creates a separate clean rule for SONIC_PYTHON_STDEB_DEBS.

Also noticed the .arch file showing up on 'git status' so
added it to .gitignore.

Signed-off-by: Greg Paussa <greg.paussa@broadcom.com>
2019-08-15 00:34:34 -07:00
arheneus@marvell.com
8de26b7bb9 [Makefile/slave docker] ARM arch doesn't support few packages (#3273)
* [Makefile/slave docker] ARM arch doesn't support few packages
  iproute2 is missing for ARM sonic-slave docker

Signed-off-by: Antony Rheneus <arheneus@marvell.com>
2019-08-06 21:33:14 -07:00
arheneus@marvell.com
d80d3d60db [Makefile/Telemtry] Removing Telemetry build for ARM arch as qemu (#3287)
crashes.

Raised go lang support
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/golang-nuts/1txPOGa4aGc

Also raised qemu bug for support as golang rejected it.
https://bugs.launchpad.net/qemu/+bug/1838946

In parallel, debugging issue to resolve it

Signed-off-by: Antony Rheneus <arheneus@marvell.com>
2019-08-06 07:38:52 -07:00
arheneus@marvell.com
e9d3858633 [build] Fixes for multi arch in Dockerfile (#3237)
Fix for rules/config as it is overridden by Make infra

Signed-off-by: Antony Rheneus <arheneus@marvell.com>
2019-07-31 15:26:00 -07:00
arheneus@marvell.com
50fe458592 [build]: SONiC buildimage ARM arch support (#2980)
ARM Architecture support in SONIC

make configure platform=[ASIC_VENDOR_ARCH] PLATFORM_ARCH=[ARM_ARCH]
SONIC_ARCH: default amd64
armhf - arm32bit
arm64 - arm64bit

Signed-off-by: Antony Rheneus <arheneus@marvell.com>
2019-07-25 22:06:41 -07:00
rajendra-dendukuri
40c8bc14cd [baseimage]: Upgrade ifupdown2 to version 1.2.8 (#3180)
* Upgrade ifupdown2 to version 1.2.8

Required by ZTP to support ZTP over IPv6 transport

Signed-off-by: Rajendra Dendukuri <rajendra.dendukuri@broadcom.com>
2019-07-19 23:09:14 -07:00
Lawrence Lee
34b841e7dd [slave.mk]: remove updategraph.service generation in slave.mk (#3153) 2019-07-17 12:13:35 -07:00
Renuka Manavalan
76bf5a0bc4 [build]: Added debug symbols to many debug dockers. (#3098)
* Added debug symbols to many debug dockers.

* For debug images *only*:
1) Archive source files into debug image
2) Archived source is copied into /src
3) Created an empty dir /debug
4) Mount both /src as ro & /debug as rw into every docker
5) Login banner will give some details on /src & /debug
6) Devs can copy core file into /debug and view it from inside a container.
7) Dev may create all gdb logs and other data directly into /debug.

* Dropped redundant REDIS_TOOLS per review comments.

* Added debug symbols to frr package and hence FRR based BGP docker.

* 1) Moved dbg_files.sh to scripts/
2) Src directories to archive are now collected from individual Makefiles.
3) Added few more debug symbols
4) Added few more debug dockers.

Here after no more changes except per review comments.

To debug:
Install required version of debug image in Switch or VM.
Copy core file into /debug of host
Get into Docker
gdb /usr/bin/<daemon> -c /debug/<your core file>
set directory /src/... <-- inside gdb to get the source

For non-in-depth debugging:

Download corresponding debug Docker image (docker-...-dbg.gz) to your VM
Load the image
Run image with entrypoint as 'bash' with dir containing core mapped in.
Run gdb on the core.
2019-07-03 22:13:55 -07:00
lguohan
40330bcbdf
[build]: wait up to 60 seconds for docker engine to start (#3067)
* [build]: wait 60 seconds for docker engine to start

On some platforms, it can take more than 1 second for docker
engine to start.

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-06-22 14:40:05 -07:00
Stepan Blyshchak
1bcd54d38c [build]: Upgrade docker to 18.09 in stretch slave (#3049)
Also add an option to use docker buildkit for image build

Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2019-06-20 09:19:33 -07:00
Renuka Manavalan
cdca062693 [build]: Build sonic-broadcom.bin using debug dockers for all stretch based dockers (#2833)
* Updated Makefile infrastructure to build debug images.
As a sample, platform/broadcom/docker-orchagent-brcm.mk is updated to add a docker-orchagent-brcm-dbg.gz target.

Now "BLDENV=stretch make target/docker-orchagent-brcm-dbg.gz" will build the debug image.

NOTE: If you don't specify NOSTRETcH=1, it implicitly calls "make stretch", which builds all stretch targets and that would include debug dockers too.

This debug image can be used in any linux box to inspect core file. If your module's external dependency can be suitably mocked, you my even manually run it inside.

"docker run -it --entrypoint=/bin/bash e47a8fb8ed38"

You may map the core file path to this docker run.

* Dropped the regular binary using DBG_PACKAGES and a small name change to help readability.

* Tweaked the changes to retain the existing behavior w.r.t INSTALL_DEBUG_TOOLS=y.

When this change ('building debug docker image transparently') is extended to all dockers, this flag would become redundant. Yet, there can be some test based use cases that rely on this flag.

Until after all the dockers gets their debug images by default and we switch all use cases of this flag to use the newly built debug images, we need to maintain the existing behavior.

* 1) slave.mk - Dropped unused Docker build args
2) Debug template builder: renamed build_dbg_j2.sh to build_debug_docker_j2.sh
3) Dropped insignifcant statement CMD from debug Docker file, as base docker has Entrypoint.

* Reverted some changes, per review comments.
"User, uid, guid, frr-uid & frr-guid" are required for all docker images, with exception of debug images.

* Get in sync with the new update that filters out dockers to be built (SONIC_STRETCH_DOCKERS_FOR_INSTALLERS) and build debug-dockers only for those to be built and debug target is available.

* Mkae a template for each target that can be shared by all platforms.
Where needed a platform entry can override the template.
This avoids duplication, hence easier to maintain.

* A small change, that can fit better with other targets too.
Just take the platform code and do the rest in template.

* Extended debug to all stretch based docker images

* 1) Combined all orchagent makefiles into one platform independent make under rules/docker-orchagent.mk
2) Extened debug image to all stretch dockers

* Changes per review comments:
1) Dropped LIBSAIREDIS_DBG from database, teamd, router-advertiser, telemetry, and platform-monitor docker*.mk files from _DBG_DEPENDS list
2) W.r.t docker make for syncd, moved DEPENDS from template to specific makefile and let the template has stuff that is applicable to all.

* 1) Corrected a copy/paste mistake

* Fixed a copy/paste bug

* The base syncd dockers follow a template, which defines the base docker as DOCKER_SYNCD_BASE instead of DOCKER_SYNCD_<platform code>. Fix the docker-syncd-<mlnx, bfn>.mk to use the new one.

[Yet to be tested locally]

* Fixed spelling mistake

* Enable build of dbg-sonic-broadcom.bin, which uses dbg-dockers in place of regular dockers, for dockers that build debug version. For dockers that do not build debug version, it uses the regular docker.

This debug bin is installable and usable in a DUT, just like a regular bin.

* Per review comments:
  1) Share a single rule for final image for normal & debug flavors (e.g. sonic-broadcom.bin & sonic-broadcom-dbg.bin)
  2) Put dbg as suffix in final image name.
  3) Compared target/sonic-broadcom.bin.logs with & w/o fix to verify integrity of sonic-broadcom.bin
  4) Compared target/sonic-broadcom.bin.logs with sonic-broadcom-dbg.bin.log for verification

This fix takes care of ONIE image only. The next PR will cover the rest.
The next PR, will also make debug image conditional with flag.

* Updated per comments.
Now that debug dockers are available, do not need a way to install debug symbols in regular dockers.

With this commit, when INSTALL_DEBUG_TOOLS=y is set, it builds debug dockers (for dockers that enable debug build) and the final image uses debug dockers. For dockers that do not enable debug build, regular dockers get used in the final image.

Note:
The debug dockers are explicitly named as <docker name>-dbg.gz. But there is no "-dbg" suffix for image.
Hence if you make two runs with and w/o INSTALL_DEBUG_TOOLS=y, you have complete set of regular dockers + debug dockers. But the image gets overwritten.
Hence if both regular & debug images are needed, make two runs, as one with INSTALL_DEBUG_TOOLS=y and one w/o. Make sure to copy/rename the final image, before making the second run.
2019-06-12 01:36:21 -07:00
Joe LeVeque
42591718b5 [slave.mk] Fix typo for STRETCH_DEBS_PATH for linux-kernel (#2971) 2019-06-05 09:31:29 -07:00
lguohan
30b37ec6fb
[build]: make sonic-slave-stretch as the default build docker (#2921)
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-05-27 15:50:51 -07:00
lguohan
8080695ecf
[docker-{sonic,syncd}-vs]: upgrade {sonic,syncd}-vs docker to stretch (#2865)
* [docker-{sonic,syncd}-vs]: upgrade sonic-vs and syncd-vs docker to stretch

* remove python-click 6.6

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-05-06 07:19:36 -07:00
Renuka Manavalan
ba0ca01ee0 [build]: Makefile: Extend to build debug docker images for all stretch dockers (#2789)
Overall goal: Build debug images for every stretch docker.

An earlier PR (#2789) made the first cut, by transforming broadcom/orchagent to build target/docker-orhagent-dbg.gz.

Changes in this PR:

Made docker-orchagent build to be platform independent.
1.1) Created rules/docker_orchagent.mk
1.2) Removed platform//docker-orchagent-*.mk
1.3) Removed the corresponding entry from platform//rules.mk

Extended the debug docker image build to stretch based syncd dockers.
2.1) For now, only mellanox & barefoot are stretch based.
2.2) All the common variable definitions are put in one place platform/template/docker-syncd-base.mk
2.3) platform/[mellanox, bfn]/docker-syncd-[mlnx, bfn].mk are updated as detailed below.
2.3.1) Set platform code and include template base file
2.3.2) Add the dependencies & debug dependencies and any update over what base template offers.

Extended all stretch based non-platform dockers to build debug dockers too.
3.1) Affected are:
docker-database.mk,
docker-platform-monitor.mk,
docker-router-advertiser.mk,
docker-teamd.mk,
docker-telemetry.mk

Next: Build debug flavor of final images with regular dockers replaced with debug dockers where available.
2019-04-19 18:49:21 -07:00
Renuka Manavalan
c846fb9ead [build]: Updated Makefile infrastructure to build debug images. (#2753)
* Updated Makefile infrastructure to build debug images.
As a sample, platform/broadcom/docker-orchagent-brcm.mk is updated to add a docker-orchagent-brcm-dbg.gz target.

Now "BLDENV=stretch make target/docker-orchagent-brcm-dbg.gz" will build the debug image.

This debug image can be used in any linux box to inspect core file. If your module's external dependency can be suitably mocked, you my even manually run it inside.

"docker run -it --entrypoint=/bin/bash e47a8fb8ed38"

You may map the core file path to this docker run.

* Dropped the regular binary using DBG_PACKAGES and a small name change to help readability.

* Tweaked the changes to retain the existing behavior w.r.t INSTALL_DEBUG_TOOLS=y.

When this change ('building debug docker image transparently') is extended to all dockers, this flag would become redundant. Yet, there can be some test based use cases that rely on this flag.

Until after all the dockers gets their debug images by default and we switch all use cases of this flag to use the newly built debug images, we need to maintain the existing behavior.
2019-04-13 17:05:18 -07:00
lguohan
158531b48d
[build]: only build dockers needed by installers (#2771)
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-04-11 19:28:41 -07:00
Ying Xie
9d6bac567b [build] force Linux to drop cache before calling kvm (#2717)
* [vm build] force Linux to drop cache before calling kvm

KVM need to allocate 2G memory for this build. The system memory might
be occupied by cache at the moment and doesn't have 2G chunk to give
out. Forcing Kernel to drop cache to boost the chance of getting 2G
memory.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>

* [make] add option to enable/disable VS build memory preparation

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-03-29 15:25:17 -07:00
Stepan Blyshchak
c0caf0c372 [dockers] Build stretch dockers only once and only in stretch slave docker (#2678)
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2019-03-19 09:16:33 -07:00
lguohan
f682e7b131
[docker-radvd]: upgrade docker radvd to stretch based (#2524)
* [docker-radvd]: upgrade docker radvd to stretch based

* install jinja>=2.10

Signed-off-by: Guohan Lu <gulv@microsoft.com>

* install pip packages for testing sonic-utilities

Signed-off-by: Guohan Lu <gulv@microsoft.com>

* set storage driver to vfs

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-02-06 21:28:07 -08:00
lguohan
f20665008c
[build]: put stretch debian packages under target/debs/stretch/ (#2519)
* [build]: put stretch debian packages under target/debs/stretch/

* in stretch build phase, all debian packages built in that stage are placed under target/debs/stretch directory.
* for python-based debian packages, since they are really the same for jessie and stretch, they are placed under target/python-debs directory.

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-02-04 22:06:37 -08:00
stepanblyschak
c998dc81d2 [slave.mk] Fix DEB_BUILD_OPTIONS passing (#2473)
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2019-01-23 18:49:44 -08:00
Nikos
7056b49af7 Routing application split config support (#2286)
* Routing application split config support

Signed-off-by: nikos <ntriantafillis@gmail.com>

* Routing application split config support
Routing application split config support

Signed-off-by: nikos <Nikos Triantafillis>
2018-11-26 18:19:12 -08:00
lguohan
64a2b1ce99
[vs]: build sonic vs kvm image (#2269)
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2018-11-20 22:32:40 -08:00
Qi Luo
465ebbafff
Build patched redis-dump-load (#2277)
* Build patched redis-dump-load
* Fix build
* Add build rule
2018-11-20 19:27:56 -08:00
lguohan
8de5aadea6
[fix]: various minor fixes (#2246)
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2018-11-10 13:39:30 -08:00
Wataru Ishida
4db98b46e4 [build]: add build option to use native docker instead of dind for build (#2215)
using native docker is faster than dind dockerd with vfs storage driver

https://github.com/Azure/sonic-buildimage/pull/2016

https://github.com/Azure/draft/issues/181

Signed-off-by: Wataru Ishida <ishida@nel-america.com>
2018-11-01 18:00:00 -07:00
Jipan Yang
c64f105a3e [build]: Add missing timestamp and build_number in docker meta label tag value (#2194)
* Add missing timestamp in docker meta label tag value

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>

* add BUILD_NUMBER as well

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2018-10-29 12:06:57 -07:00
Joe LeVeque
1e1add90f9
Remove Arista-specific service ACL solution; All platforms now use caclmgrd (#2202) 2018-10-29 10:25:18 -07:00
lguohan
9078377028
[build]: install debug symbols packages when INSTALL_DEBUG_TOOLS=y (#2175)
[build]: install debug symbols packages when INSTALL_DEBUG_TOOLS=y
2018-10-21 17:20:27 -07:00
lguohan
a401ee7fa8
[build]: do not override ENABLE_SYNCD_RPC in rules/config (#2151)
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2018-10-15 13:49:35 -07:00
lguohan
0a8e0c1238
[build]: generate build timestamp at the beginning and use it for the… (#2118)
* [build]: generate build timestamp at the beginning and use it for the rest of build

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2018-10-04 21:20:01 -07:00
Jipan Yang
0e85986088 Add Tag info to docker label meta data (#2058)
* Add Tag info to docker label meta data

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>

* Move DIRTY_SUFFIX to sonic_get_version()

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
2018-09-21 10:44:28 -07:00
lguohan
be9f3ad7c1
[build]: use vfs storage driver to build dockers (#2016)
seen issues to build dockers using aufs in ubuntu 18.04

Immedidate dockers are exported to docker file and then
imported into sonic image. Thus, whether using vfs or aufs
as the storage driver does not matter for the immediate build.

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2018-09-05 15:28:32 -07:00
Joe LeVeque
98082d56a0 [baseimage]: Download picocom version 3.1-2 from stretch-backports; No longer build from source (#1946) 2018-08-17 17:38:20 -07:00
lguohan
c059d9982a
[baseimage]: install picocom 3.1 in base image (#1943)
* [baseimage]: install picocom 3.1 in base image

Signed-off-by: Guohan Lu <gulv@microsoft.com>

* add picocom to stretch build

Signed-off-by: Guohan Lu <gulv@microsoft.com>

* fix slave.mk bug

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2018-08-17 09:06:05 -07:00
Guohan Lu
66ffca2832 [kernel]: set default option to build kernel
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2018-08-11 23:46:13 +00:00
Guohan Lu
0edb3802b0 [build]: add stretch target to build in stretch docker 2018-08-11 09:09:03 +00:00
Guohan Lu
2449fafae0 [kernel]: update kernel submodule and remove standalone igb driver 2018-08-11 09:07:59 +00:00
Guohan Lu
ff1f508f33 [baseimage]: use debian 4.9.0-3 kernel 2018-08-11 09:07:59 +00:00
Rodny Molina
c3c8f7fd7f Fix for bash's memory-leak (#1879)
* Fix for bash's memory-leak

Memory leak is observed during the execution of scripts that make use of bash-arrays. In scenarios where the offending script is executed on a regular basis (e.g. fancontrol), the leaking process may end up consuming most of the system resources.

In this PR i'm replacing bash in all the contexts where it executes (both host and dockers). The official patch for this issue is here: https://ftp.gnu.org/gnu/bash/bash-4.3-patches/bash43-040

* Fixing minor issue during code-merge

Signed-off-by: Rodny Molina <rmolina@linkedin.com>
2018-07-27 17:46:33 -07:00
Ying Xie
3c33737d58
[make] introducing new build option KERNEL_PROCURE_METHOD (#1837)
* [make] introducing new build option KERNEL_BUILD_METHOD

- Kernel could be built from source files with method 'build'
- Kernel could be downloaded from Azure storage with method 'download'

Signed-off-by: Ying Xie <ying.xie@microsoft.com>

* Replace BUILD with PROCURE

* fix typo
2018-07-25 08:14:18 -07:00
Sagar Balani
93905d3d82 [barefoot]: Support for platforms based on Barefoot Networks' device (#1796)
* Initial commit

* Add Ingrasys S9180-32X platform dirver.

Signed-off-by: Wade He <chihen.he@gmail.com>

* Add bfn.service for init barefoot.

Signed-off-by: Wade He <chihen.he@gmail.com>

* [Barefoot Beta] Add some functions and fixed some bugs.

1. Update sensors.conf.
2. Fixed IO expander init.
3. Fixed PSU EEPROM.
4. Fixed MB EEPROM.
5. Add fancontrol and fan init.
6. Add SYS LED control (sys, fan, fan tray).
7. 2.5V compute and setup max and min.
8. Fixed typo MB eeprom delete address.
9. Remove coretemp to BMC.
10. Add active CPLD.
11. Modify SFP+ GPIO slave address.
12. Modify tmp75 Near Port 32 slave address.

Signed-off-by: Wade He <chihen.he@gmail.com>

* Add bfn script in /etc/init.d/

Signed-off-by: Wade He <chihen.he@gmail.com>

* Add bfn service in debian

Signed-off-by: Wade He <chihen.he@gmail.com>

* Fixed CPLD switch LED behavior.

Signed-off-by: Wade He <chihen.he@gmail.com>

* [Barefoot Beta] Fixed sensors and hwmon order.
1. Fixed ignore sensors Vbat.
2. Reorg hwmon order.

Signed-off-by: Wade He <chihen.he@gmail.com>

* Fixed PSU1 and PSU2 EEPROM order.

Signed-off-by: Wade He <chihen.he@gmail.com>

* initial barefoot checkin october 2017

* update refpoint

* update refpoints

* update refpoints to bf-master

* update refpoint

* update refpoint to tested version

* change to platform from asic

* update refpoint for swss

* revert core creation setting

* update refpoints

* add telnet for debug shell

* update refpoints 11/17/17

* missed change in file on previous merge

* [CPLD] Fixed blink LED issue.

* Fixed blink LED mask set error.

Signed-off-by: Wade He <chihen.he@gmail.com>

* Update bf_kdrv.c for 6.0.2.39

* Update bf kernel driver

* Add bf_fun kernel module.

* Update bf_tun for fixed build error

* merge with Azure master (12/12/17)

* update swss refpoint

* update refpoint of swss

* library dependency for stack unroll

* update refpoint to bf-master

* [DHCP relay]: Fix circuit ID and remote ID bugs (#1248)

* [DHCP relay]: Fix circuit ID and remote ID bugs

* Set circuit_id_len after setting circuit_id_len to ip->name

* [Platform] Add Psuutil and update sensors.conf for S9100-32X, S8810-32Q and S9200-64X (#1272)

* Add I2C CPLD kernel module for psuutil.
 * Support psuutil script.
 * Add voltage min and max threshold.
 * Update sensors.conf for tmp75.

Signed-off-by: Wade He <chihen.he@gmail.com>

* Allow multi platform support - infra (more changes to follow)

* update relative path to include platform for clarity

* [Platform] Add Ingrasys S9130-32X and S9230-64X with Nephos Switch ASIC for "branch 201712" (#1274)

- What I did
Add switch ASIC vendor: Nephos
Add Nephos platforms: Ingrasys S9130-32X, Ingrasys S9230-64X

- How I did it
Add platform/nephos files
Add platform/nephos/sonic-platform-modules-ingrasys submodule
Add device/ingrasys/x86_64-ingrasys_s9130_32x-r0 files
Add device/ingrasys/x86_64-ingrasys_s9230_64x-r0 files
Add SONiC to support Nephos platform
Update Head of submodule src/sonic-sairedis to "3b817bb"

- How to verify it
To build SONiC installer image and docker images, run the following commands:

make configure PLATFORM=nephos
make target/sonic-nephos.bin
Check system and network feature is worked as well

- Description for the changelog
Add switch ASIC vendor and platforms for Nephos

- A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Sam Yang <yang.kaiyu@gmail.com>

* change source of files to github (from dropbox), update sairedis refpoint

* update refpoint of sairedis

* [centec] support CENTEC SAI 1.0 on 201712 branch and update e582-48x6q board   (#1269)

* [marvel]: Marvell's updates for SONiC.201712 & SAI v1.0 (#1287)

* update sairedis (fast-boot refpoint)

* fix syncd rpc make files

* update refpoint to handle Makefile change (no functional change)

* [Marvell]: Add support for SLM5401-54x device (#1307)

* Marvell's updates for SONiC.201712 & SAI v1.0
* [Platform] Add Marvell's SLM5401-54x for branch 201712

* [Broadcom]: Update Boradcom SAI package to 3.0.3.3-3 (#1312) (#1321)

- update Arista 7050-QX32S config.bcm file
- update Accton th-as771*-32x100G.config.bcm files

* update refpoint for Makefile chnage in sairedis

* update refpoint - sairedis

* update sairedis to older refpoint till we debug clean build

* export asic platform for build

* update refpoint for makefiles

* [PLATFORM] Centec update E582 driver fan/epprom/sensor (#1332)

* Upload wnc-osw1800

* Modify for Barefoot suggest

* Revert bfn-platform.mk

* Update bfn-platform-wnc.mk

Update parameter name

* Update parameter name

* initial support for WNC platform

* change switch name to "switch"

* Delete bf modules for rel_7_0

* Add Ingrasys S9180 platform

Signed-off-by: Wade He <chihen.he@gmail.com>

* Modify bfnsdk for Ingrasys S9180 platform

Signed-off-by: Wade He <chihen.he@gmail.com>

* Resolved the conflict.

* Resolved the conflict.

* Update submodule path and url.

* Delete unused file.

* Update PSU GPIO and EEPROM for psuutil.

* Add psuutil in S9180-32X

Signed-off-by: Wade He <chihen.he@gmail.com>

* update refpoint

* update refpoint

* change contact email, update refpoint

* cleanup and update kernel modules

* updates based on review

* update refpoint

* update refpoint

* fix typo in config script to check for platforms

* remove stale file

* resolve conflicts

* cleanup diffs with Azure repo and update SDK debs

* update refpoints to Azure

* address review comments

* revert refpoint of swss-common

* porting the build fix from master

* porting build fix from master

* Minor Fix

* Minor fix

* Temp to sde deb packages url

* Update sonic - sairedis,swss & swss-common refpoints

* Update git modules url path to bfn repo

* updated paths for swss, swss-common & sairedis

* Update refpoint for sonic-swss to local bfn repo

* Update URL for downloading sde debian packages

* porting fix links of debian git server from master

* porting fix links of debian git server from master

* [Ingrasys] Add platform support for S9280-64X with Barefoot ASIC

* Update ref points for swss, swss-common and sairedis repos

* Add sonic platform scripts for bfn montara/maverick

* Call sh scripts instead of calling py scripts

* Address upstream PR Comments (#10)

* Update bf-master with azure/master
* Undo changes to some files

* Revert "Address upstream PR Comments (#10)"

This reverts commit a7fddb83ca.

* Address upstream comments (#11)

* Remove all non bfn specific changes from upstream PR

* Revert "Address upstream comments (#11)"

This reverts commit 559132103e.

* Undo non bfn changes

* Little more cleanup

* Add back code removed in merge

* export CONFIGURED_PLATFORM

* Update sairedis and swss refpoints

* Address Upstream PR comment

* change deb pkg dependency from 3.16.0-4-amd64 to 3.16.0-5-amd64

* Set default tx queue len for usb0 interface to 64

* Update sairedis refpoint

* Update swss ref point

* Add bfn buffer cfg files for montara/maverick as per new design

* Update buffer cfg templates for bfn montara

* add non zero size to buffer profile
* add macro to generate port lists

* Update buffer cfg templates for bfn mavericks

* add non zero size for buffer profiles
* add port generation macro

* Add missing psmisc package

* BGP docker seems to be missing killall utility being used by fast-reboot script. This is causing non graceful termination of BGP sessions.
Adding psmisc to resolve this issue.

* Update swss ref point

* Update swss ref point

* Update sairedis refpoint

* Update sairedis refpoint

* Update sairedis refpoint

* Update sairedis refpoint

* Update refpoint for sairedis and swss

* sairedis to azure master
* swss to latest bfn bf-master

* Update gitmodules

Update url for sairedis to azure master

* Correct typo in bfn platform script

* Update swss and sairedis ref points

* Update swss ref point

* Address Review comments

* Update swws path in gitmodules to azure master
* update swss refpoint
* update base docker j2 file -remove psmisc package (could be a concern, would cause fast reboot to not work correctly will fix in another PR)

* Fix sairedis refpoint broken in by previous merge

* Remove psmisc from docker base image

* This will break fast reboot as killall is required for killing bgp process and initiating graceful termination of BGP session.
Will fix this in a seperate PR. Need this for SONIC upstreaming

* Address upstream comments

* Remove bmc interface from interface jinja template and sample output interfaces file
* Add bmc interface at boot time to network interfaces for bfn bmc based platforms

* Remove autogen ingrasys debian files

* Revert "Remove autogen ingrasys debian files"

* Buffer and qos config template fix for bfn platforms (#21)

SWI-1509 Buffer and qos config template fix for bfn platforms

* Fix qos config files for montara & mavericks (#22)

* Reference only ppg 3,4 in qos files as no profiles are attached to 0,1 in buffer configs

* Fix vs test (#23)
2018-07-24 10:23:12 -07:00
Andriy Moroz
dadc17d9e6 [Mellanox] Use MAC from EEPROM for PortChannels and VLAN Interfaces (#1793)
* Use MAC from EEPROM for PortChannels

Signed-off-by: Andriy Moroz <c_andriym@mellanox.com>

* Use MAC from EEPROM in DEVICE_METADATA

Will affect MAC for VLAN interfaces

Signed-off-by: Andriy Moroz <c_andriym@mellanox.com>

* Get MAC via decode-syseeprom

Signed-off-by: Andriy Moroz <c_andriym@mellanox.com>

* hw-management is now a service

Signed-off-by: Andriy Moroz <c_andriym@mellanox.com>

* Add error handling for MAC fetch process

Signed-off-by: Andriy Moroz <c_andriym@mellanox.com>
2018-07-23 15:51:03 -07:00
Joe LeVeque
f04f0704f7
Build python-click Debian package from version 6.7-4 source to fix CLI autocomplete/suggest (#1824) 2018-06-29 09:59:46 -07:00
Qi Luo
7ba08e5bf6
Prefix docker container name to syslog syslogtag (program name) (#1810) 2018-06-25 10:48:42 -07:00
Rodny Molina
b5f153b3ae Introducing 'debugging' and 'profiling' options in sonic build-infra (#1782) 2018-06-19 15:59:12 -07:00
Serhey Popovych
8d88455509 [baseimage]: Improve password hashing for default user account (#1748)
* [slave.mk]: Fix displaying username and password in build summary

We display contents of DEFAULT_USERNAME and DEFAULT_PASSWORD, while
image can be build with USERNAME and/or PASSWORD given on make(1)
command line. For example:

  $ make USERNAME=adm PASSWORD=mypass target/sonic-broadcom.bin

Fix by displaying USERNAME and PASSWORD variables in build summary.

Signed-off-by: Sergey Popovich <sergey.popovich@ordnance.co>

* [baseimage]: Improve default user account handling

There are couple of issues with current implementation of default
user account management in baseimage:

  1) It uses DES to encrypt accounts password. Furthermore this
     effectively limits password length to 8 symbols, even if more
     provided with PASSWORD or DEFAULT_PASSWORD from rules/config.

  2) Salt value for password is same on all builds even with different
     password increasing attack surface.

  3) During the build process password passed as command line parameter
     either as plain text (if given to make(1) as "make PASSWORD=...")
     or DES encrypted (if given to build_debian.sh) can be seen by
     non-build users using /proc/<pid>/cmdline file that has group and
     world readable permissions.

Both 1) and 2) come from:

  perl -e 'print crypt("$(PASSWORD)", "salt"),"\n"')"

that by defalt uses DES if salt does not have format $<id>$<salt>$,
where <id> is hashing function id. See crypt(3) for more details on
valid <id> values.

To address issues above we propose following changes:

  1) Do not create password by hands (e.g. using perl snippet above):
     put this job to chpasswd(8) which is aware about system wide
     password hashing policy specified in /etc/login.defs with
     ENCRYPT_METHOD (by default it is SHA512 for Debian 8).

  2) Now chpasswd(8) will take care about proper salt value.

  3) This has two steps:

    3.1) For compatibility reasons accept USERNAME and PASSWORD as
         make(1) parameters, but warn user that this is unsafe.

    3.2) Use process environment to pass USERNAME and PASSWORD variables
         from Makefile to build_debian.sh as more secure alternative to
         passing via command line parameters: /proc/<pid>/environ
         readable only by user running process or privileged users like
         root.

Before change:
--------------

  hash1
  -----
  # u='admin'
  # p="$(LANG=C perl -e 'print crypt("YourPaSs", "salt"),"\n"')"
                                      ^^^^^^^^
                                      8 symbols
  # echo "$u:$p" | chpasswd -e

  # getent shadow admin
  admin:sazQDkwgZPfSk:17680:0:99999:7:::
        ^^^^^^^^^^^^^
        Note the hash (DES encrypted password)

  hash2
  -----
  # u='admin'
  # p="$(LANG=C perl -e 'print crypt("YourPaSsWoRd", "salt"),"\n"')"
                                      ^^^^^^^^^^^^
                                      12 symbols
  # echo "$u:$p" | chpasswd -e

  # getent shadow admin
  admin:sazQDkwgZPfSk:17680:0:99999:7:::
        ^^^^^^^^^^^^^
        Hash is the same as for "YourPaSs"

After change:
-------------

  hash1
  -----
  # echo "admin:YourPaSs" | chpasswd
  # getent shadow admin
  admin:$6$1Nho1jHC$T8YwK58FYToXMFuetQta7/XouAAN2q1IzWC3bdIg86woAs6WuTg\
           ^^^^^^^^
           Note salt here
  ksLO3oyQInax/wNVq.N4de6dyWZDsCAvsZ1:17681:0:99999:7:::

  hash2
  -----
  # echo "admin:YourPaSs" | chpasswd
  # getent shadow admin
  admin:$6$yKU5g7BO$kdT02Z1wHXhr1VCniKkZbLaMPZXK0WSSVGhSLGrNhsrsVxCJ.D9\
           ^^^^^^^^
           Here salt completely different from case above
  plFpd8ksGNpw/Vb92hvgYyCL2i5cfI8QEY/:17681:0:99999:7:::

Since salt is different hashes for same password different too.

  hash1
  -----
  # LANG=C perl -e 'print crypt("YourPaSs", "\$6\$salt\$"),"\n"'
                                             ^^^^^
                                             We want SHA512 hash
  $6$salt$qkwPvXqUeGpexO1vatnIQFAreOTXs6rnDX.OI.Sz2rcy51JrO8dFc9aGv82bB\
  yd2ELrIMJ.FQLNjgSD0nNha7/

  hash2
  -----
  # LANG=C perl -e 'print crypt("YourPaSsWoRd", "\$6\$salt\$"),"\n"'
  $6$salt$1JVndGzyy/dj7PaXo6hNcttlQoZe23ob8GWYWxVGEiGOlh6sofbaIvwl6Ho7N\
  kYDI8zwRumRwga/A29nHm4mZ1

Now with same "salt" and $<id>$, and same 8 symbol prefix in password, but
different password length we have different hashes.

Signed-off-by: Sergey Popovich <sergey.popovich@ordnance.co>
2018-06-09 11:29:16 -07:00
Joe LeVeque
e900369fa3
[slave.mk] Define SPACE variable differently such that the line doesn't end in whitespace (#1676) 2018-05-10 18:52:38 -07:00
Jipan Yang
f0690b246b [build]: Fix the problem of ENABLE_SYSTEM_TELEMETRY not taking effect in rules/config (#1615)
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
2018-04-18 00:31:12 -07:00
Jipan Yang
f74de8914b [telemetry]: SONiC system telemetry Support (#1526)
* SONiC system telemetry Support

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>

* Update package name from telemetry to sonic-telemetry

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
2018-03-27 13:39:04 -07:00
sihuihan88
7639cc1b53 [pfcwd]: add flag to enable pfc wd on start (#1429) 2018-03-05 23:55:37 -08:00
Joe LeVeque
dc7abaf569 [slave.mk] Apply patches to dockers before building, if any exist (#1395) 2018-02-15 17:48:49 -08:00
byu343
ecf5c8d311 ssh and snmp allow list (#1363)
- Service ACL framework for Arista platforms
2018-02-08 17:43:52 -08:00
Marian Pritsak
43554c8b1a [slave.mk]: Clear deb_dist directory (#1344)
Python packages, having their version changed, will fail to build
because deb_dist directory contains both build directories for old and
new version, and (for some uncleaer reason) debian utilities don't know
which one to choose.

Signed-off-by: marian-pritsak <marianp@mellanox.com>
2018-01-29 08:14:01 -08:00
Joe LeVeque
134707f822
Move platform-specific hardware plugin base packages to sonic-platform-common submodule (#1301) 2018-01-17 17:11:31 -08:00
pavel-shirshov
47aba6f76d
[libsaithrift-dev]: Enable building libsaithrift-dev and pythonthrift libraries (#1296)
* Fix build of libsaithrift for broadcom

* Restore libsaithrift-dev building

* Comment out libsaithrift library on cavium and marvell. Both of them have old SAI drivers

* Revert back unintentional acton changes

* Don't run the package checks when building sairedis package
2018-01-11 20:32:14 -08:00
lguohan
15d433d975
[build]: allow to use http(s) proxy in the build (#1265)
* allow to use http(s) proxy in the build

To enable this, use following command
http_proxy=[your_proxy] https_proxy=[your_proxy] make
2017-12-23 23:34:15 -08:00
kaiyu22
63de341dd6 [Platform] Add Ingrasys S9130-32X and S9230-64X with Nephos Switch ASIC (#1245)
* Add switch ASIC vendor and platforms for Nephos

- What I did
Add switch ASIC vendor: Nephos
Add Nephos platforms: Ingrasys S9130-32X, Ingrasys S9230-64X

- How I did it
Add platform/nephos files
Add platform/nephos/sonic-platform-modules-ingrasys submodule
Add device/ingrasys/x86_64-ingrasys_s9130_32x-r0 files
Add device/ingrasys/x86_64-ingrasys_s9230_64x-r0 files
Add SONiC to support Nephos platform

- How to verify it
To build SONiC installer image and docker images, run the following commands:
make configure PLATFORM=nephos
make target/sonic-nephos.bin
Check system and network feature is worked as well

- Description for the changelog
Add switch ASIC vendor and platforms for Nephos

- A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Sam Yang <yang.kaiyu@gmail.com>

* Advance sonic-sairedis submodule to include #271 (Add Nephos ASIC)
2017-12-22 10:04:29 -08:00
Ying Xie
eadd74fb84
[build] allow user to override the default number of build jobs (#1255)
User could issue SONIC_BUILD_JOBS=<num> to set number of concurrent
build job(s) to run.

With this commit, the default setting is still left at 1.
2017-12-20 15:25:30 -08:00
Qi Luo
4b2225bca5 [docker-snmpd]: asyncsnmp depends on sonic-utilities so it is possible to import sonic_psu (#1212)
* asyncsnmp depends on sonic-utilities so it is possible to import sonic_psu

Signed-off-by: Qi Luo <qiluo-msft@users.noreply.github.com>

* Ignore sonic_utilities test during build
2017-12-07 13:08:23 -08:00
Liuqu
8383b1f256 [TACACS+]: Add support for TACACS+ Authentication (#1019)
* [TACACS+]: Add support for TACACS+ Authentication

* pam_tacplus - A TACACS+ protocol client library and PAM module to
  supports core TACACS+ functions for AAA.
* nss_tacplus - A NSS plugin for TACACS+ to extend function getpwnam,
  make the TACACS+ authenticated user which is not found in local
  could login successfully.

* Add make rules for pam_tacplus and install script
* Add a patch for pam_tacplus to disable pam-auth-update pam-tacplus
  by default
* Add a patch for pam_tacplus to inlucde and build nss_tacplus

  Signed-off-by: chenchen.qcc@alibaba-inc.com

* [TACACS+]: Add nss-tacplus as a separate src repo

* Separate nss-tacplus from pam-tacplus, modify tacacs.mk and
  makefile, add a patch to adapt to the new user map profile.
* Use the lastest stable version for pam-tacplus, add a dependent
  package in sonic-salve, add two patches to fix build error.
* Add scripts to disable tacplus by default.
* Remove hostcfgd service file

  Signed-off-by: Chenchen Qi <chenchen.qcc@alibaba-inc.com>

* [TACACS+]: Fix nss-tacplus filter some valid TACACS+ username

* The NAME_REGEX for username check in plugin nss-tacplus is
  the ANSI version "^[0-9a-zA-Z_-\ ]*$", but the regular expression
  in /etc/adduser.conf is not defined as ANSI version. To avoid
  nss-tacplus filter some valid TACACS+ username, remove username
  check.

  Signed-off-by: Chenchen Qi <chenchen.qcc@alibaba-inc.com>
2017-12-07 03:36:17 -08:00
lguohan
414e43bb74
[build]: start docker service if not already started (#1203)
docker start fails when docker service is already started
2017-12-01 19:24:55 -08:00
lguohan
f4e37a66f9 add INSTALLS, LAZY_INSTALLS, DEPENDS targets to image build (#1159)
1. _INSTALLS defines the packages to be installed in the image
at the build time.
2. _LAZY_INSTALLS defines the packages to be installed in the image
at the first run time
3. _DEPENDS defines the packages to at the build dependency of
the image
2017-11-16 12:27:03 -08:00
lguohan
a31c08047b [build]: add sonic-config-engine to sonic-utilitie build dependency (#1075) 2017-10-25 05:06:30 -07:00
lguohan
f073d8c654 [virtualswitch]: build docker-sonic-vs docker based on SAI virtual switch (#1065) 2017-10-23 22:01:42 -07:00
Shuotian Cheng
27cac94da7 [Makefile]: Fix unmatched variable name ENABLE_SYNCD_RPC (#1032)
Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
2017-10-11 17:46:26 -07:00
Joe LeVeque
8a305ce9c9 [sonic-utilities]: Build as Debian package once again (revert change to wheel) (#1018) 2017-10-06 12:46:47 -07:00
Joe LeVeque
1cd9818fe9 [build]: sonic-utilities package depends on swsssdk; build as wheel and add build dependency (#1011)
* [build]: sonic-utilities package now depends on swsssdk; add build dependency

* Now building sonic-utilities Python package in wheel format

* Update sonic-utilities submodule

* Change output wheel name to match proper format
2017-10-04 19:50:39 -07:00
Andriy Moroz
9b249ceb59 Add time stamp suffix to the dirty images version string (#958)
* Add timestamp suffix to the dirty images version string

This will allow to build different debug images without
having to update BUILD_NUMBER variable before the build

* Changed timestamp format to YYYYMMDD.HHMMSS
2017-09-27 00:04:34 -07:00
ravijo2
458093fee5 Framework to plugin Organization specific scripts during ONIE Image build (#951)
* Framework to plugin Organization specific scripts

* Framework to plugin Organization specific scripts

* Framework to plugin Organization specific scripts

* add getopt option to organization script
2017-09-19 16:23:31 -07:00
Samuel Angebault
bc73082922 [devices]: Add led plugin for Arista 7060CX-32S and 7260CX3-64 (#945)
* Bump sonic-platform-modules-arista submodule

* Allow image specific mount option for containers

* Add led plugin for DCS-7060CX-32S

* Add led plugin for DCS-7260CX3-64
2017-09-11 15:29:42 -07:00
Joe LeVeque
211689ad29 [slave.mk]: Apply series of patches to SONIC_PYTHON_STDEB_DEBS targets if they exist (#941) 2017-09-07 14:02:17 -07:00
Marian Pritsak
175b952508 [slave.mk]: Add list target (#856)
Add new target to list all available packages, files, docker images and
installers.

Signed-off-by: marian-pritsak <marianp@mellanox.com>
2017-07-30 01:34:27 +03:00
Oleksandr Ivantsiv
382d52843f [mellanox]: Move FW upgrade script to base image. (#849)
- Download files to target/files directory (instead of target/debs)
- Clean python-weels log files
2017-07-28 10:57:51 -07:00
Marian Pritsak
7d95fd7e8c [rules/functions][slave.mk]: Refine build output (#838)
Print current build configuration before run
Update screen with currently running targets (only available if TERM is
available)
Change format of printed targets

Signed-off-by: marian-pritsak <marianp@mellanox.com>
2017-07-25 09:49:39 +03:00
Marian Pritsak
cd601650bf [slave.mk]: Use host timestamp for online targets. (#826)
By default, wget tries to use timestamp received from the server. This
behavior messes up make dependencies which are also based on last
modification time. In order to avoid this, we use the current time as a
timestamp to keep dependencies in correct order.

Signed-off-by: marian-pritsak <marianp@mellanox.com>
2017-07-21 19:05:21 +03:00
lguohan
3fd2014e93 [sonic-mgmt]: move sonic-mgmt docker build into sonic-slave (#819)
need to install sonic-device-data into sonic-mgmt docker
2017-07-19 08:10:45 -07:00
Marian Pritsak
51cab5d4ed [config]: Add SONIC_CONFIG_MAKE_JOBS (#784)
* [config]: Add SONIC_CONFIG_MAKE_JOBS

This config option allows user to specify -j value that will be passed
to each package build.

Signed-off-by: marian-pritsak <marianp@mellanox.com>
2017-07-07 15:32:50 +03:00
Taoyu Li
5e6620e19e [bgp] Save bgp admin state (#690)
* [bgp] Save admin state and set default state to shutdown

* Set default behavior to no shutdown

* Add build option SHUTDOWN_BGP_ON_START

* Script change for default admin state to be on

* Address CR comments to bgp_neighbor script

* Fix script bug
2017-06-12 11:05:22 -07:00
Marian Pritsak
30db4ec191 [slave.mk]: Move initialization of ENABLE_SYNCD_RPC up (#680)
ENABLE_SYNCD_RPC should be initialized before all recipes are included
in order for them to be able to use this variable

Signed-off-by: marian-pritsak <marianp@mellanox.com>
2017-06-08 18:24:28 -07:00
Marian Pritsak
520fff5b2d Add ENABLE_SYNCD_RPC option (#642)
Under rules/config now there's additional configuration option
ENABLE_SYNCD_RPC that allows building SONIC installer with
docker-syncd-*-rpc for evaluation purposes.

Signed-off-by: marian-pritsak <marianp@mellanox.com>
2017-05-30 16:17:03 -07:00
lguohan
ef980151ae [build]: support submodule patching for SONIC_DPKG_DEBS (#619) 2017-05-22 01:08:16 -07:00
Rodny Molina
d30fbf1d72 [build]: Adding support for Free-Range-Routing stack. (#510)
- Extending SONiC building infrastructure to provide users
           with greater flexibility, by allowing them to elect a
           routing-stack different than the default one (quagga). The desired
           routing-stack will be defined in rules/config file.

         - As part of these changes I'm adding support for
           Free-Range-Routing (FRR) stack. Quagga will continue to be
           the default routing-stack.

Signed-off-by: Rodny Molina <rodny@linkedin.com>
2017-04-20 09:12:27 -07:00
lguohan
2e9b92e1c1 [build]: allow to customize USERNAME and PASSWORD in command line (#492)
* [build]: allow to customize USERNAME and PASSWORD in command line

* use default password and username if not specified in the command line
2017-04-12 11:23:48 -07:00
Marian Pritsak
6dbe979e5f [build]: Include SONiC version into installer. (#472)
* [build]: Include SONiC version into installer.

Signed-off-by: marian-pritsak <marianp@mellanox.com>

* Append dirty if contains local changes

Signed-off-by: marian-pritsak <marianp@mellanox.com>

* Update config

* Use correct name for kernel version field

* Update sysDescription.j2
2017-04-05 16:14:41 -07:00
Joe LeVeque
b7d2009b14 [base image]: Install SwSS SDK Python 2 package into base image (#461)
* [base image]: Install SwSS SDK Python packages (Python 2 & 3) into base image

* Fix typo in existing variable name

* Only install Python 2 version, as Python 3 is not installed in base image
2017-04-03 23:56:15 -07:00
Taoyu Li
fed908fc6b [config-engine] minigraph.py refactoring (#448)
* Refactor minigraph.py
See description in https://github.com/Azure/sonic-buildimage/pull/448 for detail
2017-03-30 15:25:31 -07:00
Taoyu Li
3643281594 [cfggen] Add tool to translate openconfig acl into sonic format (#388)
* Build sonic-config-engine as whl instead of deb package
* Add tool to translate openconfig acl into sonic format
2017-03-17 14:51:42 -07:00
lguohan
b165ab9e54 [build]: allow install same platform module deb on multiple platforms (#410) 2017-03-17 11:59:07 -07:00
Qi Luo
36e02f76ee Fix: support submodule patching if provided (#407) 2017-03-16 22:57:30 -07:00
Qi Luo
74b6c4074b Support submodule patching if provided (#403)
* Support submodule patching if provided
* Replace stg with quilt, working without git branch
2017-03-16 14:24:43 -07:00
Marian Pritsak
fcffbb3ce6 [slave.mk]: Add installer to clean targets. (#384)
Signed-off-by: marian-pritsak <marianp@mellanox.com>
2017-03-07 22:34:24 +02:00
Oleksandr Ivantsiv
e6c2380b6f Remove SKU from distclean target. 2017-03-02 14:08:25 +02:00
lguohan
0ed9b3ce28 [platform]: remove sku from build parameter (#350)
different sku will be contained in a single one sonic image.
no longer need to specify sku at the build time
2017-03-01 17:05:13 -08:00
Marian Pritsak
a8776033bf Merge swss and syncd into single service (#334)
Current implementation of swss and syncd causes a lot
of problems in terms of dependencies and synchronization.
Instead of handling them in separate services, we now
start and stop them both as a single entity.

Signed-off-by: marian-pritsak <marianp@mellanox.com>
2017-03-01 10:57:35 -08:00
lguohan
73fb59c52c [build]: allow single src file to build multiple independent debian p… (#349)
add_derived_package setup dependency between the main deb and derived deb.
The derived deb depends on the main deb and need to install the main deb.

add_extra_package does not setup dependency between the main deb and peer deb,
does not require to install the main deb.

* rename add_peer_packages to add_extra_packages
2017-03-01 08:32:58 -08:00
lguohan
b9b7d7a295 [installer]: support platform driver lazy installation (#340)
allow one image to support multiple switch devices, install
corresponding platform driver during the first boot time.
2017-02-27 13:08:41 -08:00
Joe LeVeque
2a551d3c60 Consolidate device-specific files; install as a Debian package (#316)
( All device-specific files now reside under /device directory in a <vendor-name>/<platform-string>/<hardware-SKU> directory structure in repo.

* Device-specific files are now packaged into a Debian package (sonic-device-data) and are now installed to /usr/share/sonic/device/<platform-string>/<hardware-SKU>/ directory on switch.
2017-02-27 00:13:36 -08:00
Marian Pritsak
8dc9dccaef [slave.mk]: Make installer executable. (#337)
Signed-off-by: marian-pritsak <marianp@mellanox.com>
2017-02-26 16:00:44 -08:00
antonpatenko
a2b4bcbb61 [docker-base]: Enabled common debug packages installation. (#292) 2017-02-21 19:04:43 -08:00
Taoyu Li
ea372cc7c1 Add get_graph service to fetch minigraph automatically (#288)
- Add a functionality to get SNMP community from DHCP (option 224)
- Add a functionality to get minigraph from http service instead of using default minigraph
  - The url for graph service is passed through DHCP option 225
  - This feature is by default disabled. Modify rule/config to enable it on build time, or modify /etc/sonic/graph_service_url on run time.
- Fix a bug that getting hostname from DHCP is not working correctly
2017-02-17 13:47:01 -08:00
Taoyu Li
f5b2dccee9 Allow build image with local changes for debug purpose (#247)
* Allow build with local change for debug purpose
Uncomment DEBUG_BUILD = y in rule.mk/config to enable local build
2017-02-14 16:56:08 -08:00
Marian Pritsak
ec584a2cc0 [sonic-utilities]: integrate into base image (#282)
Signed-off-by: marian-pritsak <marianp@mellanox.com>
2017-02-12 15:05:12 -08:00
lguohan
6119a58e4a [build_debian]: install deps in rootfs instead of just extracting the packages (#280)
packages contains post-install that needs to be performed
2017-02-10 07:39:05 -08:00
lguohan
a412fe16c4 [dockers]: use docker --squash to build dockers (#273) 2017-02-07 08:36:19 -08:00
Oleksandr Ivantsiv
53a9792014 [Makefile]: Add possibility for docker containers to install files to base image (#240)
- Add vtysh/lldpctl/sensors to baseimage
2017-02-07 00:33:20 -08:00
lguohan
8826beb597 [docker]: change hardcoded value to DOCKERFS_DIR for docker directory on the disk (#269) 2017-02-06 08:17:16 -08:00
lguohan
f6a509cbb7 Revert "Add possibility to install debug packages during build." (#259) 2017-02-03 22:14:47 -08:00
antonpatenko
26f4c85825 Add possibility to install debug packages during build. (#234)
* Enabled common debug packages installation.

* Update gitignore.

* Make debug-packages installation disabled by default.
2017-02-03 21:38:32 -08:00
Oleksandr Ivantsiv
34ea91349c one image implementation (#215)
* Single image

* Fix review comments

* Update syncd service. Add HW mgmt to Mellanox single image.

* Add single image template for Broadcom platform.

SKU should be provided during configure:
make configure PLATFORM=broadcom SKU=Force10-S6000

* Add single image template for Cavium platform.

SKU should be provided during configure:
make configure PLATFORM=cavium SKU=AS7512

* Add description to sonic_debian_extension.j2 file.
2017-01-29 11:33:33 -08:00
Marian Pritsak
66aebb329c [docker]: Change templating method to jinja2 (#115) (#153)
Move from sed to jinja2 for more readability.
2016-12-23 15:22:06 -08:00
Shuotian Cheng
e3b126893b [makefile]: Exclude p4 from building generic platform dockers (#155) 2016-12-22 21:32:07 -08:00
Shuotian Cheng
da9022971e Makefile: Add platform generic for platform neutral targets (#127) 2016-12-15 19:43:17 -08:00
lguohan
98e3479e26 build orchagent and syncd docker on broadcom platform (#118)
* build orchagent and syncd docker on broadcom platform
2016-12-14 14:33:07 -08:00
Marian Pritsak
0d9ca53c00 Enable logging for targets in SONIC_PYTHON_STDEB_DEBS (#105) 2016-12-08 07:05:19 -08:00
Marian Pritsak
e9098b99fb Build improvements (#80)
* Build improvements

Fix dependencies
Add configuration options
Automatically build sonic-slave

* Set default number of jobs to 1

* Auto generate target/debs directory

Signed-off-by: marian-pritsak <marianp@mellanox.com>

* Automatically remove sonic-slave container after exit

* Silence clean-logs

* Add SONIC_CLEAN_TARGETS to clean

* Use second expansion for clean dependencies

* Avoid creating empty log files

Remove log file on flush instead of writing empty string

* Put dpkg install inside lock

Use same lock as debian install targets do to avoid
race condition in dpkg installation

* Remove redirect to log from docker save

* Add .platform dependency to all and clean targets

* Remove header and footer from clean targets

* Disable messages for SONIC_CLEAN_TARGETS

* Exit with error if dpkg-buildpackage fails

* Set new location for debs in build_debian.sh

* Add recipe for docker-database

* Update redis version to 3.2.4

* Add support for p4 platform

* Add recipe for snmpd

* Add slave targets to phony and make all target default

* Remove build.sh from thrift

* Add versioning to team, nl, hiredis and initramfs

* Change sonic-slave to support snmpd build from sources

* Remove src/tenjin

* Add recipe for lldpd

* Add recipe for mpdecimal

* Remove hiredis directory on rebuild

* Add recipe for Mellanox hw management

* Remove generic image from all targets for Mellanox

* Add support for python wheels

* Add lldp and snmp dockers

* Sync docker-database to include libjemalloc

* Fix asyncsnmp variable name

* Change default build configuration

Redirect output to log files by default
Set number of jobs to nproc value
Do not print dependencies
Fix logging to print log of failed job into console

* Use docker inspect to check if sonic-slave image exists

* Use config in slave.mk directly

* Disable color output by default

* Remove sswsdk dependency from lldp and snmp dockers

* Fix comment in py wheels install targets

* Add dependency between two versions of sswsdk

* Add containers to mellanox platform

lldp, snmp and database containers

* Add recipe for team docker

* Add team docker to mellanox platform

* Encrypt password passed to build_debian.sh

* Update mellanox SAI version

Make version and revision setting only in main recipe

* Fix error handling in makefiles

As makefiles use .ONESHELL we should add -e
option to shell options in order to exit after any command fails

* Add recipe for platform monitor image

* Add platfotm monitor to mellanox targets

* Ignore submodules when building base image
2016-12-05 11:12:19 -08:00