Commit Graph

468 Commits

Author SHA1 Message Date
Ying Xie
d5262a3621
[first boot] sync file system after moving/copying files (#3550)
Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-10-02 13:58:34 -07:00
Wenda Ni
cf0465bf53
Adopt per-port buffer and qos profile (#3542)
Signed-off-by: Wenda Ni <wenni@microsoft.com>
2019-10-02 13:01:16 -07:00
Stepan Blyshchak
52e35a0f95 [docker_image_ctl.j2] skip hostname update if is up to date (#3529)
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2019-10-01 20:48:03 -07:00
Stephen Sun
7308d2eb97 [Mellanox] Stop pmon ahead of syncd (#3505)
Issue Overview
shutdown flow

For any shutdown flow, which means all dockers are stopped in order, pmon docker stops after syncd docker has stopped, causing pmon docker fail to release sx_core resources and leaving sx_core in a bad state. The related logs are like the following:

INFO syncd.sh[23597]: modprobe: FATAL: Module sx_core is in use.
INFO syncd.sh[23597]: Unloading sx_core[FAILED]
INFO syncd.sh[23597]: rmmod: ERROR: Module sx_core is in use
config reload & service swss.restart
In the flows like "config reload" and "service swss restart", the failure cause further consequences:

sx_core initialization error with error message like "sx_core: create EMAD sdq 0 failed. err: -16"
syncd fails to execute the create switch api with error message "syncd_main: Runtime error: :- processEvent: failed to execute api: create, key: SAI_OBJECT_TYPE_SWITCH:oid:0x21000000000000, status: SAI_STATUS_FAILURE"
swss fails to call SAI API "SAI_SWITCH_ATTR_INIT_SWITCH", which causes orchagent to restart. This will introduce an extra 1 or 2 minutes for the system to be available, failing related test cases.
reboot, warm-reboot & fast-reboot
In the reboot flows including "reboot", "fast-reboot" and "warm-reboot" this failure doesn't have further negative effects since the system has already rebooted. In addition, "warm-reboot" requires the system to be shutdown as soon as possible to meet the GR time restriction of both BGP and LACP. "fast-reboot" also requires to meet the GR time restriction of BGP which is longer than LACP. In this sense, any unnecessary steps should be avoided. It's better to keep those flows untouched.

summary
To summarize, we have to come up with a way to ensure:

shutdown pmon docker ahead of syncd for "config reload" or "service swss restart" flow;
don't shutdown pmon docker ahead of syncd for "fast-reboot" or "warm-reboot" flow in order to save time.
for "reboot" flow, either order is acceptable.
Solution
To solve the issue, pmon shoud be stopped ahead of syncd stopped for all flows except for the warm-reboot.

- How I did it

To stop pmon ahead of syncd stopped. This is done in /usr/local/bin/syncd.sh::stop() and for all shutdown sequence.
Now pmon stops ahead of syncd so there must be a way in which pmon can start after syncd started. Another point that should be taken consideration is that pmon starting should be deferred so that services which have the logic of graceful restart in fast-reboot and warm-reboot have sufficient CPU cycles to meet their deadline.
This is done by add "syncd.service" as "After" to pmon.service and startin /usr/local/bin/syncd.sh::wait()
To start pmon automatically after syncd started.
2019-09-27 10:15:46 +02:00
Stephen Sun
c34a4783e0 [build] install new platform api on host (#3282)
slave.mk: add SONIC_PLATFORM_API_PY2 as dependency of host
sonic_debian_extension.j2: install sonic_daemon_base and Mellanox-specific sonic_platform on host
mlnx-platform-api.mk: export mlnx_platform_api_py2_wheel_path for sonic_debian_extension.j2
sonic-daemon-base.mk: export daemon_base_py2_wheel_path for sonic_debian_extension.j2
daemon_base.py: hind unnecessary dependency of swss_common on host
2019-09-25 11:00:24 -07:00
Long Ou
b6a09999de [hostcfgd] hostcfgd will exit when set hostname in DEVICE_METADATA (#3394)
Signed-off-by: ouxiaolong <ouxiaolong@asterfusion.com>
2019-09-24 17:36:02 -07:00
Harish Venkatraman
9d2d617264 [SNMP] management VRF SNMP support (#2608)
* [SNMP] management VRF SNMP support

This commit adds SNMP support for Management VRF using l3mdev.
The patch included provides VRF support, there is no single
"listendevice" configuration, rather multiple agentaddress
config options can each have their own "interface" to bind to
using "ip%interface". The snmpd.conf file is accordingly
generated using the snmp.yml file and redis database info.

Adding below the comments of SNMP patch 1376
--------------------------------------------
Since the Linux kernel added support for Virtual Routing
and Forwarding (VRF) in version 4.3
(Note: these won't compile on non-linux platforms)

https://www.kernel.org/doc/Documentation/networking/vrf.txt

Linux users could not use snmpd in its current form to
bind specific listening IP addresses to specific VRF
devices. A simplified description of a VRF inteface
is an interface that is a master (a container of sorts)
that collects a set of physicalinterfaces to form a
routing table.

This set of two patches (one for V5-7-patches and one
for V5-8-patches branches) is almost identical to patch
single "listendevice" configuration. Rather, multiple
agentAddress config options can each have their own
"interface" to bind to using the <ip>%<interface>
syntax.</interface></ip>
-------------------------------------------

Signed-off-by: Harish Venkatraman <harish_venkatraman@dell.com>
2019-09-18 17:26:45 -07:00
Prince Sunny
8ca1eb289e
Install Iptables rules to set TCPMSS for 'lo' interface (#3452)
* Install Iptables rules to set TCPMSS for lo interface
* Moved implementation to hostcfgd to maintain at one place
2019-09-18 10:12:28 -07:00
sridhar-ravindran
3c0b56a709 [DELL] S6100 Support PowerCycle in Last Reboot Reason (#3403)
* [DELL] S6100 Support PowerCycle in Last Reboot Reason

* handle first time boot properly

* S6000 Last Reboot Reason Fix
2019-09-17 16:51:46 -07:00
Harish Venkatraman
31d1a76197 [baseimage]: Management vrf ntp support (#3204)
This commit adds NTP support for management VRF using L3mdev. Config vrf add
mgmt will enable management VRF, enslave the eth0 device to the master device
mgmt, stop ntp service in default, restart interfaces-configs and restart ntp
service in mgmt-vrf context. Requirement and design are covered in mgmt vrf
design document.

Signed-off-by: Harish Venkatraman <harish_venkatraman@dell.com>
2019-09-16 10:21:06 -07:00
padmanarayana
75104bb35d [sflow]: Build infrastructure changes to support sflow docker and utilities (#3251)
Introduce a new "sflow" container (if ENABLE_SFLOW is set). The new docker will include:
hsflowd : host-sflow based daemon is the sFlow agent
psample : Built from libpsample repository. Useful in debugging sampled packets/groups.
sflowtool : Locally dump sflow samples (e.g. with a in-unit collector)

In case of SONiC-VS, enable psample & act_sample kernel modules.

VS' syncd needs iproute2=4.20.0-2~bpo9+1 & libcap2-bin=1:2.25-1 to support tc-sample

tc-syncd is provided as a convenience tool for debugging (e.g. tc-syncd filter show ...)
2019-09-14 20:27:09 -07:00
Wenda Ni
81aef6b64c [Qos] use dot1p to tc mapping for backend switches (#3422)
* Use dot1p to tc mapping for backend switches

Signed-off-by: Wenda Ni <wenni@microsoft.com>

* Do not write DSCP to TC mapping into CONFIG_DB or config_db.json for
storage switches

Signed-off-by: Wenda Ni <wenni@microsoft.com>
2019-09-13 11:28:25 -07:00
Danny Allen
97c675c6d5 [cron.d] Add cron job to periodically clean-up core files (#3449)
* [cron.d] Create cron job to periodically clean-up core files
* Create script to scan /var/core and clean-up older core files
* Create cron job to run clean-up script

Signed-off-by: Danny Allen <daall@microsoft.com>

* Update interval for running cron job

* Respond to feedback

* Change syslog id
2019-09-13 10:50:31 -07:00
lguohan
95a72b4e39
[baseimage]: fix monit configuration (#3448)
- monit config broke by one monit upgrade
- abandon sed approach since it is suspestible to monit config changes
- use unixsocket instead of httpd due to a bug in 5.20.0
2019-09-12 22:48:40 -07:00
lguohan
a1158c6c18
Revert "Use dot1p to tc mapping for backend switches (#3412)" (#3421)
This reverts commit ca43dad12f.
2019-09-09 14:44:46 -07:00
Joe LeVeque
a27f12773b [baseimage]: Log message containing SONiC version to syslog at boot (#3416) 2019-09-09 14:18:23 -07:00
Wenda Ni
ca43dad12f Use dot1p to tc mapping for backend switches (#3412)
* Use dot1p to tc mapping for backend switches

Signed-off-by: Wenda Ni <wenni@microsoft.com>

* Do not write DSCP to TC mapping into CONFIG_DB or config_db.json for
storage switches

Signed-off-by: Wenda Ni <wenni@microsoft.com>
2019-09-06 11:59:47 -07:00
Danny Allen
cfcf30570b
[build_debian] Include checksum of ASIC config files in SONiC filesystem (#3384)
[build_debian] Generate checksum of ASIC config files

* Adds script to generate checksums for ASIC config files
* Adds step to build_debian that copies ASIC config checksum into SONiC filesystem

Signed-off-by: Danny Allen daall@microsoft.com
2019-09-05 19:41:35 -07:00
Dong Zhang
768beb79e1 create multiple Redis DB instances based on CONFIG at /etc/sonic/database_config.json (#2182)
this is the first step to moving different databases tables into different database instances

in this PR, only handle multiple database instances creation based on user configuration at /etc/sonic/database_config.json

we keep current method to create single database instance if no extra/new DATABASE configuration exist in database_config.json file.

if user try to configure more db instances at database_config.json , we create those new db instances along with the original db instance existing today.

The configuration is as below, later we can add more db related information if needed:
{
...
"DATABASE": {
"redis-db-01" : {
"port" : "6380",
"database": ["APPL_DB", "STATE_DB"]
},
"redis-db-02" : {
"port" : "6381",
"database":["ASIC_DB"]
},
}
...
}

The detail description is at design doc at Azure/SONiC#271

The main idea is : when database.sh started, we check the configuration and generate corresponding scripts.

rc.local service handle old_config copy when loading new images, there is no dependency between rc.local and database service today, for safety and make sure the copy operation are done before database try to read it, we make database service run after rc.local

Then database docker started, we check the configuration and generate corresponding scripts/.conf in database docker as well.

based on those conf, we create databases instances as required.

at last, we ping_pong check database are up and continue


Signed-off-by: Dong Zhang d.zhang@alibaba-inc.com
2019-08-28 11:15:10 -07:00
pavel-shirshov
8facac9149
[Fast-Reboot]: FR mode is active only first 3 minutes after start. (#3352)
* Fast reboot mode should be enabled only 3 minutes after restart

* Advance sonic-quagga submodule
2019-08-19 16:05:20 -07:00
Ying Xie
84b667fbaf
[radv service] radv service should be a cold only dependent of swss (#3348)
radv should be left alone during warm restart of swss. Otherwise it will
announce departure and cause hosts to lose default gateway.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-08-16 12:08:46 -07:00
Ying Xie
d6b4223bdd [control plane assistant] stop control plane assistant after warm reboot (#3337)
Delay saving configuration so that the control assistant configurations
won't be persisted.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-08-15 00:45:54 -07:00
Renuka Manavalan
fcdf62f5f6
Fix to ensure that tacacs servers are ordered (reverse) by priority in pam.d's config. (#3322)
Present: Servers are listed in the same order as in redis-db
Fix: Save the sort o/p, hence use sorted list to write into pam.d's conf.
     As well convert priority to integer for use by sort.
2019-08-09 11:46:46 -07:00
Ying Xie
a46df66d05
[service dependent] describe non-warm-reboot dependency outside systemd (#3311)
* [service dependent] describe non-warm-reboot dependency outside systemctl

When dependency was described with systemctl, it will kick in all the time,
including under warm reboot/restart scenarios. This is not what we always
want. For components that are capable of warm reboot/start, they need to
describe dependency in service files.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>

* [service] teamd service should not require swss service

Adding require swss will cause teamd to be killed by systemctl when swss
stops. This is not what we want in warm reboot.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>

* refactoring code

* rename functions to match other functions in the file
2019-08-08 15:45:17 -07:00
lguohan
2b28d55853
[build]: enable docker in ram option for small disk device (#3279)
when device disk is small, do not unzip dockerfs.tar.gz on disk.
keep the tar file on the disk, unzip to tmpfs in the initrd phase.

enabled this for 7050-qx32

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-08-06 23:04:00 -07:00
byu343
6add9445c8 [aboot-image]: Skip arista-hook and arista-convertfs for fast/warm-reboot (#3242) 2019-07-31 14:20:17 -07:00
Lawrence Lee
7271fe598f [build]: Move Systemd service start to systemd generator (#3172)
- What I did

 Move the enabling of Systemd services from sonic_debian_extension to a new systemd generator

- How I did it

  Create a new systemd generator to manually create symlinks to enable systemd services
  Add rules/Makefile to build generator
  Add services to be enabled to /etc/sonic/generated_services.conf to be read by the generator at boot time

Signed-off-by: Lawrence Lee <t-lale@microsoft.com>
2019-07-29 15:52:15 -07:00
arheneus@marvell.com
50fe458592 [build]: SONiC buildimage ARM arch support (#2980)
ARM Architecture support in SONIC

make configure platform=[ASIC_VENDOR_ARCH] PLATFORM_ARCH=[ARM_ARCH]
SONIC_ARCH: default amd64
armhf - arm32bit
arm64 - arm64bit

Signed-off-by: Antony Rheneus <arheneus@marvell.com>
2019-07-25 22:06:41 -07:00
Harish Venkatraman
3e69427ac0 [baseimage] management VRF support via l3mdev (#2585)
This commit adds support for New feature management VRF using L3mdev.  Added
commands to enable/disable management VRF. Config vrf add mgmt will enable
management VRF, enslave the eth0 device to the master device mgmt and restart
interfaces-configs in mgmt-vrf context.

management interface (eth0) can be configured using config interface eth0 ip
add command and removed using config interface eth0 ip remove command.

Requirement and design are covered in mgmt vrf design document.  Currently show
command displays linux command output; will update show command display in next
PR after concluding what would be the output for the show commands. Added
metric for default routes in dhcp and static, any changes for metric will be
addressed subsequently after discussing.

Signed-off-by: Harish Venkatraman <harish_venkatraman@dell.com>
2019-07-24 16:18:40 -07:00
Ying Xie
9d64ce761f
[warm reboot] save configuration after warm reboot (#3200)
* [warm reboot] save configuration after warm reboot

After warm reboot, save a copy of in memory database to config_db.json,
upgrade procedure might have removed config_db.json to force new image
to reload minigraph. However, reload minigraph is skipped during warm
reboot. Missing config_db.json would cause device to fault in next
non-upgrading cold/fast reboot.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>

* Update finalize-warmboot.sh
2019-07-24 09:59:47 -07:00
Ying Xie
401f7042a2
Revert "[database] save configuration after DB migration (#3143)" (#3199)
This reverts commit b5a4527cb0.
2019-07-22 14:13:50 -07:00
rajendra-dendukuri
40c8bc14cd [baseimage]: Upgrade ifupdown2 to version 1.2.8 (#3180)
* Upgrade ifupdown2 to version 1.2.8

Required by ZTP to support ZTP over IPv6 transport

Signed-off-by: Rajendra Dendukuri <rajendra.dendukuri@broadcom.com>
2019-07-19 23:09:14 -07:00
zzhiyuan
e4c041b57f [baseimage]: Fix process-reboot-cause possibly throwing OSError (#3159)
In case of going from previous iteration of SONiC, and the last reboot
was hardware, REBOOT_CAUSE_FILE may not be present and the service may
throw an error.
2019-07-16 08:34:11 -07:00
Ying Xie
b5a4527cb0
[database] save configuration after DB migration (#3143)
- Make sure that migrated DB contents persisted for next boot
- Make sure that db saved after warm reboot.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-07-15 20:21:02 -07:00
Stepan Blyshchak
59117d23f0 [swss.sh]: Cleanup LAG entries in STATE DB (#3114)
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2019-07-08 17:29:57 -07:00
Joe LeVeque
5e2ab9dd03
[process-reboot-cause] Handle case if platform does not yet have sonic_platform implementation (#3126) 2019-07-05 17:53:49 -07:00
Renuka Manavalan
76bf5a0bc4 [build]: Added debug symbols to many debug dockers. (#3098)
* Added debug symbols to many debug dockers.

* For debug images *only*:
1) Archive source files into debug image
2) Archived source is copied into /src
3) Created an empty dir /debug
4) Mount both /src as ro & /debug as rw into every docker
5) Login banner will give some details on /src & /debug
6) Devs can copy core file into /debug and view it from inside a container.
7) Dev may create all gdb logs and other data directly into /debug.

* Dropped redundant REDIS_TOOLS per review comments.

* Added debug symbols to frr package and hence FRR based BGP docker.

* 1) Moved dbg_files.sh to scripts/
2) Src directories to archive are now collected from individual Makefiles.
3) Added few more debug symbols
4) Added few more debug dockers.

Here after no more changes except per review comments.

To debug:
Install required version of debug image in Switch or VM.
Copy core file into /debug of host
Get into Docker
gdb /usr/bin/<daemon> -c /debug/<your core file>
set directory /src/... <-- inside gdb to get the source

For non-in-depth debugging:

Download corresponding debug Docker image (docker-...-dbg.gz) to your VM
Load the image
Run image with entrypoint as 'bash' with dir containing core mapped in.
Run gdb on the core.
2019-07-03 22:13:55 -07:00
Joe LeVeque
e5a2beb13b [reboot-cause]: Move reboot cause processing to its own service, 'process-reboot-cause' (#3102) 2019-07-03 10:38:20 -07:00
Michel Moriniaux
dc747247d1 [ARISTA] adding 7060_cs32s to eMMC exclusions (#2982)
* [ARISTA] adding 7060_cs32s to eMMC exclusions

Following PR 2774 we added the 7060-cx32s according to the guidelines of
PR 2780

This adds the 7060-cx32s to the list f devices that mount /var/log as a
tmpfs to mitigate eMMC wearout

Signed-off-by: Michel Moriniaux <m.moriniaux@criteo.com>

* [ARISTA] adding 7060_cs32s to eMMC exclusions

Following PR 2774 we added the 7060-cx32s according to the guidelines of
PR 2780

This adds the 7060-cx32s to the list f devices that mount /var/log as a
tmpfs to mitigate eMMC wearout

Signed-off-by: Michel Moriniaux <m.moriniaux@criteo.com>
2019-07-02 11:52:43 -07:00
Stepan Blyshchak
6961816dec fix fast reboot compatibility (#3083)
* fix fast reboot compatibility

We should handle both cases for backward-compatible with 201803:
 - fast-reboot
 - SONIC_BOOT_TYPE=fast-reboot

* handle review comments
* add a comment that getBootType code snippet is shared between two files
2019-06-26 12:46:58 -07:00
Jipan Yang
9a1bebe496 [telemetry]: change the service dependency from swss to database (#3072)
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
2019-06-24 12:36:16 -07:00
Joe LeVeque
319d854e46 [baseimage]: Increase TMOUT for serial port connections to 15 minutes (#3032)
Increase TMOUT value in order to close inactive serial console connections after 900 seconds (15 minutes) of inactivity
2019-06-19 00:16:01 -07:00
Qi Luo
e7b1988638
[submodule] update sonic-linux-kernel (#2985)
* [submodule] update sonic-linux-kernel
* update linux kernel version
* Fix many version strings
* update mellanox components (built with new kernel)
* [mlnx] add make files for SDK WJH libs
* Update arista driver submodule (#8)
Make the debian packaging point to a newer kernel version.
2019-06-18 10:00:16 -07:00
Kebo Liu
c927517355 [Mellanox] Inject SDK libs dependency to pmon on Mellanox platform (#3000)
* inject sdk libs to pmon
* fix wrong code
2019-06-14 17:38:24 -07:00
lguohan
8f6ae90cba
[docker]: get hostname from config db instead of minigraph (#3004)
minigraph may not be always available on the some system configuration.
Should use config db as the source of truth.
2019-06-13 22:24:09 -07:00
Renuka Manavalan
cdca062693 [build]: Build sonic-broadcom.bin using debug dockers for all stretch based dockers (#2833)
* Updated Makefile infrastructure to build debug images.
As a sample, platform/broadcom/docker-orchagent-brcm.mk is updated to add a docker-orchagent-brcm-dbg.gz target.

Now "BLDENV=stretch make target/docker-orchagent-brcm-dbg.gz" will build the debug image.

NOTE: If you don't specify NOSTRETcH=1, it implicitly calls "make stretch", which builds all stretch targets and that would include debug dockers too.

This debug image can be used in any linux box to inspect core file. If your module's external dependency can be suitably mocked, you my even manually run it inside.

"docker run -it --entrypoint=/bin/bash e47a8fb8ed38"

You may map the core file path to this docker run.

* Dropped the regular binary using DBG_PACKAGES and a small name change to help readability.

* Tweaked the changes to retain the existing behavior w.r.t INSTALL_DEBUG_TOOLS=y.

When this change ('building debug docker image transparently') is extended to all dockers, this flag would become redundant. Yet, there can be some test based use cases that rely on this flag.

Until after all the dockers gets their debug images by default and we switch all use cases of this flag to use the newly built debug images, we need to maintain the existing behavior.

* 1) slave.mk - Dropped unused Docker build args
2) Debug template builder: renamed build_dbg_j2.sh to build_debug_docker_j2.sh
3) Dropped insignifcant statement CMD from debug Docker file, as base docker has Entrypoint.

* Reverted some changes, per review comments.
"User, uid, guid, frr-uid & frr-guid" are required for all docker images, with exception of debug images.

* Get in sync with the new update that filters out dockers to be built (SONIC_STRETCH_DOCKERS_FOR_INSTALLERS) and build debug-dockers only for those to be built and debug target is available.

* Mkae a template for each target that can be shared by all platforms.
Where needed a platform entry can override the template.
This avoids duplication, hence easier to maintain.

* A small change, that can fit better with other targets too.
Just take the platform code and do the rest in template.

* Extended debug to all stretch based docker images

* 1) Combined all orchagent makefiles into one platform independent make under rules/docker-orchagent.mk
2) Extened debug image to all stretch dockers

* Changes per review comments:
1) Dropped LIBSAIREDIS_DBG from database, teamd, router-advertiser, telemetry, and platform-monitor docker*.mk files from _DBG_DEPENDS list
2) W.r.t docker make for syncd, moved DEPENDS from template to specific makefile and let the template has stuff that is applicable to all.

* 1) Corrected a copy/paste mistake

* Fixed a copy/paste bug

* The base syncd dockers follow a template, which defines the base docker as DOCKER_SYNCD_BASE instead of DOCKER_SYNCD_<platform code>. Fix the docker-syncd-<mlnx, bfn>.mk to use the new one.

[Yet to be tested locally]

* Fixed spelling mistake

* Enable build of dbg-sonic-broadcom.bin, which uses dbg-dockers in place of regular dockers, for dockers that build debug version. For dockers that do not build debug version, it uses the regular docker.

This debug bin is installable and usable in a DUT, just like a regular bin.

* Per review comments:
  1) Share a single rule for final image for normal & debug flavors (e.g. sonic-broadcom.bin & sonic-broadcom-dbg.bin)
  2) Put dbg as suffix in final image name.
  3) Compared target/sonic-broadcom.bin.logs with & w/o fix to verify integrity of sonic-broadcom.bin
  4) Compared target/sonic-broadcom.bin.logs with sonic-broadcom-dbg.bin.log for verification

This fix takes care of ONIE image only. The next PR will cover the rest.
The next PR, will also make debug image conditional with flag.

* Updated per comments.
Now that debug dockers are available, do not need a way to install debug symbols in regular dockers.

With this commit, when INSTALL_DEBUG_TOOLS=y is set, it builds debug dockers (for dockers that enable debug build) and the final image uses debug dockers. For dockers that do not enable debug build, regular dockers get used in the final image.

Note:
The debug dockers are explicitly named as <docker name>-dbg.gz. But there is no "-dbg" suffix for image.
Hence if you make two runs with and w/o INSTALL_DEBUG_TOOLS=y, you have complete set of regular dockers + debug dockers. But the image gets overwritten.
Hence if both regular & debug images are needed, make two runs, as one with INSTALL_DEBUG_TOOLS=y and one w/o. Make sure to copy/rename the final image, before making the second run.
2019-06-12 01:36:21 -07:00
Prince Sunny
231d309b69
Generate interface table to have an entry designated to default VRF. (#2848)
* Generate default VRF table for router interfaces

* Updated jinja2 template to have prefix filter
2019-06-10 14:02:55 -07:00
Myron Sosyak
3ec95e17c8 [build_templates] [hostcfgd] Keep containers hostname up to date (#2924)
* Add updateHostName function to docker_image_ctl.j2
* Add hostname specification on container creating step
* Add listener for hostname changes in hostcfgd

Signed-off-by: Myron Sosyak <msosyak@barefootnetworks.com>
2019-06-06 00:41:30 -07:00
Kebo Liu
bd519322cb [Mellanox] Expose SDK share buffer and unix socket from syncd (#2951)
* expose SDK share buffer and unix socket from syncd
* fix PR comments
* fix community comments and add TODO
2019-06-05 11:19:56 -07:00
Nazarii Hnydyn
e041b15d10 [mellanox]: Fixed config reload race. (#2930)
Signed-off-by: Nazarii Hnydyn <nazariig@mellanox.com>
2019-05-29 09:57:29 +03:00
lguohan
30b37ec6fb
[build]: make sonic-slave-stretch as the default build docker (#2921)
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-05-27 15:50:51 -07:00
Joe LeVeque
3ec3e20e5a [logrotate] Enhance robustness (#2942)
* [logrotate] Decrease frequency to every 10 minutes; kill any lingering logrotate processes

* [logrotate] Delete all *.1.gz files as firstaction; Remove note about init-system-helpers < 1.47 workaround

However, continue to send SIGHUP directly to rsyslogd process
because 'service rsyslog rotate' still doesn't work properly with
init-system-helpers version 1.48
2019-05-25 18:00:18 -07:00
Stepan Blyshchak
9523e64666 [swss.sh] flush FDB table during cold start (#2933)
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2019-05-22 22:07:29 -07:00
Ying Xie
222706120d [updategraph] set DB version after minigraph reload (#2917)
Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-05-18 22:08:41 -07:00
Samuel Angebault
aac0c24312 [device/Arista] Add support for the 7280CR3-32P4 (#2910)
* Add boot0 support for the 7280CR3

* Add platform and plugins for 7280CR3

* Add port config for 7280CR3

* Add platform_reboot for 7280CR3

* Add support for 7280CR3-32D4 based on the 7280CR3-32P4

* Update arista driver submodules

 - Introduce new 7280CR3-32P4
 - Improve to the led plugin for OSFP
2019-05-18 10:34:07 -07:00
Samuel Angebault
77cde50541 [device/Arista] Improvements to the boot of Arista devices. (#2898)
* Fix showing systemd shutdown sequence when verbose is set

* Fix creation of kernel-cmdline file

Sometimes boot0 prints error
"mv: can't preserve ownership of '/mnt/flash/image-arsonic.xxxx/kernel-cmdline': Operation not permitted"

* Improve flash space usage during installation

Some older systems only have 2GB of flash available. Installing a second
image on these can prove to be challenging.
The new installation process moves the installer swi to memory in order
to avoid free up space from the flash before uncompressing it there.
It removes all the flash space usage spike and also improves the IO
since the installation is no more reading and writting to the flash at
the same time.

* Add support of 7060CX-32S-SSD

* 7260CX3: use inventory powerCycle procedures

* 7050QX-32S: use inventory powerCycle procedures

* 7050QX-32: use inventory powerCycle procedures

* platform: arista: add common platform_reboot

Replace platform_reboot by a link to new common for devices already
using a similar script.

* 7060CX-32S: use inventory powerCycle procedures

* Install python smbus in pmon

Some platform plugin need the python smbus library to perform some actions.
This installs the dependency.
2019-05-15 12:45:05 -07:00
Renuka Manavalan
a357693f52 [tacacs]: skip accessing tacacs servers for local non-tacacs users (#2843)
* Switch the nss look up order as "compat" followed by "tacplus".
This helps use the legacy passwd file for user info and go to tacacs only if not found.
This means, we never contact tacacs for local users like "admin".
This isolates local users from any issues with tacacs servers.
W/o this fix, the sudo commands by local users could take <count of servers> * <tacacs timeout> seconds, if the tacacs servers are unreachable.

* Skip tacacs server access for local non-tacacs users.
Revert the order of 'compat tacplus' to original 'tacplus compat' as tacplus
access is required for all tacacs users, who also get created locally.
2019-05-09 14:36:32 -07:00
Ying Xie
9efcf1759a
[ebtables] install ebtables in base image and install filter rules (#2805)
- Add ebtables package, and install some filter rules:
  1. ebtables -A FORWARD -d BGA -j DROP
  2. ebtables -A FORWARD -p ARP -j DROP

Basically, we let the ARP packets in the VLAN being forwarded by the ASIC,
kernel gets a copy of these ARP packets and the forwarding from Kenerl gets
dropped. So there is always only one copy of ARP/response in the VLAN.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-05-09 09:44:41 -07:00
lguohan
5fb185cd83
[docker-frr]: bring quagga docker features to frr docker (#2870)
- use superviord to manage process in frr docker
- intro separated configuration mode for frr
- bring quagga configuration template to frr.

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-05-08 23:00:49 -07:00
Joe LeVeque
6eca27e564 [services] Restart SwSS service upon unexpected critical process exit (#2845)
* [service] Restart SwSS Docker container if orchagent exits unexpectedly

* Configure systemd to stop restarting swss if it attempts to restart more than 3 times in 20 minutes

* Move supervisor-proc-exit-listener script

* [docker-dhcp-relay] Enhance wait_for_intf.sh.j2 to utilize STATEDB

* Ensure dependent services stop/start/restart with SwSS

* Change 'StartLimitInterval' to 'StartLimitIntervalSec', as Stretch installs systemd 232 (>= v230)

* Also update journald.conf options

* Remove 'PartOf' option from unit files

* Add '$(SUPERVISOR_PROC_EXIT_LISTENER_SCRIPT)' to new shared docker-orchagent makefile

* Make supervisor-proc-exit-listener script read from 'critical_processes' file inside container

* Update critical_processes file for swss container
2019-05-01 08:02:38 -07:00
Joe LeVeque
2736da97c7 [sudoers] Add /usr/bin/teamshow to READ_ONLY_CMDS (#2846) 2019-05-01 08:01:44 -07:00
Ying Xie
6431248243
[db migrator] migrate the DB to latest schema when needed (#2808)
Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-04-30 14:46:18 -07:00
Qi Luo
6b3a26f0cc
Remove unused packages in docker images and host (#2807)
* Remove unneeded packages in docker images and host
* Remove libpython3.6 from snmp docker image
2019-04-29 17:21:24 -07:00
Ying Xie
c7af19a4db
[teamd service] start teamd service after swss (#2829)
SWSS clears DB tables, if teamd is not started after swss, there is a
race condition that swss might clear vital teamd information.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-04-26 15:12:33 -07:00
Andriy Moroz
ca7924eb27 Increase syncd start timeout (#2776)
* Increase syncd start timeout

Signed-off-by: Andriy Moroz <c_andriym@mellanox.com>

* Replace TimeoutSec to TimeoutStartSec

Signed-off-by: Andriy Moroz <c_andriym@mellanox.com>
2019-04-24 17:51:26 +03:00
zhenggen-xu
75964ef243 [baseimage]: Add fstrim service and fstrim timer by default (#2804)
This service (weekly) will let SSD firmware to do the garbage collection
after file-system deleted files. It could avoid slowness or
even READ-ONLY error due to SSD not being able to free the pages
even though the file system thinks there was a lot of space left.

Signed-off-by: Zhenggen Xu <zxu@linkedin.com>
2019-04-21 14:21:16 -07:00
Stepan Blyshchak
6a4ffef1fd [snmp.service] Make swss.service a requisite (#2790) 2019-04-16 18:32:36 -07:00
Ying Xie
8bf9247c5e
[tmpfs var/log] mount /var/log as tmpfs for some platforms (#2780)
SONiC is a heavy writer to /var/log partition, we noticed that this
behavior causes certain flash drive to become read-only over time.
To avoid this issue, we mount /var/log parition on these devices as
tmpfs.

- Mount /var/log as tmpfs
- /var/log default size is 128M
- Adjust size according to existing var-log.ext4 file size.
- Adjust size to between 5% to 10% of total memory size.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-04-14 22:46:26 -07:00
Ying Xie
f583f57af6
[service] add warmboot finializer service (#2715)
After warm reboot is done, we need to disable warm reboot flag and
tear down anything setup for warm reboot and persisted across.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-04-12 15:45:58 -07:00
Renuka Manavalan
6d7ecc426c [hostcfgd] -- Fix the default for failthrough as false.
This implies that by default, if TACACS is configured properly and it reported auth_err, then don't try fail through to traditional unix authentication through /etc/passwd.

If this failthrough is intended, make it explicit through "sudo config aaa authentication failthrough enable"

Removed an unused variable "aaa.fallback"

Tested manually. Note the presence of 'auth_err=die' in all cases except when failthrough is explicitly enabled.

admin@str-s6000-acs-13:~$ sudo config aaa authentication failthrough default; date
Wed Apr  3 23:05:18 UTC 2019
admin@str-s6000-acs-13:~$ ls -lrt /etc/pam.d/common-auth-sonic ; grep 123 /etc/pam.d/common-auth-sonic
-rw-r--r-- 1 root root 1316 Apr  3 23:05 /etc/pam.d/common-auth-sonic
auth    [success=done new_authtok_reqd=done default=ignore auth_err=die]        pam_tacplus.so server=100.127.20.22:49 secret=testing123 login=login timeout=5 try_first_pass
auth    [success=done new_authtok_reqd=done default=ignore auth_err=die]        pam_tacplus.so server=100.127.20.21:49 secret=testing123 login=login timeout=5 try_first_pass

admin@str-s6000-acs-13:~$ sudo config aaa authentication failthrough enable; date ; h4 "AAA|authentication"
Wed Apr  3 23:06:37 UTC 2019
admin@str-s6000-acs-13:~$ ls -lrt /etc/pam.d/common-auth-sonic ; grep 123 /etc/pam.d/common-auth-sonic
-rw-r--r-- 1 root root 1294 Apr  3 23:06 /etc/pam.d/common-auth-sonic
auth    [success=done new_authtok_reqd=done default=ignore]     pam_tacplus.so server=100.127.20.22:49 secret=testing123 login=login timeout=5 try_first_pass
auth    [success=done new_authtok_reqd=done default=ignore]     pam_tacplus.so server=100.127.20.21:49 secret=testing123 login=login timeout=5 try_first_pass

admin@str-s6000-acs-13:~$ sudo config aaa authentication failthrough disable; date ; h4 "AAA|authentication"
Wed Apr  3 23:07:09 UTC 2019
admin@str-s6000-acs-13:~$ ls -lrt /etc/pam.d/common-auth-sonic ; grep 123 /etc/pam.d/common-auth-sonic
-rw-r--r-- 1 root root 1321 Apr  3 23:07 /etc/pam.d/common-auth-sonic
auth    [success=done new_authtok_reqd=done default=ignore auth_err=die]        pam_tacplus.so server=100.127.20.22:49 secret=testing123 login=login timeout=5 try_first_pass
auth    [success=done new_authtok_reqd=done default=ignore auth_err=die]        pam_tacplus.so server=100.127.20.21:49 secret=testing123 login=login timeout=5 try_first_pass
2019-04-03 23:16:56 +00:00
Ying Xie
00a0f22f38
Revert "[teamd service] teamd service should start after syncd (#2724)" (#2733)
This reverts commit 0d1efb131c.
2019-04-03 08:20:44 -07:00
paavaanan
b56124bf48 removing dhcp- turn- off option from initrd (#2555)
* removing dhcp changes from initrd

* removing mgmt-intf-dhcp file
2019-04-02 15:48:04 -07:00
Ying Xie
0d1efb131c
[teamd service] teamd service should start after syncd (#2724)
* [teamd service] teamd service should start after syncd

Signed-off-by: Ying Xie <ying.xie@microsoft.com>

* combine after lines
2019-04-01 15:40:22 -07:00
Qi Luo
9c83b5480d
[security] Do not generate ssh server keys for non RSA protocols (#2718) 2019-03-29 15:27:33 -07:00
Ying Xie
698b248a13
[docker script] skip docker mount point checking for database container (#2683)
database container doesn't mount hwsku folder.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-03-19 20:14:07 -07:00
Renuka Manavalan
ae05579c67 [baseos]: Install ipaddress python package that has deprecated current ipaddr. … (#2674)
* Install ipaddress python package that has deprecated current ipaddr. ipaddress has backport to python2.7

* Install python ipaddress module as required by route_check.py sonic utility. BTW, ipaddress deprecates ipaddr and ipaddress has python2 backport

* Revert the old chaneg per review comments.

Signed-off-by: Renuka Manavalan <remanava@microsoft.com>
2019-03-18 11:12:47 -07:00
Pavlo Yadvichuk
11c2e9ee3d [barefoot]: Allow configuration of platform-specific interfaces used for internal purposes (#2631)
- Why it is required
since SONiC master switches ifupdown package to the new implementation (ifupdown2), it is required to change the configuration of a platform-specific interface for wedge100bf_32x and wedge100bf_65x platforms (bc of ifupdown2 doesn't support auto mode for inet6 protocol).

Also, need to make some refactoring and remove if platform == smth then.. from the system level scripts.

- What I did

removed customization of /usr/bin/interfaces-config.sh
explicitly created directory /etc/network/interfaces.d
added "source" to the /etc/network/interfaces generation template (to include platform-specific interfaces processing)
added platform-specific interfaces config itself (for wedge100bf_32x and wedge100bf_65x)
fixed testcase in sonic-config-engine
- How to verify it

build image for wedge100bf_32x
perform sudo config reload -y on new installation
check the correct configuration of usb0 interface
- Description for the changelog

Allow configuration of platform-specific interfaces
2019-03-09 06:22:32 -08:00
Joe LeVeque
2bb5400948 [services] Services which start containers now use 'docker wait' instead of 'docker attach' (#2661) 2019-03-08 10:59:41 -08:00
Wenda Ni
f9c9fa8ba1 [qos]: Map tc 1, 2, 5, and 6 back to pg 0 (#2650)
Lossy traffic does not need to be mapped to different ingress PGs. They can all share the same ingress PG.

Signed-off-by: Wenda Ni <wenni@microsoft.com>
2019-03-08 02:23:32 -08:00
Nazarii Hnydyn
b22fe37670 [mellanox]: Upgraded hw-management V.2.0.0160. (#2643)
Signed-off-by: Nazarii Hnydyn <nazariig@mellanox.com>
2019-03-06 18:51:46 -08:00
Wenda Ni
784bf77a92 Add hook to allow customizing link cable lengths
Signed-off-by: Wenda Ni <wenni@microsoft.com>
2019-03-05 22:06:00 +00:00
Ying Xie
66f5202b9f
[swss/syncd] cold start syncd service in swss in attach method (#2639)
start() is called by service startPre method, which is blocking. Starting
syncd service here is causing deadlock.

attach() is called by service start method, which is non-blocking.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-03-04 16:46:55 -08:00
RAMA CHANDRA REDDY GADDAM
b9edb7153d [aaa] Fix common-auth-sonic.j2 template issue (#2613) 2019-03-02 15:36:35 -08:00
Joe LeVeque
5eb7872a07 [services] Ensure swss and syncd services start before dependent services (#2634)
* [services] Ensure swss and syncd services start before dependent services

* Add 'attach' functions to scripts which get installed to /usr/local/bin so that services only reference the one script each

* Add 'After=swss.service' to syncd.service
2019-03-02 15:28:34 -08:00
yurypm
d632569a6a Add initramfs hook for Arista devices (#2595)
We are going to use initramfs hook for firmware upgrades
To install Arista hook:
- create folder /mnt/flash/<image dir>/platform/hooks/boot1/ from Aboot or
  /host/<image dir>/platform/hooks/boot1/ from Sonic
- add executable script to created folder
2019-02-27 10:28:04 -08:00
Ying Xie
3086f4f391
Revert "[baseimage] Delay ntp-config service to start after 5 minutes (#2494)" (#2590)
This reverts commit 33fe8d298e.
2019-02-21 10:04:54 -08:00
Nikos
1158277533 [frr]: staticd terminating due to inadequate permissions (#2580)
Signed-off-by: nikos <ntriantafillis@gmail.com>
2019-02-19 21:50:19 -08:00
lguohan
572db1e0a9
[swss]: flush asic db in swss.sh for non warm-boot (#2582)
need to flush asic db in swss.sh instead of syncd.sh

orchagent might already started in swss.sh and put commands
into asic db before asic db is flushed in syncd.sh. This
causes race condition such as INIT_VIEW not passing to syncd.

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-02-19 21:48:43 -08:00
Jipan Yang
ff74daaf13 Move warm_restart enable/disable config to stateDB WARM_RESTART_ENABLE_TABLE (#2538)
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
2019-02-19 17:06:56 -08:00
Renuka Manavalan
fa7c46611e [hostcfgd]: Promote logs for update-notifications-from-DB from DEBUG to INFO (#2576)
* Add a log message for each notification of add/del TACACS server.

Signed-off-by: Renuka Manavalan <remanava@microsoft.com>

* Moved another syslog message from DEBUG to INFO to be able to see those notifications.

All these changes are to help with a one-time-seen-bug, that hostcfgd did not act upon changes to redis for TACACS servers. We could not repro the bug.

Signed-off-by: Renuka Manavalan <remanava@microsoft.com>
2019-02-16 10:17:13 -08:00
Stepan Blyshchak
2dd769bf46 [syncd.sh] Don't stop sxdkernel during warm shutdown on Mellanox platform (#2572)
/etc/init.d/sxdkernel stop may take up to 15 sec which has impact on
control plane downtime

Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2019-02-15 16:08:08 -08:00
Nazarii Hnydyn
d53df059d4 [devices]: Added new SN3700/SN3700C Mellanox platforms (#2548)
* [mlnx-msn3700]: Added MSN3700 platform.

Signed-off-by: Nazarii Hnydyn <nazariig@mellanox.com>

* [mlnx-msn3700]: Upgrade FW burn: use ASIC auto detect.

Signed-off-by: Nazarii Hnydyn <nazariig@mellanox.com>

* [mlnx-msn3700]: Updated HW-MGMT/FW/MFT/SAI/SDK.

Signed-off-by: Nazarii Hnydyn <nazariig@mellanox.com>

* [mlnx-msn3700]: Added MSN3700C platform.

Signed-off-by: Nazarii Hnydyn <nazariig@mellanox.com>
2019-02-13 23:08:04 -08:00
Ying Xie
44551d0fb5
[swss/syncd] log swss/syncd service script activities (#2545)
Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-02-10 11:56:31 -08:00
zzhiyuan
6037707abc [devices]: Add device data for Arista 7060PX/DX4-32 (#2534)
* Add boot0 definition for Arista 7060PX4-32 and 7060DX4-32

* Add port configuration for Arista 7060PX4-32

* Add plugins for Arista 7060PX4-32

* Add platform_reboot for Arista 7060PX4-32

* Add Arista 7060DX4-32 as symlink of 7060PX4-32

* Add sensors configuration and fancontrol for Arista 7060PX4-32

* Update arista-driver submodules for barefoot/broadcom

* Add platform_reboot script for Alhambra

* Rook fancontrol CPLD rename
2019-02-08 22:02:01 -08:00
Nadiia Stetskovych
bb5a171ffc [minigraph]: Do not fail for minigraphs which do not have neighbors listed in <Devices> section (#2522)
Signed-off-by: Nadiya.Stetskovych <nstetskovych@barefootnetworks.com>
2019-02-04 22:43:08 -08:00
lguohan
f20665008c
[build]: put stretch debian packages under target/debs/stretch/ (#2519)
* [build]: put stretch debian packages under target/debs/stretch/

* in stretch build phase, all debian packages built in that stage are placed under target/debs/stretch directory.
* for python-based debian packages, since they are really the same for jessie and stretch, they are placed under target/python-debs directory.

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-02-04 22:06:37 -08:00
zhenggen-xu
982eddfaa4 [updategraph] After system upgrade, restore files/directories with original attributes etc. (#2368)
* [updategraph] After system upgrade, restore files/directories with
original attributes etc.
Restore a few more files that was missed before.
Restore FRR configuration directory if exists on old system

Signed-off-by: Zhenggen Xu <zxu@linkedin.com>

* Removed deployment_id_asn_map.yml from copy list

Signed-off-by: Zhenggen Xu <zxu@linkedin.com>
2019-02-02 12:50:19 -08:00
lguohan
9c2d7240ea
[vs]: Force10-S6000 buffer settings for virtual switch (#2515)
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-02-01 11:18:02 -08:00
Prince Sunny
39e12a1d82 [swss]: Change VrfMgrd startup order, cleanup VRF_TABLE from state DB (#2510) 2019-01-31 23:28:31 -08:00
Wenda Ni
58adf06cc0 [QoS]: Link pg 2 and 6 to lossy buffer profile (#2511)
* Link pg 2 and 6 to lossy buffer profile

Signed-off-by: Wenda <wenni@microsoft.com>
2019-01-31 23:27:58 -08:00