Commit Graph

756 Commits

Author SHA1 Message Date
judyjoseph
642479f75d
[multi-Asic] Add support for multi-asic to swssloglevel (#5316)
* Support for multi-asic platform for swssloglevel command

admin@str-acs-1:~$ swssloglevel 
Usage: /usr/bin/swssloglevel -n [0 to 3] [OPTION]... 

* Update to use the env file to get the PLATFORM string.
2020-09-16 11:29:05 -07:00
Tamer Ahmed
15f5d47338
[dhcpmon] Print Both Snapshot And Current Counters (#5374)
Printing both snapshot and current counter sets will make it easier to pinpoint
which message type(s) is/are not being relayed. This PR prints both counter sets.
Also, this PR defines gnu11 as a C standard to compile with in order to avoid
making changes when porting to 201811 branch.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-15 15:27:36 -07:00
Joe LeVeque
12c94a7431
[lldpmgrd] Inherit DaemonBase class from sonic-py-common package (#5370)
Eliminate duplicate logging and signal handling code by inheriting from DaemonBase class in sonic-py-common package.
2020-09-15 10:55:55 -07:00
Tamer Ahmed
1bf6fdc6d2
[dhcpmon] Monitor Mgmt Interface For DHCP Packets (#5317)
When BGP routes are missing, DHCP packets get relayed over mgmt
interface. This results in dhcpmon alerting that DHCP packets are
not being relayed. This is PR include mgmt interface as uplink
device, and so, if DHCP packet gets relayed over mgmt interface,
regular dhcpmon alert will not be issues. Instead, dhcpmon will
check the mgmt interface counts and issue a separate alert regarding
packets travelling through mgmt network.

In addition, this PR includes the following enhancements:
1. Add SIGUSR1 handler that prints out current packet counts
2. Increase alert grace window to 3 minutes from currently 2 minutes
3. Time is now computed more accurately
4. Print vlan name before counters

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-09 18:37:01 -07:00
Joe LeVeque
5b3b4804ad
[dockers][supervisor] Increase event buffer size for dependent-startup (#5247)
When stopping the swss, pmon or bgp containers, log messages like the following can be seen:

```
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,061 ERRO pool dependent-startup event buffer overflowed, discarding event 34
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,063 ERRO pool dependent-startup event buffer overflowed, discarding event 35
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,064 ERRO pool dependent-startup event buffer overflowed, discarding event 36
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,066 ERRO pool dependent-startup event buffer overflowed, discarding event 37
```

This is due to the number of programs in the container managed by supervisor, all generating events at the same time. The default event queue buffer size in supervisor is 10. This patch increases that value in all containers in order to eliminate these errors. As more programs are added to the containers, we may need to further adjust these values. I increased all buffer sizes to 25 except for containers with more programs or templated supervisor.conf files which allow for a variable number of programs. In these cases I increased the buffer size to 50. One final exception is the swss container, where the buffer fills up to ~50, so I increased this buffer to 100.

Resolves https://github.com/Azure/sonic-buildimage/issues/5241
2020-09-08 23:36:38 -07:00
Qi Luo
d4fc8e5b22
[redis] Use redis-server and redis-tools in blob storage to prevent upstream link broken (#5340)
* [redis] Use redis-server and redis-tools in blob storage to prevent
upstream link broken
* Use curl instead of wget
* Explicitly install dependencies
2020-09-08 19:30:14 -07:00
abdosi
eff745fbb1
Fix the issue as reported in (#5315)
https://github.com/Azure/sonic-buildimage/issues/5255

Root Cause: Waiting on Restore count != 0 can lead to race condition
between orchagent process and swssconfig.sh.

Ideally check of  Restore count != 0 is not needed as the State DB
cannot be flushed as if it was flushed then Warm Restart or swss-restart
should not be true also.
2020-09-04 09:34:26 -07:00
Joe LeVeque
fb8f09a116
[radvd] No longer build from source; Install vanilla Debian package once again (#5242)
Remove radvd Makefile and patch, change docker-router-advertiser Dockerfile template to simply install the vanilla radvd package using apt-get.

- In PR https://github.com/Azure/sonic-buildimage/pull/2795, we started building radvd from source and patching it to prevent it from erroring out when advertising an MTU of 9100 which was greater than the MTU size configured on the bridge interface (1500), which was due to a limitation in the 4.9 Linux kernel.
- Master branch is now using Linux kernel 4.19. As of 4.18, the kernel supports setting a bridge MTU to a value > 1500.
- PR https://github.com/Azure/sonic-swss/pull/1393 modified vlanmgrd to take advantage of this and now configures the MTU of bridge interfaces in SONiC to the proper size of 9100. Therefore, we no longer need to patch radvd. Since we no longer need to patch radvd, we no longer need to build it from source, so we can save build time by going back to simply installing the vanilla radvd Debian package in the router-advertiser container.
2020-09-01 13:53:36 -07:00
arlakshm
17e78715ae
[Multi-ASIC]:Update the template to add ipinip entry for Loopback4096 (#5235)
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
The following changes are done. 
- Multi asic platform have 2 Loopback interfaces, Loopback0 and Loopback4096. IPinIP decap entries need to be added for both of them. Update the ipinip.json.j2 template to add decap entries for Loopback4096.
- Add corressponding unit test
2020-08-31 17:35:48 -07:00
Prince Sunny
4338d8293f
Skip vnet-vxlan interfaces from generating networks (#5251)
* Skip Vnet interface from generating networks
2020-08-27 14:14:04 -07:00
Mykola F
834a29cb66
[enable counters] Enable port buffer drops by default and update MLNX SAI submodule (#5059)
* Enable port buffer drops by default
* [Mellanox] Update SAI_Implementation

Signed-off-by: Mykola Faryma <mykolaf@mellanox.com>
2020-08-25 08:48:56 -07:00
shi-su
f3feb56c8a
Add switch for synchronous mode (#5237)
Add a master switch so that the sync/async mode can be configured.
Example usage of the switch:
1.  Configure mode while building an image
    `make ENABLE_SYNCHRONOUS_MODE=y <target>`
2. Configure when the device is running 
    Change CONFIG_DB with `sonic-cfggen -a '{"DEVICE_METADATA":{"localhost": {"synchronous_mode": "enable"}}}' --write-to-db`
    Restart swss with `systemctl restart swss`
2020-08-24 14:04:10 -07:00
Joe LeVeque
97d44214cf
[docker-radv] Fix startup issues (#5230)
**- Why I did it**

PR https://github.com/Azure/sonic-buildimage/pull/4599 introduced two bugs in the startup of the router advertiser container:

1. References to the `wait_for_intf.sh` script were changed to `wait_for_link.sh`, but the actual script was not renamed
2. The `ipv6_found` Jinja2 variable added to the supervisor config file goes out of scope before it is read.

**- How I did it**
1. Rename the `wait_for_intf.sh` script to `wait_for_link.sh`
2. Use the Jinja2 "namespace" construct to fix the scope issue

**- How to verify it**

Ensure all processes in the radv container start properly under the correct conditions (i.e., whether or not there is at least one VLAN with an IPv6 address assigned).
2020-08-21 13:12:01 -07:00
Tamer Ahmed
adcca53b8d
[radv] Reduce Calls to SONiC Cfggen (#5178)
Calls to sonic-cfggen is CPU expensive. This PR reduces calls to
sonic-cfggen to one call during startup when starting radv service.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-17 15:48:04 -07:00
Tamer Ahmed
89f3206a3f
[swss] Reduce Calls to SONiC Cfggen (#5177)
Calls to sonic-cfggen is CPU expensive. This PR reduces calls to
sonic-cfggen to one call during startup when starting swss service.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-17 15:47:52 -07:00
Tamer Ahmed
a10c5bfd02
[frr] Reduce Calls to SONiC Cfggen (#5176)
Calls to sonic-cfggen is CPU expensive. This PR reduces calls to
sonic-cfggen to two calls during startup when starting frr service.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-17 15:47:42 -07:00
Tamer Ahmed
3a10e9c6fa
[dhcp-relay] Reduce Calls to SONiC Cfggen (#5175)
Calls to sonic-cfggen is CPU expensive. This PR reduces calls to
sonic-cfggen to one call during startup when starting dhcp-relay
service.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-17 15:47:14 -07:00
Tamer Ahmed
cc970627d0
[snmp]: Reduce Calls to SONiC Cfggen (#5166)
Calls to sonic-cfggen is CPU expensive. This PR reduces calls to
sonic-cfggen to once calle during snmp startup

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-17 15:46:29 -07:00
Joe LeVeque
6132ae34fe
[build] Build/install remaining platform daemons as Python wheel packages (#5188)
As part of migrating all Python-based package installers to wheel format rather than Debian packages. Also to allow for easily building a Python 3 version of the package in the near future. ledd and psud were converted in earlier PRs. This PR converts the remainder:

- pcied
- syseepromd
- thermalctld
- xcvrd
2020-08-15 08:42:11 -07:00
Joe LeVeque
c3202d8982
[build] Build/install sonic-psud as a Python wheel package (#5182)
As part of migrating all Python-based package installers to wheel format rather than Debian packages. Also to allow for easily building a Python 3 version of the package in the near future.
2020-08-14 11:11:45 -07:00
Joe LeVeque
fc9e97fc3d
[build] Build/install sonic-ledd as a Python wheel package (#5168)
As part of migrating all Python-based package installers to wheel format rather than Debian packages. Also to allow for easily building a Python 3 version of the package in the near future.

- Also remove some references to sonic-daemon-base which I previously missed and add missing sonic-py-common dependency for sonic-pcied.
2020-08-13 11:26:43 -07:00
Tamer Ahmed
7872b4e196
[platform] Add Support For Environment Variable File (#5010)
* [platform] Add Support For Environment Variable

This PR adds the ability to read environment file from /etc/sonic.
the file contains immutable SONiC config attributes such as platform,
hwsku, version, device_type. The aim is to minimize calls being made
into sonic-cfggen during boot time.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-07-31 17:59:09 -07:00
abdosi
ec435b955c
Changes to add template support for copp.json. (#5053)
* Changes to add template support for copp.json.
This is needed so that we can install differnt type of
Traps based on Device Role (Tor/Leaf/Mgmt/etc...).

Initial use case is to install DHCP/DHCPv6 tarp only
for tor router.

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>

* Fixed based on review comments.

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>

* Fixed based on review comment.
2020-07-31 14:14:21 -07:00
joyas-joseph
f0dfe36953
[docker-fpm-frr]: Upgrade docker-fpm-frr to buster (#4920)
Verify that /etc/apt/sources.list points to buster using docker exec bgp cat /etc/apt/sources.list

BGP neighborship is established.

root@sonic:~# show ip bgp summary 

IPv4 Unicast Summary:
BGP router identifier 10.1.0.1, local AS number 65100 vrf-id 0
BGP table version 1
RIB entries 1, using 184 bytes of memory
Peers 1, using 20 KiB of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
6.1.1.1         4        100      96      96        0    0    0 01:32:04            0

Total number of neighbors 1
root@sonic:~#  

Signed-off-by: Joyas Joseph <joyas_joseph@dell.com>
2020-07-29 14:19:03 -07:00
Sujin Kang
02a98add92
Add pcied to PMON docker to monitor the PCIe device status (#5000)
* Add pcied to PMON container

* remove tailing spaces

* update pmon submodule

* review comments

* rebase to the latest
2020-07-29 11:27:49 -07:00
Qi Luo
48b5792b07
[redis] Upgrade redis version (#5060)
buster-backports updated and the old version disappeared
2020-07-28 20:50:31 -07:00
Joe LeVeque
2600747f0e
[docker-pmon] Fix copy of fancontrol config file (#5037)
Copy proper fancontrol config file to the proper destination. Also some minor refactoring for code reuse to help prevent issues like this in the future.

Fixes a bug introduced by #4599
2020-07-28 00:23:21 -07:00
pavel-shirshov
89184038fd
[docker-fpm-frr]: Start bgpd after zebra was started (#5038)
fixes https://github.com/Azure/sonic-buildimage/issues/5026
Explanation:
In the log from the issue I found:
```
I see following in the log
Jul 22 21:13:06.574831 vlab-01 WARNING bgp#bgpd[49]: [EC 33554499] sendmsg_nexthop: zclient_send_message() failed
```
Analyzing source code I found that the error message could be issues only when `zclient_send_rnh()` return less than 0.
```
	ret = zclient_send_rnh(zclient, command, p, exact_match,
			       bnc->bgp->vrf_id);
	/* TBD: handle the failure */
	if (ret < 0)
		flog_warn(EC_BGP_ZEBRA_SEND,
			  "sendmsg_nexthop: zclient_send_message() failed");
```
I checked [zclient_send_rnh()](88351c8f6d/lib/zclient.c (L654)) and found that this function will return the exit code which the function gets from [zclient_send_message()](88351c8f6d/lib/zclient.c (L266)) But the latter function could return not 0 in two cases:
1.	bgpd didn’t connect to the zclient socket yet [code](88351c8f6d/lib/zclient.c (L269))
2.	The socket was closed. But in this case we would receive the error message in the log. (And I can find the message in the log when we reboot sonic) [code](88351c8f6d/lib/zclient.c (L277))

Also I see from the logs that client connection was set later we had the issue in bgpd.

Bgpd.log
```
Jul 22 21:13:06.574831 vlab-01 WARNING bgp#bgpd[49]: [EC 33554499] sendmsg_nexthop: zclient_send_message() failed
```
Vs
Zebra.log
```
Jul 22 21:13:12.713249 vlab-01 NOTICE bgp#zebra[48]: client 25 says hello and bids fair to announce only static routes vrf=0
Jul 22 21:13:12.820352 vlab-01 NOTICE bgp#zebra[48]: client 30 says hello and bids fair to announce only bgp routes vrf=0
Jul 22 21:13:12.820352 vlab-01 NOTICE bgp#zebra[48]: client 33 says hello and bids fair to announce only vnc routes vrf=0
```
So in our case we should start zebra first. Wait until it is started and then start bgpd and other daemons.

**- How I did it**

I changed a graph to start daemons in the following order:
1. First start zebra
2. Then starts staticd and bgpd
3. Then starts vtysh -b and bgpeoi after bgpd is started.
2020-07-25 03:48:47 -07:00
anish-n
da017f4ec9
[bgpcfgd]: Add Vlan prefix list to the FRR templates (#5005)
add the Vlan prefix list to the FRR templates
2020-07-21 19:26:19 -07:00
Stepan Blyshchak
16a37d8c17
[dockers] update mellanox syncd and pmon to buster (#4818)
Upgrade to libsensors5

Updated sonic-sairedis pointer:
    d54bfb4 [SAI] update pointer (#636)
    1885a8c [syncd] Fix notification on shutdown request (#635)
    9e57ba2 Fixing hostif For Genetlink host interfaces (#633)
    449a092 sonic-sairedis: Add support to sonic-sairedis for gearbox phys (#632)

Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2020-07-18 03:46:15 -07:00
joyas-joseph
78945766fc
[docker-iccpd]: Upgrade docker-iccpd to buster (#4984)
Signed-off-by: Joyas Joseph <joyas_joseph@dell.com>
2020-07-18 00:12:59 -07:00
Nazarii Hnydyn
a3f4c31193
[orchagent]: Fix platform string export. (#4993)
Signed-off-by: Nazarii Hnydyn <nazariig@mellanox.com>
2020-07-18 00:05:10 -07:00
joyas-joseph
18bfa6df08
[docker-nat]: upgrade docker-nat to buster (#4943)
move iptables to 1.8.2-4 (version in buster)

Signed-off-by: Joyas Joseph <joyas_joseph@dell.com>
2020-07-15 22:48:09 -07:00
taochengyi
1ca47da40d
[build][arm] Adding a backport source to arm to resolve docker-base-stretch install redis-tools=5:5.0.3-3~bpo9+2 failure issue (#4950) 2020-07-15 12:23:20 -07:00
Tamer Ahmed
5f3c4fac4b
[docker-orchagent] Call sonic-cfggen Once (#4936)
Optimizing number of calls made to sonic-cfggen during service
start up as it adds to total system boot up time.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>

**- Why I did it**
sonic-cfggen call is slow and it adds to system start up time

**- How I did it**
places all required variable into single template and called into sonic-cfggen using this template

**- How to verify it**
***-Test 1***
there is an average saving of .5 to 1 sec between old script and new script
```
root@str-s6000-acs-14:/# time ./orchagent_old.sh
/usr/bin/orchagent -d /var/log/swss -b 8192 -m f4:8e:38:16:bc:8d

real	0m3.546s
user	0m2.365s
sys	0m0.585s

root@str-s6000-acs-14:/# time ./orchagent_new.sh
/usr/bin/orchagent -d /var/log/swss -b 8192 -m f4:8e:38:16:bc:8d

real	0m2.058s
user	0m1.650s
sys	0m0.363s
```
***-Test 2***
Built an image with this change and orchagent is running with intended params:
```
admin@str-s6000-acs-14:~$ ps -ef | grep orchagent
root      2988  1901  1 02:09 pts/0    00:00:02 /usr/bin/orchagent -d /var/log/swss -b 8192 -m f4:8e:38:16:bc:8d
```

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-07-14 15:36:47 -07:00
joyas-joseph
71e93d921c [docker-team]: upgrade docker-teamd to buster (#4914)
Signed-off-by: Joyas Joseph <joyas_joseph@dell.com>
2020-07-12 18:08:52 +00:00
Stephen Sun
0db05e35b0 [watermark] Fix error: BUFFER_POOL_WATERMARK isn't enabled by default (#4882)
* Fix error: watermarkstat -t buffer_pool doesn't work

Signed-off-by: Stephen Sun <stephens@mellanox.com>
2020-07-12 18:08:52 +00:00
Tamer Ahmed
3197c2dfac [mgmt-framework] Call sonic-cfggen Once (#4937)
Optimizing number of calls made to sonic-cfggen during service
start up as it adds to total system boot up time.

***-Test 1***
there is an average saving of 1 to 1.5 sec between old script and new script
```
root@str-s6000-acs-14:/# time /usr/bin/rest-server-old.sh
Generating temporary TLS server certificate ...
2020/07/09 19:03:33 wrote cert.pem
2020/07/09 19:03:33 wrote key.pem
REST_SERVER_ARGS = -ui /rest_ui -logtostderr -cert /tmp/cert.pem -key /tmp/key.pem
/usr/sbin/rest_server -ui /rest_ui -logtostderr -cert /tmp/cert.pem -key /tmp/key.pem

real	0m8.790s
user	0m7.993s
sys	0m0.584s
root@str-s6000-acs-14:/# time /usr/bin/rest-server-new.sh
Generating temporary TLS server certificate ...
2020/07/09 19:03:45 wrote cert.pem
2020/07/09 19:03:45 wrote key.pem
REST_SERVER_ARGS = -ui /rest_ui -logtostderr -cert /tmp/cert.pem -key /tmp/key.pem
/usr/sbin/rest_server -ui /rest_ui -logtostderr -cert /tmp/cert.pem -key /tmp/key.pem

real	0m6.940s
user	0m5.670s
sys	0m0.386s
```
***-Test 2***
Built an image with this change and rest server is running with params as described in test 1 above
```
admin@str-s6000-acs-14:~$ ps -ef | grep rest_server
root      3301  2866  2 02:09 pts/0    00:00:10 /usr/sbin/rest_server -ui /rest_ui -logtostderr -cert /tmp/cert.pem -key /tmp/key.pem

```

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-07-12 18:08:52 +00:00
pra-moh
24f684da68 [docker-ptf] add gnmi python client (#4928)
For telemetry regression test we need gnmi client to be present on ptfdocker. Gnmi-server will be present on SONiC DuT. Further, we can access gnmi_get from ptfdocker inside pytest to verify gnmi server streaming data successfully or not.
2020-07-12 18:08:52 +00:00
Tamer Ahmed
ceace4b605 [telemetry] Fix telemetry vars template path (#4938)
The template is referenced relative to the script path and this could
results in errors in case script is run from root. Add explicit
path to the template file name.
Also, moving telemetry_var template to template dir.
And remove double quotes from around json dict.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-07-12 18:08:52 +00:00
Ying Xie
d499a266c0 [mgmt docker] move pycryptodome installation to the end of the docker building (#4917)
* [mgmt docker] move pycryptodome installation to the end of the docker building

Signed-off-by: Ying Xie <ying.xie@microsoft.com>

* pin down the version to current: 3.9.8

* comment
2020-07-12 18:08:52 +00:00
Akhilesh Samineni
525029e3d8 [NAT]: Update the conntrack entries timeout to Max value after warmboot (#4596)
Signed-off-by: Akhilesh Samineni <akhilesh.samineni@broadcom.com>

All new NAT conntrack entries are added to kernel with max entry timeout of 432000 and setting the same timeout during system warm reboot also
2020-07-12 18:08:52 +00:00
joyas-joseph
7a6fca2f98 [docker-sflow]: upgrade docker-sflow on buster (#4904) 2020-07-12 18:08:52 +00:00
Tamer Ahmed
f4eae5dabd [telemetry] Call sonic-cfggen Once (#4901)
sonic-cfggen call is slow and this is taking place in the SONiC
boot up process. The change uses templates to assemble all required
vars into single template file. With this change, telemetry now calls
once into sonic-cfggen.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-07-12 18:08:52 +00:00
Ying Xie
6f11833ffa Revert "[sonic mgmt docker] lock pycryptodome version to 3.9.7 (#4913)" (#4915)
This reverts commit f427d2eecf.
2020-07-12 18:08:51 +00:00
Ying Xie
caa3323e9d [sonic mgmt docker] lock pycryptodome version to 3.9.7 (#4913)
Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2020-07-12 18:08:51 +00:00
arlakshm
97fa2c087b "[config]: Multi ASIC loopback changes (#4895)
Resubmitting the changes for (#4825) with fixes for sonic-bgpcdgd test failures
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
2020-07-12 18:08:51 +00:00
lguohan
e2e57d32d6 [docker-orchagent]: upgrade docker-orchagent to buster (#4889)
also update submodule

* 01f810f 2020-07-02 | fix compiling issue for gcc8.3 (#1339) [lguohan]
* 9b13120 2020-07-03 | Fix in script to avoid orchagent crash when port down followed by fdb delete (#1340) [rupesh-k]
* 9b01844 2020-07-01 | [qosorch] Update QoS scheduler params for shaping features (#1296) [Michael Li]
* 86b5e99 2020-07-02 | [mirrororch] Port Mirroring implementation (#1314) [rupesh-k]
* c05601c 2020-06-24 | [portsyncd]: add debug message if a port cannot be found in port able (#1328) [lguohan]
* a0b6412 2020-06-23 | COPP_DEL_fix: DEL for one trap group from SONIC is resetting all the trap IDs (#1273) [SinghMinu]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-07-12 18:08:51 +00:00
Guohan Lu
f8da3e4c69 Revert "[config]: Loopback Interface changes for multi ASIC devices (#4825)"
This reverts commit cae65a451c.
2020-07-12 18:08:51 +00:00
arlakshm
002335a3d5 [config]: Loopback Interface changes for multi ASIC devices (#4825)
* Loopback IP changes for multi ASIC devices
multi ASIC will have 2 Loopback Interfaces

- Loopback0 has globally unique IP address, which is advertised by the multi ASIC device to its peers.
This way all the external devices will see this device as a single device.
- Loopback4096 is assigned an IP address which has a scope is within the device. Each ASIC has a different ip address for Loopback4096. This ip address will be used as Router-Id by the bgp instance on multi ASIC devices.

This PR implements this change for multi ASIC devices

Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
2020-07-12 18:08:51 +00:00
pavel-shirshov
2b137fb540 Tests of FRR templates which rendered by sonic-cfggen (#4875)
* Tests of FRR templates which rendered by sonic-cfggen
2020-07-12 18:08:51 +00:00
abdosi
fc6bcff52b [sonic-buildimage] Changes to make network specific sysctl common for both host and docker namespace (#4838)
* [sonic-buildimage] Changes to make network specific sysctl
common for both host and docker namespace (in multi-npu).

This change is triggered with issue found in multi-npu platforms
where in docker namespace
net.ipv6.conf.all.forwarding was 0 (should be 1) because of
which RS/RA message were triggered and link-local router were learnt.

Beside this there were some other sysctl.net.ipv6* params whose value
in docker namespace is not same as host namespace.

So to make we are always in sync in host and docker namespace
created common file that list all sysctl.net.* params and used
both by host and docker namespace. Any change will get applied
to both namespace.

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>

* Address Review Comments and made sure to invoke augtool
only one and do string concatenation of all set commands

* Address Review Comments.
2020-07-12 18:08:51 +00:00
judyjoseph
1af68b3aa6 Support for connecting to DB in namespace via TCP port in multi-asic platform. (#4779)
* Support for connecting to DB in namespace via IP:port ( using docker bridge network ) for applications in multi-asic platform.

* Added the default IP as 127.0.0.1 if the IPaddress derivation from interface fails.
Moved the localhost loopback IP binding logic into the supervisor.j2 file.
2020-07-12 18:08:51 +00:00
abdosi
15440b6e43
Changes to make default route programming correct in multi-npu platforms (#4774)
* Changes to make default route programming
correct in multi-asic platform where frr is not running
in host namespace. Change is to set correct administrative distance.
Also make NAMESPACE* enviroment variable available for all dockers
so that it can be used when needed.

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>

* Fix review comments

* Review comment to check to add default route
only if default route exist and delete is successful.
2020-06-29 11:38:46 -07:00
yozhao101
1c32933c7d
[docker] Correct the lldp-syncd program name in critical_process file. (#4862)
The program name in critical_processes file must match the program name defined in supervisord.conf file.

Signed-off-by: Yong Zhao <yozhao@microsoft.com>
2020-06-28 11:08:30 -07:00
pavel-shirshov
1eb3dfe541
[docker-teamd]: Introducing tlm_teamd: telemetry for teamd (#4824)
**- What I did**
1. Updated submodule sonic-swss to bring tlm_teamd to the buildimage.
2. Updated supervisord for the teamd
3. Updated critical process list (not sure that tlm_teamd is critical for now)

**- How to verify it**
Build an image and run. Check that tlm_teamd is running and STATE_DB has information in the LAG_INTERFACE, and :LAG_MEMBER_INTERFACE
```
admin@sonic:~$ redis-cli -n 6 hgetall 'LAG_TABLE|PortChannel16'
 1) "state"
 2) "ok"
 3) "team_device.ifinfo.dev_addr"
 4) "4c:76:25:f5:48:80"
 5) "setup.kernel_team_mode_name"
 6) "loadbalance"
 7) "team_device.ifinfo.ifindex"
 8) "6"
 9) "runner.fast_rate"
10) "false"
11) "runner.active"
12) "true"
13) "setup.pid"
14) "35"
15) "runner.fallback"
16) "false"
```

```
admin@sonic:~$ redis-cli -n 6 hgetall 'LAG_MEMBER_TABLE|PortChannel16|Ethernet16'
 1) "runner.selected"
 2) "true"
 3) "runner.aggregator.selected"
 4) "true"
 5) "runner.aggregator.id"
 6) "26"
 7) "runner.actor_lacpdu_info.state"
 8) "61"
 9) "runner.state"
10) "current"
11) "runner.actor_lacpdu_info.system"
12) "4c:76:25:f5:48:80"
13) "runner.partner_lacpdu_info.state"
14) "61"
15) "link.up"
16) "true"
17) "ifinfo.dev_addr"
18) "4c:76:25:f5:48:80"
19) "ifinfo.ifindex"
20) "26"
21) "link_watches.list.link_watch_0.up"
22) "true"
23) "runner.actor_lacpdu_info.port"
24) "17"
25) "runner.partner_lacpdu_info.port"
26) "1"
27) "runner.partner_lacpdu_info.system"
28) "52:54:00:ff:34:1b"
```
2020-06-27 01:22:23 -07:00
Qi Luo
6849a0351c
[redis] Install vanilla redis packages for Buster and Stretch; upgrade Buster to 6.0.5 (#4732)
upgrade redis server to 5:6.0.5-1~bpo10+1
2020-06-27 01:17:20 -07:00
yozhao101
4fa81b4f8d
[dockers] Update critical_processes file syntax (#4831)
**- Why I did it**
Initially, the critical_processes file contains either the name of critical process or the name of group.
For example, the critical_processes file in the dhcp_relay container contains a single group name
`isc-dhcp-relay`. When testing the autorestart feature of each container, we need get all the critical
processes and test whether a  container can be restarted correctly if one of its critical processes is
killed. However, it will be difficult to differentiate whether the names in the critical_processes file are
the critical processes or group names. At the same time, changing the syntax in this file will separate the individual process from the groups and also makes it clear to the user.

Right now the critical_processes file contains two different kind of entries. One is "program:xxx" which indicates a critical process. Another is "group:xxx" which indicates a group of critical processes
managed by supervisord using the name "xxx". At the same time, I also updated the logic to
parse the file critical_processes in supervisor-proc-event-listener script.

**- How to verify it**
We can first enable the autorestart feature of a specified container for example `dhcp_relay` by running the comman `sudo config container feature autorestart dhcp_relay enabled` on DUT. Then we can select a critical process from the command `docker top dhcp_relay` and use the command `sudo kill -SIGKILL <pid>` to kill that critical process. Final step is to check whether the container is restarted correctly or not.
2020-06-25 21:18:21 -07:00
Shuba Viswanathan
921d132a32
[sonic-mgmt]: Support for pytest-html to control logs better (#4791)
The current stdout file which also includes the dut logs are very verbose and noisy.

We have manually installed it in the sonic-mgmt docker in our organization and tuned the pytest settings to produce very helpful and concise logs.

pytest-html plugins can be used to post-process the output in various ways based on our different and unique organizational needs.

Hence proposing to add this pkt to the docker file
2020-06-25 17:45:16 -07:00
pavel-shirshov
d592e9b0f8
Tests for bgpcfgd templates (#4841)
* Tests for bgpcfgd templates
2020-06-25 14:54:02 -07:00
lguohan
cebb85b161
[docker-orchagent]: start portsyncd before orchagent (#4845)
when portsyncd starts, it first enumerates all front panel ports
and marks them as old interfaces. Then, for new front panel ports
it checks if their indexes exist in previous sets. If yes, it will
treats them as old interfaces and ignore them.

The reason we have this check is because broadcom SAI only removes
front panel ports after sai switch init.

So, if portsyncd starts after orchagent, new interfaces could be
created before portsyncd and treated as old interface.

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-06-24 22:48:37 -07:00
padmanarayana
5cacc2004c
Add a port index mapper service for sFlow (#4794)
* Add a port index mapper service for sFlow
2020-06-23 22:23:08 -07:00
joyas-joseph
b48d274f69
[docker-dhcp-relay]: convert dhcp-relay docker to buster (#4671)
Upgrade isc-dhcp to 4.4.1-2 (buster version)
Update libevent dependency for dhcpmon to 2.1-6

Signed-off-by: Joyas Joseph <joyas_joseph@dell.com>
2020-06-22 15:34:21 -07:00
pavel-shirshov
0d863c39ac
[bgpcfgd]: make a package for bgpcfgd (#4813) 2020-06-20 21:01:24 -07:00
Tamer Ahmed
211d1e7e2e
[fast-reboot] Back up FDB/ARP/Default routes (#4795)
FDB/ARP/Default routes files are deleted after swssconfig. This
makes debugging/validation of device conversion hard. This PR
saves those files in order to facilitate debugging of device conversion.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-06-20 15:30:53 -07:00
Sachin Holla
3dc7992a6e
[mgmt-framework]: REST server cert configurations (#4799)
REST and telemetry servers were using "DEVICE_METADATA|x509" table for
server certificate configurations. This table has been deprecated now.
Enhanced REST server startup script to read server certificate file
path configurations from REST_SERVER table. Three more attributes -
server_crt, server_key and ca_crt are introduced as described in
https://github.com/Azure/SONiC/pull/550.

For backard compatibility, certificate configurations are read from
old "DEVICE_METADATA|x509" table if they (server_crt, server_key and
ca_crt) are not present in REST_SERVER table.

Fixes bug https://github.com/Azure/sonic-buildimage/issues/4291

Signed-off-by: Sachin Holla <sachin.holla@broadcom.com>
2020-06-19 00:46:03 -07:00
Danny Allen
364511aa36
[mgmt docker] Clean up docker-sonic-mgmt dockerfile (#4759)
- Alphabetize dependencies to prevent duplicates
- Remove unneccesary git clone

Signed-off-by: Danny Allen <daall@microsoft.com>
2020-06-16 11:32:25 -07:00
arlakshm
80298fae1f
[bgp]:Add redistribution connected for ipv6 also for Frontend ASICs (#4767)
* fix redistribution connected for ipv6 also

Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
2020-06-16 06:45:45 -07:00
Arun Saravanan Balachandran
a748daeaed
[docker-sonic-mgmt]: import patch to support 'become' and 'become_user' arguments in pytest-ansible (#4681) 2020-06-12 16:21:10 -07:00
joyas-joseph
1714e621be
[docker-radv]: Convert radv docker to buster (#4727)
* Set radvd version to match buster version(2.17-2)

Signed-off-by: Joyas Joseph <joyas_joseph@dell.com>
2020-06-12 16:10:23 -07:00
Ying Xie
4b39193a13
[sonic-mgmt] upgrade ansible to 2.7.12 (#4751)
Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2020-06-11 23:23:27 -07:00
Joe LeVeque
5d5d5739c2
[dockers] Rename 'docker-snmp-sv2' to 'docker-snmp' (#4699)
The `-sv2` suffix was used to differentiate SNMP Dockers when we transitioned from "SONiCv1" to "SONiCv2", about four years ago. The old Docker materials were removed long ago; there is no need to keep this suffix. Removing it aligns the name with all the other Dockers.

Also edit Monit configuration to detect proper snmp-subagent command line in Buster, and make snmpd command line matching more robust.
2020-06-11 16:04:23 -07:00
Ying Xie
2e93a92bd9
[sonic-mgmt] upgrade paramilo to version 2.7.1 (#4750)
spytest requires higher paramiko version. Fix it to 2.7.1.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2020-06-11 14:04:51 -07:00
Joe LeVeque
9b27efdcc2
[dockers] Rename 'docker-lldp-sv2' to 'docker-lldp' (#4700)
The -sv2 suffix was used to differentiate SNMP Dockers when we transitioned from "SONiCv1" to "SONiCv2", about four years ago. The old Docker materials were removed long ago; there is no need to keep this suffix. Removing it aligns the name with all the other Dockers.
2020-06-09 09:09:56 -07:00
Mykola F
49a93743a4
[enable counters] enable RIF flex counter by default (#4655)
**- Why I did it**
We need RIF counters to be enabled by default. Flex Counter does probe for supported counters. If a platform does not support RIF counters, SAI will return NOT_SUPPORTED and Flex Counter will stop polling the counter.

**- How to verify it**
After fresh install rif counter gropup is enabled by default:

    $ counterpoll show
    Type                  Interval (in ms)    Status
    --------------------  ------------------  --------
    QUEUE_STAT            default (10000)     enable
    PORT_STAT             default (1000)      enable
    RIF_STAT              default (1000)      enable
    QUEUE_WATERMARK_STAT  default (10000)     enable
    PG_WATERMARK_STAT     default (10000)     enable

Signed-off-by: Mykola Faryma <mykolaf@mellanox.com>
2020-06-04 09:52:43 -07:00
taochengyi
ccd08f10dd
[build]: fix mgmt-framework build failure on ARM64 (#4674)
PIP installs grpcio-tools via source code

Co-authored-by: taocy <taocy2@centecnetworks.com>
2020-05-31 03:09:39 -07:00
joyas-joseph
cae67728f5
[docker-database]: Upgrade docker-database to buster (#4665)
Signed-off-by: Joyas Joseph <joyas_joseph@dell.com>
2020-05-29 03:29:49 -07:00
Guohan Lu
767152f09b [docker-sonic-mgmt]: fix pip version to 20.1.1
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-05-26 10:29:13 +00:00
taocy
00e7e14927 Install the libraries that j2cli relies on from source, for arm arch. 2020-05-25 13:15:19 +00:00
Guohan Lu
ddd6368e64 [docker-database]: do not generate pidfile for rsyslogd
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-05-22 11:01:28 -07:00
Guohan Lu
6cd8d5bb68 [docker-snmp-sv2]: use service dependency in supervisord to start services
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-05-22 11:01:28 -07:00
Guohan Lu
2e42a4ba0f [docker-dhcp-relay]: use service dependency in supervisord to start services 2020-05-22 11:01:28 -07:00
Guohan Lu
448d1cdd49 [docker-teamd]: use service dependency in supervisord to start services 2020-05-22 11:01:28 -07:00
Guohan Lu
38499ab0f8 [docker-mgmt-framework]: use service dependency in supervisord to start services
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-05-22 11:01:28 -07:00
Guohan Lu
1cf417ed1b [docker-telemetry]: use service dependency in supervisord to start services
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-05-22 11:01:28 -07:00
Guohan Lu
15c6282eac [docker-restapi]: use service dependency in supervisord to start services 2020-05-22 11:01:28 -07:00
Guohan Lu
8da46d26c3 [docker-pmon]: use service dependency in supervisord to start services 2020-05-22 11:01:28 -07:00
Guohan Lu
1636be4b68 [docker-sflow]: use service dependency in supervisord to start services 2020-05-22 11:01:28 -07:00
Guohan Lu
b8da6c3588 [docker-orchagent]: use service dependency in supervisord to start services 2020-05-22 11:01:28 -07:00
Guohan Lu
7ea6d9dc8f [docker-radvd]: use service dependency in supervisord to start services 2020-05-22 11:01:28 -07:00
Guohan Lu
1f7602b275 [docker-lldp-sv2]: use service dependency in supervisord to start services 2020-05-22 11:01:28 -07:00
Guohan Lu
267b0b7aa8 [docker-iccpd]: use service dependency in supervisord to start services
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-05-22 11:01:28 -07:00
Guohan Lu
0c005fdb1c [docker-nat]: use service dependency in supervisord to start servicesx
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-05-22 11:01:28 -07:00
Guohan Lu
2c7e55ae98 [docker-frr]: use service dependency in supervisord to start services
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-05-22 11:01:28 -07:00
Guohan Lu
c915c3cbd6 [docker-base]: remove dummy password for supervisord control
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-05-22 11:01:28 -07:00
Guohan Lu
75fe8888d2 [docker-base]: add supervisord-dependent-startup plugin for supervisord 2020-05-22 11:01:28 -07:00
Tyler Li
2398992d52
[iccpd] build iccpd deb by auto tools (#4540)
* [iccpd] build iccpd deb by auto tools
2020-05-21 09:12:51 -07:00
judyjoseph
4ba2f608c1
Adding new BGP peer groups PEER_V4_INT and PEER_V6_INT. (#4620)
* Adding new BGP peer groups PEER_V4_INT and PEER_V6_INT. The internal BGP sessions
will be added to this peer group while the external BGP sessions will be added
to the exising PEER_V4 and PEER_V6 peer group.

* Check for "ASIC" keyword in the hostname to identify the internal neighbors.
2020-05-20 20:52:11 -07:00
joyas-joseph
9084ac50fb
[docker-telemetry]: upgrade telemetry docker to buster (#4515)
Signed-off-by: Joyas Joseph <joyas_joseph@dell.com>
2020-05-19 03:12:50 -07:00
Kelly Chen
e0f8aa71d6
[lldpmgrd] only log error in is_port_up() after port init done (#4606)
lldpmgrd listens for changes to the PORT table in the CONFIG_DB and APP_DB in order to handle alias/description config change. It checks if port is up or down by looking into the oper-status for in APP_DB PORT TABLE. If it cannot find it in the App DB, it will log error. 

During initializing, it is possible that there is a port change in CONFIG_DB and but the not ready in APP_DB. 

The change here is to only log error in is_port_up() after port init done.
2020-05-19 02:52:48 -07:00
Qi Luo
af95d57fa9
[docker-base-buster]: Install python 3.7 into docker-base-buster (#4603)
* Install python3 in docker-base-buster, so all derived image will benefit

Signed-off-by: Qi Luo <qiluo-msft@users.noreply.github.com>
2020-05-17 04:26:50 -07:00
joyas-joseph
9814da1e21
[docker-lldp-sv2]: upgrade docker-lldp-sv2 to buster (#4598)
Signed-off-by: Joyas Joseph <joyas_joseph@dell.com>
2020-05-15 14:51:53 -07:00
joyas-joseph
9dea816532
Convert docker-snmp-sv2 to buster (#4529)
* Fix libsnmp-base compilation failure
* Convert docker-snmp-sv2 to buster
* Define install_python3_wheels
* Address review comments
* Address review comments
* Advance snmpagent submodule
* Bump net-snmp to the Buster version
* Revert "Fix libsnmp-base compilation failure"
* use azure storage url
2020-05-14 10:23:37 -07:00
arlakshm
d3c28a45d9
Change to enable redistribute connected on Frontend asics instead of backend asics (#4588)
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
2020-05-13 08:56:01 -07:00
abdosi
a96f9ecee9
Changes for LLDP docker to support multi-npu platforms (#4530)
* Changes for LLDP for Multi NPU Platoforms:-
a) Enable LLDP for Host namespace for Management Port
b) Make sure Management IP is avaliable in per asic namespace
   needed for LLDP Chassis configuration
c) Make sure chassis mac-address is correct in per asic namespace
d) Do not run lldp on eth0 of per asic namespace and avoid chassis
   configuration for same
e) Use Linux hostname instead from Device Metadata for lldp chassis
   configuration since in multi-npu platforms device metadata hostname
   will be differnt

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>

* Address Review Comment with following changes:
a) Use Device Metadata hostname even in per namespace conatiner.
   updated minigraph parsing for same to have hostname as system
   hostname and add new key for asic name

b) Minigraph changes to have MGMT_INTERFACE Key in per asic/namespace
   config also as needed for LLDP for setting chassis management IP.

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>

* Address Review Comments
2020-05-11 11:05:44 -07:00
judyjoseph
acf465b43b
Multi DB with namespace support, Introducing the database_global.json… (#4477)
* Multi DB with namespace support, Introducing the database_global.json file
for supporting accessing DB's in other namespaces for service running in
linux host

* Updates based on comments

* Adding the j2 templates for database_config and database_global files.

* Updating to retrieve the redis DIR's to be mounted from database_global.json file.

* Additional check to see if asic.conf file exists before sourcing it.

* Updates based on PR comments discussion.

* Review comments update

* Updates to the argument "-n" for namespace used in both context of parsing minigraph and multi DB access.

* Update with the attribute "persistence_for_warm_boot" that was added to database_config.json file earlier.

* Removing the database_config.json file to avioid confusion in future.
We use the database_config.json.j2 file to generate database_config.json files dynamically.

* Update the comments for sudo usage in docker_image_ctrl.j2

* Update with the new logic in PING PONG tests using sonic-db-cli. With this we wait till the
PONG response is received when redis server is up.

* Similar changes in swss and syncd scripts for the PING tests with sonic-db-cli

* Updated with a missing , in the database_config.json.j2 file, Do pip install of j2cli in docker-base-buster.
2020-05-08 21:24:05 -07:00
abdosi
fc28af7ce9
[bgpcfgd]: Fix for BGP peer not coming up even after config BGP startup all (#4547)
Issue was key not correct to look into self.peer. It need to be tuple of
(vrf,nbr). Updated for both add/del
2020-05-06 19:30:31 -07:00
Dong Zhang
340cf826a6
[MultiDB] use sonic-db-cli PING and fix wrong multiDB API in NAT (#4541) 2020-05-06 15:41:28 -07:00
arlakshm
2db87669c2
[bgp]: align the bgp templates with new minigraph for multi NPU platforms (#4488)
- change the references to 'type' field to 'sub_role'
- change the references to 'InternalFrontend' and 'InternalBackend' to 'FrontEnd' and 'BackEnd' respectively
- add a statement to reflect route-reflector for backend asics
- add a change to set "next-hop-self force" configuration for internal BGP session in multi asic platform.

Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
2020-05-06 14:58:02 -07:00
Wei Bai
46ef6defa7
[docker-sonic-mgmt] Add IxNetwork python client (#4533)
* Add IxNetwork python client to sonic mgmt docker
2020-05-05 18:40:20 -07:00
Sujin Kang
cbc75fe4c8
[pmon]: Fix the continous syseepromd autorestart issue on 201911 (#4478)
- Remove syseepromd from the critical process of pmon docker
- Fix supervisor autorestart configuration of syseepromd
2020-04-30 15:51:34 -07:00
judyjoseph
93ade1d96b
[Multi-ASIC] To pass the asic instance ID to orchagent, Advance the swss, swss-common submodules. (#4465)
* Multi-ASIC platforms will have the ID field in the DEVICE_METADATA, which will be pulled and
will be used when starting the orchagent process with the new option [-i INST_ID]
This is currently added only for Broadcom ASIC based platforms

* Making the asic instance ID passing global across asics/platforms.
Also changed the config DB id field to asic_id

* Minor updates

* Advance sonic-swss submodule

* Advance swss_common submodule as well due to dependencies
2020-04-28 20:32:22 -07:00
joyas-joseph
fc55329b22
[mgmt-framework]: convert mgmt-framework to use buster docker (#4480)
This change adds support to build dockers using buster as base.
- Define docker-base-buster using docker-base-stretch as starting point
- Define docker-config-engine-buster using docker-config-engine-stretch as starting point.
- sonic-mgmt-framework docker is updated to build using buster as base

Signed-off-by: Joyas Joseph <joyas_joseph@dell.com>
2020-04-28 15:57:13 -07:00
Prince Sunny
4adf243260
Fix docker restapi start issue, undefined error (#4475) 2020-04-24 12:43:56 -07:00
pavel-shirshov
057ced0391
[bgpcfgd]: Split one bgp mega-template to chunks. (#4143)
The one big bgp configuration template was splitted into chunks.

Currently we have three types of bgp neighbor peers:

general bgp peers. They are represented by CONFIG_DB::BGP_NEIGHBOR table entries
dynamic bgp peers. They are represented by CONFIG_DB::BGP_PEER_RANGE table entries
monitors bgp peers. They are represented by CONFIG_DB::BGP_MONITORS table entries
This PR introduces three templates for each peer type:

bgp policies: represent policieas that will be applied to the bgp peer-group (ip prefix-lists, route-maps, etc)
bgp peer-group: represent bgp peer group which has common configuration for the bgp peer type and uses bgp routing policy from the previous item
bgp peer-group instance: represent bgp configuration, which will be used to instatiate a bgp peer-group for the bgp peer-type. Usually this one is simple, consist of the referral to the bgp peer-group, bgp peer description and bgp peer ip address.
This PR redefined constant.yml file. Now this file has a setting for to use or don't use bgp_neighbor metadata. This file has more parameters for now, which are not used. They will be used in the next iteration of bgpcfgd.

Currently all tests have been disabled. I'm going to create next PR with the tests right after this PR is merged.

I'm going to introduce better bgpcfgd in a short time. It will include support of dynamic changes for the templates.

FIX:: #4231
2020-04-23 09:42:22 -07:00
Junchao-Mellanox
c730f3e207
[Mellanox] thermal control enhancement for dynamic minimum fan speed and PSU fan speed policy (#4403) 2020-04-21 08:09:53 -07:00
Kebo Liu
860cb265ac
[PMON] Extend pmon daemon start control to lm-sensors and fancontrol (#4447) 2020-04-21 08:00:48 -07:00
taochengyi
744d33d502
[devices]: Centec x86 goldengate platforms and devices update (#4248)
* Centec x86 goldengate update (#8)
2020-04-17 00:38:48 -07:00
Sumukha Tumkur Vani
12400a447c
[docker-restapi]: Fix authentication in restapi (#4383)
- enabling http/https endpoint and mounting the certificates directory
- updating the sonic-restapi submodule
2020-04-15 02:08:45 -07:00
Kebo Liu
cfa112ace8
[Mellanox] Extend mellanox platform API to report SFP error event (#4365)
* extend mellanox platform API to report SFP error event
* remove unnecessary loop code
* install enum34 to pmon to support using Enum
2020-04-14 10:20:06 -07:00
Danny Allen
2872d80231
[docker-ptf] Add support for spytest to ptf container (#4410)
- Install apt and pip dependencies
- Define traffic generator service

Signed-off-by: Danny Allen <daall@microsoft.com>
2020-04-13 16:56:19 -07:00
Danny Allen
4f8f1d45e4
[docker-sonic-mgmt] Merge spytest dependencies into mgmt docker (#4411)
Signed-off-by: Danny Allen <daall@microsoft.com>
2020-04-12 13:35:28 -07:00
Dong Zhang
de5a04ad18
[MultiDB] : add persistence field for each redis instance (#4254)
- add "persistence" field for each redis instance in database_config.json
- we will use this information to decide if saving redis instance data while warm/fast reboot
   - before multiDB changes, SONiC uses "redis-cli save " to save all the data into rdb file on default instance on port 6379
   - with multiDB changes, we plan to implement "sonic-db-cli save" to save all data to corresponding rdb files on all listed redis instances which has "persistence" field set "yes"
2020-04-07 21:01:39 -07:00
wangshengjun
711445c98e
[lldp]add the excutable mode for 'lldpcli' (#4385)
Signed-off-by: wangshengjun <wangshengjun@asterfusion.com>
2020-04-07 12:49:34 -07:00
lguohan
60b16495cc
[docker-base-stretch]: move common packages into docker-base-stretch (#4371)
libpython2.7, libdaemon0, libdbus-1-3, libjansson4 are common
across different containers. move them into docker-base-stretch

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-04-05 13:29:34 -07:00
xumia
78b93a8897
[docker-sonic-mgmt]: Fix virtual environment bug (#4370) 2020-04-04 18:17:55 -07:00
shine4chen
524cf9e56a
MCLAG feature for SONIC (#2514)
* MCLAG feature for sonic

* MCLAG feature for sonic

* remove binary file

* remove unused dockerfile

update docker-iccpd to stretch-based container

Signed-off-by: shine.chen <shine.chen@nephosinc.com>

* minor fix for isolation port setting

Signed-off-by: shine.chen <shine.chen@nephosinc.com>

* iccpd docker would start on demand

Signed-off-by: shine.chen <shine.chen@nephosinc.com>

* Add x attribute on mclagdctl file

Signed-off-by: shine.chen <shine.chen@nephosinc.com>

* add warm-reboot support for MCLAG

Signed-off-by: shine.chen <shine.chen@nephosinc.com>

* merge to master branch and reformat iccpd file

Signed-off-by: shine.chen <shine.chen@nephosinc.com>

* fix some bugs and make peer-link configuration optional

Signed-off-by: shine.chen <shine.chen@nephosinc.com>

* refactor code per Brcm review

Signed-off-by: shine.chen <shine.chen@nephosinc.com>

* correct a typo

Signed-off-by: shine.chen <shine.chen@nephosinc.com>

* * optimize iccpd arp/mac sync process
* refine code according to brcm opinoin
* unify function return value

Signed-off-by: shine.chen <shine.chen@nephosinc.com>

* * optimize warm-reboot process
* estabish iccpd connection with configurated src-ip

Signed-off-by: shine.chen <shine.chen@nephosinc.com>

* fix a typo

Signed-off-by: shine.chen <shine.chen@nephosinc.com>

* optimize some code
* add some debug info
* optimize bridge mac setting
* fix vlan mac sync issue on standby node

Signed-off-by: shine.chen <shine.chen@mediatek.com>

* optimize some code

Signed-off-by: shine.chen <shine.chen@mediatek.com>

* fix some bugs for warm-reboot

Signed-off-by: shine.chen <shine.chen@mediatek.com>

* refine log level

Signed-off-by: shine.chen <shine.chen@mediatek.com>

* refine iccpd syslog & skip arp packet whose src ip is local ip

Signed-off-by: shine.chen <shine.chen@mediatek.com>

* remove iccpd dependency with teamd

Signed-off-by: shine.chen <shine.chen@mediatek.com>

* print log level when dump mclag status

Signed-off-by: shine.chen <shine.chen@mediatek.com>

* revise per community review

Signed-off-by: shine.chen <shine.chen@mediatek.com>

Co-authored-by: shine.chen <shine.chen@nephosinc.com>
Co-authored-by: shine.chen <shine.chen@mediatek.com>
2020-04-04 15:24:06 -07:00
Ze Gan
ea38d061c7
Fixbug: EVPN issue in FRR template (#4260)
* Fixbug: EVPN issue in FRR template
2020-04-02 18:17:08 -07:00
SuvarnaMeenakshi
4b8067e913
Multi-ASIC implementation (#3888)
Changes made to support multi-asic platform. Added multi-instance support for swss, syncd, database, bgp, teamd and lldp.
2020-03-31 10:06:19 -07:00
xumia
2c97321545
[sonic-mgmt]: Support virtual environment for ansible 2.0.0.2 (#4325)
. env-201811/bin/activate

The ansible 2.0.0.2 will be used.
2020-03-30 17:50:57 -07:00
lguohan
a0d213cc37
[telemetry]: move default certs location from device metadata to telemetry (#4307)
maintains backward compatibility to search original x509 location
when telemetry table does not have certs

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-03-24 00:04:36 -07:00
RayWang910012
f90bf8fe62
[monit]: monit_telemetry which will have error when telemetry is in secure mode (#4286)
When telemetry is in secure mode ,the monitor will have error log of the match string "--insecure". So I modify to be compatiable with insecure mode and secure mode.

Co-authored-by: Ubuntu <ubuntu@ip-10-5-1-21.ap-south-1.compute.internal>
2020-03-21 18:48:17 -07:00
Sujin Kang
01f3f9286f
[fancontrol] Restart process upon unexpected exit, not entire pmon container (#4101)
* fancontrol restart

* Cleanup the default setting for exitcodes

* Remove the unnecessary stopwaitsecs default settin
2020-03-19 17:24:22 -07:00
lguohan
9b48c22449
[docker-ptf]: install ntp related package (#4275)
setup ntp server to do the test

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-03-18 11:01:36 -07:00
Junchao-Mellanox
be549db395
Add thermal control support for SONiC (#3949) 2020-03-09 10:41:10 -07:00
Joe LeVeque
64a6989d02
[Services] Restart NAT service upon unexpected critical process exit. (#4208) 2020-03-05 15:27:21 -08:00
lguohan
09c0563692
[docker-ptf]: install tacacs+ server to test tacacs (#4224)
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-03-05 01:37:23 -08:00
Danny Allen
f906a758cd
[spytest] Create new sonic-mgmt docker for running spytests (#4212)
Signed-off-by: Danny Allen <daall@microsoft.com>
2020-03-04 11:47:31 -08:00
yozhao101
23ff55a709
[Services] Restart BGP service upon unexpected critical process exit. (#4207) 2020-03-03 16:50:32 -08:00
Stepan Blyshchak
41ae7a2102
[snmp] remove hostname change as it share uts namespace with host (#4206)
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2020-02-28 15:57:03 -08:00
Prince Sunny
fbc24b4279
[orchagent] Use mac address from config_db instead of from eth0 (#4166)
* Use mac address from config_db instead of eth0
2020-02-20 19:16:14 -08:00
Partha Dutta
2cfdb28367
[mgmt-framework]: Removing explicit libyang plugin path settings (#4144)
Removing explicit libyang plugin path settings. Libyang debian package installs the extensions in standard location.

Libyang debian package installs the extensions in standard location as per build time configuration (install prefix is set in patch/libyang.patch). If plugin directory is not set, libyang explicitly searches the plugins in LYEXT_PLUGINS_DIR which is populated during build. The plugins are installed in LYEXT_PLUGINS_DIR itself through debian package installation.

$ grep -nr LYEXT_PLUGINS_DIR src/
src/plugin_config.h.in:18:#define LYEXT_PLUGINS_DIR "@EXTENSIONS_PLUGINS_DIR_MACRO@" /**< directory with YANG extension plugins */
src/plugins.c:420: pluginsdir = LYEXT_PLUGINS_DIR;
2020-02-13 13:47:16 -08:00
yozhao101
729f343f77
[Services] Restart database service upon unexpected critical process exit. (#4138)
* [database] Implement the auto-restart feature for database container.

Signed-off-by: Yong Zhao <yozhao@microsoft.com>

* [database] Remove the duplicate dependency in service files. Since we
already have updategraph ---> config_setup ---> database, we do not need
explicitly add database.service in all other container service files.

Signed-off-by: Yong Zhao <yozhao@microsoft.com>

* [event listener] Reorganize the line 73 in event listener script.

Signed-off-by: Yong Zhao <yozhao@microsoft.com>

* [database] update the file sflow.service.j2 to remove the duplicate
dependency.

Signed-off-by: Yong Zhao <yozhao@microsoft.com>

* [event listener] Add comments in event listener.

Signed-off-by: Yong Zhao <yozhao@microsoft.com>

* [event listener] Update the comments in line 56.

Signed-off-by: Yong Zhao <yozhao@microsoft.com>

* [event listener] Add parentheses for if statement in line 76 in event listener.

Signed-off-by: Yong Zhao <yozhao@microsoft.com>
2020-02-11 14:03:02 -08:00
zhenggen-xu
ae491a9c47 [docker-lldp] Fix lldpcli issue when description has special characters (#4133)
Before the fix:
lldpcli configure ports Ethernet96 lldp portidsubtype local 'Eth1/1' description 50G|sonic1|Eth1/3/2
bash: sonic1: command not found
bash: Eth1/3/2: No such file or directory

After fix:
lldpcli configure ports Ethernet96 lldp portidsubtype local 'Eth1/1' description '50G|sonic1|Eth1/3/2'
run successfully.

Signed-off-by: Zhenggen Xu <zxu@linkedin.com>
2020-02-09 20:34:55 -08:00
yozhao101
91e5fb5602
[Service] Enable/disable container auto-restart based on configuration. (#4073) 2020-02-07 12:34:07 -08:00
Xin Wang
3c12b32529
[docker-sonic-mgmt]: Add the snmp tool to the sonic-mgmt docker (#4110)
The snmp tool is required for interacting with certain type
of PDU hosts in platform PSU/power related testing. This
change is to have the snmp tool pre-built in the sonic-mgmt
docker image.

Signed-off-by: Xin Wang <xinw@mellanox.com>
2020-02-05 01:08:27 -08:00
Guohan Lu
fb535bb2ba [docker-sonic-mgmt]: set wheel version to 0.33.6 to fix sonic-mgmt build break
looks like version 0.42 has build issues

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2020-01-31 12:38:46 +00:00
judyjoseph
c95d8e4d47
[teamd]: increase startsecs to 5 seconds for teamsyncd (#4083)
Updating the startsecs=5sec for teamsyncd to make the time for which the process needs to stay up before declaring the startup successfull.
2020-01-30 00:46:55 -08:00
Kiran Kumar Kella
97165a0d69
Changes in sonic-buildimage to support the NAT feature (#3494)
* Changes in sonic-buildimage for the NAT feature
- Docker for NAT
- installing the required tools iptables and conntrack for nat

Signed-off-by: kiran.kella@broadcom.com

* Add redis-tools dependencies in the docker nat compilation

* Addressed review comments

* add natsyncd to warm-boot finalizer list

* addressed review comments

* using swsscommon.DBConnector instead of swsssdk.SonicV2Connector

* Enable NAT application in docker-sonic-vs
2020-01-29 17:40:43 -08:00
kannankvs
7cb63008d7
mvrf_avoid_snmp_yml_config: made changes to pass SNMP config from con… (#4057)
* mvrf_avoid_snmp_yml_config: made changes to pass SNMP config from confiDB to snmpd.conf without using snmp.yml
* added a missing if condition
2020-01-28 17:41:21 -08:00