Commit Graph

1408 Commits

Author SHA1 Message Date
Qi Luo
42e78a19c1
[sonic-swss-common] Update submodule (#5969)
Update the sonic-swss-common submodule. The following are the commits in the submodule.
```
95f9e11 2020-11-19 | [pyext] allow to catch exceptions raised in python (#415) [Stepan Blyshchak]
5a718f9 2020-11-18 | [swig] translate C++ `del` to python `delete` (#416) [Qi Luo]
40b255b 2020-11-12 | Fix: SWIG dict.get() should have optional default value parameter (#413) [Qi Luo]
91e484d 2020-11-07 | Reduce notice logging (#412) [Qi Luo]
f5945ae 2020-11-05 | Mux Cable schema definitions for interaction between linkmanager and xcvrd (#411) [vdahiya12]
602f9c2 2020-11-05 | [lua] load lua script on demand (#409) [Dong Zhang]
d88412b 2020-11-04 | Rename hdel to del when using multiple keys as param (#410) [Kamil Cudnik]
e0c229a 2020-11-04 | CHASSIS_STATE_DB on control-card for chassis state (#395) [mprabhu-nokia]
a4e3ac8 2020-11-04 | Chassisd config table to store admin state (#388) [mprabhu-nokia]
```
2020-11-19 10:57:07 -08:00
pavel-shirshov
a92732fe5d
[bgpcfgd]: Fixes for BBR (#5956)
* Add explicit default state into the constants.yml
* Enable/disable only peer-groups, available in the config
* Retrieve updates from frr before using configuration

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-11-19 00:07:58 -08:00
Praveen Chaudhary
2fe79c2c61
[yang_models]: yang models for device_metadata, device_neighbor, flex… (#4442)
Changes:
1.) yang models for device_metadata, device_neighbor, flex_counters, crm, versions.
2.) Test cases for above yang models.
Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com:
2020-11-18 01:59:55 -08:00
Joe LeVeque
6c4092d0a2
[sonic-platform-common] Update submodule (#5953)
Commits include:

* src/sonic-platform-common e4883e5...5db8c84 (1):
  > [sonic-platform-common] Fix name error and import error  (#143)
2020-11-18 01:37:05 -08:00
Joe LeVeque
ced11615a4
[sonic-py-common] Add 'universal_newlines=True' arg to all Popen calls (#5919)
The behavior of `subprocess.Popen()` changed in Python 3 such that stdin, stdout and stderr are treated as bytes by default. Adding the `universal_newlines=True` argument changes this behavior to return strings, matching the behavior of Python 2. The change is backward-compatible with Python 2, as well.
2020-11-16 17:07:37 -08:00
Lawrence Lee
261a81d379
[minigraph.py]: Parse mux cable table (#5876)
Parse device IPv6 loopback addresses when available
Add MUX_CABLE table to config DB:

'MUX_CABLE': {
	'Ethernet4': {
    	'state': 'auto',
        'server_ipv4': '10.10.10.1/32',
        'server_ipv6': 'fe80::0001/128'
	}
}
2020-11-16 13:14:34 -08:00
Joe LeVeque
e526d8895f
[sonic-platform-common][sonic-platform-daemons] Update submodules (#5924)
This should also fix the vsimage check build.

Commits include:

* src/sonic-platform-common a659219...e4883e5 (7):
  > [PsuBase] Reference inherited member constant (#142)
  > Thermalctld APIs for recording min and max temp (#131)
  > Common power consumption and supply APIs for modular chassis (#136)
  > [sonic-platform-base] Introduce APIs for modular chassis support (#124)
  > Remove shebangs from non-executable Python files (#140)
  > [sonic_eeprom] Make compatible with Python 2 and 3 (#127)
  > Add unit test infrastructure (#139)

* src/sonic-platform-daemons 600d043...be21d91 (9):
  > [psud] Import of mock libraries determined by environment variable (#117)
  > [chassisd][thermalctld] Set testing env var at top of file, not in setup_function() (#114)
  > [chassisd][thermalctld] Import os module, now needed for env var checking (#113)
  > [chassisd][thermalctld] Import mock libraries based on environment variable (#112)
  > [xcvrd] Fix the console-script xcvrd startup  (#111)
  > [Thermalctld] Update thermal info to CHASSIS_STATE_DB (#101)
  > PSUd changes to compute power-budget for Modular chassis (#104)
  > Introduce chassisd to monitor status of cards on chassis (#97)
  > [xcvrd] support for integrating Credo Y cable Ports initialization and status updates with xcvrd.  (#105)
2020-11-15 02:37:38 -08:00
pavel-shirshov
af654944bd
[bgp]: Update TSA functionality (#5906)
Fixed TSA bugs:
1. TSA didn't advertise Loopback ipv6 address
2. TSA and TSB changed BGP dynamic and BGP monitors sessions

**- How to verify it**
Build an image and run on your DUT.
```
admin@str-s6100-acs-1:~$ TSA
System Mode: Normal -> Maintenance
admin@str-s6100-acs-1:~$ vtysh -c 'show bgp ipv4 neighbors 10.0.0.1 advertised-routes'
BGP table version is 6, local router ID is 10.1.0.32, vrf id 0
Default local pref 100, local AS 64601
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 10.1.0.32/32     0.0.0.0                  0         32768 i

Total number of prefixes 1
admin@str-s6100-acs-1:~$ vtysh -c 'show bgp ipv6 neighbors fc00::a advertised-routes'
BGP table version is 6, local router ID is 10.1.0.32, vrf id 0
Default local pref 100, local AS 64601
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> fc00:1::/64      ::                       0         32768 i

Total number of prefixes 1
admin@str-s6100-acs-1:~$ TSB
System Mode: Maintenance -> Normal
```

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-11-13 17:54:20 -08:00
Lawrence Lee
bf50562670
[minigraph.py]: Parse peer switch info from minigraph (#5869)
* Create new `PEER_SWITCH` table in config DB with info from minigraph
* Add `subtype` field to `DEVICE_METADATA` table and set value to `DualToR` if device is in a dual ToR setup
2020-11-13 15:31:51 -08:00
Joe LeVeque
56fa3cf494
[sonic-bgpcfgd] Call Python 3 version of sonic-cfggen for testing (#5847)
sonic-bgpcfgd build fails in the absence of Python 2, as it attempts to explicitly call sonic-cfggen using `/usr/bin/python2.7`. Also, it attempts to call sonic-cfggen using a local, relative path. Since the sonic-config-engine package is not installed, neither are its dependencies.

Now, we configure the Python 3 sonic-config-engine as a dependency of sonic-bgpcfgd, which ensures the Python 3 sonic-config-engine package and its dependencies are installed before sonic-bgpcfgd is built/tested.
2020-11-13 11:39:16 -08:00
Ying Xie
48822be2aa
[dbsycnd] update submodule head of dbsyncd (#5916)
[lldp daemon] improve lldp daemon event when neighbor capability is not sent (#30)

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2020-11-13 03:12:12 -08:00
Joe LeVeque
d0c7bd8158
[sonic-py-swsssdk][sonic-snmpagent][sonic-utilities] Update submodules (#5849)
Submodule updates include the following commits. Also adding sonic-swsscommon build dependencies where necessary.

* src/sonic-py-swsssdk 1ea30d2...9d9f0c6 (1):
  > [SonicV2Connector] make decode_responses=True as default, so python2 application need no code change when migrated to python3(#93)

* src/sonic-snmpagent 6e4a796...57e54d9 (3):
  > Interact with Redis by str instead of bytes, migrate to SonicV2Connector with `decode_responses=True` (#171)
  > Add a test case for LLDP_LOC_CHASSIS.lldp_loc_man_addr has only one IPv6 address (#167)
  > [LLDP]: Update init_db to load global database config (#166)

* src/sonic-utilities acfa824...b693cf6 (11):
  > Remove SKU create pytest output directory before execution of the script (#1226)
  > [show][techsupport][multi-ASIC] Add support to collect tech support on multi ASIC platform (#1193)
  > [show] Fix `show ip bgp sum` (#1194)
  > [sonic_sku_create] Move tests from sonic-utilities-tests/ folder to tests/ folder (#1222)
  > Replace swsssdk.SonicV2Connector with swsscommon.SonicV2Connector (SWIG wrapper of C++ implementation) in production code (#1217)
  > Copy missing values from INIT_CFG to config_db as part of db_migration task (#1209)
  > [connect][clear] Support --devicename option for connect/clear line commands (#1214)
  > [consutil][show] Remove root need from show line command (#1218)
  > [Mellanox] SKU creator Tool (#1163)
  > SONiC installer - fix string formatting during image type check (#1197)
  > [show/fgnhg.py] Fix whitespace issues (#1211)
2020-11-12 17:58:56 -08:00
Guohan Lu
a98cf22162 [submodule]: downgrade sonic-platform-daemons
remove commit

850d0c6 2020-11-09 | [xcvrd] support for integrating Credo Y cable Ports initialization and status updates with xcvrd.  (#105) (HEAD, origin/master, origin/HEAD) [vdahiya12]

fix issue #5907

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-11-11 17:14:35 -08:00
Kebo Liu
4342cb0cea
[submodule]: update sonic-linux-kernel (#5894)
Pick up new kernel patches:
-  a28b399db609e9cab6748df28cf1482537f31560  [Mellanox] Add patches for i2c API 
- 18236531cb74d34fa5945d1c766a1a35b658569f [Mellanox] Add kernel patches with hw-mgmt V.7.0010.1300 
- cb4712f97dbfb4b7a5f867034317e1aa3ac75b45 Improve `patch/kconfig-exclusions`
2020-11-11 16:55:09 -08:00
carl-nokia
dbb6373e85
[Marvell] Utilize eeprom for system mac and remove dependency on get_hwsku (#5882)
Utilize eeprom for system mac and remove dependency on get_hwsku

Co-authored-by: Carl Keene <keene@nokia.com>
2020-11-11 15:11:00 -08:00
pavel-shirshov
83d090f42c
[teamd]: Remove extensive debug output for libteamdctl (#5870) 2020-11-10 19:12:51 -08:00
pavel-shirshov
cdc6879c3e
[bgpcfgd]: Implement BBR template test (#5850)
Add the test to check that all templates, which use CONFIG_DB__BBR are configured in constants.yml
2020-11-10 18:44:20 -08:00
Lawrence Lee
d0f16c0d79
Make backend device checking more robust (#5730)
Treat devices that are ToRRouters (ToRRouters and BackEndToRRouters) the same when rendering templates
 Except for BackEndToRRouters belonging to a storage cluster, since these devices have extra sub-interfaces created
Treat devices that are LeafRouters (LeafRouters and BackEndLeafRouters) the same when rendering templates

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2020-11-10 15:06:35 -08:00
Prince Sunny
1eaaf64ed2
Set preference for forced mgmt routes (#5844)
When forced mgmt routes are present, the issue fixed as part of #5754 is not complete. 
Added a preference(priority) field to forced mgmt route ip rules
2020-11-10 14:20:13 -08:00
Joe LeVeque
aee389e402
[sonic-config-engine] openconfig_acl.py: Make compliant with Python 3 (#5873)
Python 3 is more strict with `__slots__`. As per the [documentation](https://docs.python.org/3/reference/datamodel.html#notes-on-using-slots):

> \_\_slots\_\_ are implemented at the class level by creating descriptors (Implementing Descriptors) for each variable name. As a result, class attributes cannot be used to set default values for instance variables defined by \_\_slots\_\_; otherwise, the class attribute would overwrite the descriptor assignment.

This was apparently missed when making sonic-config-engine compliant with Python 3, and errors like the following would be seen:

```
tests/acl_loader_test.py:10: in <module>
    from acl_loader.main import *
acl_loader/main.py:8: in <module>
    import openconfig_acl
/usr/local/lib/python3.7/dist-packages/openconfig_acl.py:24: in <module>
    class yc_state_openconfig_acl__acl_state(PybindBase):
E   ValueError: '_pybind_generated_by' in __slots__ conflicts with class variable
```
2020-11-10 10:26:16 -08:00
madhanmellanox
b177ed9efd
Accomadating lower case ACL rules for Control Plane ACLs (#5748)
To make Control plane ACLs handle case insensitive ACL rules. Currently, it handles only upper case ACL rules.

Co-authored-by: Madhan Babu <madhan@arc-build-server.mtr.labs.mlnx>
2020-11-10 10:16:32 -08:00
judyjoseph
f2b22b5cd1
[multi-ASIC] util changes with the BGP_INTERNAL_NEIGHBOR table (#5874)
Reintroduce #5760, along with the fix needed in the template file for python3 compatibility.
2020-11-10 09:34:56 -08:00
Joe LeVeque
4669c3241b
Remove shebangs from various setup.py files (#5867)
setup.py files are not executable, and thus have no need for shebangs.

Also remove unnecessary comments
2020-11-10 00:07:37 -08:00
vdahiya12
e95483471e
[submodule]: update sonic-platform-daemons (#5868)
850d0c6 [xcvrd] support for integrating Credo Y cable Ports initialization and status updates with xcvrd.  (#105)
600d043 [xcvrd] Add enum dependence back; Install 'enum34' conditionally based on Python version (#107)
af79326 [xcvrd] Remove log errors in single ASIC platforms with init Global config (#108)

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
2020-11-10 00:06:48 -08:00
judyjoseph
b5121dcfd4
Revert "[multi-ASIC] util changes with the BGP_INTERNAL_NEIGHBOR table. (#5760)" (#5871)
This reverts commit c972052594.
2020-11-09 14:30:13 -08:00
judyjoseph
c972052594
[multi-ASIC] util changes with the BGP_INTERNAL_NEIGHBOR table. (#5760)
- Why I did it
Update the routine is_bgp_session_internal() by checking the BGP_INTERNAL_NEIGHBOR table.
Additionally to address the review comment #5520 (comment)
Add timer settings as will in the internal session templates and keep it minimal as these sessions which will always be up.
Updates to the internal tests data + add all of it to template tests.

- How I did it
Updated the APIs and the template files.

- How to verify it
Verified the internal BGP sessions are displayed correctly with show commands with this API is_bgp_session_internal()
2020-11-09 11:10:10 -08:00
Volodymyr Samotiy
3fa724d216
[submodule] Update sonic-swss submodule (#5825)
62e6c5a [bitmap_vnet] Remove BMTOR implementation (#1494)
582beae [FPMSYNCD] EVPN Type5 prefix handling in FPMSYNCD (#1326)
d9ca6e2 [vstest] Resolve intermittent timing failure in test_fgnhg.py (#1495)
a89b8ed [sub intf] Inherit parent port mtu changes (#1479)

Signed-off-by: Volodymyr Samotiy <volodymyrs@nvidia.com>
2020-11-08 23:57:50 -08:00
Longxiang Lyu
92704e8a93
[bgpmon] Fix dict_keys unsubscriptable error (#5859)
Simply looping through `dict_key` object instead of index-subscribing to
fix this.

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>
2020-11-08 13:42:38 -08:00
Lawrence Lee
23b0e07d48
[minigraph.py]: Parse IP-in-IP tunnels from minigraph (#5742)
Take tunnel info from `<TunnelInterface>` tag in minigraph, and create tables in config_DB:

```
"TUNNEL": {
    "MUX_TUNNEL_0": {
        "tunnel_type": "IPINIP",
        "dst_ip": "26.1.1.10",
        "dscp_mode": "uniform",
        "encap_ecn_mode": "standard",
        "ecn_mode": "copy_from_outer",
        "ttl_mode": "pipe"
    }
}
```

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2020-11-07 13:18:40 -08:00
Praveen Chaudhary
6156cb2805
[sonic-yang-mgmt] Build PY3 & PY2 packages (#5559)
Moving sonic-yang-mgmt to PY3 to support move of sonic-utilities to PY3.

Signed-off-by: Praveen Chaudhary<pchaudhary@linkedin.com>
2020-11-07 13:03:41 -08:00
Joe LeVeque
04d0e8ab00
[hostcfgd] Convert to Python 3; Add to sonic-host-services package (#5713)
To consolidate host services and install via packages instead of file-by-file, also as part of migrating all of SONiC to Python 3, as Python 2 is no longer supported.
2020-11-07 12:48:19 -08:00
Petro Bratash
32a832a8ac
[lldp]: Add verification IPv4 address on LLDP conf Jinja2 Template (#5699)
Fix #5812

LLDP conf Jinja2 Template does not verify IPv4 address and can use IPv6 version. This issue does not effect control LLDP daemon. Issue can be reproduced via `test_snmp_lldp` test. LLDP conf Jinja2 Template selects first item from the list of mgmt interfaces.

TESTBED_1 LLDP conf

```
# cat /etc/lldpd.conf 
configure ports eth0 lldp portidsubtype local eth0
configure system ip management pattern FC00:3::32
configure system hostname dut-1
```
TESTBED_2  LLDP conf

```
# cat /etc/lldpd.conf
configure ports eth0 lldp portidsubtype local eth0
configure system ip management pattern 10.22.24.61
configure system hostname dut-2
```
TESTBED_1  MGMT_INTERFACE

```
$ redis-cli -n 4 keys "*" | grep MGMT_INTERFACE
MGMT_INTERFACE|eth0|10.22.24.53/23
MGMT_INTERFACE|eth0|FC00:3::32/64
```
TESTBED_2  MGMT_INTERFACE

```
$ redis-cli -n 4 keys "*" | grep MGMT_INTERFACE
MGMT_INTERFACE|eth0|FC00:3::32/64
MGMT_INTERFACE|eth0|10.22.24.61/23

```

Signed-off-by: Petro Bratash <petrox.bratash@intel.com>
2020-11-07 10:30:41 -08:00
pavel-shirshov
cdcd20a7b5
[BGP]: Convert ip address to network address for the LOCAL_VLAN filter (#5832)
* [BGP]: Convert ip address to network address for the LOCAL_VLAN prefix filter
2020-11-06 17:47:08 -08:00
dflynn-Nokia
174b741dfd
[submodule]: update sonic-linux-kernel (#5829)
pick up follow commits:

  > hwmon-lm75: backport support for PCT2075 thermal sensor (#165)
  > [arista] Reassign prefetch memory per platform (#163)
  > Add .gitignore file (#157)
  > Enable PCA9541 I2C mux module (#160)
2020-11-06 12:37:41 -08:00
Danny Allen
cea364aa77
[utilities] Update sonic-utilities submodule (#5814)
Add a switch to route_check to control whether write log to syslog (#1215)
[consutil] Store console port state in STATE_DB (#1208)
[sfputil] Display 'N/A' for non-SFP ports (#1078)
Show FG_NHG CLI Commands Added (#1056)
[show] remove 'device2interface_dict' in show interface neighbor expected (#1188)
Add license file, remove third-party licenses file (#1192)
[fwutil]: Set min log priority to INFO. (#1191)
[CLI][show][platform] Added ASIC count in the output. (#1185)
fdbshow and nbrshow use SonicV2Connector with decode_responses=True, and remove all the decode() (#1187)
Remove stdeb.cfg; no longer used now that we build as wheel (#1182) 
[counterpoll] Disable Counter Poll When Entering Fast Reboot (#1174) 
Fixes the issue with show interface counters and for pfc and queue counters. (#1180)
[config/show] Add CLI support for proxy arp (#1168)
[consutil] Add brief option to show line command (#1176)
Modify fast-reboot script to use BGP service script to stop bgp service (#1177)
[config/console] Support update console configuration related commands (#1166)
[consutil] Fix issue where the show line command crash if no ttyUSB exists (#1173)
[watermarkstat] Add unit tests for watermarkstat show commands (#1157)
Fix exception for ipaddress in python2 (#1164) 
[celestica] consutil to support customize tty device name (#1155)

Signed-off-by: Danny Allen <daall@microsoft.com>
2020-11-06 00:58:49 -08:00
Joe LeVeque
2c9454e86a
[sonic-device-data] Convert Python files to Python 3 (#5816)
- Convert config_checker, media_checker and platform_json_checker scripts to Python 3
- Reorganize imports per PEP8 standard
- Two blank lines precede functions per PEP8 standard
2020-11-05 11:17:08 -08:00
pavel-shirshov
13f8e9ce5e
[bgpcfgd]: Convert bgpcfgd and bgpmon to python3 (#5746)
* Convert bgpcfgd to python3

Convert bgpmon to python3
Fix some issues in bgpmon

* Add python3-swsscommon as depends

* Install dependencies

* reorder deps

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-11-05 10:01:43 -08:00
Praveen Chaudhary
1f9132db60
[sonic-yang]: Yang model update fields (#5577)
Changes:
-- ACL stage allow lower and upper.
-- ACL include services leaf-list.
-- PORT include pfc_asym leaf.
-- PORT fec alloe none as per code.
-- 3 Tests for above changes.

Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com
2020-11-05 00:25:33 -08:00
Qi Luo
4ed2ff805f
[swss-common]: Advance submodule (#5799)
ec96868 2020-11-03 | Fix: treat DBConnector timeout=0 as infinite timeout (#408) [Qi Luo]
b4b8334 2020-11-03 | Add lua script for redis multi keys api hmset and del (#406) [Kamil Cudnik]
2020-11-04 05:49:52 -08:00
Qi Luo
215ce13890
[swss-common]: Advance submodule (#5780)
9b0e955 2020-10-30 | [schema]: Add MACsec support (#403) [Ze Gan]
f24dc97 2020-10-28 | Implementation of System ports initialization, Interface & Neighbor Synchronization... (#380) [minionatwork]
91e0885 2020-10-27 | Mux cable schema definitions (#398) [Prince Sunny]
d0cedea 2020-10-27 | Change log level (#402) [Prince Sunny]
64b3cfe 2020-10-24 | SonicV2Connector supports host and decode_responses in constructor parameters (#401) [Qi Luo]
f8b0065 2020-10-23 | Implement FieldValueMap update method (#400) [Qi Luo]
2020-11-03 14:28:42 -08:00
Joe LeVeque
84d3a26000
[sonic-py-swsssdk] Update submodule (#5757)
Commits included:

* src/sonic-py-swsssdk 748c404...1ea30d2 (1):
  > Fix bug: ConfigDBConnector.get_table does not work in python3 (#92)
2020-11-02 20:31:14 -08:00
lguohan
9d7355287f
[submodule]: update sairedis (#5772)
* b458e6f 2020-11-02 | [syncd] Disable use bulk api by default (#688) (HEAD, origin/master, origin/HEAD) [Kamil Cudnik]
* d978789 2020-11-01 | [syncd] Set modify redis flag in RedisSelectableChannel (#689) [Kamil Cudnik]
* 5df11f5 2020-11-01 | [syncd] Lower bulk missing api message level from error to info (#687) [Kamil Cudnik]
* dc73a1d 2020-10-30 | [saiplayer] Fix log messages (#686) [Kamil Cudnik]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-11-02 16:59:47 -08:00
lguohan
98d370bcef
[submodule]: swss/sairedis module update (#5765)
swss:

* d7643f2 2020-11-01 | [tlm_teamd]: Make the destionation for std::transform() to use std::back_inserter() for allocating new space for the copied objects (#1490) (HEAD, origin/master, origin/HEAD) [pavel-shirshov]
* 7fa7cd6 2020-10-31 | [vstest]: stablize fgnhg test (#1491) [lguohan]
* 9b0696e 2020-10-29 | Create vnet tunnel map only if it doesn't exist (#1482) [Prince Sunny]
* 0481e99 2020-10-29 | [acl] Update CRM to include LAG bindings for ACL tables (#1487) [Danny Allen]

sairedis

* 5df11f5 2020-11-01 | [syncd] Lower bulk missing api message level from error to info (#687) (HEAD, origin/master, origin/HEAD) [Kamil Cudnik]
* dc73a1d 2020-10-30 | [saiplayer] Fix log messages (#686) [Kamil Cudnik]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-11-01 20:08:55 -08:00
Blueve
698b5544c9
[openssh] Introduce custom openssh-server package for supporting reverse console SSH (#5717)
* Build and install openssh from source
* Copy openssh deb package to dest folder
* Update make rule
* Update sonic debian extension
* Append empty line before EOF
* Update openssh patch
* Add openssh-server to base image dependency
* Fix indent type
* Fix comments
* Use commit id instead of tag id and add comment

Signed-off-by: Jing Kan jika@microsoft.com
2020-11-02 10:31:15 +08:00
lguohan
c8a00eda95
[mgmt ip]: mvrf ip rule priority change to 32765 (#5754)
Fix Azure/SONiC#551

When eth0 IP address is configured, an ip rule is getting added for eth0 IP address through the interfaces.j2 template. 

This eth0 ip rule creates an issue when VRF (data VRF or management VRF) is also created in the system.
When any VRF (data VRF or management VRF) is created, a new rule is getting added automatically by kernel as "1000: from all lookup [l3mdev-table]".
This l3mdev IP rule is never getting deleted even if VRF is deleted.

Once if this l3mdev IP rule is added, if user configures IP address for the eth0 interface, interfaces.j2 adds an eth0 IP rule as "1000:from 100.104.47.74 lookup default ". Priority 1000 is automatically chosen by kernel and hence this rule gets higher priority than the already existing rule "1001:from all lookup local ".

This results in an issue "ping from console to eth0 IP does not work once if VRF is created" as explained in Issue 551.
More details and possible solutions are explained as comments in the Issue551.

This PR is to resolve the issue by always fixing the low priority 32765 for the IP rule that is created for the eth0 IP address.
Tested with various combinations of VRF creation, deletion and IP address configuration along with ping from console to eth0 IP address.

Co-authored-by: Kannan KVS <kannan_kvs@dell.com>
2020-10-31 20:45:59 -07:00
gechiang
908787d2a2
Added new method get_back_end_interface_set() to speed up back-end in… (#5731)
Added new MultiASIC util method "get_back_end_interface_set()" to speed up back-end interface check by allowing caller to cache the back-end intf into a set. This way the caller can use this set for all subsequent back-end interface check requests  instead of each time need to read from redis DB which become a scaling issue for cases such as checking for thousands of nexthop routes for filtering purpose.
2020-10-31 18:06:06 -07:00
abdosi
dddf96933c
[monit] Adding patch to enhance syslog error message generation for monit alert action when status is failed. (#5720)
Why/How I did:

Make sure first error syslog is triggered based on FAULT TOLERANCE condition.

Added support of repeat clause with alert action. This is used as trigger
for generation of periodic syslog error messages if error is persistent

Updated the monit conf files with repeat every x cycles for the alert action
2020-10-31 17:29:49 -07:00
Shi Su
279943c11f
[sonic-swss] Update submodule (#5745)
Update the sonic-swss submodule. The following are the commits in the submodule.

[neighorch] Remove pending DEL operation after SET operation for the same key
2265f548386929b7827d1079efd453128f1ec1f9

[NAT]: Update nat entries to use nat_type to support DNAT Pool changes.
8696e939f973895ead4731ad499a72f257a3b510

[intfsorch] Init proxy_arp variable while adding router interface.
1da3c773762fa637a5ea47017715361bede50a4a
2020-10-31 02:26:19 -07:00
Joe LeVeque
e111204206
[caclmgrd] Convert to Python 3; Add to sonic-host-services package (#5739)
To consolidate host services and install via packages instead of file-by-file, also as part of migrating all of SONiC to Python 3, as Python 2 is no longer supported, convert caclmgrd to Python 3 and add to sonic-host-services package
2020-10-29 16:29:12 -07:00
Volodymyr Boiko
fd7e2a12bc
[submodule-update][sonic-platform-daemon] Update submodule (#5741)
95b1696 [xcvrd] Remove dependence on enum; Add 'sonic-py-common' as dependencies in setup.py (#106)
61ed24e [thermalctld] Print exception using repr(e) to get more information (#103)
8507085 [psud] Fix psud logging (#98)

Signed-off-by: Volodymyr Boyko <volodymyrx.boiko@intel.com>
2020-10-29 09:46:55 -07:00
Shi Su
5ee5c13f32
Enable synchronous mode by default and add in minigraph parser (#5735) 2020-10-29 09:15:12 -07:00
Dong Zhang
d95e1969c8
[swsssdk] update submodule for adding new MultiDB API (#5737) 2020-10-28 21:47:01 -07:00
judyjoseph
6088bd59de
[multi-ASIC] BGP internal neighbor table support (#5520)
* Initial commit for BGP internal neighbor table support.
  > Add new template named "internal" for the internal BGP sessions
  > Add a new table in database "BGP_INTERNAL_NEIGHBOR"
  > The internal BGP sessions will be stored in this new table "BGP_INTERNAL_NEIGHBOR"

* Changes in template generation tests with the introduction of internal neighbor template files.
2020-10-28 16:41:27 -07:00
Shi Su
09d5a62fad
[sonic-sairedis] Update submodule (#5728)
Update the sonic-sairedis submodule. The following are the commits in the submodule.

[syncd_init_common.sh] Use template file to retrieve vars (#683)
0bf336a3e895167357d5d2e5a988471e115522e8

[syncd/FlexCounter]:Fix last remove bug (#679)
4d21a264d5956501bf69ad3a89ea2ebccd369654
2020-10-27 20:42:57 -07:00
Lawrence Lee
a639021af2
[minigraph.py]: Parse VLAN MAC address from minigraph when present (#5726) 2020-10-27 17:20:55 -07:00
Danny Allen
e0b09d0998
[swss] Update swss submodule (#5719)
Signed-off-by: Danny Allen <daall@microsoft.com>
2020-10-27 14:38:43 -07:00
vdahiya12
dfe005545a
[sonic-platform-common] update submodule (#5721)
a659219 [SONIC_SFP] adding abstract methods for reading and writing the eeprom address space within platform api (#126)
848f4a6 Add third-party licenses (#138)
c2ecd9a Add license file (#137)
403747a [sonic-platform-common] Add new platform API for SONiC Physical MIB Extension feature (#134)
19b8545 [sonic_y_cable] fix the unpacking (#135)

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
2020-10-27 09:36:26 -07:00
Joe LeVeque
9e34003136
[sonic-config-engine] Clean up dependencies, pin versions; install Python 3 package in Buster container (#5656)
To clean up the image build procedure, and let setuptools/pip[3] implicitly install Python dependencies. Also use ipaddress package instead of ipaddr.
2020-10-26 13:48:50 -07:00
shlomibitton
e66d49a57c
[LLDP] Fix for LLDP advertisements being sent with wrong information. (#5493)
* Fix for LLDP advertisments being sent with wrong information.
Since lldpd is starting before lldpmgr, some advertisment packets might sent with default value, mac address as Port ID.
This fix hold the packets from being sent by the lldpd until all interfaces are well configured by the lldpmgrd.

Signed-off-by: Shlomi Bitton <shlomibi@nvidia.com>

* Fix comments

* Fix unit-test output caused a failure during build

* Add 'run_cmd' function and use it

* Resume lldpd even if port init timeout reached
2020-10-26 19:38:09 +02:00
judyjoseph
c14b41dc30
[submodule]: Advance sonic-swss-common and sonic-sairedis module (#5703)
Advance sonic-swss-common submodule by adding the following  commits 

3ec30ef Deprecate RedisClient and remove unused header file (#399)
165a679 Schema update for BGP internal neighbor table (#389)
262e330 Fix SonicV2Connector interfaces (#396)

Advance sonic-sairedis submodule by adding the following  commits 

bc3e044  [Sai]: Change Sai::set log to level INFO (#680)
b16bc8b Clean code: remove unused header file (#678)
40439b4 [syncd] Remove depreacated dependency on swss::RedisClient (#681)
1b6fc2e [syncd] Add supports of bulk api in syncd (#656)
a9f69c1 [syncd] Add to handle FDB MOVE notification (#670)
c7ef5e9 [gbsyncd] exit with zero when platform has no gearbox (#676)
57228fd [gbsyncd]: add missing python dependency (#675)
02a57a6 [vs] Add CRM SAI attributes to virtual switch interface (#673)
609445a fix boot type for fast boot (#674)
1325cdf Add support for saiplayer bulk API and add performance timers (#666)
1d84b90 Add ZeroMQ communication channel between sairedis and syncd (#659)
017056a Support System ports config (#657)
0f3668f Enable fabric counter for syncd's FlexCounter (#669)
2020-10-26 09:19:16 -07:00
Lawrence Lee
c4f9bec562
[minigraph.py]: Add support for parsing mux cable (#5676)
Find LogicalLinks in minigraph and parse the port information. A new field called `mux_cable` is added to each port's entry in the Port table in config DB:

```
PORT|Ethernet0: {
	"alias": "Ethernet4/1"
	...
	"mux_cable": "true"
}
```

If a mux cable is present on a port, the value for `mux_cable` will be `"true"`. If no mux cable is present, the attribute will either be omitted (default behavior) or set to `"false"`.
2020-10-26 08:52:20 -07:00
Nazarii Hnydyn
ba6f012cc6
[sonic-py-common]: Fix syslog implicit min priority override (#5707)
Current implementation of logger class is based on standard python syslog library.  
Thus, logger class can be instantiated in different places and share the same context across the entire process.  
This means that reducing log severity level will affect other modules which use logging facility.

**- Why I did it**
* To fix syslog implicit min priority override

**- How I did it**
* Added per instance log severity check

**- How to verify it**
1. Run code snippet
```
from sonic_py_common import logger

log1 = logger.Logger(log_identifier='myApp1')
log1.set_min_log_priority_debug()
log1.log_error("=> this is error")
log1.log_warning("=> this is warning")
log1.log_notice("=> this is notice")
log1.log_info("=> this is info")
log1.log_debug("=> this is debug")

log2 = logger.Logger(
    log_identifier='myApp2',
    log_facility=logger.Logger.LOG_FACILITY_DAEMON,
    log_option=(logger.Logger.LOG_OPTION_NDELAY | logger.Logger.LOG_OPTION_PID)
)
log2.log_error("=> this is error")
log2.log_warning("=> this is warning")
log2.log_notice("=> this is notice")
log2.log_info("=> this is info")
log2.log_debug("=> this is debug")
```
2. Sample output:
```
Oct 23 15:08:30.447301 sonic ERR myApp1: => this is error
Oct 23 15:08:30.447908 sonic WARNING myApp1: => this is warning
Oct 23 15:08:30.448305 sonic NOTICE myApp1: => this is notice
Oct 23 15:08:30.448696 sonic INFO myApp1: => this is info
Oct 23 15:08:30.449063 sonic DEBUG myApp1: => this is debug

Oct 23 15:08:30.449442 sonic ERR myApp2[19178]: => this is error
Oct 23 15:08:30.449819 sonic WARNING myApp2[19178]: => this is warning
Oct 23 15:08:30.450183 sonic NOTICE myApp2[19178]: => this is notice
```

Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>
2020-10-24 12:36:22 -07:00
Joe LeVeque
3a4435eb53
Add sonic-host-services and sonic-host-services-data packages (#5694)
**- Why I did it**

Install all host services and their data files in package format rather than file-by-file

**- How I did it**

- Create sonic-host-services Python wheel package, currently including procdockerstatsd
  - Also add the framework for unit tests by adding one simple procdockerstatsd test case
- Create sonic-host-services-data Debian package which is responsible for installing the related systemd unit files to control the services in the Python wheel. This package will also be responsible for installing any Jinja2 templates and other data files needed by the host services.
2020-10-23 09:52:29 -07:00
judyjoseph
ace7f24cba
[docker-teamd]: Add teamd as a depedent service to swss (#5628)
**- Why I did it**
On teamd docker restart, the swss and syncd needs to be restarted as there are dependent resources present.

**- How I did it**
Add the teamd as a dependent service for swss
Updated the docker-wait script to handle service and dependent services separately.
Handle the case of warm-restart for the dependent service   

**- How to verify it**

Verified the following scenario's with the following testbed 
VM1 ----------------------------[DUT 6100] -----------------------VM2,  ping traffic continuous between VMs

1. Stop teamd docker alone  
      >  swss, syncd dockers seen going away
      >  The LAG reference count error messages seen for a while till swss docker stops.
      >  Dockers back up.

2. Enable WR mode for teamd. Stop teamd docker alone  
      >  swss, syncd dockers not removed.
      >  The LAG reference count error messages not seen
      >  Repeated stop teamd docker test - same result, no effect on swss/syncd.

3. Stop swss docker. 
      >  swss, teamd, syncd goes off - dockers comes back correctly, interfaces up

4. Enable WR mode for swss . Stop swss docker 
      >  swss goes off not affecting syncd/teamd dockers.

5. Config reload 
      > no reference counter error seen, dockers comes back correctly, with interfaces up

6. Warm reboot, observations below
	 > swss docker goes off first 
	 > teamd + syncd goes off to the end of WR process.
 	 > dockers comes back up fine.
	 > ping traffic between VM's was NOT HIT

7. Fast reboot, observations below
	 > teamd goes off first ( **confirmed swss don't exit here** )
	 > swss goes off next 
	 > syncd goes away at the end of the FR process
	 > dockers comes back up fine.
	 > there is a traffic HIT as per fast-reboot

8. Verified in multi-asic platform, the tests above other than WR/FB scenarios
2020-10-23 00:41:16 -07:00
Joe LeVeque
73825e4d4d
[system-health] Update .gitignore file (#5688)
Touch up .gitignore file to properly ignore all files generated when building a Python wheel package
2020-10-22 11:58:27 -07:00
pavel-shirshov
c94f93f046
[bgpcfgd]: Dynamic BBR support (#5626)
**- Why I did it**
To introduce dynamic support of BBR functionality into bgpcfgd.
BBR is adding  `neighbor PEER_GROUP allowas-in 1' for all BGP peer-groups which points to T0
Now we can add and remove this configuration based on CONFIG_DB entry 

**- How I did it**
I introduced a new CONFIG_DB entry:
 - table name: "BGP_BBR"
 - key value: "all". Currently only "all" is supported, which means that all peer-groups which points to T0s will be updated
 - data value: a dictionary: {"status": "status_value"}, where status_value could be either "enabled" or "disabled"

Initially, when bgpcfgd starts, it reads initial BBR status values from the [constants.yml](https://github.com/Azure/sonic-buildimage/pull/5626/files#diff-e6f2fe13a6c276dc2f3b27a5bef79886f9c103194be4fcb28ce57375edf2c23cR34). Then you can control BBR status by changing "BGP_BBR" table in the CONFIG_DB (see examples below).

bgpcfgd knows what peer-groups to change fron [constants.yml](https://github.com/Azure/sonic-buildimage/pull/5626/files#diff-e6f2fe13a6c276dc2f3b27a5bef79886f9c103194be4fcb28ce57375edf2c23cR39). The dictionary contains peer-group names as keys, and a list of address-families as values. So when bgpcfgd got a request to change the BBR state, it changes the state only for peer-groups listed in the constants.yml dictionary (and only for address families from the peer-group value).

**- How to verify it**
Initially, when we start SONiC FRR has BBR enabled for PEER_V4 and PEER_V6:
```
admin@str-s6100-acs-1:~$ vtysh -c 'show run' | egrep 'PEER_V.? allowas'
  neighbor PEER_V4 allowas-in 1
  neighbor PEER_V6 allowas-in 1
```

Then we apply following configuration to the db:
```
admin@str-s6100-acs-1:~$ cat disable.json                
{
        "BGP_BBR": {
            "all": {
                "status": "disabled"
            }
        }
}


admin@str-s6100-acs-1:~$ sonic-cfggen -j disable.json -w 
```
The log output are:
```
Oct 14 18:40:22.450322 str-s6100-acs-1 DEBUG bgp#bgpcfgd: Received message : '('all', 'SET', (('status', 'disabled'),))'
Oct 14 18:40:22.450620 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-f', '/tmp/tmpmWTiuq']'.
Oct 14 18:40:22.681084 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V4 soft in']'.
Oct 14 18:40:22.904626 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V6 soft in']'.
```

Check FRR configuraiton and see that no allowas parameters are there:
```
admin@str-s6100-acs-1:~$ vtysh -c 'show run' | egrep 'PEER_V.? allowas' 
admin@str-s6100-acs-1:~$
```

Then we apply enabling configuration back:
```
admin@str-s6100-acs-1:~$ cat enable.json 
{
        "BGP_BBR": {
            "all": {
                "status": "enabled"
            }
        }
}

admin@str-s6100-acs-1:~$ sonic-cfggen -j enable.json -w 
```
The log output:
```
Oct 14 18:40:41.074720 str-s6100-acs-1 DEBUG bgp#bgpcfgd: Received message : '('all', 'SET', (('status', 'enabled'),))'
Oct 14 18:40:41.074720 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-f', '/tmp/tmpDD6SKv']'.
Oct 14 18:40:41.587257 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V4 soft in']'.
Oct 14 18:40:42.042967 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V6 soft in']'.
```


Check FRR configuraiton and see that the BBR configuration is back:
```
admin@str-s6100-acs-1:~$ vtysh -c 'show run' | egrep 'PEER_V.? allowas'
  neighbor PEER_V4 allowas-in 1
  neighbor PEER_V6 allowas-in 1
```

*** The test coverage ***
Below is the test coverage
```
---------- coverage: platform linux2, python 2.7.12-final-0 ----------
Name                             Stmts   Miss  Cover
----------------------------------------------------
bgpcfgd/__init__.py                  0      0   100%
bgpcfgd/__main__.py                  3      3     0%
bgpcfgd/config.py                   78     41    47%
bgpcfgd/directory.py                63     34    46%
bgpcfgd/log.py                      15      3    80%
bgpcfgd/main.py                     51     51     0%
bgpcfgd/manager.py                  41     23    44%
bgpcfgd/managers_allow_list.py     385     21    95%
bgpcfgd/managers_bbr.py             76      0   100%
bgpcfgd/managers_bgp.py            193    193     0%
bgpcfgd/managers_db.py               9      9     0%
bgpcfgd/managers_intf.py            33     33     0%
bgpcfgd/managers_setsrc.py          45     45     0%
bgpcfgd/runner.py                   39     39     0%
bgpcfgd/template.py                 64     11    83%
bgpcfgd/utils.py                    32     24    25%
bgpcfgd/vars.py                      1      0   100%
----------------------------------------------------
TOTAL                             1128    530    53%
```

**- Which release branch to backport (provide reason below if selected)**

- [ ] 201811
- [x] 201911
- [x] 202006
2020-10-22 11:04:21 -07:00
abdosi
ef8a48a728
Everflow and Dataacl were getting binded to Backplane port-channels. (#5691)
Issue was because we were relying on port_alias_asic_map dictionary
but that dictionary can't be used as alias name format has changed.

Fix the port alias mapping as what is needed.

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2020-10-22 10:29:46 -07:00
Prince Sunny
63d40865ab
[Submodule] sonic-restapi (#5679)
37a7c61 - 2020-10-16 : Add to local route DB if ifname present in route add API (#51) [Sumukha Tumkur Vani]
2b4b6d7 - 2020-10-01 : Fix typo (#50) [Sumukha Tumkur Vani]
f0a527b - 2020-09-24 : Use RESTAPI_DB, cache reset_status (#49) [Prince Sunny]
2020-10-22 08:28:02 -07:00
pavel-shirshov
d19d1dd569
[bgpcfgd]: Change prefix-list generation for "Allow prefix" feature (#5639)
**- Why I did it**
I was asked to change "Allow list" prefix-list generation rule.
Previously we generated the rules using following method:
``` 
For each {prefix}/{masklen} we would generate the prefix-rule
permit {prefix}/{masklen} ge {masklen}+1
Example:
Prefix 1.2.3.4/24 would have following prefix-list entry generated
permit 1.2.3.4/24 ge 23
```
But we discovered the old rule doesn't work for all cases we have.

So we introduced the new rule:
```
For ipv4 entry,  
For mask  < 32 , we will add ‘le 32’ to cover all  prefix masks to be sent by T0  
For mask =32 , we will not add any ‘le mask’ 
For ipv6 entry, we will add le 128 to cover all the prefix mask to be sent by T0  
For mask < 128 , we will add ‘le 128’ to cover all prefix masks to be sent by T0 
For mask = 128 , we will not add any ‘le mask’ 
```    

**- How I did it**
I change prefix-list entry generation function. Also I introduced a test for the changed function.

**- How to verify it**
1. Build an image and put it on your dut.

2. Create a file test_schema.conf with the test configuration
```
{
    "BGP_ALLOWED_PREFIXES": {
        "DEPLOYMENT_ID|0|1010:1010": {
            "prefixes_v4": [
                "10.20.0.0/16",
                "10.50.1.0/29"
            ],
            "prefixes_v6": [
                "fc01:10::/64",
                "fc02:20::/64"
            ]
        },
        "DEPLOYMENT_ID|0": {
            "prefixes_v4": [
                "10.20.0.0/16",
                "10.50.1.0/29"
            ],
            "prefixes_v6": [
                "fc01:10::/64",
                "fc02:20::/64"
            ]
        }
    }
}
```

3. Apply the configuration by command 
```
sonic-cfggen -j test_schema.conf --write-to-db
```

4. Check that your bgp configuration has following prefix-list entries:
```
admin@str-s6100-acs-1:~$ show runningconfiguration bgp | grep PL_ALLOW
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V4 seq 10 deny 0.0.0.0/0 le 17
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V4 seq 20 permit 127.0.0.1/32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V4 seq 30 permit 10.20.0.0/16 le 32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V4 seq 40 permit 10.50.1.0/29 le 32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V4 seq 10 deny 0.0.0.0/0 le 17
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V4 seq 20 permit 127.0.0.1/32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V4 seq 30 permit 10.20.0.0/16 le 32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V4 seq 40 permit 10.50.1.0/29 le 32
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V6 seq 10 deny ::/0 le 59
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V6 seq 20 deny ::/0 ge 65
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V6 seq 30 permit fc01:10::/64 le 128
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V6 seq 40 permit fc02:20::/64 le 128
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V6 seq 10 deny ::/0 le 59
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V6 seq 20 deny ::/0 ge 65
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V6 seq 30 permit fc01:10::/64 le 128
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V6 seq 40 permit fc02:20::/64 le 128

``` 

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-10-20 00:38:09 -07:00
Danny Allen
cdb82b9fc2
[swss] Update swss and swss-common submodule pointers (#5630)
[swss]
[acl] Replace IP_PROTOCOL with NEXT_HEADER for IPv6 ACL tables (#1458)
[acl] Refactor port OID retrieval into aclorch (#1462)
Fix issue #5157 by identifying the dependency among objects and avoiding releasing an object still being referenced (#1440)
[mock tests] Update MockDBConnector to match new swsscommon interface (#1465)

[swss-common]
netlink: Setting nl_socket buffer size to 3M from 2M (#391)
Added support in Swig file to cast Selectable object to Subscriber Table object (#394)
[warm reboot] Warm Reboot Support for EVPN VXLAN (#350)
Implement DBInterface/SonicV2Connector in C++ (#387)
Fix memory leak if a RedisCommand object were to be reused (#392)

Signed-off-by: Danny Allen <daall@microsoft.com>
2020-10-16 14:28:55 -07:00
Stepan Blyshchak
8df0e2b6ce
[sonic-cfggen] store jinja2 cache in log level db. (#5646)
This PR makes two changes:
    - Store Jinja2 cache in LOGLEVEL DB instead of STATE DB
    - Store bytecode cache encoded in base64

Tested with the following command: "redis-dump -d 3 -k JINJA2_CACHE"

Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
2020-10-16 12:00:57 -07:00
Joe LeVeque
d8363a99dc
[sonic-platform-common] Update submodule (#5625)
Includes the following commits:

* src/sonic-platform-common 111dcf7...b316f8d (5):
  > [sonic_platform_base] fix an indentation in fan_drawer_base.py (#133)
  > [SONIC_Y_CABLE] Adding new Y cable Package for accessing  Y cable QSFP's eeprom and configuring Y cable's mux through eeprom (#128)
  > Merge pull request #132 from vdahiya12/adding_changes_for_base_api
  > [ChassisBase][ModuleBase] Remove redundant 'get_serial_number()' method (#130)
  > Merge pull request #125 from vdahiya12/changes_for_mux_driver
2020-10-16 11:17:13 -07:00
Qi Luo
a5adb577f3
[sonic-snmpagent]: Advance submodule (#5614)
6e4a796 2020-10-12 | Fix: correctly handle that lldp_loc_man_addr contains only IPv6 address without IPv4 address (#164) [Qi Luo]
0f772ce 2020-09-28 | [ciscoSwitchQosMIB]: Remove invocation of update_data function during (#161) [SuvarnaMeenakshi]
2020-10-14 10:54:24 -07:00
Joe LeVeque
88c1d66c27
[python-click] No longer build our own package, let pip/setuptools install vanilla (#5549)
We were building our own python-click package because we needed features/bug fixes available as of version 7.0.0, but the most recent version available from Debian was in the 6.x range.

"Click" is needed for building/testing and installing sonic-utilities. Now that we are building sonic-utilities as a wheel, with Click specified as a dependency in the setup.py file, setuptools will install a more recent version of Click in the sonic-slave-buster container when building the package, and pip will install a more recent version of Click in the host OS of SONiC when installing the sonic-utilities package. Also, we don't need to worry about installing the Python 2 or 3 version of the package, as the proper one will be installed as necessary.
2020-10-14 10:16:35 -07:00
Nazarii Hnydyn
0155a15813
[submodule]: Advance swss submodule. (#5611)
- [portsorch]: Set proper initial forwarding state for LAG member #1448

Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>
2020-10-14 08:58:50 -07:00
pavel-shirshov
812e1a3489
[bgp]: Enable next-hop-tracking through default (#5600)
**- Why I did it**
FRR introduced [next hop tracking](http://docs.frrouting.org/projects/dev-guide/en/latest/next-hop-tracking.html) functionality. 
That functionality requires resolving BGP neighbors before setting BGP connection (or explicit ebgp-multihop command). Sometimes (BGP MONITORS) our neighbors are not directly connected and sessions are IBGP. In this case current configuration prevents FRR to establish BGP connections.  Reason would be "waiting for NHT". To fix that we need either add static routes for each not-directly connected ibgp neighbor, or enable command `ip nht resolve-via-default`

**- How I did it**
Put `ip nht resolve-via-default` into the config

**- How to verify it**
Build an image. Enable BGP_MONITOR entry and check that entry is Established or Connecting in FRR

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-10-13 22:21:28 -07:00
vdahiya12
3cd1d8e923
[sonic-py-common] Add platform.json to port_config files candidates only if interfaces are present within platform.json (#5538)
* [sonic-py-common] add platform.json to port_config

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
2020-10-13 16:58:06 -07:00
lguohan
4a390e93cf
[submodule]: sonic-utilities (#5606)
* b6af9f4 2020-10-12 | Fix expected neighbor when multiple ports connect to same neighbor (#1162) [lguohan]
* a71c72b 2020-10-12 | [show] Add 'show' CLI for system-health feature (#971) [shlomibitton]
* 561d133 2020-10-10 | [README.md] Add instructions for setting up a build/test environment (#1161) [Joe LeVeque]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-10-13 15:25:45 -07:00
Junchao-Mellanox
1c97a03b81
[system-health] Add support for monitoring system health (#4835)
* system health first commit

* system health daemon first commit

* Finish healthd

* Changes due to lower layer logic change

* Get ASIC temperature from TEMPERATURE_INFO table

* Add system health make rule and service files

* fix bugs found during manual test

* Change make file to install system-health library to host

* Set system LED to blink on bootup time

* Caught exceptions in system health checker to make it more robust

* fix issue that fan/psu presence will always be true

* fix issue for external checker

* move system-health service to right after rc-local service

* Set system-health service start after database service

* Get system up time via /proc/uptime

* Provide more information in stat for CLI to use

* fix typo

* Set default category to External for external checker

* If external checker reported OK, save it to stat too

* Trim string for external checker output

* fix issue: PSU voltage check always return OK

* Add unit test cases for system health library

* Fix LGTM warnings

* fix demo comments: 1. get boot up timeout from monit configuration file; 2. set system led in library instead of daemon

* Remove boot_timeout configuration because it will get from monit config file

* Fix argument miss

* fix unit test failure

* fix issue: summary status is not correct

* Fix format issues found in code review

* rename th to threshold to make it clearer

* Fix review comment: 1. add a .dep file for system health; 2. deprecated daemon_base and uses sonic-py-common instead

* Fix unit test failure

* Fix LGTM alert

* Fix LGTM alert

* Fix review comments

* Fix review comment

* 1. Add relevant comments for system health; 2. rename external_checker to user_define_checker

* Ignore check for unknown service type

* Fix unit test issue

* Rename user define checker to user defined checker

* Rename user_define_checkers to user_defined_checkers for configuration file

* Renmae file user_define_checker.py -> user_defined_checker.py

* Fix typo

* Adjust import order for config.py

Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com>

* Adjust import order for src/system-health/health_checker/hardware_checker.py

Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com>

* Adjust import order for src/system-health/scripts/healthd

Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com>

* Adjust import orders in src/system-health/tests/test_system_health.py

* Fix typo

* Add new line after import

* If system health configuration file not exist, healthd should exit

* Fix indent and enable pytest coverage

* Fix typo

* Fix typo

* Remove global logger and use log functions inherited from super class

* Change info level logger to notice level

Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com>
2020-10-12 11:12:49 +03:00
Joe LeVeque
5e97574e5d
[sonic-utilities] Update submodule (#5578)
* src/sonic-utilities 6e58dff...e1244a5 (5):
  > [setup.py] Pin external runtime dependencies at known working versions (#1151)
  > [config] Reload Monit configuration after potential hostname changes (#1132)
  > [NAT]:Updated natshow script to support DNAT Pool changes (#1001)
  > [pfcstat] Create new copies of Rx and Tx dictionaries (#1150)
  > [master]Fix show ip interface neighbor info not correct when add second ip (#1134)
2020-10-10 14:14:58 -07:00
Shi Su
c1add9a9c5
[sonic-sairedis] Update submodule (#5568)
Signed-off-by: shi-su <ssu@microsoft.com>
2020-10-08 23:37:36 -07:00
anish-n
42d9a44e05
Minigraph resource type changes (#5198)
* Parse sub_role from minigraph into DEVICE_METADATA
* Change minigraph sub_role to resource_type
2020-10-08 15:45:21 -07:00
Mahesh Maddikayala
744612d269
[ECMP][Multi-ASIC] Have different ECMP seed value on each ASIC (#5357)
* Calculate ECMP hash seed based on ASIC ID on multi ASIC platform. Each ASIC will have a unique ECMP hash seed value.
2020-10-08 09:05:37 -07:00
Petro Bratash
895f4e0bf7
[sonic-utilities] Submodule update (#5558)
- Fix show queue watermark command fail
     Azure/sonic-utilities@6e58dff

Signed-off-by: Petro Bratash <petrox.bratash@intel.com>
2020-10-07 11:32:38 -07:00
zzhiyuan
3da996bbf8
[arista]: Add disable_pcie_firmware_check soc property (#5543)
This is to fix pcie firmware check assert in Broadcom SDK once the SAI changes merges. This will be in the future but adding the soc property in the broadcom config now.

Co-authored-by: Zhi Yuan Carl Zhao <zyzhao@arista.com>
2020-10-06 18:35:15 -07:00
pavel-shirshov
63c1afbc68
[bgpcfgd]: Extract classes into their own files. Run bgpcfgd as a module (#5535)
1. Rename app module to bgpcfgd
2. Extract classes from one file to the module
2020-10-06 08:37:49 -07:00
Danny Allen
90d84143e8
[submodule update] Update sonic-utilities submodule pointer (#5545)
- [acl-loader] Revert fix for IP protocol == 0 (#1142)
- Show sflow interface to display all interfaces enabled for sflow (#1143)
- [sflow_test.py]: tests for config sflow commands. (#1112)
- [config/console] Support add/del console port setting commands (#1136)
- [cli][bgp]use vtysh in the show ip bgp summary command (#1137)
- [README.md] Describe new package creation schema (#1131)
- [consutil] Remove actual baud and refactor lib for future change (#1130)

Signed-off-by: Danny Allen <daall@microsoft.com>
2020-10-05 19:54:21 -07:00
Lawrence Lee
8c344095a8
[docker-orchagent]: Add NDP Proxy Daemon (#5517)
* Install ndppd during image build, and copy config files to image
* Configure proxy settings based on config DB at container start
* Pipe ndppd output to logger inside container to log output in syslog
2020-10-05 08:48:13 -07:00
Dong Zhang
6b0690adbb
[sonic-py-swsssdk] update submodule for using correct connect parameter number (#5524)
* [MultiDB] fix wrong parameter number in interface.py (#88)

Signed-off-by: Dong Zhang d.zhang@alibaba-inc.com
2020-10-03 23:03:13 -07:00
Mahesh Maddikayala
32a517f583
[sonic-swss] Update sonic-swss submodule (#5533) 2020-10-03 09:47:51 -07:00
pavel-shirshov
ffae82f8be
[bgp] Add 'allow list' manager feature (#5513)
implements a new feature: "BGP Allow list."

This feature allows us to control which IP prefixes are going to be advertised via ebgp from the routes received from EBGP neighbors.
2020-10-02 10:06:04 -07:00
abdosi
c0bbb7b63d
Fix python expception of missing subprocess (#5503)
Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2020-10-02 09:09:38 -07:00
abdosi
dda98023e3
[Submodule update] sonic swss (#5489)
be51ebc533cbe92a885c584b4611ffc92d10ea36 Add IPv6 key item support to request parser (#1449)
76e22516bffe45d577ab9cce4f5d96687140fe75 When teamd feature state is disabled the Netdevice created by teamd were (#1450)
6aa97ce7982b78a26803908c0ce12302e51df793 Use .clear() after std::move() (#1444)
d5757db5d896aa2405f6cbf9f4c150f46b27aa4e Add libzmq to README dependencies (#1447)
c7b262ee61973a48943603810c8246a2471d1862 Add libzmq to Makefiles (#1443)
0b2e59ab6b909e5ea06013da03215f0d1af1e618 [drop counters] Clarify log messages for initial counter setup (#1445)
003cf24353c845ef708d192b7da8d571537671e1 [dvs] Refactor and add buffer pool wm test (#1446)
2f5d2d99dfae506333bddda9de2f289f92bb889b [acl] Remove Ethertype from L3V6 qualifiers (#1433)
f7b974f7fc6474a6493a7b6d0676e17caffa6165 Fix issue: bufferorch only pass the first attribute to sai when setting attribute (#1442)

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2020-10-02 09:06:30 -07:00
Qi Luo
30f5557d36
Fix generate_l2_config: don't override hostname or device role (ToRRouter) (#5510)
* Fix generate_l2_config: don't override hostname because sonic-cfggen may not read from Redis. Fix test_l2switch_template test case to test preset l2 feature.
* Improve test script: compare json files with sort_keys
* Revert changes on sample_output
* Remove members field in VLAN section. Fix test assertTrue statement.
2020-10-01 23:24:35 -07:00
pavel-shirshov
1f0f751d4d
[bgpcfgd]: Don't use neighbor metadata in bgpmon sessions (#5506)
**- Why I did it**
BGP_MONITORS sessions don't have corresponding DEVICE_NEIGHBOR_METADATA CONFIG_DB entries in the minigraphs. Prevent bgpcfgd to wait on such entries for BGP_MONITORS sessions.

**- How I did it**
Set constructor argument to False that means - don't wait for device neighbors metadata info for BGP_MONITORS

**- How to verify it**
Build an image, write on your device, use a minigraph with BGP_MONITORS sessions. Check that sessions are populated in the config.
2020-09-30 22:35:52 -07:00
Dong Zhang
b01879b44b
[sonic-py-swsssdk] update submodule for changing redis_client index from db_id to db_name (#5512) 2020-09-30 13:42:20 -07:00
Tamer Ahmed
57a584997a [cfggen] Iterative Version Of Deep Update
Avoiding recursive update of maps as it consumes stack frames. This
PR introduces iterative version of deep_update method.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Tamer Ahmed
110f7b7817 [cfggen] Build Python 2 And Python 3 Wheel Packages
This builds Python 2&3 wheel packages for sonic-cfggen script.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Tamer Ahmed
99ffce6b57 [cfggen] Move Python 2 Test Data to Py2 Folder
Rendering templates show different orders when rendered using Python
2 vs Python 3. This PR prepare for Python 3 packaging by creating
new dir 'py2' for Python 2 rendered test cases.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Tamer Ahmed
6754635010 [cfggen] Make Jinja2 Template Python 3 Compatible
Jinja2 templates rendered using Python 3 interpreter, are required
to conform with Python 3 new semantics.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Tamer Ahmed
9122d16dd2 [cfggen] Remove NatSorted
Natural sorting of SONiC config gen output consumes lot of CPU cycles.
The sole use of natsorted was to make test comparison easier and so,
the natsorting logic is now relocated to the test suite. As a result
sonic-cfggen gained nearly 1 sec per call since we no longer import
natsorted module!

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Guohan Lu
e412338743 Revert "[bgp] Add 'allow list' manager feature (#5309)"
This reverts commit 6eed0820c8.
2020-09-28 22:00:29 -07:00
bingwang-ms
e3f8159606
[sonic_py_common] Fix exception in daemon_base.py (#5472)
There is a syntax error (self is missing for log_info and log_warning) in signal_handler in daemon_base.py, which causes an exception
2020-09-27 11:31:45 -07:00
pavel-shirshov
6eed0820c8
[bgp] Add 'allow list' manager feature (#5309)
implements a new feature: "BGP Allow list."

This feature allows us to control which IP prefixes are going to be advertised via ebgp from the routes received from EBGP neighbors.
2020-09-27 10:47:43 -07:00
Joe LeVeque
dd02437d7f
[sonic-platform-common] Update submodule (#5470)
* src/sonic-platform-common 7255d3a...111dcf7 (2):
  > [EEPROM] Add new function part_number_str to TlvInfoDecoder (#121)
  > Fix fp_port_index error to get correct index type (#111)
2020-09-25 19:18:05 -07:00
Sangita Maity
ccc3d75535
[portconfig] Fallback to port_config.ini if hwsku.json is not available (#5434)
**- Why I did it**
As discussed, This PR covers the below points.
1. Give precedence to platform.json only if both platform.json and  hwsku.json file exist.  In case only platform.json exists, we don’t allow breakout for that HWSKU and fallback to port_config.ini.
 
**- How I did it**
check for `hwsku.json` file presence under get_path_to_port_config_file function.

Signed-off-by: Sangita Maity <sangitamaity0211@gmail.com>
2020-09-25 12:47:45 -07:00
Sumukha Tumkur Vani
b5bcfef013
Update conf DB with CA cert & rename ca_crt field (#5448) 2020-09-25 09:20:09 -07:00
Syd Logan
0311a4a037
Add gearbox phy device files and a new physyncd docker to support VS gearbox phy feature (#4851)
* buildimage: Add gearbox phy device files and a new physyncd docker to support VS gearbox phy feature

* scripts and configuration needed to support a second syncd docker (physyncd)
* physyncd supports gearbox device and phy SAI APIs and runs multiple instances of syncd, one per phy in the device
* support for VS target (sonic-sairedis vslib has been extended to support a virtual BCM81724 gearbox PHY).

HLD is located at b817a12fd8/doc/gearbox/gearbox_mgr_design.md

**- Why I did it**

This work is part of the gearbox phy joint effort between Microsoft and Broadcom, and is based
on multi-switch support in sonic-sairedis.

**- How I did it**

Overall feature was implemented across several projects. The collective pull requests (some in late stages of review at this point):

https://github.com/Azure/sonic-utilities/pull/931 - CLI (merged)
https://github.com/Azure/sonic-swss-common/pull/347 - Minor changes (merged)
https://github.com/Azure/sonic-swss/pull/1321 - gearsyncd, config parsers, changes to orchargent to create gearbox phy on supported systems
https://github.com/Azure/sonic-sairedis/pull/624 - physyncd, virtual BCM81724 gearbox phy added to vslib

**- How to verify it**

In a vslib build:

root@sonic:/home/admin# show gearbox interfaces status
  PHY Id    Interface        MAC Lanes    MAC Lane Speed        PHY Lanes    PHY Lane Speed    Line Lanes    Line Lane Speed    Oper    Admin
--------  -----------  ---------------  ----------------  ---------------  ----------------  ------------  -----------------  ------  -------
       1   Ethernet48  121,122,123,124               25G  200,201,202,203               25G       204,205                50G    down     down
       1   Ethernet49  125,126,127,128               25G  206,207,208,209               25G       210,211                50G    down     down
       1   Ethernet50      69,70,71,72               25G  212,213,214,215               25G           216               100G    down     down

In addition, docker ps | grep phy should show a physyncd docker running.

  Signed-off-by: syd.logan@broadcom.com
2020-09-25 08:32:44 -07:00
Mahesh Maddikayala
224cc3953f
[CLI][Multi ASIC] update get_all_namespace to return current namespace (#5446) 2020-09-24 11:32:16 -07:00
SuvarnaMeenakshi
8c8773bee6
[sonic-snmpagent]: Advance submodule (#5454)
- Implementation changes for CiscoBgp4MIB (#158)

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
2020-09-24 11:12:23 -07:00
Syd Logan
623d5c08c1
[submodule]: update sonic-swss-common/sonic-sairedis (#5449)
Update to pick up changes needed for VS gearbox

- sonic-sairedis
  -  meta/saiserialize.cpp - Fix typo at SAI_ATTR_VALUE_TYPE_ACL_FIELD_DATA_UINT32 (#662)
  -  syncd/scripts/gbsyncd_start.sh - Rename physyncd_startup.sh/physyncd_startup.py to gbsyncd_startup.sh/gbsyncd_startup.py (#661)
  - syncd/scripts/gbsyncd_startup.py - Rename physyncd_startup.sh/physyncd_startup.py to gbsyncd_startup.sh/gbsyncd_startup.py (#661)

- sonic-swss-common
  - common/database_config.json -  Add restapi DB (#386)
  - common/redispipeline.h -  Throw proper exceptions when talking with Redis (#384)
  - common/redisreply.cpp -  Throw proper exceptions when talking with Redis (#384)
  - common/redisreply.h -  Throw proper exceptions when talking with Redis (#384)
  -  common/schema.h -  Add restapi DB (#386)
  - common/table.cpp - Add restapi DB (#386)

Signed-off-by Syd Logan syd.logan@broadcom.com
2020-09-24 01:08:55 -07:00
Joe LeVeque
ec9281be76
[sonic-platform-common][sonic-platform-daemons][sonic-snmpagent][sonic-utilities] Update submodules (#5435)
Update Python-based submodules to include updated dependencies in setup.py files.

* src/sonic-platform-common 14c6e53...7255d3a (2):
  > [setup.py] Update dependenices (#120)
  > [sfputilbase|sfputilhelper] fix value unpack error due to port_alias_asic_map in portconfig.py file (#116)

* src/sonic-platform-daemons e1842b2...1aaffcc (2):
  > Add 'wheel' package to 'setup_requires' list (#95)
  > [thermalctld] Fix issue: thermalctld should be auto restarted when being killed (#94)

* src/sonic-snmpagent 1a2b62a...5957460 (1):
  > [setup.py] Add 'wheel' to 'setup_requires' (#160)

* src/sonic-utilities 2244d7b...2f79ac1 (4):
  > [show] Add missing import to fix `show ip interfaces` (#1126)
  > Fixed config load_minigrpah not working for Multi-asic platfroms. (#1123)
  > Add CLI for configuring synchronous mode (#1094)
  > [setup.py] Update dependencies (#1125)
2020-09-23 19:24:59 -07:00
gechiang
04c709d27f
Fix bgpmon.py packaging issue for console_scripts entry point (#5436)
* Fix bgpmon.py packaging issue for console_scripts entry point

* renamed directory from bgpmon_proj to bgpmon
2020-09-23 09:40:49 -07:00
Qi Luo
03f82a0107
[submodule]: update sonic-swss-common/sonic-sairedis/sonic-swss (#5406) 2020-09-21 14:49:56 -07:00
Joe LeVeque
3987cbd80a
[sonic-utilities] Build and install as a Python wheel package (#5409)
We are moving toward building all Python packages for SONiC as wheel packages rather than Debian packages. This will also allow us to more easily transition to Python 3.

Python files are now packaged in "sonic-utilities" Pyhton wheel. Data files are now packaged in "sonic-utilities-data" Debian package.

**- How I did it**
- Build and install sonic-utilities as a Python package
- Remove explicit installation of wheel dependencies, as these will now get installed implicitly by pip when installing sonic-utilities as a wheel
- Build and install new sonic-utilities-data package to install data files required by sonic-utilities applications
- Update all references to sonic-utilities scripts/entrypoints to either reference the new /usr/local/bin/ location or remove absolute path entirely where applicable

Submodule updates:

* src/sonic-utilities aa27dd9...2244d7b (5):
  > Support building sonic-utilities as a Python wheel package instead of a Debian package (#1122)
  > [consutil] Display remote device name in show command (#1120)
  > [vrf] fix check state_db error when vrf moving (#1119)
  > [consutil] Fix issue where the ConfigDBConnector's reference is missing (#1117)
  > Update to make config load/reload backward compatible. (#1115)

* src/sonic-ztp dd025bc...911d622 (1):
  > Update paths to reflect new sonic-utilities install location, /usr/local/bin/ (#19)
2020-09-20 20:16:42 -07:00
gechiang
128def6969
Add bgpmon to be started as a new daemon under BGP docker (#5329)
* Add bgpmon under sonic-bgpcfgd to be started as a new daemon under BGP docker

* Added bgpmon to be monitored by Monit so that if it crashed, it gets alerted

* use console_scripts entry point to package bgpmon
2020-09-20 14:32:09 -07:00
Junchao-Mellanox
35de31b741
[sonic-platform-daemons] Update submodule (#5386)
e1842b2 Change STATE_DB key (PCIE_STATUS|PCIE_DEVICES -> PCIE_DEVICES) (#93)
a6c0071 [thermalctld] Fix issue: fan status should not be True when fan is absent (#92)
2020-09-16 10:36:40 -07:00
madhanmellanox
1b25b98f43
[submodule]: updating sonic-utilities changes (#5383)
Updating submodule for sonic-utilities to get the changes to Azure master. The following were the commits that were part of this submodule.

Handling deletion of Portchannel before deletion of its members 219d78bda596cf78b71e5f6930545fb43f9c6c8b

removed a extra space in the condifitional statement 828719ed89112f317f9e78fff8264416176736f0

added Note about portchannel deletion in Command-Reference guide b36079f1d3f4086e42cfdca2090ebc76d811ad10

removed ctx.fail() and replaced it with click.echo() to display the error. Also, references of portchannel is replaced with port channel in the command reference guide b716b97e1aaf890ba9ef4ed0bb2931f4c2c030d2

Co-authored-by: Madhan Babu <madhan@arc-build-server.mtr.labs.mlnx>
2020-09-16 09:12:13 -07:00
Samuel Angebault
9bf4b0a93e
[baseimage]: Change the loopback mask from /8 to /16 (#5353)
As per the VOQ HLDs, internal networking between the linecards and supervisor is required within a chassis.
Allocating 127.X/16 subnets for private communication within a chassis is a good candidate.
It doesn't require any external IP allocation as well as ensure that the traffic will not leave the chassis.

References:
https://github.com/Azure/SONiC/pull/622
https://github.com/Azure/SONiC/pull/639

**- How I did it**

Changed the `interfaces.j2` file to add `127.0.0.1/16` as the `lo` ip address.
Then once the interface is up, the post-up command removes the `127.0.0.1/8` ip address.
The order in which the netmask change is made matters for `127.0.0.1` to be reachable at all times.

**- How to verify it**

```
root@sonic:~# ip address show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/16 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
```

Co-authored-by: Baptiste Covolato <baptiste@arista.com>
2020-09-15 15:29:48 -07:00
Tamer Ahmed
15f5d47338
[dhcpmon] Print Both Snapshot And Current Counters (#5374)
Printing both snapshot and current counter sets will make it easier to pinpoint
which message type(s) is/are not being relayed. This PR prints both counter sets.
Also, this PR defines gnu11 as a C standard to compile with in order to avoid
making changes when porting to 201811 branch.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-15 15:27:36 -07:00
Qi Luo
17c923c15b
[submodule]: update sonic-swss-common (#5362)
cc2f80b 2020-09-11 | Fix SubscriberStateTable::hasCachedData formula for a timing risk
2020-09-12 00:17:23 -07:00
Petro Bratash
c44279a365
[sonic-platform-daemons] Update submodule (#5355)
- 1893c40495
  - Fix the xcvrd theowing error on sfprecover function on getKeys() not valid.
- 65fa4430fe
  - Merge pull request #90 from abdosi/multiasic-fix
- 7f812c93b1
  - [xcvrd] Don't log unnecessary messages upon empty transceiver change event (#53)
- 096920273b
  - Fix pcied daemon failure (#91)

Signed-off-by: Petro Bratash <petrox.bratash@intel.com>
2020-09-11 13:03:53 -07:00
Tamer Ahmed
1bf6fdc6d2
[dhcpmon] Monitor Mgmt Interface For DHCP Packets (#5317)
When BGP routes are missing, DHCP packets get relayed over mgmt
interface. This results in dhcpmon alerting that DHCP packets are
not being relayed. This is PR include mgmt interface as uplink
device, and so, if DHCP packet gets relayed over mgmt interface,
regular dhcpmon alert will not be issues. Instead, dhcpmon will
check the mgmt interface counts and issue a separate alert regarding
packets travelling through mgmt network.

In addition, this PR includes the following enhancements:
1. Add SIGUSR1 handler that prints out current packet counts
2. Increase alert grace window to 3 minutes from currently 2 minutes
3. Time is now computed more accurately
4. Print vlan name before counters

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-09 18:37:01 -07:00
Renuka Manavalan
e9918bac07
[sonic-utilities] Update submodule (#5337)
Update to current head in master. Latest: commit 96cb3594a6afc64edcef7351623258287099118d
2020-09-09 09:04:21 -07:00
Joe LeVeque
5b3b4804ad
[dockers][supervisor] Increase event buffer size for dependent-startup (#5247)
When stopping the swss, pmon or bgp containers, log messages like the following can be seen:

```
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,061 ERRO pool dependent-startup event buffer overflowed, discarding event 34
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,063 ERRO pool dependent-startup event buffer overflowed, discarding event 35
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,064 ERRO pool dependent-startup event buffer overflowed, discarding event 36
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,066 ERRO pool dependent-startup event buffer overflowed, discarding event 37
```

This is due to the number of programs in the container managed by supervisor, all generating events at the same time. The default event queue buffer size in supervisor is 10. This patch increases that value in all containers in order to eliminate these errors. As more programs are added to the containers, we may need to further adjust these values. I increased all buffer sizes to 25 except for containers with more programs or templated supervisor.conf files which allow for a variable number of programs. In these cases I increased the buffer size to 50. One final exception is the swss container, where the buffer fills up to ~50, so I increased this buffer to 100.

Resolves https://github.com/Azure/sonic-buildimage/issues/5241
2020-09-08 23:36:38 -07:00
judyjoseph
1558c04fa8
Update sonic-swss submodule (#5322)
65f63c1 Sflow fixes during DEL processing (#1427)
e54948a [teamsyncd][teammgrd] Graceful exit after receiving SIGTERM (#1407)
2020-09-06 08:16:20 -07:00
vdahiya12
679d8df36f
[sonic-utilities][sonic-py-common] Move logic to get port config file path to sonic-py-common and update sonic-utilities to comply (#5264)
* [sonic-utilities]update submodule with fix

This PR addresses fixes in sonic-py-common to imitate the behavior inside
sonic-cfggen. Essentially this is a fix for accessing the port-config file.
First check if there is a platform.json file for config generation
and then for legacy port_config.ini.
Also updating the sub-module sonic-utilities.

Fix pfcwd stats crash with invalid queue name (#1077)
[show][bgp]Display the Total number of neighbors in the show ip bgp(v6) summary. (#1079)
[config] Update SONiC Environment Vars When Loading Minigraph (#1073)
Multi asic platform changes for interface, portchannel commands (#878)
Update Command-Reference.md (#1075)
[filter-fdb] Fix Filter FDB With IPv6 Present in Config DB (#1059)
[config] Remove _get_breakout_cfg_file_name helper function (#1069)
[SHOW][BGP] support show ip(v6) bgp summary for multi asic platform (#1064)
[fanshow] Display other fan status, such as Updating (#1014)
Add ip_prefix len based on proxy_arp status (#1046)
Enable the platform specific ssd firmware upgrade during reboot (#954)
[show][cli[show interface portchannel support for Multi ASIC (#1005)
support show interface commands for multi ASIC platforms (#1006)

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
2020-09-04 10:19:12 -07:00
judyjoseph
200d49eabb
sonic-platform-daemons submodule update (#5311)
[ledd][multi-ASIC] Update to ledd based on sonic-swss-common updates (#87)
remove sonic-utilities dependency from pcied (#88)
2020-09-04 08:58:20 -07:00
Danny Allen
fe01f0c285
[submodule] Update swsssdk submodule (#5295)
Signed-off-by: Danny Allen <daall@microsoft.com>
2020-09-03 00:21:25 -07:00
abdosi
dd908c2ee2
[sonic-swsscommon] submodule update with commit's (#5300)
[schema] Make schema header support C project (#373)
Removed DB specific get api's from Selectable class (#378)

With the change as part of #378 caclmgrd need to be updated
to use new client side Get API to access namespace.

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2020-09-02 18:09:03 -07:00
Joe LeVeque
fb8f09a116
[radvd] No longer build from source; Install vanilla Debian package once again (#5242)
Remove radvd Makefile and patch, change docker-router-advertiser Dockerfile template to simply install the vanilla radvd package using apt-get.

- In PR https://github.com/Azure/sonic-buildimage/pull/2795, we started building radvd from source and patching it to prevent it from erroring out when advertising an MTU of 9100 which was greater than the MTU size configured on the bridge interface (1500), which was due to a limitation in the 4.9 Linux kernel.
- Master branch is now using Linux kernel 4.19. As of 4.18, the kernel supports setting a bridge MTU to a value > 1500.
- PR https://github.com/Azure/sonic-swss/pull/1393 modified vlanmgrd to take advantage of this and now configures the MTU of bridge interfaces in SONiC to the proper size of 9100. Therefore, we no longer need to patch radvd. Since we no longer need to patch radvd, we no longer need to build it from source, so we can save build time by going back to simply installing the vanilla radvd Debian package in the router-advertiser container.
2020-09-01 13:53:36 -07:00
sdddean
426b6aaf5b
[iccpd] Fix the bug of iccp cmd show. (#5214)
The cmd "mclagdctl dump state" would goes wrong when there are two or more mclag_groups configured. The field "MCLAG Interface" can not be displayed in some group.

Signed-off-by: Sun Dandan <sundandan@asterfusion.com>
2020-09-01 13:07:13 -07:00
arlakshm
17e78715ae
[Multi-ASIC]:Update the template to add ipinip entry for Loopback4096 (#5235)
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
The following changes are done. 
- Multi asic platform have 2 Loopback interfaces, Loopback0 and Loopback4096. IPinIP decap entries need to be added for both of them. Update the ipinip.json.j2 template to add decap entries for Loopback4096.
- Add corressponding unit test
2020-08-31 17:35:48 -07:00
Sumukha Tumkur Vani
3d6ef3e790
Update restapi (#5267)
* Add available routes data to heartbeat API
* Accept subnet routes via tunnel, skip self loopback nexthop
2020-08-31 10:43:32 -07:00
judyjoseph
ca3e71da0a
[sonic-py-common] Relocate some functions from sonic-utilities (#5269)
* Relocate interface related common functions to py-common
* Add unit tests for interface API's.
2020-08-30 08:24:29 -07:00
shi-su
f973d90c76
[sonic-sairedis] Update submodule (#5243)
Signed-off-by: shi-su <ssu@microsoft.com>
2020-08-29 18:03:52 -07:00
judyjoseph
c3dbffa847
[sonic-platform-daemons] Update submodule (#5262)
Update the patform daemons with new commits 

commit e9628b6b5a4683c7366fc99ee19ce727546fbabb
    Merge pull request #64 from judyjoseph/multi_asic_ledd_xcvr
    * platform daemon (Xcvrd, Ledd) changes for multi asic platform
      * Updates in ledd daemon to use namespaces and get the namespace from selector object.
      * Updates to xcvrd daemon to use the asic_id in talking to the right DB.
      * Updated based on new sonic-py-common API's
      * Invoke initializeGlobalConfig() in the SfpUpdate/DomInfoUpdate processes as well.

commit 415b8c457625c514aff0f8ecbdbbb655414d8067
    [thermalctld] Optimize the thermal policy loop to make it execute every 60 seconds (#77)
    
commit 3d1f3196fd9c9942134e4926de7d248743e9589d
    Update FAN_INFO in psud to avoid inconsistant output of show platform psud and show platform fan (#81)
2020-08-29 10:25:46 -07:00
judyjoseph
123a5640bd
Update platform-common submodule (#5261)
Updating platform-common submodule with these commits

commit 14c6e53ecb861e124e2b45a7b65875ffac1b949e 
    [sff8472.py] Make hex keys all lowercase (#115)
    Alpha chars in hex-based keys should be lowercase

commit b60f46cd1fb0ced1ffbff382e0125517f8c74b9e
    Sfputil base and helper class changes for multi-ASIC (#100)
    * Sfputil base and helper class changes for multi-ASIC
          > adding the logical interface to asic id mapping
    * Updated based on new sonic-py-common API's.
2020-08-29 10:25:06 -07:00
Renuka Manavalan
5a166cc60c
[sonic-py-common] Add function to get hostname (#5265)
Added a function to get hostname. It get it from DEVICE_METADATA|localhost: hostname.
2020-08-28 13:47:44 -07:00
Prince Sunny
4338d8293f
Skip vnet-vxlan interfaces from generating networks (#5251)
* Skip Vnet interface from generating networks
2020-08-27 14:14:04 -07:00
Tamer Ahmed
a8016a6987
[cfggen] Use Redis Pipeline (#5250)
This PR enables cfggen to readr/write from Redis DB using pipelines.
Pipelines enables batch read/write from/to Redis DB.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-26 15:05:54 -07:00
Tamer Ahmed
a38105237c
[py-swsssdk] Submodule Update (#5249)
Change:
  c25d492 Merge pull request #83 from tahmed-dev/taahme/add-redis-pipeline-operation
  198d143 review comments - part of [configdb] Add Ability to Query/Update Redis Using Pipelines
  994851c review comments - part of [configdb] Add Ability to Query/Update Redis Using Pipelines
  2d2b7e1 making lgtm happy - part of [configdb] Add Ability to Query/Update Redis Using Pipelines
  fa9093c [configdb] Add Ability to Query/Update Redis Using Pipelines

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-26 09:09:43 -07:00
judyjoseph
394aa5f75f
Platform monitor changes in daemon_base for multi_asic (#4932)
Adding namespace support for db connect API.
2020-08-25 09:33:43 -07:00
Joe LeVeque
58db2d53e3
[sonic-py-common] Add unit test framework (#5238)
**- Why I did it**

To install the framework for adding unit tests to the sonic-py-common package and report coverage.

** How I did it **

- Incorporate pytest and pytest-cov into sonic-py-common package build
- Updgrade version of 'mock' installed to version 3.0.5, the last version which supports Python 2. This fixes a bug where the file object returned from `mock_open()` was not iterable (see https://bugs.python.org/issue32933)
- Add support for Python 3 setuptools and pytest in sonic-slave-buster environment
- Add tests for `device_info.get_machine_info()` and `device_info.get_platform()` functions
- Also add a .gitignore in the root of the sonic-py-common directory, move all related ignores from main .gitignore file, and add ignores for files and dirs generated by pytest-cov
2020-08-24 10:35:22 -07:00
Joe LeVeque
87713aaa1b
[sonic-swss] Update submodule (#5231)
* src/sonic-swss d2bab10...c4949a2 (34):
  > [dvs] Add new common issues and TOC to DVS README (#1405)
  > Avoid adding loopback interface (ip link add) when setting nat zone on loopback interface (#1411)
  > [portsorch] add buffer drop FC group (#1368)
  > [dvs/chassis] Bring up SONiC interfaces in virtual chassis (#1410)
  > [chassis/dvs] Add support for virtual chassis to DVS testbed (#1345)
  > [sonic-swsss] Fix the issue of field "next_hop_ip" not getting updated in state DB in ERSPAN Mirror (#1375)
  > [intfmgr] Fix OA crash issue due to link local configurations (#1195)
  > Fix the issue when persistent DVS is used to run pytest which has number of front-panel ports < 32 (#1373)
  > [dvs] Refactor AsicDbValidator (#1402)
  > [fec] Get FEC mode when port is already admin down (#1403)
  > [fec] added logic that put port down before applying fec onfiguration (#1399)
  > [dvs] Add performance test for adding and deleting routes (#1392)
  > Ignore IPv6 link-local and multicast entries as Vnet routes (#1401)
  > [vlanmgr] Support Jumbo Frame By Default (#1393)
  > Fix log/syslog not being correct when last test fails for given module (#1395)
  > Get initial speed from ASIC DB  (#1390)
  > [dvs] Add options to limit CPU usage (#1394)
  > [intfsorch] Retrieve Port object before setting NAT zone on router interfaces. (#1372)
  > [.gitignore] Ignore gearsyncd binary (#1381)
  > Added Max Nexthopgroup/ECMP Count supported by device into State DB. (#1383)
  > [dvs] Upload logs even if failure occurs during startup (#1389)
  > [rates] fix issue with rates init (#1387)
  > [dvs] Validate that SWSS is ready to receive input before starting tests (#1385)
  > [dvs] Convert sflow and speed tests to use dvslib (#1382)
  > [dvs_acl] Refactor and document dvs_acl library (#1378)
  > [dvs] Fix install instructions in README (#1379)
  > [dvs] Update README with new flags, options, and known issues (#1380)
  > swss: gearsyncd should return 0 on exit (#1376)
  > Remove 00-copp.config.json from swss debian package. (#1366)
  > fix undefined var in rates lua scripts (#1365)
  > [fdborch] Fixed Orchagent crash in FDB flush on port disable. (#1369)
  > [tlm_teamd]: Try to add LAG again, when teamd is not ready first time (#1347)
  > [vs] Incorporate python3 best practices into DVSLib (#1357)
  > [dvs] Mark unstable tests as xfail (#1356)
2020-08-22 18:41:32 -07:00
judyjoseph
cbb46e426b
Add common functions applicable to single/multi asic platforms (#5224)
* Add common functions applicable to single/multi asic platforms
* Raise exception if invalid namespace is given as input.
2020-08-22 16:24:44 -07:00
nirenjan
bb57ccecd4
[sonic-host-service]: Add SONiC Host Services infrastructure (#4840)
- Why I did it

When SONiC is configured with the management framework and/or telemetry services, the applications running inside those containers need to access some functionality on the host system. The following is a non-exhaustive list of such functionality:

Image management
Configuration save and load
ZTP enable/disable and status
Show tech support
- How I did it

The host service is a Python process that listens for requests via D-Bus. It will then service those requests and send a response back to the requestor.

This PR only introduces the host service infrastructure. Applications that need access to the host services must add applets that will register on D-Bus endpoints to service the appropriate functionality.

- How to verify it

- Description for the changelog

Add SONiC Host Service for container to execute select commands in host

Signed-off-by: Nirenjan Krishnan <Nirenjan.Krishnan@dell.com>
2020-08-21 15:34:14 -07:00
Joe LeVeque
a1335aa1f4
[sonic-utilities] Update submodule (#5233)
* src/sonic-utilities d5fdd74...17fb378 (7):
  > [sonic-installer] Import re module (#1061)
  > [fast-reboot]: Fix fail to execute fast-reboot problem (#1047)
  > [config] Reduce Calls to SONiC Cfggen (#1052)
  > [filter-fdb] Call Filter FDB Main From Within Test Code (#1051)
  > [sflow_test.py]: Fix show sflow display. (#1054)
  > Change fast-reboot script to use swss and radv service script (#1036)
  > Common functions for show CLI support on multi ASIC (#999)
2020-08-21 13:58:46 -07:00
Kelly Chen
570dbf52f6
[iccpd] Fix uninitialized variable. (#5112)
To declare *tb[] but do not initialize it, it might be very risky. We get iccpd exception during processing arp/nd event. Initialize it to {0};
2020-08-21 00:27:04 -07:00
judyjoseph
ba3f5ca094
Advance swss-common submodule. (#5222)
9a7c9d Dbconnector namespace support (#376)
c32f0b5 add state db entry for fgnhg route entry (#374)
2020-08-20 15:10:19 -07:00
Joe LeVeque
ce2c0781ab
[sonic-config-engine] Update .gitignore (#5223)
- Ignore directories generated by building Python wheel package
- Move all sonic-config-engine ignores from the root .gitignore to src/sonic-config-engine/.gitignore
2020-08-20 11:09:10 -07:00
Mahesh Maddikayala
fdea6fb439
[sonic-py-common][multi ASIC] API to get a list of frontend ports (#5221)
* [sonic-py-common][multi ASIC] utility to get a list of frontend ports from a given list of ports
2020-08-20 09:14:00 -07:00
Joe LeVeque
da69d57a8d
[redis-dump-load] Update submodule (#5215)
* src/redis-dump-load 832a645...7585497 (2):
  > Merge pull request #63 from jleveque/update_gitignore
  > Merge pull request #59 from breser/redis-load-empty
2020-08-19 11:23:42 -07:00
Tamer Ahmed
ec11308a66
[cfggen] Conform With Python 3 Syntax (#5154)
Preparing sonic-cfggen for migration to Python 3.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-19 09:29:40 -07:00
SuvarnaMeenakshi
f5746f3189
[submodule]: Advance sonic-snmpagent. (#5213)
Update sonic-snmpagent submodule to include below commits:
1a2b62a [Namespace]: Fix SAI_ID key used in cpfcIfTable and csqIfQosGroupStatsTable implementation (#138)
d06f00c [pytest/coverage]: add coverage support (#156)
90e9f2e [Namespace]: Simplify sync_d functions to use higher order (#154)
b5815d9 [LLDP]: Modify OID index of LLDPRemTableUpdater MIB (#155)
d5f2b92 [Multiasic]: Provide namespace support for ipNetToMediaPhysAddress (#129)
166c221 [Namespace]: Fix interface counters in RFC 1213 (#145)

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
2020-08-19 04:27:09 -07:00
Tamer Ahmed
f840013ed9
[cfggen] Allow Write To Redis DB With Template/Batch Mode (#5203)
Argument to write to config-db is not allowed when using template.
This PR allows cfggen to write to redis db when using template
mode.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-18 18:24:36 -07:00
judyjoseph
52fe7ae365
[sonic-py-common] Add interface utilities (#5113)
* Add sonic_interface.py in sonic-py-common for sonic interface utilities to keep this SONIC PREFIX naming convention in one place in py-common and all modules/applications use the functions defined here.
2020-08-18 10:20:26 -07:00
arlakshm
4487abdfcb
Fix to prevent classification all portchannels is External (#5202)
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
Fix the checks in the API is_port_channel_internal
2020-08-17 17:22:30 -07:00
Joe LeVeque
fa4af0d372
[sonic-config-engine] Ensure all test output files are ignored (#5197)
Also align the naming and cleanup of generated output files in test_cfggen.py with the rest of the tests.
2020-08-15 23:09:11 -07:00
Praveen Chaudhary
32539a1bda
[sonic-vlan.yang]: Correct Pattern for VLAN. (#5165)
Changes:
-- Correct Pattern for VLAN.
-- Add Special Test support.
-- Add first special test to load VLAN 1-4094.
-- Add lanes in ports.
-- Make Sure None test gets no output from libyang while config load.

[yangModelTesting.py]: INCORRECT VLAN_NAME FIELD IN VLAN TABLE test.

Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com
2020-08-15 12:38:15 -07:00
Joe LeVeque
df11f17843
[sonic-platform-daemons] Update submodule (#5190)
src/sonic-platform-daemons 0231b49...80661c7 (1):
Update .gitignore file (#86)
2020-08-15 12:34:43 -07:00
Stephen Sun
5c02078118
[submodule] Advance submodule head for sonic-utilities (#5184)
[db_migrator] Support migrating database regarding buffer configuration for all Mellanox switches (#993)
    [show/vlan]: fix show vlan config (#1050)
    [PFCWD] Fix issue with "pfcwd show stats" command during SONiC init (#1018)
    [config] Eliminate port breakout-related globals (#1045)
    [teamshow]: refactor teamshow to use state db information (#1049)
    [watermarkstat] Import print_function from __future__ (#1048)
    [show]: split show interfaces commands into separate file (#1044)
    Python 3 compliance (#1043)
    Update all references to new 'sonic-installer' file name (#1033)
    [config/show]: split vlan into separate file (#1038)
    [pytest/coverage]: add coverage for all scripts (#1041)
    enable watchdog before running platform specific reboot plugin (#1037)
    [tests]:add coverage for config/main/utilities_common module (#1039)
    [cli/feature]: split feature command into a separate file (#1034)

Signed-off-by: Stephen Sun <stephens@nvidia.com>

Co-authored-by: Stephen Sun <stephens@mellanox.com>
2020-08-14 11:14:35 -07:00
arlakshm
6c895513ac
Common functions for Multi ASIC (#4973)
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan arlakshm@microsoft.com

The following common APIs are added for multi ASIC
- an  API to check if a given port is a internal or external port
- an  API to check if a given port-channel is internal or external
- an API to check if a bgp-session is internal or external
- an  API to connect to the config and other dbs in the a given namespace
- added common APIs to the sonic_py_common library.
- update the sample port-config.ini with role column and add corresponding test to verify the ports configuration is - generated properly.
2020-08-14 07:36:00 -07:00
Joe LeVeque
893d0d8638
[sonic-platform-common] Update submodule (#5173)
* src/sonic-platform-common f3f3573...433dd1a (1):
  > Parse 'Extended Specification compliance' as independent field (#112)
2020-08-13 19:15:40 -07:00
Prince Sunny
3fee094760
Enable restapi, update sonic-restapi (#5169)
* Enable restapi if included in image
* [Submodule update] sonic-restapi
2020-08-13 11:29:49 -07:00
shi-su
a974611fbb
[sonic-sairedis] Update submodule (#5156) 2020-08-12 19:49:10 -07:00
Tamer Ahmed
f9edf6e5cc
[cfggen] Extend Template Argument to Support Batch Mode (#4941)
Calls to cfggen take considerable time. With batch mode, we will have the ability
to reduce number of calls from services.

Example of the batch mode command:
sonic-cfggen -t template-1.j2 -t template-2.j2,config-db -t template-3.j2,config-db -t template-4.j2,file1 -t template-5.j2,file2 --write-to-db.

template-1.j2 will be rendered to stdout since it is missing the dest part. stdout is default
config-db is a special keyword that will inject the rendered template into internal data structure. The internal data structure gets written to redis-db with --write-to-db switch. In the case the user would like to write to a file named config-db, it could be given as /config-db or ./config-db

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-12 15:13:06 -07:00
Petro Bratash
9f7a8d59a1
[sonic-platform-daemons] Update submodule (#5159)
- [pcied] Fix path to pcie.yaml in pcied daemon (#84) 
>  0231b49ae84cf18c3a8d0c855215c467f28d48d3

Signed-off-by: Petro Bratash <petrox.bratash@intel.com>
2020-08-12 12:01:35 -07:00
Joe LeVeque
47a866ecbd
[sonic-platform-daemons] Update submodule (#5152)
* src/sonic-platform-daemons da3dfbb...7a30ae0 (1):
  > Python 3 compliance (#85)
2020-08-11 14:57:40 -07:00
SuvarnaMeenakshi
3d7b6faf91
[systemd-generator]: Remove creation of NUM_ASIC environment (#5122)
NUM_ASIC environment variable was added so that it could be used by other utilities.
This is not being used by any other utility or docker, hence removing the addition of NUM_ASIC environment variable.
Also, the environment variable was added by adding the variable value to /etc/environ file.
Upon each reboot, this file gets updated with the NUM_ASIC value but the existing value was not removed.
This causes multiple lines getting appended in /etc/environ file upon each reboot.

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
2020-08-11 13:58:20 -07:00
Sumukha Tumkur Vani
9c364fbf93
Bring up all neighbor ports in DeviceInterfaceLinks (#5071)
* Bring up FPGA ports and test it

* Bring up those ports in neighbors dict

* Revert delete of a line

* Add test

* change code comment

* Change test name

* Revert submodule update
2020-08-11 07:29:33 -07:00
Joe LeVeque
2b5e418e2e
Remove sonic-daemon-base package (#5131)
sonic-daemon-base package has been deprecated in favor of the sonic-py-common package. All related functionality has been moved there.
2020-08-09 21:27:36 -07:00
Joe LeVeque
966ac85e24
[sonic-config-engine] Remove sonic_device_util.py (#5130)
sonic_device_util.py has been deprecated in favor of the sonic-py-common package. All related functionality has been moved there.
2020-08-09 21:26:54 -07:00
Joe LeVeque
78baecef80
[sonic-utilities] Update submodule (#5129)
* src/sonic-utilities 9700e45...621aad0 (9):
  > [Python] Migrate applications/scripts to import sonic-py-common package (#1008)
  > [config qos] QoS and Buffer config genration for multi ASIC platforms (#978)
  > [cli]: pass db connector as click context (#1029)
  > [PDDF] Make utilities compatible to platform API 2.0, in accordance with pddf_2.0 (#940)
  > [tests] Add unit tests for 'show platform ...' commands (#1021)
  > Add secure fast/warm-reboot support for Aboot (#994)
  > [sonic-installer] Create Envvars File for Incoming Image (#1011)
  > [config load]: do not stop/reset/reload service if it is disabled (#1028)
  > [tests]: move feature_test.py into tests folder (#1027)
2020-08-08 18:21:12 -07:00
Joe LeVeque
5e62d8d324
[sonic-platform-daemons] Update submodule (#5128)
* src/sonic-platform-daemons 8e0704e...da3dfbb (1):
  > [xcvrd][thermalctld] Add logger instances for helper functions and classes (#82)
2020-08-08 11:33:14 -07:00
Joe LeVeque
567da44a85
[sonic-swss-common] Update submodule (#5126)
* src/sonic-swss-common ed9c497...00bcd1d (6):
  > [warmboot] add disable warmstart timer value (#370)
  > Update .gitignore (#372)
  > Fix swss::exec return value (#368)
  > Revert "DBConnector classes to understand the namespace. (#364)" (#369)
  > DBConnector classes to understand the namespace. (#364)
  > Added new table in APPL_Db for DNAT Pool changes (#367)
2020-08-08 01:14:24 -07:00
Joe LeVeque
6b72dee3d3
[sonic-ztp] Update submodule (#5120)
* src/sonic-ztp c959371...dd025bc (2):
  > Update all references to new 'sonic-installer' file name (#18)
  > Filter out non-printable characters read from syseeprom (#17)
2020-08-07 14:37:46 -07:00
Joe LeVeque
d272802511
[sonic-platform-common][sonic-platform-daemons] Update submodules (#5103)
* src/sonic-platform-common be1cc24...f3f3573 (3):
  > Remove sonic-config-engine dependency from setup.py (#109)
  > Migrate from sonic-daemon-base package to sonic-py-common package (#103)
  > [Cables] Add support for 'Extended Specification Compliance' for QSFP cables (#108)
  > [sfputilbase.py] Add application_advertisement NA field to transceiver_info_dict for platform api 1.0 (#104)

* src/sonic-platform-daemons 49d145c...8e0704e (1):
  > Proper fix for thermalctld using sonic-py-common (#80)
  > Fix thermalctld tests which were broken by the transition to sonic-py-common (#79)
  > Migrate from sonic-daemon-base package to sonic-py-common package (#74)
2020-08-06 16:11:13 -07:00
lguohan
082c26a27d
[build]: combine feature and container feature table (#5081)
1. remove container feature table
2. do not generate feature entry if the feature is not included
   in the image
3. rename ENABLE_* to INCLUDE_* for better clarity
4. rename feature status to feature state
5. [submodule]: update sonic-utilities

* 9700e45 2020-08-03 | [show/config]: combine feature and container feature cli (#1015) (HEAD, origin/master, origin/HEAD) [lguohan]
* c9d3550 2020-08-03 | [tests]: fix drops_group_test failure on second run (#1023) [lguohan]
* dfaae69 2020-08-03 | [lldpshow]: Fix input device is not a TTY error (#1016) [Arun Saravanan Balachandran]
* 216688e 2020-08-02 | [tests]: rename sonic-utilitie-tests to tests (#1022) [lguohan]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-08-05 13:23:12 -07:00
Joe LeVeque
172354d36a
[sonic-py-common] get_platform(): Refactor method of retrieving platform identifier (#5094)
Applications running in the host OS can read the platform identifier from /host/machine.conf. When loading configuration, sonic-config-engine *needs* to read the platform identifier from machine.conf, as it it responsible for populating the value in Config DB.

When an application is running inside a Docker container, the machine.conf file is not accessible, as the /host directory is not mounted. So we need to retrieve the platform identifier from Config DB if get_platform() is called from inside a Docker 
container. However, we can't simply check that we're running in a Docker container because the host OS of the SONiC virtual switch is running inside a Docker container. So I refactored `get_platform()` to:
    1. Read from the `PLATFORM` environment variable if it exists (which is defined in a virtual switch Docker container)
    2. Read from machine.conf if possible (works in the host OS of a standard SONiC image, critical for sonic-config-engine at boot)
    3. Read the value from Config DB (needed for Docker containers running in SONiC, as machine.conf is not accessible to them)

- Also fix typo in daemon_base.py
- Also changes to align `get_hwsku()` with `get_platform()`
2020-08-04 23:51:35 -07:00
Sangita Maity
0a409ff65e
DPB Breakout Cli Test with proper setup (#3910)
- Created the VS setup to test DPB functionality
- Created "platform.json" for VS docker
- Added test case for Breakout CLI

Signed-off-by: Sangita Maity <sangitamaity0211@gmail.com>
2020-08-04 12:07:44 -07:00
Volodymyr Boiko
804b246cbb
[submodule][sonic-platform-daemons] Update submodule (#5091)
[xcvrd] Fix bailing out on platforms that do not support QSFP-DD (#78)
Fix psud and test_thermalctld (#75)
[Thermal] Fix for 'show platform fan' command (#71)

Signed-off-by: Volodymyr Boyko <volodymyrx.boiko@intel.com>
2020-08-04 09:55:02 -07:00
Joe LeVeque
3b89e5d467
[Python] Migrate applications/scripts to import sonic-py-common package (#5043)
As part of consolidating all common Python-based functionality into the new sonic-py-common package, this pull request:
1. Redirects all Python applications/scripts in sonic-buildimage repo which previously imported sonic_device_util or sonic_daemon_base to instead import sonic-py-common, which was added in https://github.com/Azure/sonic-buildimage/pull/5003
2. Replaces all calls to `sonic_device_util.get_platform_info()` to instead call `sonic_py_common.get_platform()` and removes any calls to `sonic_device_util.get_machine_info()` which are no longer necessary (i.e., those which were only used to pass the results to `sonic_device_util.get_platform_info()`.
3. Removes unused imports to the now-deprecated sonic-daemon-base package and sonic_device_util.py module

This is the next step toward resolving https://github.com/Azure/sonic-buildimage/issues/4999

Also reverted my previous change in which device_info.get_platform() would first try obtaining the platform ID string from Config DB and fall back to gathering it from machine.conf upon failure because this function is called by sonic-cfggen before the data is in the DB, in which case, the db_connect() call will hang indefinitely, which was not the behavior I expected. As of now, the function will always reference machine.conf.
2020-08-03 11:43:12 -07:00
Tamer Ahmed
7872b4e196
[platform] Add Support For Environment Variable File (#5010)
* [platform] Add Support For Environment Variable

This PR adds the ability to read environment file from /etc/sonic.
the file contains immutable SONiC config attributes such as platform,
hwsku, version, device_type. The aim is to minimize calls being made
into sonic-cfggen during boot time.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-07-31 17:59:09 -07:00
pavel-shirshov
03d3aa4fc5
Fix for ipv6 local-addr problem (#4876)
Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-07-31 10:40:38 -07:00
pavel-shirshov
e22de3a659
[bgpcfgd]: Clarify error messages on reset Loopback0 ip address (#5062)
To clarify error messages in case the ip address for Loopback is already set. It doesn't make sense to call correct ip address as ambiguous in this case
2020-07-30 02:46:19 -07:00
Joe LeVeque
97f15576fb
[sonic-py-common] Add task_base.py (#5049)
Add new task_base.py module from sonic-daemon-base into sonic-py-common package
2020-07-28 15:07:36 -07:00
gechiang
cdc652148c
[sonic-sairedis] Update Submodule (#5052)
Advanced Submodule head of Sonic-Sairedis and picked up the following changes:

Workaround getSwitchType failure due to NPU SAI not yet supporting it (#647)
https://github.com/Azure/sonic-sairedis/pull/647

Adding support for new SAI API sai_query_attribute_capability() (#645)
https://github.com/Azure/sonic-sairedis/pull/645

[syncd] ARM32 bit fixes, for 64bit printf format specifier. (#643)  
Signed-off-by: Rajkumar Pennadam Ramamoorthy <rpennadamram@marvell.com>
https://github.com/Azure/sonic-sairedis/pull/643

[syncd] Add support for synchronous bulk api (#640)
https://github.com/Azure/sonic-sairedis/pull/640
2020-07-28 14:57:16 -07:00
Mahesh Maddikayala
ee4197e9f8
+ Modified buffer config template to include internal ASIC with 5m ca… (#4959)
+ Modified buffer config template to include internal ASIC with 5m cable length
+ added unit test to verify the changes.
2020-07-27 10:57:07 -07:00
Joe LeVeque
c0d1616f89
Introduce sonic-py-common package (#5003)
Consolidate common SONiC Python-language functionality into one shared package (sonic-py-common) and eliminate duplicate code.

The package currently includes three modules:

- daemon_base
- device_info
- logger
2020-07-26 23:15:41 -07:00
Nazarii Hnydyn
a0ba0436fd
[submodule]: Advance platform common/daemons submodules. (#5033)
**sonic-platform-common:**

Commits on Jul 07, 2020
Changes in fan and psu base classes (1.0 platform API) related to pdd… 17292e4
Commits on Jul 10, 2020
update get_pcie_check() to use sysfs instead of lspci output (#95) d4eb804
Commits on Jul 11, 2020
[eeprom] Fix UnboundLocalError (#93) 7c8bed1
Commits on Jul 14, 2020
[Transceiver] Add parser for QSFP-DD cable type and dictionaries for … be1cc24 

**sonic-platform-daemons:**

Commits on Jul 06, 2020
[psud] Store PSU temperature and voltage information to database (#61) ef9716a
Commits on Jul 14, 2020
[xcvrd] Add support for QSFP-DD cables (#66) c530587
Commits on Jul 18, 2020
Initial version of pcied (#60) e665ee8
[xcvrd] Return non-zero error code on SFP error (#67) 4f42a79
Commits on Jul 19, 2020
[README.md] Add LTGM badges (#69) 23757a3
Remove unused imports (#70) 029d5a5 

Signed-off-by: Nazarii Hnydyn <nazariig@mellanox.com>
2020-07-24 19:24:19 -07:00
Joe LeVeque
639ad1895d
[sonic-utilities] Update submodule (#5023)
* src/sonic-utilities 82dfe50...995cf39 (1):
  > [config] Restart telemetry service upon config (re)load (#992)
2020-07-23 00:42:23 -07:00
shi-su
adacf1addd
[sonic-swss-common] Update submodule (#5020)
Update the sonic-swss-common submodule. The following are the commits in the submodule.

Fix sai api loglevel
ed9c497316e73bc5d4a6dfbe9f3507d5e92c15f3

Add support for attribute capability query in lua script
f58ea1e7a692eadd8695b4dcf89290c5e9018aae

Copp changes
4e50be7728d1e68582ac4dc369323ec5ab7b81dd

fix pytest for python3
c76d3b3a6dacae7fde14622c41e41b717876924a

use sysconfig instead of distutils.sysconfig
1772581f9b9f25e5ab2b790d7d517ea2b2010a78

CONFIG_DB table name for BUM Storm-control feature
c39c958bea445e650353651fcbae5c228114f86d

Fine Grained ECMP schema changes
1974d2e1547464a3b0279b5d63fc40eda94fa780
2020-07-22 22:00:20 -07:00
Joe LeVeque
2890670de2
[sonic-telemetry][sonic-mgmt-common] Update submodules (#4986)
* src/sonic-telemetry fa8d498...3bd7ca3 (4):
  > Update gnmi deps (#40)
  > [testdata] Update SFP keys to align with new standard (#39)
  > Fixed the parameters for subscribe APIs (#38)
  > Azure ro mode (#34)

* src/sonic-mgmt-common 444aa9a...cc01ce4 (4):
  > Make gnmi dep version the same as in telemetry repo (#17)
  > Cleanup translib and cvl go test cases (#13)
  > Package update and enhancements/fixes in YGOT, and Request Binder (#12)
  > Translib phase I changes (#11)

Note: sonic-telemetry submodule update is dependent upon sonic-mgmt-common submodule update, thus updating both in this patch
2020-07-22 11:51:15 -07:00
anish-n
da017f4ec9
[bgpcfgd]: Add Vlan prefix list to the FRR templates (#5005)
add the Vlan prefix list to the FRR templates
2020-07-21 19:26:19 -07:00
anish-n
07d559103f
[bgpcfgd]: Add fix to bgpcfgd to ignore NEIGHBOR_METADATA entries for dynamic peers (#5008)
This fix removes the requirement to have a NEIGHBOR_METADATA for dynamic peers. The change is made since it is not necessary for NEIGHBOR_METADATA entries be present for the dynamic neighbors
2020-07-21 02:55:42 -07:00
Joe LeVeque
6ef199ffdd
[sonic-utilities] Update submodule (#5004)
* src/sonic-utilities a23479e...82dfe50 (3):
  > [sonic-installer] Update group name for 'verify-next-image' subcommand (#/995)
  > [acl_loader] Fix bugs in acl_loader (#991)
  > [show] Add support for QSFP-DD cables on 'show' command (#989)

This submodule update resolves https://github.com/Azure/sonic-utilities/issues/997 by including PR https://github.com/Azure/sonic-utilities/pull/995
2020-07-20 11:21:24 -07:00
lguohan
c6027ab46f
[submodule]: update sonic-swss (#5002)
* d2bab10 2020-07-19 | [vstest]: use BytesIO for file operations [lguohan]
* 982c9a3 2020-07-18 | [vstest]: fix vstest for python3 (#1354) [lguohan]
* ffa0dc3 2020-07-16 | [vstest]: clean up processes in server namespace at start (#1353) [lguohan]
* b16368c 2020-07-15 | [aclorch.cpp] Handle all the ACL redirect requests in AclRuleL3::validateAddAction()  (#1278) [madhanmellanox]
* 310e0aa 2020-07-15 | [portsorch,intfsorch] add port, rif rates FC groups (#1201) [Mykola F]
* 5ddea37 2020-07-09 |  [NAT]: Update NAT conntrack entries from natmgr instead of natorch (#1274) [Akhilesh Samineni]
* 94c622f 2020-07-07 | [aclorch] Use IPv6 Next Header internally for protocol number on MLNX platform (#1343) [Danny Allen]
*   82d36c4 2020-07-07 | support swss vstest in python3 [lguohan]
|\
| * ff04e6d 2020-06-27 | [doc]: update instruction to run vstest under python3 [Guohan Lu]
| * c90b281 2020-07-05 | [vstest]: reuse dvs setReadOnlyAttr in test_crm.py [Guohan Lu]
| * 8807b40 2020-07-05 | [vstest]: fix string format compatibility issue for python2 and swig [Guohan Lu]
| * 26efbcf 2020-07-04 | [vstest]: change float division to integer division [Guohan Lu]
| * 6adaf2e 2020-07-04 | [vstest]: change time.clock() to time.time() [Guohan Lu]
| * 9c71203 2020-07-04 | [vstest]: let redis decodes the response to be string instead of bytes [Guohan Lu]
| * c7c63ee 2020-06-27 | [vstest]: change from platform to distro [Guohan Lu]
| * d7ff1ad 2020-06-27 | [vstest]: upgrade swss vs tests to python3 [Guohan Lu]
* | 2ebd44e 2020-07-03 | [sonic-swss] ARMHF warning fixes (#1325) [arheneus@marvell.com]
* | 10ad70c 2020-07-02 | [swss] Add support for gearbox phys (#1321) [Syd Logan]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-07-20 02:59:55 -07:00
vdahiya12
c9983df062
[daemon_base] fix to not reregister signal handler (#4998)
* [daemon_base] fix to not reregister signal handler

-src/sonic-daemon-base/sonic_daemon_base/daemon_base.py
Problem:
Currently all daemons inherit from daemon_base class, and for
signal handling functionality they register the signal_handler() by
overriding the siganl_handler() in daemon_base by their own
implmentation.
But some sonic_platform instances also can invoke the daemon_base
constructor while trying to instantiate the common utilities
for example
platform_chassis = sonic_platform.platform.Platform().get_chassis()
This will cause the re registration of signal_handler which will
cause base class signal_handler() to be invoked when the daemon
gets a signal, whereas their own signal_handler should have been
invoked.

Fix:
We only register the siganl_handler once, and if signal_handler has
been registered, not re register it.

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>

* [daemon_base] fix to not reregister signal handler

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
2020-07-19 20:58:52 -07:00
Stepan Blyshchak
16a37d8c17
[dockers] update mellanox syncd and pmon to buster (#4818)
Upgrade to libsensors5

Updated sonic-sairedis pointer:
    d54bfb4 [SAI] update pointer (#636)
    1885a8c [syncd] Fix notification on shutdown request (#635)
    9e57ba2 Fixing hostif For Genetlink host interfaces (#633)
    449a092 sonic-sairedis: Add support to sonic-sairedis for gearbox phys (#632)

Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2020-07-18 03:46:15 -07:00