Commit Graph

3888 Commits

Author SHA1 Message Date
ajbalogh
a9fc866ba5
dockers/docker-sonic-mgmt/Dockerfile.js: Add keysight ixnetwork-open-… (#5762)
*Why I did it
Provide access to the ixnetwork-open-traffic-generator pypi package

*How I did it
Added a pip install entry for the package in the Dockerfile.j2

*How to verify it
pip show ixnetwork-open-traffic-generator

*Description for the changelog
Install of ixnetwork-open-traffic-generator pypi package is required for proposed rdma tests.
2020-11-09 14:12:39 -08:00
Ying Xie
6a8ebef449
[Arista 7050cx3] add dummy MMU configurations for Arista-7050CX3-32S-C32 (#5798)
- Why I did it
On Arista-7050CX3-32S-C32, there are constant stream of errors like following.

Nov 3 21:56:24.415190 str2-7050cx3-acs-06 NOTICE swss#orchagent: :- registerInWdDb: No lossless TC found on port Ethernet68

Which causes:

loganalyzer to claim test failed.
leaving the system without MMU configuration. Which couldn't be good for any IO test.
- How I did it
Added these MMU configuraions are copied from another platform and guaranteed to be incorrect for hwsku Arista-7050CX3-32S-C32.

Adding them so that we have A MMU configuration and the system won't throw a whole bunch of errors and leave MMU unconfigured. The correct MMU configuration will come later.

This configuration is definitely not suitable for testing system performance or QoS behavior.

Signed-off-by: Ying Xie ying.xie@microsoft.com

- How to verify it
Test will have chance to pass. Ran a few test that would fail otherwise.
2020-11-09 13:27:36 -08:00
abdosi
4f82463670
[multi-asic] Fixed the docker mount point check for multi-asic (#5848)
API getMount() API was not updated to handle multi-asic platforms
Updated API getMount() to return abspath() for Docker Mount Point
and use that one for mount point comparison

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2020-11-09 13:03:00 -08:00
abdosi
84aa99d04b
[multi-asic] teamdctl support for multi-asic (#5851)
Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2020-11-09 12:31:33 -08:00
judyjoseph
c972052594
[multi-ASIC] util changes with the BGP_INTERNAL_NEIGHBOR table. (#5760)
- Why I did it
Update the routine is_bgp_session_internal() by checking the BGP_INTERNAL_NEIGHBOR table.
Additionally to address the review comment #5520 (comment)
Add timer settings as will in the internal session templates and keep it minimal as these sessions which will always be up.
Updates to the internal tests data + add all of it to template tests.

- How I did it
Updated the APIs and the template files.

- How to verify it
Verified the internal BGP sessions are displayed correctly with show commands with this API is_bgp_session_internal()
2020-11-09 11:10:10 -08:00
Joe LeVeque
e0fdf45ad0
[update_chassisdb_config] Convert to Python 3 (#5838)
- Convert update_chassisdb_config script to Python 3
- Reorganize imports per PEP8 standard
- Two blank lines precede functions per PEP8 standard
2020-11-09 08:35:36 -08:00
Longxiang Lyu
385dfc4921
[monit] Fix status error due to shebang change (#5865)
lldpmgrd, bgpcfgd, and bgpmon are reported error status not running due
to recent change of shebang to use `Python3`. Modifying the argument of
`process_checker` to follow this change.

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>
2020-11-09 01:52:22 -08:00
Guohan Lu
ad2e18e856 [baseimage]: install psutil for python3
psutil is needed by process_checker which is using python3

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-11-09 00:29:10 -08:00
Volodymyr Samotiy
3fa724d216
[submodule] Update sonic-swss submodule (#5825)
62e6c5a [bitmap_vnet] Remove BMTOR implementation (#1494)
582beae [FPMSYNCD] EVPN Type5 prefix handling in FPMSYNCD (#1326)
d9ca6e2 [vstest] Resolve intermittent timing failure in test_fgnhg.py (#1495)
a89b8ed [sub intf] Inherit parent port mtu changes (#1479)

Signed-off-by: Volodymyr Samotiy <volodymyrs@nvidia.com>
2020-11-08 23:57:50 -08:00
Blueve
fb6af9d6d7
[broadcom][cel][udev] Update console usb devices permission (#5840)
* Add MODE "0666" to all console usb tty

Signed-off-by: Jing Kan jika@email.com
2020-11-09 08:54:21 +08:00
Longxiang Lyu
92704e8a93
[bgpmon] Fix dict_keys unsubscriptable error (#5859)
Simply looping through `dict_key` object instead of index-subscribing to
fix this.

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>
2020-11-08 13:42:38 -08:00
Garrick He
9fda295cd0
[sflow] Clean-up sFlow container and port_index_mapper.py script (#5846)
* Fix some spelling error and add addition check in port_index_mapper.py script.
* Remove unneeded pyroute2 python module from sFlow container

Signed-off-by: Garrick He <garrick_he@dell.com>
2020-11-07 20:23:01 -08:00
Lawrence Lee
23b0e07d48
[minigraph.py]: Parse IP-in-IP tunnels from minigraph (#5742)
Take tunnel info from `<TunnelInterface>` tag in minigraph, and create tables in config_DB:

```
"TUNNEL": {
    "MUX_TUNNEL_0": {
        "tunnel_type": "IPINIP",
        "dst_ip": "26.1.1.10",
        "dscp_mode": "uniform",
        "encap_ecn_mode": "standard",
        "ecn_mode": "copy_from_outer",
        "ttl_mode": "pipe"
    }
}
```

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2020-11-07 13:18:40 -08:00
Praveen Chaudhary
6156cb2805
[sonic-yang-mgmt] Build PY3 & PY2 packages (#5559)
Moving sonic-yang-mgmt to PY3 to support move of sonic-utilities to PY3.

Signed-off-by: Praveen Chaudhary<pchaudhary@linkedin.com>
2020-11-07 13:03:41 -08:00
Joe LeVeque
04d0e8ab00
[hostcfgd] Convert to Python 3; Add to sonic-host-services package (#5713)
To consolidate host services and install via packages instead of file-by-file, also as part of migrating all of SONiC to Python 3, as Python 2 is no longer supported.
2020-11-07 12:48:19 -08:00
Joe LeVeque
9e7e092610
[Monit process_checker] Convert to Python 3 (#5836)
Convert process_checker script to Python 3
2020-11-07 12:46:23 -08:00
Petro Bratash
32a832a8ac
[lldp]: Add verification IPv4 address on LLDP conf Jinja2 Template (#5699)
Fix #5812

LLDP conf Jinja2 Template does not verify IPv4 address and can use IPv6 version. This issue does not effect control LLDP daemon. Issue can be reproduced via `test_snmp_lldp` test. LLDP conf Jinja2 Template selects first item from the list of mgmt interfaces.

TESTBED_1 LLDP conf

```
# cat /etc/lldpd.conf 
configure ports eth0 lldp portidsubtype local eth0
configure system ip management pattern FC00:3::32
configure system hostname dut-1
```
TESTBED_2  LLDP conf

```
# cat /etc/lldpd.conf
configure ports eth0 lldp portidsubtype local eth0
configure system ip management pattern 10.22.24.61
configure system hostname dut-2
```
TESTBED_1  MGMT_INTERFACE

```
$ redis-cli -n 4 keys "*" | grep MGMT_INTERFACE
MGMT_INTERFACE|eth0|10.22.24.53/23
MGMT_INTERFACE|eth0|FC00:3::32/64
```
TESTBED_2  MGMT_INTERFACE

```
$ redis-cli -n 4 keys "*" | grep MGMT_INTERFACE
MGMT_INTERFACE|eth0|FC00:3::32/64
MGMT_INTERFACE|eth0|10.22.24.61/23

```

Signed-off-by: Petro Bratash <petrox.bratash@intel.com>
2020-11-07 10:30:41 -08:00
lguohan
e6796da141
[init_cfg.json.j2]: only enable gbsyncd feature for vs platform (#5815)
currently only vs platform has gdbsyncd feature built

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-11-07 00:46:18 -08:00
pavel-shirshov
cdcd20a7b5
[BGP]: Convert ip address to network address for the LOCAL_VLAN filter (#5832)
* [BGP]: Convert ip address to network address for the LOCAL_VLAN prefix filter
2020-11-06 17:47:08 -08:00
dflynn-Nokia
174b741dfd
[submodule]: update sonic-linux-kernel (#5829)
pick up follow commits:

  > hwmon-lm75: backport support for PCT2075 thermal sensor (#165)
  > [arista] Reassign prefetch memory per platform (#163)
  > Add .gitignore file (#157)
  > Enable PCA9541 I2C mux module (#160)
2020-11-06 12:37:41 -08:00
Stepan Blyshchak
9bc693ce6e
[hostcfgd] If feature state entry not in the cache, add a default state (#5777)
Our use case is to register new features in runtime. The previous change which introduced the cache broke this capability and caused hostcfgd crash.

Signed-off-by: Stepan Blyshchak <stepanb@nvidia.com>
2020-11-06 10:24:31 -08:00
Joe LeVeque
ad555d9ffd
[restore_nat_entries.py] Convert to Python 3 (#5788)
- Convert restore_nat_entries.py script to Python 3
- Use logger from sonic-py-common for uniform logging
- Reorganize imports alphabetically per PEP8 standard
- Two blank lines precede functions per PEP8 standard
2020-11-06 10:15:49 -08:00
Garrick He
27a911f16e
[sflow] Fix port_index_mapper.py script; Convert to Python 3 (#5800)
**- Why I did it**
A memory issue was discovered during system test for scaling. The issue is documented here: https://docs.pyroute2.org/ipdb.html

> One of the major issues with IPDB is its memory footprint. It proved not to be suitable for environments with thousands of routes or neighbours. Being a design issue, it could not be fixed, so a new module was started, NDB, that aims to replace IPDB. IPDB is still more feature rich, but NDB is already more fast and stable.

**- How I did it**
- Rewrote the port_index_mapper.py script to use dB events.
- Convert to Python 3
2020-11-06 10:15:06 -08:00
Joe LeVeque
51292330e9
[enable_counters.py] Convert to Python 3 (#5789)
**- Why I did it**

As part of moving all SONiC code from Python 2 (no longer supported) to Python 3

**- How I did it**

- Convert enable_counters.py script to Python 3
- Reorganize imports per PEP8 standard
- Two blank lines precede functions per PEP8 standard
2020-11-06 09:00:19 -08:00
Danny Allen
cea364aa77
[utilities] Update sonic-utilities submodule (#5814)
Add a switch to route_check to control whether write log to syslog (#1215)
[consutil] Store console port state in STATE_DB (#1208)
[sfputil] Display 'N/A' for non-SFP ports (#1078)
Show FG_NHG CLI Commands Added (#1056)
[show] remove 'device2interface_dict' in show interface neighbor expected (#1188)
Add license file, remove third-party licenses file (#1192)
[fwutil]: Set min log priority to INFO. (#1191)
[CLI][show][platform] Added ASIC count in the output. (#1185)
fdbshow and nbrshow use SonicV2Connector with decode_responses=True, and remove all the decode() (#1187)
Remove stdeb.cfg; no longer used now that we build as wheel (#1182) 
[counterpoll] Disable Counter Poll When Entering Fast Reboot (#1174) 
Fixes the issue with show interface counters and for pfc and queue counters. (#1180)
[config/show] Add CLI support for proxy arp (#1168)
[consutil] Add brief option to show line command (#1176)
Modify fast-reboot script to use BGP service script to stop bgp service (#1177)
[config/console] Support update console configuration related commands (#1166)
[consutil] Fix issue where the show line command crash if no ttyUSB exists (#1173)
[watermarkstat] Add unit tests for watermarkstat show commands (#1157)
Fix exception for ipaddress in python2 (#1164) 
[celestica] consutil to support customize tty device name (#1155)

Signed-off-by: Danny Allen <daall@microsoft.com>
2020-11-06 00:58:49 -08:00
Joe LeVeque
13ff7b38d5
[docker-wait-any] Convert to Python 3, install dependency in host OS (#5784)
- Convert docker-wait-any script to Python 3
- Install Python 3 Docker Engine API in host OS
2020-11-05 11:23:00 -08:00
Joe LeVeque
d8045987a6
[core_uploader.py] Convert to Python 3; Use logger from sonic-py-common for uniform logging (#5790)
- Convert core_uploader.py script to Python 3
- Use logger from sonic-py-common for uniform logging
- Reorganize imports alphabetically per PEP8 standard
- Two blank lines precede functions per PEP8 standard
- Remove unnecessary global variable declarations
2020-11-05 11:19:26 -08:00
Joe LeVeque
2c9454e86a
[sonic-device-data] Convert Python files to Python 3 (#5816)
- Convert config_checker, media_checker and platform_json_checker scripts to Python 3
- Reorganize imports per PEP8 standard
- Two blank lines precede functions per PEP8 standard
2020-11-05 11:17:08 -08:00
pavel-shirshov
13f8e9ce5e
[bgpcfgd]: Convert bgpcfgd and bgpmon to python3 (#5746)
* Convert bgpcfgd to python3

Convert bgpmon to python3
Fix some issues in bgpmon

* Add python3-swsscommon as depends

* Install dependencies

* reorder deps

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-11-05 10:01:43 -08:00
Joe LeVeque
522a071ffb
[core_cleanup.py] Convert to Python 3; Fix bug; Improve code reuse (#5781)
- Convert to Python 3
- Fix bug: `CORE_FILE_DIR` previously was set to `os.path.basename(__file__)`, which would resolve to the script name. Fix this by hardcoding to `/var/core/` instead
- Remove locally-define logging functions; use Logger class from sonic-py-common instead
2020-11-05 10:01:12 -08:00
Nazarii Hnydyn
160544f5b1
[Mellanox] Update SAI to v.1.17.6. (#5767)
Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>
2020-11-05 14:52:50 +02:00
Praveen Chaudhary
1f9132db60
[sonic-yang]: Yang model update fields (#5577)
Changes:
-- ACL stage allow lower and upper.
-- ACL include services leaf-list.
-- PORT include pfc_asym leaf.
-- PORT fec alloe none as per code.
-- 3 Tests for above changes.

Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com
2020-11-05 00:25:33 -08:00
Roy Lee
ce6286eb84
[device/accton] Remove the use of python pickle package (#5475)
Pickle is applied to save the order of i2c adapters at installation.
With pickle removed, it just checks the order of i2c buses every time it needs.
2020-11-04 16:24:53 -08:00
Joe LeVeque
d3262d10f7
[generate_asic_config_checksum.py] Convert to Python 3 (#5783)
- Convert script to Python 3
    - Need to open file in binary mode before hashing due to new string data type in Python 3 being unicode by default. This should probably have been done regardless.
- Reorganize imports alphabetically
- When running the script, don't explicitly call `python`. Instead let the program loader use the interpreter specified in the shebang (which is now `python3`).
2020-11-04 15:06:44 -08:00
Joe LeVeque
ba7fda7fd1
[docker-platform-monitor] Install Python 2 'enum34' package to fix Arista platforms (#5779)
Recent changes to dependencies caused the 'enum34' package to cease being installed for Python 2 in the PMon container. This broke Arista platforms, where the Arista sonic_platform package imports 'enum'. This is because on Arista devices, the sonic_platform wheel is not installed in the container. Instead, the installation directory is mounted from the host OS. However, this method doesn't ensure all dependencies are installed in the container.
2020-11-04 11:23:03 -08:00
Kalimuthu-Velappan
194c7a97cd
[build]: Fix for - execvp: /bin/bash: Argument list too long error (#5809)
Fix #5692
It fixes the folling error when DPKG cache enabled.

make: execvp: /bin/bash: Argument list too long
make: *** [slave.mk:389: target/debs/buster/sonic-device-data_1.0-1_all.deb] Error 127

- Why I did it
Number of dependent file list is more than 2K which causes bash arg long error.

- How I did it
Used file operation to store the dependency list.
2020-11-04 11:12:26 -08:00
dflynn-Nokia
ac3a605c75
[build]: ARM build: Download redis-tools and redis-server from sonicstorage (#5797)
Prevent intermittent build failures when building Sonic for the ARM platform architecture due to version upgrades of the redis-tools and redis-server packages.

Modify select Dockerfile templates to download the redis-tools and redis-server packages from sonicstorage rather than from debian.org.

This PR has been made possible by the inclusion of ARM versions of redis-tools and redis-server into sonicstorage as described in Issue# 5701
2020-11-04 09:31:06 -08:00
Qi Luo
4ed2ff805f
[swss-common]: Advance submodule (#5799)
ec96868 2020-11-03 | Fix: treat DBConnector timeout=0 as infinite timeout (#408) [Qi Luo]
b4b8334 2020-11-03 | Add lua script for redis multi keys api hmset and del (#406) [Kamil Cudnik]
2020-11-04 05:49:52 -08:00
Qi Luo
215ce13890
[swss-common]: Advance submodule (#5780)
9b0e955 2020-10-30 | [schema]: Add MACsec support (#403) [Ze Gan]
f24dc97 2020-10-28 | Implementation of System ports initialization, Interface & Neighbor Synchronization... (#380) [minionatwork]
91e0885 2020-10-27 | Mux cable schema definitions (#398) [Prince Sunny]
d0cedea 2020-10-27 | Change log level (#402) [Prince Sunny]
64b3cfe 2020-10-24 | SonicV2Connector supports host and decode_responses in constructor parameters (#401) [Qi Luo]
f8b0065 2020-10-23 | Implement FieldValueMap update method (#400) [Qi Luo]
2020-11-03 14:28:42 -08:00
Joe LeVeque
e3164d5fb4
[lldpmgrd] Convert to Python 3 (#5785)
- Convert lldpmgrd to Python 3
- Install Python 3 swsscommon package in docker-lldp
2020-11-03 12:50:11 -08:00
Andriy Kokhan
0a1c5792a1
[BFN] Updated SDK packages to 20201023 (#5708)
- BFN platform was affected by ACL changes that add IPV6_NEXT_HEADER support.
- Bugfixes

Signed-off-by: Andriy Kokhan <akokhan@barefootnetworks.com>
2020-11-03 11:26:26 -08:00
Joe LeVeque
84d3a26000
[sonic-py-swsssdk] Update submodule (#5757)
Commits included:

* src/sonic-py-swsssdk 748c404...1ea30d2 (1):
  > Fix bug: ConfigDBConnector.get_table does not work in python3 (#92)
2020-11-02 20:31:14 -08:00
Kebo Liu
1158701edc
add pcied config files for mellanox platform (#5669)
This PR has a dependency on community change to move PCIe config files from $PLATFORM/plugin folder to $PLATFORM/ folder
- Why I did it
To support PCIed daemon on Mellanox platforms
- How I did it
Add PCIed config yaml files for all Mellanox platforms
Update pmon daemon config files for SimX platforms
2020-11-02 19:45:36 -08:00
lguohan
9d7355287f
[submodule]: update sairedis (#5772)
* b458e6f 2020-11-02 | [syncd] Disable use bulk api by default (#688) (HEAD, origin/master, origin/HEAD) [Kamil Cudnik]
* d978789 2020-11-01 | [syncd] Set modify redis flag in RedisSelectableChannel (#689) [Kamil Cudnik]
* 5df11f5 2020-11-01 | [syncd] Lower bulk missing api message level from error to info (#687) [Kamil Cudnik]
* dc73a1d 2020-10-30 | [saiplayer] Fix log messages (#686) [Kamil Cudnik]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-11-02 16:59:47 -08:00
Lawrence Lee
10ab46f7a0
Revert "[docker-base]: Rate limit priority INFO and lower in syslog" (#5763)
* This was a temporary fix for orchagent spamming log messages and causing rate limiting, leading to critical messages being dropped for the syslog. No longer needed since Azure/sonic-sairedis#680 was merged.
2020-11-02 08:49:40 -08:00
Junchao-Mellanox
1be9c4a33a
[Mellanox] Update SDK 4.4.1956 and FW *.2008.1956 (#5768)
* [Mellanox] Update SDK 4.4.1956 and FW *.2008.1956

* Update submoudle pointer for Switch-SDK-drivers
2020-11-02 09:56:02 +02:00
lguohan
98d370bcef
[submodule]: swss/sairedis module update (#5765)
swss:

* d7643f2 2020-11-01 | [tlm_teamd]: Make the destionation for std::transform() to use std::back_inserter() for allocating new space for the copied objects (#1490) (HEAD, origin/master, origin/HEAD) [pavel-shirshov]
* 7fa7cd6 2020-10-31 | [vstest]: stablize fgnhg test (#1491) [lguohan]
* 9b0696e 2020-10-29 | Create vnet tunnel map only if it doesn't exist (#1482) [Prince Sunny]
* 0481e99 2020-10-29 | [acl] Update CRM to include LAG bindings for ACL tables (#1487) [Danny Allen]

sairedis

* 5df11f5 2020-11-01 | [syncd] Lower bulk missing api message level from error to info (#687) (HEAD, origin/master, origin/HEAD) [Kamil Cudnik]
* dc73a1d 2020-10-30 | [saiplayer] Fix log messages (#686) [Kamil Cudnik]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-11-01 20:08:55 -08:00
Blueve
698b5544c9
[openssh] Introduce custom openssh-server package for supporting reverse console SSH (#5717)
* Build and install openssh from source
* Copy openssh deb package to dest folder
* Update make rule
* Update sonic debian extension
* Append empty line before EOF
* Update openssh patch
* Add openssh-server to base image dependency
* Fix indent type
* Fix comments
* Use commit id instead of tag id and add comment

Signed-off-by: Jing Kan jika@microsoft.com
2020-11-02 10:31:15 +08:00
Joe LeVeque
f2a258aca9
[docker-platform-monitor] Check if sonic_platform is available before installed (#5764)
On Arista platforms, sonic_platform packages are not installed in the PMon container, but are rather mounted into the container from the host OS. Therefore, pip show sonic_platform will fail in the PMon container. This change will first check if we can import sonic_platform. If this fails, it will then fall back to checking if the package is installed. If both fail, it will attempt to install the package.
2020-11-01 01:48:07 -08:00
lguohan
c8a00eda95
[mgmt ip]: mvrf ip rule priority change to 32765 (#5754)
Fix Azure/SONiC#551

When eth0 IP address is configured, an ip rule is getting added for eth0 IP address through the interfaces.j2 template. 

This eth0 ip rule creates an issue when VRF (data VRF or management VRF) is also created in the system.
When any VRF (data VRF or management VRF) is created, a new rule is getting added automatically by kernel as "1000: from all lookup [l3mdev-table]".
This l3mdev IP rule is never getting deleted even if VRF is deleted.

Once if this l3mdev IP rule is added, if user configures IP address for the eth0 interface, interfaces.j2 adds an eth0 IP rule as "1000:from 100.104.47.74 lookup default ". Priority 1000 is automatically chosen by kernel and hence this rule gets higher priority than the already existing rule "1001:from all lookup local ".

This results in an issue "ping from console to eth0 IP does not work once if VRF is created" as explained in Issue 551.
More details and possible solutions are explained as comments in the Issue551.

This PR is to resolve the issue by always fixing the low priority 32765 for the IP rule that is created for the eth0 IP address.
Tested with various combinations of VRF creation, deletion and IP address configuration along with ping from console to eth0 IP address.

Co-authored-by: Kannan KVS <kannan_kvs@dell.com>
2020-10-31 20:45:59 -07:00