[DPB][MLNX][YANG] fixed range of max speed
- Why I did it
All Mellanox platforms require DPB modes with a specific set of speeds example
- How I did it
Extended regex pattern inside YANG model.
Supported platforms: SN2010, SN2100, SN2410, SN2700, SN3420, SN3700, SN3700C, SN3800, SN4600C, SN4410, SN4700
- How to verify it
Manually tested DPB CLI on all platform with all modes
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
Avoid sonic-cfggen crashing when a server does not have a configured loopback address in the minigraph
Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
- Why I did it
Management framework (CVL) needs to call lyd_check_mandatory_tree() for validation and hence exported lyd_check_mandatory_tree() as an API.
- How I did it
Added "API" keyword before lyd_check_mandatory_tree() definition.
- How to verify it
There is no functionality code change here and no specific steps to verify it. Management framework (CVL) should be able to call this function and no patching and compilation error should be seen.
- Description for the changelog
Added "API" keyword before export lyd_check_mandatory_tree() function definition.
**- Why I did it**
Prior to SONiC using Debian Buster, we needed to build Python 3.5 or newer from source for installation in the SNMP container, becuase it wasn't available from the Debian repository for Jessie or Stretch. Now that all containers are based on Buster, we simply install Python 3.7 from the Debian repository in the host as well as all containers. We are no longer building Python 3 from source, so the Makefile is unused and we no longer need to install build dependencies in the slave containers.
**- How I did it**
- Remove Python 3 makefile
- No longer install Python 3 build dependencies in the slave containers.
Update sonic-linux-kernel pointer to pick up new commits:
- Backport patches to increase critical threshold for ASIC and validate transceiver temperature a7c1af7c44edde90dff49d672071139043bcdb65 548e8e0be4
- [ci]: Set up CI with Azure Pipelines 548e8e0be49692050ea4071d5e9945816bc5aacc a7c1af7c44
Signed-off-by: Kebo Liu <kebol@nvidia.com>
* Fix py3 version changed even version control enabled issue
* Add some comments and simplify the script
* Add the comment to explain how to get the not hooked command
Server IPv4 loopbacks do not always arrive with /32 prefix, which is a requirement for the MUX_CABLE table in config DB
Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Fix#119
when parallel build is enable, multiple dpkg-buildpackage
instances are running at the same time. /var/lib/dpkg is shared
by all instances and the /var/lib/dpkg/updates could be corrupted
and cause the build failure.
the fix is to use overlay fs to mount separate /var/lib/dpkg
for each dpkg-buildpackage instance so that they are not affecting
each other.
Signed-off-by: Guohan Lu <lguohan@gmail.com>
To make the peer switch hostname easily accessible from config DB. Add peer_switch field to DEVICE_METADATA table
Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
The HLD about MACsec feature is at :
https://github.com/Azure/SONiC/blob/master/doc/macsec/MACsec_hld.md
- How to verify it
This PR doesn't set MACsec container automatically start, You should manually start the container by docker run docker-macsec
wpa_supplicant binary can be found at MACsec container.
This PR depends on the PR, WPA_SUPPLICANT, and The MACsec container will be set as automatically start by later PR.
Signed-off-by: zegan <zegan@microsoft.com>
* Use 20 and 30 route-map entries instead of 2 and 3 for TSA
* Added support for dynamic "Allow list" default action.
Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
Pass universal_newlines=True parameter to subprocess.Popen(); no longer use .encode('utf-8') on resulting stdout.
This was missed in #5886
Note: I would prefer to use text=True instead of universal_newlines=True, as the former is an alias only available in Python 3 and is more understandable than the latter. However, Even though the setup.py file for this package only specifies Python 3, the LGTM tool finds other Python 2 code in the repo and validates the code as Python 2 code and alerts that text=True is an invalid parameter. Will stick with universal_newlines=True for now. Once all Python code in the repo has been converted to Python 3, I will change all universal_newlines=True to text=True.
src/sonic-platform-common 9935fca...8664efc (2):
Make sonic_sfp Python2 and Python3 compatible (#157)
[sffbase.py] Fix to make Python 3-compatible (#156)
src/sonic-platform-daemons e6c786b...81318f7 (1):
[psud] Fix issue where PSU Fan info is not updated in State DB (#137)
Fixes#6341
To incldue following changes:
- [ci]: add build for arm64 and armhf (#757)
- Use template hgetall, because we will tune the return types of library functions (#759)
- [syncd] Fix bulk multi attrs for same key db update (#761)
Signed-off-by: Ying Xie <ying.xie@microsoft.com>
- Why I did it - Reboot cause prints "Non-Hardware (N/A)" instead of showing the software reboot cause.
The issue is mishandling of hardware reboot cause in determine-reboot-cause script.
- How I did it
Fixed the handling for Non Hardware reboot cause. Ignore if Non-Hardware is present in the hardware_reboot_cause output. Added some code refactoring for simplicity.
- How to verify it - With fix, the hardware reboot cause is ignored (if it is non hw):
The details are as follows:
1. 0010-Ported-fix-for-length-exceeded-from-lldp-community.patch
Patch taken from 78243478dc
lib: remove limit on system description length
The limit was introduced in 9c49ced while fixing a memory leak.
The state data is used to ensure we don't interleave operations. We
need to handle the case where the value is truncated because it is
larger than the allocated size.
Fix issue https://github.com/lldpd/lldpd/issues/408
2. 0011-fix-med-location-len.patch
Patch taken from 5c3479463a
lib: fix LLDP-MED location parsing in liblldpctl
Some bounds were not checked correctly when parsing LLDP-MED civic
location fields. This triggers out-of-bound reads (no write) in
lldpcli, ultimately leading to a crash.
Fix https://github.com/lldpd/lldpd/pull/420
Signed-off-by: sudhanshukumar22 <sudhanshu.kumar@broadcom.com>
In the emulated armhf environment, the function readdir()returns NULL on a ext4 file system directory. When running the libyang1 test cases, it will require to load the plugins from the files (such as metadata.so), because the readdir() is failing, the plugins can’t be loaded in the emulated armhf environment, so it causes libyang1 test error. This error is a combination of the following reasons.
• Emulation of a 32-bit target from a 64-bit host –> qemu from x86_64 to armhf
• Glibc version > 2.27 – Debian buster is using glibc 2.28
- How I did it
Enabled large file support by setting _FILE_OFFSET_BITS=64 for libyang1.
Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>
Including commits in sonic-swss-common repo:
b423b9c Add support for hexists call (#432) [Tony Titus]
0982996 Remove extension of tableNameSeparatorMap (#430) [Qi Luo]
d16cc76 [build]: add azure pipeline build badge (#429) [lguohan]
f2aaf55 Set up CI with Azure Pipelines (#428) [lguohan]
What: Modify unit test to not call any platform dependent api in test_find_hardware_reboot_cause.
- Why I did it
MELLANOX build is failing for the recent PRs. The errors are due to platform library being invoked in a unit test for determine-reboot-cause script.
Verified by running unit tests and a successful Mellanox build.
Co-authored-by: Vaibhav Hemant Dixit <vadixit@microsoft.com>
Why I did it
To support FG_ECMP scenarios
- How I did it
Modified minigraph parser to parse ECMP fields in the case they are present in minigraph
- How to verify it
Loaded ensuing config_db file on a DUT to verify the fields are parsed and configure device correctly
Mellanox already supports multiple destination IPs in IPinIP tunnel configuration, thus removing mellanox
exception for IPinIP configuration.
- How I did it
Removed "dst_ip" field generation in mellanox platform condition.
Sorted the "dst_ip" list, so that it is easier to test against sample configuration in unit tests.
Aligned unit test sample.
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
frr does not advertise route if local route is not reachable, as a result
loopback route /64 is not advertised to the neighbors. Add static route
allows frr to advertise the route to its peers
Signed-off-by: Guohan Lu <lguohan@gmail.com>
Introduce tunnel manager daemon. Start the process as part of swss container
Submodule update for swss:
9ed3026 - 2020-12-24 : [NAT] ACL Rule with DO_NOT_NAT action is getting failed. (#1502) [Akhilesh Samineni]
c39a4b1 - 2020-12-23 : Mux/IPTunnel orchagent changes (#1497) [Prince Sunny]
bc8df0e - 2020-12-23 : Add support for headroom pool watermark (#1567) [Neetha John]
Changes:
-- Add Default value for leaf-lists in ACL and portchannel.
-- Remove members in VLAN.
-- Test modifications.
Signed-off-by: Praveen Chaudhary<pchaudhary@linkedin.com>
* src/sonic-swss c7ee75f...cadf28f (24):
> Revert "Add support for headroom pool watermark (#1453)"
> [VxlanOrch] pytest for EVPN VXLAN (#1318)
> [restore_neighbors] python3 support for restore_neighbors.py (#1542)
> [buffermgmt] more build error fixes when compiling for armhf (32-bit) (#1559)
> Sflow fix to avoid NULL in field. (#1531)
> [fgnhgorch] Fg Nhg link handling (#1537)
> [dpb]: make sure port is in admin down state before remove port. (#1513)
> [FPMSYNCD/FDBSYNCD] EVPN Type-5 route removing prefix-len for host route and removing junk character present in the mac (#1553)
> Added support for EVPN L3 VXLAN as described in the PR Azure/SONiC#437 (#1267)
> [crm]: Typecast to unit64_t to avoid divide by 0 during overflow (#1550)
> [vxlanmgr] Fix build error when compiling for armhf (32-bit) (#1552)
> [Dynamic buffer calc] Support dynamic buffer calculation (#1338)
> [dvs] Clean-up dvs_database and dvs_common (#1541)
> [VxlanMgr] changes for EVPN VXLAN (#1266)
> Statistics support for Tx and Rx counters of different frame sizes (#1536)
> [orchagent/phy]: Add firmware info propagation (#1540)
> [vxlanorch] Use PRI instead of %l to avoid warnings in 32-bit arch (#1539)
> [FDBSYNCD] Added support for EVPN as described in the PR Azure/SONiC#437 (#1276)
> [everflow] Add retry mechanism for mirror sessions and policers (#1486)
> Enable ACL table type mirror_v6 for Innovium Platform (#1527)
> [fgnhgorch] Change format specifier %lu to %zu for size_t (#1529)
> [dvs] Fix issue where concurrent netns operations cause test setup to fail (#1535)
> Add support for headroom pool watermark (#1453)
> Change gAsicInstance to type string with max length limit (#1526)
importlib-resources v4.0.0 was released today (2020-12-23) and drops support for Python 2. This caused the sonic-config-engine Python 2 wheel build to fail.
Reference: https://pypi.org/project/importlib-resources/
Pin 'importlib-resources' package to v3.3.1 for Python 2
Unrelated: remove pinned version of zipp for sonic-bgpcfgd because we no longer build a Python 2 version of that package
* First cut image update for kubernetes support.
With this,
1) dockers dhcp_relay, lldp, pmon, radv, snmp, telemetry are enabled
for kube management
init_cfg.json configure set_owner as kube for these
2) Each docker's start.sh updated to call container_startup.py to register going up
As part of this call, it registers the current owner as local/kube and its version
The images are built with its version ingrained into image during build
3) Update all docker's bash script to call 'container start/stop/wait' instead of 'docker start/stop/wait'.
For all locally managed containers, it calls docker commands, hence no change for locally managed.
4) Introduced a new ctrmgrd service, that helps with transition between owners as kube & local and carry over any labels update from STATE-DB to API server
5) hostcfgd updated to handle owner change
6) Reboot scripts are updatd to tag kube running images as local, so upon reboot they run the same image.
7) Added kube_commands.py to handle all updates with Kubernetes API serrver -- dedicated for k8s interaction only.
- Why I did it
Latest master image crashes when loading minigraph
Fixing #6265
- How I did it
Avoid converting 'None' to ipaddress.
- How to verify it
On a system crashing with the issue, manually patch minigraph.py with the change in PR and load minigraph succeeded.
Signed-off-by: Ying Xie ying.xie@microsoft.com
Added source interface support for NTP.
Also made NTP start on Mgmt-VRF by default when configured.
**- How I did it**
1) Updated hostcfg to listen to global config NTP and NTP_SERVER tables and restart ntp when ever the configuration changes. NTP table includes source interface configuration.
2) The ntp script updated to by default start on Mgmt-VFT when configured.
Signed-off-by: Prabhu Sreenivasan <prabhu.sreenivasan@broadcom>
* [Mellanox] Update SAI to 1.18.0
* [Mellanox] Update SDK to 4.4.2112
* Updated Mellanox SAI to 1.18.0.2
* Updated bcmsai debians to use SAI 1.7.1
* Updated Mellanox to use SAI 1.7.1
* Updated submodule sonic-sairedis using SAI 1.7.1
Co-authored-by: Vineet Mittal <vmittalmittal@microsoft.com>
Co-authored-by: Nazarii Hnydyn <nazariig@nvidia.com>
[vs] Add workaround for clean up macsec ports (#752)
[logfile]: Add handling of Sairedis rec filename (#747)
Update README.md
[meta] Fix stat_mode enums to sai_bulk_op_error_mode_t (#753)
[syncd][tests] Add syncd deprecated attribute value test (#751)
[vs] Skip MACsec clean up if /sbin/ip is not accessible (#750)
Configure enable -Wcast-align=strict when supported by compiler (#749)
[syncd] Translate depreacated attr enum values to new ones (#746)
[sairedis]vs SAI support for voq neighbor (#725)
[syncd] Translate removed RIDs in fdb notification (#734)
[syncd] Move syncd classes to syncd namespace (#742)
[vs] Use /sbin/ip absolute path for ip command in MACsecManager (#744)
[saidiscovery] Update saidiscovery to use VendorSai object and metadata (#736)
Remove Winline warning since it depends on external headers (#741)
[meta] Enable strict cast-align warning (#738)
[vs] Use meta class instead info when using unittests (#740)
[vs] Support flush entry type all on virtual switch (#735)
[vslib]: Add MACsec state to state base (#722)
[README.md] Update installation steps (#730)
Switch Capability support (#728)
[vs] Fail switch create when warm boot requested and no warm boot state (#739)
Dynamic Port breakout fix the crash, port down event processing after<80> (#727)
Code clean (#721)
Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>
- Why I did it
In some build machine, it may be not able to run as root during the build, only has root authority in docker containers.
- How I did it
Remove the sudo in Makefile
- How to verify it
cd src/sonic-build-hooks
make all
- Why I did it
The change is done to make sure the system initialization is done before updating the feature states
- How I did it
use the command "systemctl is-system-running --wait" to wait till system has finished booting up before updating the feature states
Commits include:
- 9935fca5 Fixed eeprom_tlvinfo.py to make it Python2/Python3 compatible (#155)
- 7e23e63 [y_cable] Skip ImportError in y_cable.py (#154)
Refactor determine-reboot cause code. Fix errors seen during determine-reboot-cause when sonic_platform package is not installed.
Add error handling for healthd service when sonic_platform package is not installed.
Tested on KVM where sonic_platform is not present, and the errors are not seen anymore in syslog.
This PR updates the following commits in master for
sonic-platform-daemons
b0be7ca [xcvrd] add unit test infrastructure and unit tests for xcvrd (#133)
8555077 [xcvrd] removing the delete notification message logic for command probe in mux cable driver (#134)
Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
* Parse device type from <ElementType> first in <PngDec>
* Fall back to <Device> type attribute if no <ElementType> is found
Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
- Why I did it
The sai.profile file in kvm images overrides the warmboot file with path /var/cache/sai_warmboot.bin. Since the directory /var/cache is not mounted in syncd, it will be cleared in an image upgrade, the warm-reboot image upgrade will fail if the file is put in the directory.
Fix#6183
- How I did it
Remove the path that overrides the default path. The warmboot file path will then be the default value /var/warmboot/sai-warmboot.bin. Since /var/warmboot/ is mounted by /host/warmboot/ in the host, it could survive an image upgrade.
- How to verify it
Tested warm reboot upgrading kvm image locally.
[dhcp-relay]: Prevent Buffer Overrun Of Malformed DHCP Packet
The add/strip relay agent options does not take into account the buffer
length and so it is possible to overrun the buffer. The issue will
result in contents from previous packet being added to the current one.
signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
- Why I did it
Advance swss submodule to pick up latest changes.
- How I did it
Including folowing changes:
[portsorch] adjust port initialized event back to notice (#1532)
Signed-off-by: Ying Xie ying.xie@microsoft.com
* Introduced a list console_device_types which contains the device types that support console management feature
* Inject CONSOLE_SWITCH:console_mgmt table with enabled:yes or enabled:no
Signed-off-by: Jing Kan jika@microsoft.com
* src/sonic-platform-daemons 73e6ddd...4da0bfc (3):
> Align style with PEP8 standards (#128)
> Support python3 for xcvrd, psud, thermalctld and syseepromd (#132)
> Import mock psu object for testing LED (#119)
This update brings in the following commits.
86c1108 Enable arm architecture to build in addition to amd64 (#37)
4acb2c3 fix bugs and enhance Transformer (#35)
49e5a22 ygot related enhancements and fixes (#34)
51224de Fix ietf yang search path for cvl schema builds (#32)
3c6cdb3 CVL Changes #8: 'must' and 'when' expression evaluation (#31)
dabf231 CVL Changes #7: 'leafref' evaluation (#28)
6f9535f CVL Changes #6: Customized Xpath Engine integration (#27)
5e2466b DB-Layer fixes/enhancements (#26)
9a27302 CVL Changes #4: Implementation of new CVL APIs (#22)
dbf1093 Translib support for authorization, yang versioning and Delete flag (#21)
80f369e CVL Changes #5: YParser enhancement (#23)
904ce18 CVL Changes #3: Multi-db instance support (#20)
9d24a34 CVL Changes #2: YValidator infra changes for evaluating xpath expression (#19)
f3fc40f CVL Changes #1: Initial CVL code reorganization and common infra changes (#18)
4922601 Bulk and RPC API support in translib (#16)
1d730df RFC7895 yang module library implementation (#15)
Update sonic-platform-common submodule:
* Make eeprom_tlvinfo.py Python3 compatible
**- Why I did it**
To get the latest changes which fix some python2 -> python3 migration errors.
[dataset] Add dataset "system uptime" into non-db client. (#52)
Adding new data set to query Sonic OS version. (#50)
[gnmi_server] Disregard EOF status for STREAM subs (#48)
Signed-off-by: Yong Zhao <yozhao@microsoft.com>
[sairedis] Advance sairedis pointer to support cmd zmq flag
[meta] Use memcpy instead of cast to prevent strict-aliasing error (#723)
[vslib]Add MACsec forward and filters to HostInterfaceInfo (#719)
[vslib] Add StateBase function for MACsec (#717)
Add support for default zmq synchronous mode flag (#711)
[syncd] Code clean (#720)
[sairedis] Remove custom bulk fdb methods (#710)
[vslib]Add MACsec Filters (#713)
[vslib]Add MACsec Forwarder (#714)
[vslib]Add MACsec Manager (#715)
Add helper functions, findObjects and dumpObject (#716)
Code clean refactor (#712)
[vslib] Fix CorePortIndexMap log line (#708)
[meta] Use custom hash in SaiObjectCollection (#709)
Fix LGTM localtime function warnings (#707)
[vs] VoQ Switch objects initialization - Local Port OID mapping to System Ports (#703)
Code style refactor (#705)
[vs] Initialization of VOQ switch objects (#702)
[vs] SAI support for VOQ switches - Switch State Initialization (#701)
Add MACsec meta methods (#704)
[vs] SAI support for VOQ switches (#698)
[vs] SAI support for VOQ switches - Core Port Index Map File parser (#700)
[vs] SAI support for VoQ switch - Core Port Index Map Container (#699)
[syncd][sairedis] Change pub/sub model to push/pull in zmq notification (#695)
[syncd] Use lua script to update db when using bulk api (#690)
[syncd] Fix bulk api object type for next hop group members (#685)
Add FlexCounter for MACsec SA (#684)
- Kdump configurations stored and manipulated in ConfigDB are now processed
by hostcfgd and applied asynchronously
Signed-off-by: Rajendra Dendukuri <rajendra.dendukuri@broadcom.com>
This PR updates the submodule sonic-platform-common, sonic-platform-daemons
For sonic-platform-common update following commits are added
a8823a3 [sonic_y_cable] add check_mux_direction api for y_cable (#151)
7c0301d Modular Chassis - APIs for midplane monitoring (#148)
6f7d8a0 Fix some LGTM alerts; Add LGTM badges to README.md (#146)
for sonic-platform-daemons update following commits are added
73e6ddd [xcvrd] replace check_active_linked_tor_side api to check_mux_direction (#131)
Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
swss:
- [port_rates][rif_rates] Fix port_rates.lua (#1520)
- Improve the way of handling BUFFER_PG during PFC storm (#1480)
- [Orchagent] Vxlanorch and Portsorch changes for EVPN VXLAN (#1264)
- [sub intf] ecmp hardware convergence acceleration at parent port oper status changes (#1492)
- [dvs/copp] Adjust policer checks in VS tests for default COPP policers (#1519)
- Flush ARP/neighbor entry on FDB flush when port L2-L3 (#1506)
- [intfmgr]: Add GARP support (#1503)
swss-common:
- [Dynamic buffer calc] Support dynamic buffer calculation (#361)
- README.md: Update build requirements (#414)
Signed-off-by: Stephen Sun <stephens@nvidia.com>
* src/sonic-utilities 6fabbed...3749f5e (7):
> [config vlan] Stop, reset-failed, then start dhcp_relay service (#1269)
> [show] Break some groups out into their own modules (#1259)
> Convert the output of 'arp -n' from bytes to str. (#1279)
> Fixing python porting errors in fdbshow, natshow and nbrshow (#1280)
> [pfcwd] Use floor division in Python3 (#1277)
> Modify fast-reboot script to use teamd service script (#1251)
> Add `show reboot-cause history` cli (#1210)
this PR updates the following Commits in sonic-platform-daemons
ab39059 [xcvrd] Fix y_cable state updates from 'failure' to 'unknown' on error conditions/events (#129)
5d74356 Clean some unused variables, check for multiAsic before calling initializeGlobalConfig (#130)
12b3628 [psud] Make sure psu leds are set on the first run (#125)
Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
Changes for supporting vstest for VOQ system ports. The changes include:
(1)Use of chassis_db.json is avoided since the SYSTEM_PORT is made
available in virtual chassis linecard's default_config.json which will
be loaded during bootup
(2)Core port index map file is introduced and is copied from virtual chassis
directory to hwsku direcory by start.sh
(3)vs sai profile is modified to include core port index map file name
Signed-off-by: vedganes <vedavinayagam.ganesan@nokia.com>
The issue was a typo introduced in #6006. In that change, the BGP allow list
configuration manager was updated to use a method of common ConfigMgr
for restarting peer groups. However, the method name 'restart_peers' was
used instead of the correct 'restart_peer_groups'.
This change updated the managers_allow_list.py to use correct method
'restart_peer_groups' for restarting peer groups.
Signed-off-by: Xin Wang <xiwang5@microsoft.com>
- Why I did it
'always_enabled' feature can still be disabled/enabled.
- How I did it
When checking if a feature is 'always_enabled', check the cached state to prevent new change to be accepted.
Fix an issue where cache value is updated before all the check is done.
Restore 'always_enabled' value in config db if someone wants to change.
Signed-off-by: Ying Xie ying.xie@microsoft.com
- How to verify it
Without the fix, 'always_enabled' feature can be enabled or disabled without cli protection. With the protection, the change will be rejected properly.
6fabbedc8ba0f7961bec3ec0adf7bb9d5c71eb2b [consutil][test] Add unit tests for consutil clear command (#1274)
a4f663ea0d7b52eeb76d903b19119c34ab86b1e9 [show/fgnhg] Fix crash bug (#1272)
9f276b21b06423ad11d75fbe17f46a43723f62a5 [config][loopback] Fix Bug that cannot add loopback interface by CLI (#1276)
a5b78cf968e4e5c49b764f5a76bdea6d6532c8d3 [config][show] CLI support for interacting with muxcable (#1221)
8f3b22ed11e166ed24dcd2fb1ac61e227140c2af [connect][sonic-clear] Align the exit code with consutil for line commands (#1256)
cfb7a2269406e8cca354db8a579c401e97f66e9f Feature table cli command update (#1271)
5b8da561b8712e9785622a53a2898671ada9c76e Added Multi-ASIC support for show ip(v6) route (#1216)
a7f39b04eb4b14f56f14f9851d59b81ecd5285f8 Improve MockRedis _encode(): so it will work on all types of value (#1265)
8427caf28ea60af97d8eb1eb3ccb75e99bc54184 [config_mgmt.py]: Remove sort_data call, we do not need it now. (#1246)
9dc58eaa76d38a40f33d534b13442284f598852a [sonic-utilities] Fix show vnet [neighbors | routes [all | tunnel]] crashed issue (#1263)
Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
Made changes so that Lldp docker start using py3 of sonic-db-syncd
submodule update sonic-db-syncd
5cc29a1b32d8d1f4dfbc967bfea2727c50a49c76 (HEAD -> master, origin/master, origin/HEAD) Changes to convert sonic-dbsyncd from python 2 to 3
Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
- Why I did it
Fix issue #6043
- How I did it
We are disabling in container frr log. The log entries are sent to base image and are logged in /var/log/quagga/bgpd.log.
However, we need to remove the whole outchannel config block to avoid an error message raised by rsyslogd.
- How to verify it
Without the change, test_autorestart bgp container will fail on loganalyer errors. With the change, restarting bgp container is no longer generating error message and the test will pass.
The log generated by frr continued appearing in /var/log/quagga/bgpd.log
* [bgpcfgd]: Batch bgp updates.
vtysh -f command is slow. It is sometimes takes about 3 seconds.
When we need to run many vtysh -f commands that slows down the system.
Batch vtysh -f updates.
* Use correct file to import run_command
Submodule updates include the following commits:
* src/sonic-utilities 9dc58ea...f9eb739 (18):
> Remove unnecessary calls to str.encode() now that the package is Python 3; Fix deprecation warning (#1260)
> [generate_dump] Ignoring file/directory not found Errors (#1201)
> Fixed porstat rate and util issues (#1140)
> fix error: interface counters is mismatch after warm-reboot (#1099)
> Remove unnecessary calls to str.decode() now that the package is Python 3 (#1255)
> [acl-loader] Make list sorting compliant with Python 3 (#1257)
> Replace hard-coded fast-reboot with variable. And some typo corrections (#1254)
> [configlet][portconfig] Remove calls to dict.has_key() which is not available in Python 3 (#1247)
> Remove unnecessary conversions to list() and calls to dict.keys() (#1243)
> Clean up LGTM alerts (#1239)
> Add 'requests' as install dependency in setup.py (#1240)
> Convert to Python 3 (#1128)
> Fix mock SonicV2Connector in python3: use decode_responses mode so caller code will be the same as python2 (#1238)
> [tests] Do not trim from PATH if we did not append to it; Clean up/fix shebangs in scripts (#1233)
> Updates to bgp config and show commands with BGP_INTERNAL_NEIGHBOR table (#1224)
> [cli]: NAT show commands newline issue after migrated to Python3 (#1204)
> [doc]: Update Command-Reference.md (#1231)
> Added 'import sys' in feature.py file (#1232)
* src/sonic-py-swsssdk 9d9f0c6...1664be9 (2):
> Fix: no need to decode() after redis client scan, so it will work for both python2 and python3 (#96)
> FieldValueMap `contains`(`in`) will also work when migrated to libswsscommon(C++ with SWIG wrapper) (#94)
- Also fix Python 3-related issues:
- Use integer (floor) division in config_samples.py (sonic-config-engine)
- Replace print statement with print function in eeprom.py plugin for x86_64-kvm_x86_64-r0 platform
- Update all platform plugins to be compatible with both Python 2 and Python 3
- Remove shebangs from plugins files which are not intended to be executable
- Replace tabs with spaces in Python plugin files and fix alignment, because Python 3 is more strict
- Remove trailing whitespace from plugins files
Added new flag value 'always_enabled' for the state and auto-restart field of feature table
init_cfg.json is updated to initialize state field of database/swss/syncd/teamd feature and auto-restart field of database feature
as always_enabled
Once the state/auto-restart value is initialized as "always_enabled" it is immutable and cannot be change via feature config commands. (config feature..) PR#Azure/sonic-utilities#1271
hostcfgd will not take any action if state field value is 'always_enabled'
Since we have always_enabled field for auto-restart updated supervisor-proc-exit-listener
not to have special check for database and always rely on value from Feature table.
In the emulated armhf environment, the function readdir()returns NULL on a ext4 file system directory. When running the libyang test cases, it will require to load the plugins from the files (such as metadata.so), because the readdir() is failing, the plugins can’t be loaded in the emulated armhf environment, so it causes libyang test error. This error is a combination of the following reasons.
• Emulation of a 32-bit target from a 64-bit host –> qemu from x86_64 to armhf
• Glibc version > 2.27 – Debian buster is using glibc 2.28
Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>
- Why I did it
The l2switch.j2 template does not include all fields for PORT. This could be incompatible with the 201911 image or later.
- How I did it
Update l2switch.j2 template and add a unit test.
- Why I did it
Add reboot history to State db so that can be used telemetry service
- How I did it
Split the process-reboot-cause service to determine-reboot-cause and process-reboot-cause
determine-reboot-cause to determine the reboot cause
process-reboot-cause to parse the reboot cause files and put the reboot history to state db
Moved to sonic-host-service* packages
- How to verify it
Performed unit test and tested on DUT
- Why I did it
frr is creating /var/log/frr/frr.log inside the frr docker and letting it grow. It will eventually exhaust hard drive space.
To fixe issue #5965
- How I did it
Remove rsyslog file outchannel so that frr won't generate /var/log/frr/frr.log inside the docker.
- How to verify it
Manually removed the outchannel and restart BGP docker, making sure that /var/log/frr/frr.log is no longer created inside the docker.
While restarting bgp docker, observed that base image /var/log/quagga/bgpd.log continued to grow and captured all FRR logs.
- Fix test_updater_thermal_check_min_max()
f6e5e5902e
- [sonic-platform-daemons] Add more physical entity information to DB for physical entity mib
14e586d676
Signed-off-by: Petro Bratash <petrox.bratash@intel.com>
Fix 259 alerts reported by the LGTM tool:
- 245 for Unused import
- 7 for Testing equality to None
- 5 for Duplicate key in dict literal
- 1 for Module is imported more than once
- 1 for Unused local variable
**- Why I did it**
We were building a custom version of Supervisor because I had added patches to prevent hangs and crashes if the system clock ever rolled backward. Those changes were merged into the upstream Supervisor repo as of version 3.4.0 (http://supervisord.org/changes.html#id9), therefore, we should be able to simply install the vanilla package via pip. This will also allow us to easily move to Python 3, as Python 3 support was added in version 4.0.0.
**- How I did it**
- Remove Makefiles and patches for building supervisor package from source
- Install Python 3 supervisor package version 4.2.1 in Buster base container
- Also install Python 3 version of supervisord-dependent-startup in Buster base container
- Debian package installed binary in `/usr/bin/`, but pip package installs in `/usr/local/bin/`, so rather than update all absolute paths, I changed all references to simply call `supervisord` and let the system PATH find the executable to prevent future need for changes just in case we ever need to switch back to build a Debian package, then we won't need to modify these again.
- Install Python 2 supervisor package >= 3.4.0 in Stretch and Jessie base containers
Update the sonic-swss submodule. The following are the commits in the submodule.
[dvs] Re-add runcmd_async command
5ba548cd759be4f0b1d1830ea038bf3c84e6f3f6
[dvs] Re-add copy_file method to DVS
86e11715819ac183a68aaf1deed6beabb259384b
Add a check for warm-restart, and do a clear only when warm-restart is enable.
fcb6c9de918d93967440b78bf7d5cd3a51bf7129
[Routeorch] Fix next hop group reference count in bulk operation
7a92100c1c1a344e67d4501cb7b98de557d0ea52
[orchagent][port] In case of successful port creation set log level to INFO
fea7ade74941a529505681e384f3437e873b1f7a
Copp Manager Changes
7b76d2e201f9f7d26cec7460f1335f7762b4ad01
[orchagent] Arm 32-bit arch compilation warning Fixes
bed79702c9da7957b775a0c960f67dd426db34d5
Revert: swss: flush g_asicState after each event is done #570
b9084a720d41daf1d717bcbe06f56e7a95111fa6
[dvs] Clean-up conftest.py
d6e15e955c91a6f077a913a617a336a314c6c91b
Update the sonic-swss-common submodule. The following are the commits in the submodule.
```
95f9e11 2020-11-19 | [pyext] allow to catch exceptions raised in python (#415) [Stepan Blyshchak]
5a718f9 2020-11-18 | [swig] translate C++ `del` to python `delete` (#416) [Qi Luo]
40b255b 2020-11-12 | Fix: SWIG dict.get() should have optional default value parameter (#413) [Qi Luo]
91e484d 2020-11-07 | Reduce notice logging (#412) [Qi Luo]
f5945ae 2020-11-05 | Mux Cable schema definitions for interaction between linkmanager and xcvrd (#411) [vdahiya12]
602f9c2 2020-11-05 | [lua] load lua script on demand (#409) [Dong Zhang]
d88412b 2020-11-04 | Rename hdel to del when using multiple keys as param (#410) [Kamil Cudnik]
e0c229a 2020-11-04 | CHASSIS_STATE_DB on control-card for chassis state (#395) [mprabhu-nokia]
a4e3ac8 2020-11-04 | Chassisd config table to store admin state (#388) [mprabhu-nokia]
```
* Add explicit default state into the constants.yml
* Enable/disable only peer-groups, available in the config
* Retrieve updates from frr before using configuration
Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
Changes:
1.) yang models for device_metadata, device_neighbor, flex_counters, crm, versions.
2.) Test cases for above yang models.
Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com:
The behavior of `subprocess.Popen()` changed in Python 3 such that stdin, stdout and stderr are treated as bytes by default. Adding the `universal_newlines=True` argument changes this behavior to return strings, matching the behavior of Python 2. The change is backward-compatible with Python 2, as well.
This should also fix the vsimage check build.
Commits include:
* src/sonic-platform-common a659219...e4883e5 (7):
> [PsuBase] Reference inherited member constant (#142)
> Thermalctld APIs for recording min and max temp (#131)
> Common power consumption and supply APIs for modular chassis (#136)
> [sonic-platform-base] Introduce APIs for modular chassis support (#124)
> Remove shebangs from non-executable Python files (#140)
> [sonic_eeprom] Make compatible with Python 2 and 3 (#127)
> Add unit test infrastructure (#139)
* src/sonic-platform-daemons 600d043...be21d91 (9):
> [psud] Import of mock libraries determined by environment variable (#117)
> [chassisd][thermalctld] Set testing env var at top of file, not in setup_function() (#114)
> [chassisd][thermalctld] Import os module, now needed for env var checking (#113)
> [chassisd][thermalctld] Import mock libraries based on environment variable (#112)
> [xcvrd] Fix the console-script xcvrd startup (#111)
> [Thermalctld] Update thermal info to CHASSIS_STATE_DB (#101)
> PSUd changes to compute power-budget for Modular chassis (#104)
> Introduce chassisd to monitor status of cards on chassis (#97)
> [xcvrd] support for integrating Credo Y cable Ports initialization and status updates with xcvrd. (#105)
Fixed TSA bugs:
1. TSA didn't advertise Loopback ipv6 address
2. TSA and TSB changed BGP dynamic and BGP monitors sessions
**- How to verify it**
Build an image and run on your DUT.
```
admin@str-s6100-acs-1:~$ TSA
System Mode: Normal -> Maintenance
admin@str-s6100-acs-1:~$ vtysh -c 'show bgp ipv4 neighbors 10.0.0.1 advertised-routes'
BGP table version is 6, local router ID is 10.1.0.32, vrf id 0
Default local pref 100, local AS 64601
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 10.1.0.32/32 0.0.0.0 0 32768 i
Total number of prefixes 1
admin@str-s6100-acs-1:~$ vtysh -c 'show bgp ipv6 neighbors fc00::a advertised-routes'
BGP table version is 6, local router ID is 10.1.0.32, vrf id 0
Default local pref 100, local AS 64601
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> fc00:1::/64 :: 0 32768 i
Total number of prefixes 1
admin@str-s6100-acs-1:~$ TSB
System Mode: Maintenance -> Normal
```
Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
* Create new `PEER_SWITCH` table in config DB with info from minigraph
* Add `subtype` field to `DEVICE_METADATA` table and set value to `DualToR` if device is in a dual ToR setup
sonic-bgpcfgd build fails in the absence of Python 2, as it attempts to explicitly call sonic-cfggen using `/usr/bin/python2.7`. Also, it attempts to call sonic-cfggen using a local, relative path. Since the sonic-config-engine package is not installed, neither are its dependencies.
Now, we configure the Python 3 sonic-config-engine as a dependency of sonic-bgpcfgd, which ensures the Python 3 sonic-config-engine package and its dependencies are installed before sonic-bgpcfgd is built/tested.
Submodule updates include the following commits. Also adding sonic-swsscommon build dependencies where necessary.
* src/sonic-py-swsssdk 1ea30d2...9d9f0c6 (1):
> [SonicV2Connector] make decode_responses=True as default, so python2 application need no code change when migrated to python3(#93)
* src/sonic-snmpagent 6e4a796...57e54d9 (3):
> Interact with Redis by str instead of bytes, migrate to SonicV2Connector with `decode_responses=True` (#171)
> Add a test case for LLDP_LOC_CHASSIS.lldp_loc_man_addr has only one IPv6 address (#167)
> [LLDP]: Update init_db to load global database config (#166)
* src/sonic-utilities acfa824...b693cf6 (11):
> Remove SKU create pytest output directory before execution of the script (#1226)
> [show][techsupport][multi-ASIC] Add support to collect tech support on multi ASIC platform (#1193)
> [show] Fix `show ip bgp sum` (#1194)
> [sonic_sku_create] Move tests from sonic-utilities-tests/ folder to tests/ folder (#1222)
> Replace swsssdk.SonicV2Connector with swsscommon.SonicV2Connector (SWIG wrapper of C++ implementation) in production code (#1217)
> Copy missing values from INIT_CFG to config_db as part of db_migration task (#1209)
> [connect][clear] Support --devicename option for connect/clear line commands (#1214)
> [consutil][show] Remove root need from show line command (#1218)
> [Mellanox] SKU creator Tool (#1163)
> SONiC installer - fix string formatting during image type check (#1197)
> [show/fgnhg.py] Fix whitespace issues (#1211)
remove commit
850d0c6 2020-11-09 | [xcvrd] support for integrating Credo Y cable Ports initialization and status updates with xcvrd. (#105) (HEAD, origin/master, origin/HEAD) [vdahiya12]
fix issue #5907
Signed-off-by: Guohan Lu <lguohan@gmail.com>
Treat devices that are ToRRouters (ToRRouters and BackEndToRRouters) the same when rendering templates
Except for BackEndToRRouters belonging to a storage cluster, since these devices have extra sub-interfaces created
Treat devices that are LeafRouters (LeafRouters and BackEndLeafRouters) the same when rendering templates
Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
When forced mgmt routes are present, the issue fixed as part of #5754 is not complete.
Added a preference(priority) field to forced mgmt route ip rules
Python 3 is more strict with `__slots__`. As per the [documentation](https://docs.python.org/3/reference/datamodel.html#notes-on-using-slots):
> \_\_slots\_\_ are implemented at the class level by creating descriptors (Implementing Descriptors) for each variable name. As a result, class attributes cannot be used to set default values for instance variables defined by \_\_slots\_\_; otherwise, the class attribute would overwrite the descriptor assignment.
This was apparently missed when making sonic-config-engine compliant with Python 3, and errors like the following would be seen:
```
tests/acl_loader_test.py:10: in <module>
from acl_loader.main import *
acl_loader/main.py:8: in <module>
import openconfig_acl
/usr/local/lib/python3.7/dist-packages/openconfig_acl.py:24: in <module>
class yc_state_openconfig_acl__acl_state(PybindBase):
E ValueError: '_pybind_generated_by' in __slots__ conflicts with class variable
```
To make Control plane ACLs handle case insensitive ACL rules. Currently, it handles only upper case ACL rules.
Co-authored-by: Madhan Babu <madhan@arc-build-server.mtr.labs.mlnx>
850d0c6 [xcvrd] support for integrating Credo Y cable Ports initialization and status updates with xcvrd. (#105)
600d043 [xcvrd] Add enum dependence back; Install 'enum34' conditionally based on Python version (#107)
af79326 [xcvrd] Remove log errors in single ASIC platforms with init Global config (#108)
Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
- Why I did it
Update the routine is_bgp_session_internal() by checking the BGP_INTERNAL_NEIGHBOR table.
Additionally to address the review comment #5520 (comment)
Add timer settings as will in the internal session templates and keep it minimal as these sessions which will always be up.
Updates to the internal tests data + add all of it to template tests.
- How I did it
Updated the APIs and the template files.
- How to verify it
Verified the internal BGP sessions are displayed correctly with show commands with this API is_bgp_session_internal()
Take tunnel info from `<TunnelInterface>` tag in minigraph, and create tables in config_DB:
```
"TUNNEL": {
"MUX_TUNNEL_0": {
"tunnel_type": "IPINIP",
"dst_ip": "26.1.1.10",
"dscp_mode": "uniform",
"encap_ecn_mode": "standard",
"ecn_mode": "copy_from_outer",
"ttl_mode": "pipe"
}
}
```
Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
To consolidate host services and install via packages instead of file-by-file, also as part of migrating all of SONiC to Python 3, as Python 2 is no longer supported.
Fix#5812
LLDP conf Jinja2 Template does not verify IPv4 address and can use IPv6 version. This issue does not effect control LLDP daemon. Issue can be reproduced via `test_snmp_lldp` test. LLDP conf Jinja2 Template selects first item from the list of mgmt interfaces.
TESTBED_1 LLDP conf
```
# cat /etc/lldpd.conf
configure ports eth0 lldp portidsubtype local eth0
configure system ip management pattern FC00:3::32
configure system hostname dut-1
```
TESTBED_2 LLDP conf
```
# cat /etc/lldpd.conf
configure ports eth0 lldp portidsubtype local eth0
configure system ip management pattern 10.22.24.61
configure system hostname dut-2
```
TESTBED_1 MGMT_INTERFACE
```
$ redis-cli -n 4 keys "*" | grep MGMT_INTERFACE
MGMT_INTERFACE|eth0|10.22.24.53/23
MGMT_INTERFACE|eth0|FC00:3::32/64
```
TESTBED_2 MGMT_INTERFACE
```
$ redis-cli -n 4 keys "*" | grep MGMT_INTERFACE
MGMT_INTERFACE|eth0|FC00:3::32/64
MGMT_INTERFACE|eth0|10.22.24.61/23
```
Signed-off-by: Petro Bratash <petrox.bratash@intel.com>
Add a switch to route_check to control whether write log to syslog (#1215)
[consutil] Store console port state in STATE_DB (#1208)
[sfputil] Display 'N/A' for non-SFP ports (#1078)
Show FG_NHG CLI Commands Added (#1056)
[show] remove 'device2interface_dict' in show interface neighbor expected (#1188)
Add license file, remove third-party licenses file (#1192)
[fwutil]: Set min log priority to INFO. (#1191)
[CLI][show][platform] Added ASIC count in the output. (#1185)
fdbshow and nbrshow use SonicV2Connector with decode_responses=True, and remove all the decode() (#1187)
Remove stdeb.cfg; no longer used now that we build as wheel (#1182)
[counterpoll] Disable Counter Poll When Entering Fast Reboot (#1174)
Fixes the issue with show interface counters and for pfc and queue counters. (#1180)
[config/show] Add CLI support for proxy arp (#1168)
[consutil] Add brief option to show line command (#1176)
Modify fast-reboot script to use BGP service script to stop bgp service (#1177)
[config/console] Support update console configuration related commands (#1166)
[consutil] Fix issue where the show line command crash if no ttyUSB exists (#1173)
[watermarkstat] Add unit tests for watermarkstat show commands (#1157)
Fix exception for ipaddress in python2 (#1164)
[celestica] consutil to support customize tty device name (#1155)
Signed-off-by: Danny Allen <daall@microsoft.com>
- Convert config_checker, media_checker and platform_json_checker scripts to Python 3
- Reorganize imports per PEP8 standard
- Two blank lines precede functions per PEP8 standard
* Convert bgpcfgd to python3
Convert bgpmon to python3
Fix some issues in bgpmon
* Add python3-swsscommon as depends
* Install dependencies
* reorder deps
Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
Changes:
-- ACL stage allow lower and upper.
-- ACL include services leaf-list.
-- PORT include pfc_asym leaf.
-- PORT fec alloe none as per code.
-- 3 Tests for above changes.
Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com
ec96868 2020-11-03 | Fix: treat DBConnector timeout=0 as infinite timeout (#408) [Qi Luo]
b4b8334 2020-11-03 | Add lua script for redis multi keys api hmset and del (#406) [Kamil Cudnik]
swss:
* d7643f2 2020-11-01 | [tlm_teamd]: Make the destionation for std::transform() to use std::back_inserter() for allocating new space for the copied objects (#1490) (HEAD, origin/master, origin/HEAD) [pavel-shirshov]
* 7fa7cd6 2020-10-31 | [vstest]: stablize fgnhg test (#1491) [lguohan]
* 9b0696e 2020-10-29 | Create vnet tunnel map only if it doesn't exist (#1482) [Prince Sunny]
* 0481e99 2020-10-29 | [acl] Update CRM to include LAG bindings for ACL tables (#1487) [Danny Allen]
sairedis
* 5df11f5 2020-11-01 | [syncd] Lower bulk missing api message level from error to info (#687) (HEAD, origin/master, origin/HEAD) [Kamil Cudnik]
* dc73a1d 2020-10-30 | [saiplayer] Fix log messages (#686) [Kamil Cudnik]
Signed-off-by: Guohan Lu <lguohan@gmail.com>
* Build and install openssh from source
* Copy openssh deb package to dest folder
* Update make rule
* Update sonic debian extension
* Append empty line before EOF
* Update openssh patch
* Add openssh-server to base image dependency
* Fix indent type
* Fix comments
* Use commit id instead of tag id and add comment
Signed-off-by: Jing Kan jika@microsoft.com
FixAzure/SONiC#551
When eth0 IP address is configured, an ip rule is getting added for eth0 IP address through the interfaces.j2 template.
This eth0 ip rule creates an issue when VRF (data VRF or management VRF) is also created in the system.
When any VRF (data VRF or management VRF) is created, a new rule is getting added automatically by kernel as "1000: from all lookup [l3mdev-table]".
This l3mdev IP rule is never getting deleted even if VRF is deleted.
Once if this l3mdev IP rule is added, if user configures IP address for the eth0 interface, interfaces.j2 adds an eth0 IP rule as "1000:from 100.104.47.74 lookup default ". Priority 1000 is automatically chosen by kernel and hence this rule gets higher priority than the already existing rule "1001:from all lookup local ".
This results in an issue "ping from console to eth0 IP does not work once if VRF is created" as explained in Issue 551.
More details and possible solutions are explained as comments in the Issue551.
This PR is to resolve the issue by always fixing the low priority 32765 for the IP rule that is created for the eth0 IP address.
Tested with various combinations of VRF creation, deletion and IP address configuration along with ping from console to eth0 IP address.
Co-authored-by: Kannan KVS <kannan_kvs@dell.com>
Added new MultiASIC util method "get_back_end_interface_set()" to speed up back-end interface check by allowing caller to cache the back-end intf into a set. This way the caller can use this set for all subsequent back-end interface check requests instead of each time need to read from redis DB which become a scaling issue for cases such as checking for thousands of nexthop routes for filtering purpose.
Why/How I did:
Make sure first error syslog is triggered based on FAULT TOLERANCE condition.
Added support of repeat clause with alert action. This is used as trigger
for generation of periodic syslog error messages if error is persistent
Updated the monit conf files with repeat every x cycles for the alert action
Update the sonic-swss submodule. The following are the commits in the submodule.
[neighorch] Remove pending DEL operation after SET operation for the same key
2265f548386929b7827d1079efd453128f1ec1f9
[NAT]: Update nat entries to use nat_type to support DNAT Pool changes.
8696e939f973895ead4731ad499a72f257a3b510
[intfsorch] Init proxy_arp variable while adding router interface.
1da3c773762fa637a5ea47017715361bede50a4a
To consolidate host services and install via packages instead of file-by-file, also as part of migrating all of SONiC to Python 3, as Python 2 is no longer supported, convert caclmgrd to Python 3 and add to sonic-host-services package
95b1696 [xcvrd] Remove dependence on enum; Add 'sonic-py-common' as dependencies in setup.py (#106)
61ed24e [thermalctld] Print exception using repr(e) to get more information (#103)
8507085 [psud] Fix psud logging (#98)
Signed-off-by: Volodymyr Boyko <volodymyrx.boiko@intel.com>
* Initial commit for BGP internal neighbor table support.
> Add new template named "internal" for the internal BGP sessions
> Add a new table in database "BGP_INTERNAL_NEIGHBOR"
> The internal BGP sessions will be stored in this new table "BGP_INTERNAL_NEIGHBOR"
* Changes in template generation tests with the introduction of internal neighbor template files.
Update the sonic-sairedis submodule. The following are the commits in the submodule.
[syncd_init_common.sh] Use template file to retrieve vars (#683)
0bf336a3e895167357d5d2e5a988471e115522e8
[syncd/FlexCounter]:Fix last remove bug (#679)
4d21a264d5956501bf69ad3a89ea2ebccd369654
a659219 [SONIC_SFP] adding abstract methods for reading and writing the eeprom address space within platform api (#126)
848f4a6 Add third-party licenses (#138)
c2ecd9a Add license file (#137)
403747a [sonic-platform-common] Add new platform API for SONiC Physical MIB Extension feature (#134)
19b8545 [sonic_y_cable] fix the unpacking (#135)
Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
* Fix for LLDP advertisments being sent with wrong information.
Since lldpd is starting before lldpmgr, some advertisment packets might sent with default value, mac address as Port ID.
This fix hold the packets from being sent by the lldpd until all interfaces are well configured by the lldpmgrd.
Signed-off-by: Shlomi Bitton <shlomibi@nvidia.com>
* Fix comments
* Fix unit-test output caused a failure during build
* Add 'run_cmd' function and use it
* Resume lldpd even if port init timeout reached
Advance sonic-swss-common submodule by adding the following commits
3ec30ef Deprecate RedisClient and remove unused header file (#399)
165a679 Schema update for BGP internal neighbor table (#389)
262e330 Fix SonicV2Connector interfaces (#396)
Advance sonic-sairedis submodule by adding the following commits
bc3e044 [Sai]: Change Sai::set log to level INFO (#680)
b16bc8b Clean code: remove unused header file (#678)
40439b4 [syncd] Remove depreacated dependency on swss::RedisClient (#681)
1b6fc2e [syncd] Add supports of bulk api in syncd (#656)
a9f69c1 [syncd] Add to handle FDB MOVE notification (#670)
c7ef5e9 [gbsyncd] exit with zero when platform has no gearbox (#676)
57228fd [gbsyncd]: add missing python dependency (#675)
02a57a6 [vs] Add CRM SAI attributes to virtual switch interface (#673)
609445a fix boot type for fast boot (#674)
1325cdf Add support for saiplayer bulk API and add performance timers (#666)
1d84b90 Add ZeroMQ communication channel between sairedis and syncd (#659)
017056a Support System ports config (#657)
0f3668f Enable fabric counter for syncd's FlexCounter (#669)
Find LogicalLinks in minigraph and parse the port information. A new field called `mux_cable` is added to each port's entry in the Port table in config DB:
```
PORT|Ethernet0: {
"alias": "Ethernet4/1"
...
"mux_cable": "true"
}
```
If a mux cable is present on a port, the value for `mux_cable` will be `"true"`. If no mux cable is present, the attribute will either be omitted (default behavior) or set to `"false"`.
Current implementation of logger class is based on standard python syslog library.
Thus, logger class can be instantiated in different places and share the same context across the entire process.
This means that reducing log severity level will affect other modules which use logging facility.
**- Why I did it**
* To fix syslog implicit min priority override
**- How I did it**
* Added per instance log severity check
**- How to verify it**
1. Run code snippet
```
from sonic_py_common import logger
log1 = logger.Logger(log_identifier='myApp1')
log1.set_min_log_priority_debug()
log1.log_error("=> this is error")
log1.log_warning("=> this is warning")
log1.log_notice("=> this is notice")
log1.log_info("=> this is info")
log1.log_debug("=> this is debug")
log2 = logger.Logger(
log_identifier='myApp2',
log_facility=logger.Logger.LOG_FACILITY_DAEMON,
log_option=(logger.Logger.LOG_OPTION_NDELAY | logger.Logger.LOG_OPTION_PID)
)
log2.log_error("=> this is error")
log2.log_warning("=> this is warning")
log2.log_notice("=> this is notice")
log2.log_info("=> this is info")
log2.log_debug("=> this is debug")
```
2. Sample output:
```
Oct 23 15:08:30.447301 sonic ERR myApp1: => this is error
Oct 23 15:08:30.447908 sonic WARNING myApp1: => this is warning
Oct 23 15:08:30.448305 sonic NOTICE myApp1: => this is notice
Oct 23 15:08:30.448696 sonic INFO myApp1: => this is info
Oct 23 15:08:30.449063 sonic DEBUG myApp1: => this is debug
Oct 23 15:08:30.449442 sonic ERR myApp2[19178]: => this is error
Oct 23 15:08:30.449819 sonic WARNING myApp2[19178]: => this is warning
Oct 23 15:08:30.450183 sonic NOTICE myApp2[19178]: => this is notice
```
Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>
**- Why I did it**
Install all host services and their data files in package format rather than file-by-file
**- How I did it**
- Create sonic-host-services Python wheel package, currently including procdockerstatsd
- Also add the framework for unit tests by adding one simple procdockerstatsd test case
- Create sonic-host-services-data Debian package which is responsible for installing the related systemd unit files to control the services in the Python wheel. This package will also be responsible for installing any Jinja2 templates and other data files needed by the host services.
**- Why I did it**
On teamd docker restart, the swss and syncd needs to be restarted as there are dependent resources present.
**- How I did it**
Add the teamd as a dependent service for swss
Updated the docker-wait script to handle service and dependent services separately.
Handle the case of warm-restart for the dependent service
**- How to verify it**
Verified the following scenario's with the following testbed
VM1 ----------------------------[DUT 6100] -----------------------VM2, ping traffic continuous between VMs
1. Stop teamd docker alone
> swss, syncd dockers seen going away
> The LAG reference count error messages seen for a while till swss docker stops.
> Dockers back up.
2. Enable WR mode for teamd. Stop teamd docker alone
> swss, syncd dockers not removed.
> The LAG reference count error messages not seen
> Repeated stop teamd docker test - same result, no effect on swss/syncd.
3. Stop swss docker.
> swss, teamd, syncd goes off - dockers comes back correctly, interfaces up
4. Enable WR mode for swss . Stop swss docker
> swss goes off not affecting syncd/teamd dockers.
5. Config reload
> no reference counter error seen, dockers comes back correctly, with interfaces up
6. Warm reboot, observations below
> swss docker goes off first
> teamd + syncd goes off to the end of WR process.
> dockers comes back up fine.
> ping traffic between VM's was NOT HIT
7. Fast reboot, observations below
> teamd goes off first ( **confirmed swss don't exit here** )
> swss goes off next
> syncd goes away at the end of the FR process
> dockers comes back up fine.
> there is a traffic HIT as per fast-reboot
8. Verified in multi-asic platform, the tests above other than WR/FB scenarios
**- Why I did it**
To introduce dynamic support of BBR functionality into bgpcfgd.
BBR is adding `neighbor PEER_GROUP allowas-in 1' for all BGP peer-groups which points to T0
Now we can add and remove this configuration based on CONFIG_DB entry
**- How I did it**
I introduced a new CONFIG_DB entry:
- table name: "BGP_BBR"
- key value: "all". Currently only "all" is supported, which means that all peer-groups which points to T0s will be updated
- data value: a dictionary: {"status": "status_value"}, where status_value could be either "enabled" or "disabled"
Initially, when bgpcfgd starts, it reads initial BBR status values from the [constants.yml](https://github.com/Azure/sonic-buildimage/pull/5626/files#diff-e6f2fe13a6c276dc2f3b27a5bef79886f9c103194be4fcb28ce57375edf2c23cR34). Then you can control BBR status by changing "BGP_BBR" table in the CONFIG_DB (see examples below).
bgpcfgd knows what peer-groups to change fron [constants.yml](https://github.com/Azure/sonic-buildimage/pull/5626/files#diff-e6f2fe13a6c276dc2f3b27a5bef79886f9c103194be4fcb28ce57375edf2c23cR39). The dictionary contains peer-group names as keys, and a list of address-families as values. So when bgpcfgd got a request to change the BBR state, it changes the state only for peer-groups listed in the constants.yml dictionary (and only for address families from the peer-group value).
**- How to verify it**
Initially, when we start SONiC FRR has BBR enabled for PEER_V4 and PEER_V6:
```
admin@str-s6100-acs-1:~$ vtysh -c 'show run' | egrep 'PEER_V.? allowas'
neighbor PEER_V4 allowas-in 1
neighbor PEER_V6 allowas-in 1
```
Then we apply following configuration to the db:
```
admin@str-s6100-acs-1:~$ cat disable.json
{
"BGP_BBR": {
"all": {
"status": "disabled"
}
}
}
admin@str-s6100-acs-1:~$ sonic-cfggen -j disable.json -w
```
The log output are:
```
Oct 14 18:40:22.450322 str-s6100-acs-1 DEBUG bgp#bgpcfgd: Received message : '('all', 'SET', (('status', 'disabled'),))'
Oct 14 18:40:22.450620 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-f', '/tmp/tmpmWTiuq']'.
Oct 14 18:40:22.681084 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V4 soft in']'.
Oct 14 18:40:22.904626 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V6 soft in']'.
```
Check FRR configuraiton and see that no allowas parameters are there:
```
admin@str-s6100-acs-1:~$ vtysh -c 'show run' | egrep 'PEER_V.? allowas'
admin@str-s6100-acs-1:~$
```
Then we apply enabling configuration back:
```
admin@str-s6100-acs-1:~$ cat enable.json
{
"BGP_BBR": {
"all": {
"status": "enabled"
}
}
}
admin@str-s6100-acs-1:~$ sonic-cfggen -j enable.json -w
```
The log output:
```
Oct 14 18:40:41.074720 str-s6100-acs-1 DEBUG bgp#bgpcfgd: Received message : '('all', 'SET', (('status', 'enabled'),))'
Oct 14 18:40:41.074720 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-f', '/tmp/tmpDD6SKv']'.
Oct 14 18:40:41.587257 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V4 soft in']'.
Oct 14 18:40:42.042967 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V6 soft in']'.
```
Check FRR configuraiton and see that the BBR configuration is back:
```
admin@str-s6100-acs-1:~$ vtysh -c 'show run' | egrep 'PEER_V.? allowas'
neighbor PEER_V4 allowas-in 1
neighbor PEER_V6 allowas-in 1
```
*** The test coverage ***
Below is the test coverage
```
---------- coverage: platform linux2, python 2.7.12-final-0 ----------
Name Stmts Miss Cover
----------------------------------------------------
bgpcfgd/__init__.py 0 0 100%
bgpcfgd/__main__.py 3 3 0%
bgpcfgd/config.py 78 41 47%
bgpcfgd/directory.py 63 34 46%
bgpcfgd/log.py 15 3 80%
bgpcfgd/main.py 51 51 0%
bgpcfgd/manager.py 41 23 44%
bgpcfgd/managers_allow_list.py 385 21 95%
bgpcfgd/managers_bbr.py 76 0 100%
bgpcfgd/managers_bgp.py 193 193 0%
bgpcfgd/managers_db.py 9 9 0%
bgpcfgd/managers_intf.py 33 33 0%
bgpcfgd/managers_setsrc.py 45 45 0%
bgpcfgd/runner.py 39 39 0%
bgpcfgd/template.py 64 11 83%
bgpcfgd/utils.py 32 24 25%
bgpcfgd/vars.py 1 0 100%
----------------------------------------------------
TOTAL 1128 530 53%
```
**- Which release branch to backport (provide reason below if selected)**
- [ ] 201811
- [x] 201911
- [x] 202006
Issue was because we were relying on port_alias_asic_map dictionary
but that dictionary can't be used as alias name format has changed.
Fix the port alias mapping as what is needed.
Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
**- Why I did it**
I was asked to change "Allow list" prefix-list generation rule.
Previously we generated the rules using following method:
```
For each {prefix}/{masklen} we would generate the prefix-rule
permit {prefix}/{masklen} ge {masklen}+1
Example:
Prefix 1.2.3.4/24 would have following prefix-list entry generated
permit 1.2.3.4/24 ge 23
```
But we discovered the old rule doesn't work for all cases we have.
So we introduced the new rule:
```
For ipv4 entry,
For mask < 32 , we will add ‘le 32’ to cover all prefix masks to be sent by T0
For mask =32 , we will not add any ‘le mask’
For ipv6 entry, we will add le 128 to cover all the prefix mask to be sent by T0
For mask < 128 , we will add ‘le 128’ to cover all prefix masks to be sent by T0
For mask = 128 , we will not add any ‘le mask’
```
**- How I did it**
I change prefix-list entry generation function. Also I introduced a test for the changed function.
**- How to verify it**
1. Build an image and put it on your dut.
2. Create a file test_schema.conf with the test configuration
```
{
"BGP_ALLOWED_PREFIXES": {
"DEPLOYMENT_ID|0|1010:1010": {
"prefixes_v4": [
"10.20.0.0/16",
"10.50.1.0/29"
],
"prefixes_v6": [
"fc01:10::/64",
"fc02:20::/64"
]
},
"DEPLOYMENT_ID|0": {
"prefixes_v4": [
"10.20.0.0/16",
"10.50.1.0/29"
],
"prefixes_v6": [
"fc01:10::/64",
"fc02:20::/64"
]
}
}
}
```
3. Apply the configuration by command
```
sonic-cfggen -j test_schema.conf --write-to-db
```
4. Check that your bgp configuration has following prefix-list entries:
```
admin@str-s6100-acs-1:~$ show runningconfiguration bgp | grep PL_ALLOW
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V4 seq 10 deny 0.0.0.0/0 le 17
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V4 seq 20 permit 127.0.0.1/32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V4 seq 30 permit 10.20.0.0/16 le 32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V4 seq 40 permit 10.50.1.0/29 le 32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V4 seq 10 deny 0.0.0.0/0 le 17
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V4 seq 20 permit 127.0.0.1/32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V4 seq 30 permit 10.20.0.0/16 le 32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V4 seq 40 permit 10.50.1.0/29 le 32
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V6 seq 10 deny ::/0 le 59
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V6 seq 20 deny ::/0 ge 65
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V6 seq 30 permit fc01:10::/64 le 128
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V6 seq 40 permit fc02:20::/64 le 128
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V6 seq 10 deny ::/0 le 59
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V6 seq 20 deny ::/0 ge 65
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V6 seq 30 permit fc01:10::/64 le 128
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V6 seq 40 permit fc02:20::/64 le 128
```
Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
[swss]
[acl] Replace IP_PROTOCOL with NEXT_HEADER for IPv6 ACL tables (#1458)
[acl] Refactor port OID retrieval into aclorch (#1462)
Fix issue #5157 by identifying the dependency among objects and avoiding releasing an object still being referenced (#1440)
[mock tests] Update MockDBConnector to match new swsscommon interface (#1465)
[swss-common]
netlink: Setting nl_socket buffer size to 3M from 2M (#391)
Added support in Swig file to cast Selectable object to Subscriber Table object (#394)
[warm reboot] Warm Reboot Support for EVPN VXLAN (#350)
Implement DBInterface/SonicV2Connector in C++ (#387)
Fix memory leak if a RedisCommand object were to be reused (#392)
Signed-off-by: Danny Allen <daall@microsoft.com>
This PR makes two changes:
- Store Jinja2 cache in LOGLEVEL DB instead of STATE DB
- Store bytecode cache encoded in base64
Tested with the following command: "redis-dump -d 3 -k JINJA2_CACHE"
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
Includes the following commits:
* src/sonic-platform-common 111dcf7...b316f8d (5):
> [sonic_platform_base] fix an indentation in fan_drawer_base.py (#133)
> [SONIC_Y_CABLE] Adding new Y cable Package for accessing Y cable QSFP's eeprom and configuring Y cable's mux through eeprom (#128)
> Merge pull request #132 from vdahiya12/adding_changes_for_base_api
> [ChassisBase][ModuleBase] Remove redundant 'get_serial_number()' method (#130)
> Merge pull request #125 from vdahiya12/changes_for_mux_driver
6e4a796 2020-10-12 | Fix: correctly handle that lldp_loc_man_addr contains only IPv6 address without IPv4 address (#164) [Qi Luo]
0f772ce 2020-09-28 | [ciscoSwitchQosMIB]: Remove invocation of update_data function during (#161) [SuvarnaMeenakshi]
We were building our own python-click package because we needed features/bug fixes available as of version 7.0.0, but the most recent version available from Debian was in the 6.x range.
"Click" is needed for building/testing and installing sonic-utilities. Now that we are building sonic-utilities as a wheel, with Click specified as a dependency in the setup.py file, setuptools will install a more recent version of Click in the sonic-slave-buster container when building the package, and pip will install a more recent version of Click in the host OS of SONiC when installing the sonic-utilities package. Also, we don't need to worry about installing the Python 2 or 3 version of the package, as the proper one will be installed as necessary.
**- Why I did it**
FRR introduced [next hop tracking](http://docs.frrouting.org/projects/dev-guide/en/latest/next-hop-tracking.html) functionality.
That functionality requires resolving BGP neighbors before setting BGP connection (or explicit ebgp-multihop command). Sometimes (BGP MONITORS) our neighbors are not directly connected and sessions are IBGP. In this case current configuration prevents FRR to establish BGP connections. Reason would be "waiting for NHT". To fix that we need either add static routes for each not-directly connected ibgp neighbor, or enable command `ip nht resolve-via-default`
**- How I did it**
Put `ip nht resolve-via-default` into the config
**- How to verify it**
Build an image. Enable BGP_MONITOR entry and check that entry is Established or Connecting in FRR
Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
* system health first commit
* system health daemon first commit
* Finish healthd
* Changes due to lower layer logic change
* Get ASIC temperature from TEMPERATURE_INFO table
* Add system health make rule and service files
* fix bugs found during manual test
* Change make file to install system-health library to host
* Set system LED to blink on bootup time
* Caught exceptions in system health checker to make it more robust
* fix issue that fan/psu presence will always be true
* fix issue for external checker
* move system-health service to right after rc-local service
* Set system-health service start after database service
* Get system up time via /proc/uptime
* Provide more information in stat for CLI to use
* fix typo
* Set default category to External for external checker
* If external checker reported OK, save it to stat too
* Trim string for external checker output
* fix issue: PSU voltage check always return OK
* Add unit test cases for system health library
* Fix LGTM warnings
* fix demo comments: 1. get boot up timeout from monit configuration file; 2. set system led in library instead of daemon
* Remove boot_timeout configuration because it will get from monit config file
* Fix argument miss
* fix unit test failure
* fix issue: summary status is not correct
* Fix format issues found in code review
* rename th to threshold to make it clearer
* Fix review comment: 1. add a .dep file for system health; 2. deprecated daemon_base and uses sonic-py-common instead
* Fix unit test failure
* Fix LGTM alert
* Fix LGTM alert
* Fix review comments
* Fix review comment
* 1. Add relevant comments for system health; 2. rename external_checker to user_define_checker
* Ignore check for unknown service type
* Fix unit test issue
* Rename user define checker to user defined checker
* Rename user_define_checkers to user_defined_checkers for configuration file
* Renmae file user_define_checker.py -> user_defined_checker.py
* Fix typo
* Adjust import order for config.py
Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com>
* Adjust import order for src/system-health/health_checker/hardware_checker.py
Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com>
* Adjust import order for src/system-health/scripts/healthd
Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com>
* Adjust import orders in src/system-health/tests/test_system_health.py
* Fix typo
* Add new line after import
* If system health configuration file not exist, healthd should exit
* Fix indent and enable pytest coverage
* Fix typo
* Fix typo
* Remove global logger and use log functions inherited from super class
* Change info level logger to notice level
Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com>
* src/sonic-utilities 6e58dff...e1244a5 (5):
> [setup.py] Pin external runtime dependencies at known working versions (#1151)
> [config] Reload Monit configuration after potential hostname changes (#1132)
> [NAT]:Updated natshow script to support DNAT Pool changes (#1001)
> [pfcstat] Create new copies of Rx and Tx dictionaries (#1150)
> [master]Fix show ip interface neighbor info not correct when add second ip (#1134)
This is to fix pcie firmware check assert in Broadcom SDK once the SAI changes merges. This will be in the future but adding the soc property in the broadcom config now.
Co-authored-by: Zhi Yuan Carl Zhao <zyzhao@arista.com>
- [acl-loader] Revert fix for IP protocol == 0 (#1142)
- Show sflow interface to display all interfaces enabled for sflow (#1143)
- [sflow_test.py]: tests for config sflow commands. (#1112)
- [config/console] Support add/del console port setting commands (#1136)
- [cli][bgp]use vtysh in the show ip bgp summary command (#1137)
- [README.md] Describe new package creation schema (#1131)
- [consutil] Remove actual baud and refactor lib for future change (#1130)
Signed-off-by: Danny Allen <daall@microsoft.com>
* Install ndppd during image build, and copy config files to image
* Configure proxy settings based on config DB at container start
* Pipe ndppd output to logger inside container to log output in syslog
implements a new feature: "BGP Allow list."
This feature allows us to control which IP prefixes are going to be advertised via ebgp from the routes received from EBGP neighbors.
be51ebc533cbe92a885c584b4611ffc92d10ea36 Add IPv6 key item support to request parser (#1449)
76e22516bffe45d577ab9cce4f5d96687140fe75 When teamd feature state is disabled the Netdevice created by teamd were (#1450)
6aa97ce7982b78a26803908c0ce12302e51df793 Use .clear() after std::move() (#1444)
d5757db5d896aa2405f6cbf9f4c150f46b27aa4e Add libzmq to README dependencies (#1447)
c7b262ee61973a48943603810c8246a2471d1862 Add libzmq to Makefiles (#1443)
0b2e59ab6b909e5ea06013da03215f0d1af1e618 [drop counters] Clarify log messages for initial counter setup (#1445)
003cf24353c845ef708d192b7da8d571537671e1 [dvs] Refactor and add buffer pool wm test (#1446)
2f5d2d99dfae506333bddda9de2f289f92bb889b [acl] Remove Ethertype from L3V6 qualifiers (#1433)
f7b974f7fc6474a6493a7b6d0676e17caffa6165 Fix issue: bufferorch only pass the first attribute to sai when setting attribute (#1442)
Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
* Fix generate_l2_config: don't override hostname because sonic-cfggen may not read from Redis. Fix test_l2switch_template test case to test preset l2 feature.
* Improve test script: compare json files with sort_keys
* Revert changes on sample_output
* Remove members field in VLAN section. Fix test assertTrue statement.
**- Why I did it**
BGP_MONITORS sessions don't have corresponding DEVICE_NEIGHBOR_METADATA CONFIG_DB entries in the minigraphs. Prevent bgpcfgd to wait on such entries for BGP_MONITORS sessions.
**- How I did it**
Set constructor argument to False that means - don't wait for device neighbors metadata info for BGP_MONITORS
**- How to verify it**
Build an image, write on your device, use a minigraph with BGP_MONITORS sessions. Check that sessions are populated in the config.
Avoiding recursive update of maps as it consumes stack frames. This
PR introduces iterative version of deep_update method.
signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
Rendering templates show different orders when rendered using Python
2 vs Python 3. This PR prepare for Python 3 packaging by creating
new dir 'py2' for Python 2 rendered test cases.
singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
Jinja2 templates rendered using Python 3 interpreter, are required
to conform with Python 3 new semantics.
singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
Natural sorting of SONiC config gen output consumes lot of CPU cycles.
The sole use of natsorted was to make test comparison easier and so,
the natsorting logic is now relocated to the test suite. As a result
sonic-cfggen gained nearly 1 sec per call since we no longer import
natsorted module!
singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
implements a new feature: "BGP Allow list."
This feature allows us to control which IP prefixes are going to be advertised via ebgp from the routes received from EBGP neighbors.
* src/sonic-platform-common 7255d3a...111dcf7 (2):
> [EEPROM] Add new function part_number_str to TlvInfoDecoder (#121)
> Fix fp_port_index error to get correct index type (#111)
**- Why I did it**
As discussed, This PR covers the below points.
1. Give precedence to platform.json only if both platform.json and hwsku.json file exist. In case only platform.json exists, we don’t allow breakout for that HWSKU and fallback to port_config.ini.
**- How I did it**
check for `hwsku.json` file presence under get_path_to_port_config_file function.
Signed-off-by: Sangita Maity <sangitamaity0211@gmail.com>
* buildimage: Add gearbox phy device files and a new physyncd docker to support VS gearbox phy feature
* scripts and configuration needed to support a second syncd docker (physyncd)
* physyncd supports gearbox device and phy SAI APIs and runs multiple instances of syncd, one per phy in the device
* support for VS target (sonic-sairedis vslib has been extended to support a virtual BCM81724 gearbox PHY).
HLD is located at b817a12fd8/doc/gearbox/gearbox_mgr_design.md
**- Why I did it**
This work is part of the gearbox phy joint effort between Microsoft and Broadcom, and is based
on multi-switch support in sonic-sairedis.
**- How I did it**
Overall feature was implemented across several projects. The collective pull requests (some in late stages of review at this point):
https://github.com/Azure/sonic-utilities/pull/931 - CLI (merged)
https://github.com/Azure/sonic-swss-common/pull/347 - Minor changes (merged)
https://github.com/Azure/sonic-swss/pull/1321 - gearsyncd, config parsers, changes to orchargent to create gearbox phy on supported systems
https://github.com/Azure/sonic-sairedis/pull/624 - physyncd, virtual BCM81724 gearbox phy added to vslib
**- How to verify it**
In a vslib build:
root@sonic:/home/admin# show gearbox interfaces status
PHY Id Interface MAC Lanes MAC Lane Speed PHY Lanes PHY Lane Speed Line Lanes Line Lane Speed Oper Admin
-------- ----------- --------------- ---------------- --------------- ---------------- ------------ ----------------- ------ -------
1 Ethernet48 121,122,123,124 25G 200,201,202,203 25G 204,205 50G down down
1 Ethernet49 125,126,127,128 25G 206,207,208,209 25G 210,211 50G down down
1 Ethernet50 69,70,71,72 25G 212,213,214,215 25G 216 100G down down
In addition, docker ps | grep phy should show a physyncd docker running.
Signed-off-by: syd.logan@broadcom.com
We are moving toward building all Python packages for SONiC as wheel packages rather than Debian packages. This will also allow us to more easily transition to Python 3.
Python files are now packaged in "sonic-utilities" Pyhton wheel. Data files are now packaged in "sonic-utilities-data" Debian package.
**- How I did it**
- Build and install sonic-utilities as a Python package
- Remove explicit installation of wheel dependencies, as these will now get installed implicitly by pip when installing sonic-utilities as a wheel
- Build and install new sonic-utilities-data package to install data files required by sonic-utilities applications
- Update all references to sonic-utilities scripts/entrypoints to either reference the new /usr/local/bin/ location or remove absolute path entirely where applicable
Submodule updates:
* src/sonic-utilities aa27dd9...2244d7b (5):
> Support building sonic-utilities as a Python wheel package instead of a Debian package (#1122)
> [consutil] Display remote device name in show command (#1120)
> [vrf] fix check state_db error when vrf moving (#1119)
> [consutil] Fix issue where the ConfigDBConnector's reference is missing (#1117)
> Update to make config load/reload backward compatible. (#1115)
* src/sonic-ztp dd025bc...911d622 (1):
> Update paths to reflect new sonic-utilities install location, /usr/local/bin/ (#19)
* Add bgpmon under sonic-bgpcfgd to be started as a new daemon under BGP docker
* Added bgpmon to be monitored by Monit so that if it crashed, it gets alerted
* use console_scripts entry point to package bgpmon
e1842b2 Change STATE_DB key (PCIE_STATUS|PCIE_DEVICES -> PCIE_DEVICES) (#93)
a6c0071 [thermalctld] Fix issue: fan status should not be True when fan is absent (#92)
Updating submodule for sonic-utilities to get the changes to Azure master. The following were the commits that were part of this submodule.
Handling deletion of Portchannel before deletion of its members 219d78bda596cf78b71e5f6930545fb43f9c6c8b
removed a extra space in the condifitional statement 828719ed89112f317f9e78fff8264416176736f0
added Note about portchannel deletion in Command-Reference guide b36079f1d3f4086e42cfdca2090ebc76d811ad10
removed ctx.fail() and replaced it with click.echo() to display the error. Also, references of portchannel is replaced with port channel in the command reference guide b716b97e1aaf890ba9ef4ed0bb2931f4c2c030d2
Co-authored-by: Madhan Babu <madhan@arc-build-server.mtr.labs.mlnx>
As per the VOQ HLDs, internal networking between the linecards and supervisor is required within a chassis.
Allocating 127.X/16 subnets for private communication within a chassis is a good candidate.
It doesn't require any external IP allocation as well as ensure that the traffic will not leave the chassis.
References:
https://github.com/Azure/SONiC/pull/622https://github.com/Azure/SONiC/pull/639
**- How I did it**
Changed the `interfaces.j2` file to add `127.0.0.1/16` as the `lo` ip address.
Then once the interface is up, the post-up command removes the `127.0.0.1/8` ip address.
The order in which the netmask change is made matters for `127.0.0.1` to be reachable at all times.
**- How to verify it**
```
root@sonic:~# ip address show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/16 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
```
Co-authored-by: Baptiste Covolato <baptiste@arista.com>
Printing both snapshot and current counter sets will make it easier to pinpoint
which message type(s) is/are not being relayed. This PR prints both counter sets.
Also, this PR defines gnu11 as a C standard to compile with in order to avoid
making changes when porting to 201811 branch.
singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
When BGP routes are missing, DHCP packets get relayed over mgmt
interface. This results in dhcpmon alerting that DHCP packets are
not being relayed. This is PR include mgmt interface as uplink
device, and so, if DHCP packet gets relayed over mgmt interface,
regular dhcpmon alert will not be issues. Instead, dhcpmon will
check the mgmt interface counts and issue a separate alert regarding
packets travelling through mgmt network.
In addition, this PR includes the following enhancements:
1. Add SIGUSR1 handler that prints out current packet counts
2. Increase alert grace window to 3 minutes from currently 2 minutes
3. Time is now computed more accurately
4. Print vlan name before counters
signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
When stopping the swss, pmon or bgp containers, log messages like the following can be seen:
```
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,061 ERRO pool dependent-startup event buffer overflowed, discarding event 34
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,063 ERRO pool dependent-startup event buffer overflowed, discarding event 35
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,064 ERRO pool dependent-startup event buffer overflowed, discarding event 36
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,066 ERRO pool dependent-startup event buffer overflowed, discarding event 37
```
This is due to the number of programs in the container managed by supervisor, all generating events at the same time. The default event queue buffer size in supervisor is 10. This patch increases that value in all containers in order to eliminate these errors. As more programs are added to the containers, we may need to further adjust these values. I increased all buffer sizes to 25 except for containers with more programs or templated supervisor.conf files which allow for a variable number of programs. In these cases I increased the buffer size to 50. One final exception is the swss container, where the buffer fills up to ~50, so I increased this buffer to 100.
Resolves https://github.com/Azure/sonic-buildimage/issues/5241
* [sonic-utilities]update submodule with fix
This PR addresses fixes in sonic-py-common to imitate the behavior inside
sonic-cfggen. Essentially this is a fix for accessing the port-config file.
First check if there is a platform.json file for config generation
and then for legacy port_config.ini.
Also updating the sub-module sonic-utilities.
Fix pfcwd stats crash with invalid queue name (#1077)
[show][bgp]Display the Total number of neighbors in the show ip bgp(v6) summary. (#1079)
[config] Update SONiC Environment Vars When Loading Minigraph (#1073)
Multi asic platform changes for interface, portchannel commands (#878)
Update Command-Reference.md (#1075)
[filter-fdb] Fix Filter FDB With IPv6 Present in Config DB (#1059)
[config] Remove _get_breakout_cfg_file_name helper function (#1069)
[SHOW][BGP] support show ip(v6) bgp summary for multi asic platform (#1064)
[fanshow] Display other fan status, such as Updating (#1014)
Add ip_prefix len based on proxy_arp status (#1046)
Enable the platform specific ssd firmware upgrade during reboot (#954)
[show][cli[show interface portchannel support for Multi ASIC (#1005)
support show interface commands for multi ASIC platforms (#1006)
Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
[schema] Make schema header support C project (#373)
Removed DB specific get api's from Selectable class (#378)
With the change as part of #378 caclmgrd need to be updated
to use new client side Get API to access namespace.
Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
Remove radvd Makefile and patch, change docker-router-advertiser Dockerfile template to simply install the vanilla radvd package using apt-get.
- In PR https://github.com/Azure/sonic-buildimage/pull/2795, we started building radvd from source and patching it to prevent it from erroring out when advertising an MTU of 9100 which was greater than the MTU size configured on the bridge interface (1500), which was due to a limitation in the 4.9 Linux kernel.
- Master branch is now using Linux kernel 4.19. As of 4.18, the kernel supports setting a bridge MTU to a value > 1500.
- PR https://github.com/Azure/sonic-swss/pull/1393 modified vlanmgrd to take advantage of this and now configures the MTU of bridge interfaces in SONiC to the proper size of 9100. Therefore, we no longer need to patch radvd. Since we no longer need to patch radvd, we no longer need to build it from source, so we can save build time by going back to simply installing the vanilla radvd Debian package in the router-advertiser container.
The cmd "mclagdctl dump state" would goes wrong when there are two or more mclag_groups configured. The field "MCLAG Interface" can not be displayed in some group.
Signed-off-by: Sun Dandan <sundandan@asterfusion.com>
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
The following changes are done.
- Multi asic platform have 2 Loopback interfaces, Loopback0 and Loopback4096. IPinIP decap entries need to be added for both of them. Update the ipinip.json.j2 template to add decap entries for Loopback4096.
- Add corressponding unit test
Update the patform daemons with new commits
commit e9628b6b5a4683c7366fc99ee19ce727546fbabb
Merge pull request #64 from judyjoseph/multi_asic_ledd_xcvr
* platform daemon (Xcvrd, Ledd) changes for multi asic platform
* Updates in ledd daemon to use namespaces and get the namespace from selector object.
* Updates to xcvrd daemon to use the asic_id in talking to the right DB.
* Updated based on new sonic-py-common API's
* Invoke initializeGlobalConfig() in the SfpUpdate/DomInfoUpdate processes as well.
commit 415b8c457625c514aff0f8ecbdbbb655414d8067
[thermalctld] Optimize the thermal policy loop to make it execute every 60 seconds (#77)
commit 3d1f3196fd9c9942134e4926de7d248743e9589d
Update FAN_INFO in psud to avoid inconsistant output of show platform psud and show platform fan (#81)
Updating platform-common submodule with these commits
commit 14c6e53ecb861e124e2b45a7b65875ffac1b949e
[sff8472.py] Make hex keys all lowercase (#115)
Alpha chars in hex-based keys should be lowercase
commit b60f46cd1fb0ced1ffbff382e0125517f8c74b9e
Sfputil base and helper class changes for multi-ASIC (#100)
* Sfputil base and helper class changes for multi-ASIC
> adding the logical interface to asic id mapping
* Updated based on new sonic-py-common API's.
This PR enables cfggen to readr/write from Redis DB using pipelines.
Pipelines enables batch read/write from/to Redis DB.
signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
Change:
c25d492 Merge pull request #83 from tahmed-dev/taahme/add-redis-pipeline-operation
198d143 review comments - part of [configdb] Add Ability to Query/Update Redis Using Pipelines
994851c review comments - part of [configdb] Add Ability to Query/Update Redis Using Pipelines
2d2b7e1 making lgtm happy - part of [configdb] Add Ability to Query/Update Redis Using Pipelines
fa9093c [configdb] Add Ability to Query/Update Redis Using Pipelines
signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
**- Why I did it**
To install the framework for adding unit tests to the sonic-py-common package and report coverage.
** How I did it **
- Incorporate pytest and pytest-cov into sonic-py-common package build
- Updgrade version of 'mock' installed to version 3.0.5, the last version which supports Python 2. This fixes a bug where the file object returned from `mock_open()` was not iterable (see https://bugs.python.org/issue32933)
- Add support for Python 3 setuptools and pytest in sonic-slave-buster environment
- Add tests for `device_info.get_machine_info()` and `device_info.get_platform()` functions
- Also add a .gitignore in the root of the sonic-py-common directory, move all related ignores from main .gitignore file, and add ignores for files and dirs generated by pytest-cov
* src/sonic-swss d2bab10...c4949a2 (34):
> [dvs] Add new common issues and TOC to DVS README (#1405)
> Avoid adding loopback interface (ip link add) when setting nat zone on loopback interface (#1411)
> [portsorch] add buffer drop FC group (#1368)
> [dvs/chassis] Bring up SONiC interfaces in virtual chassis (#1410)
> [chassis/dvs] Add support for virtual chassis to DVS testbed (#1345)
> [sonic-swsss] Fix the issue of field "next_hop_ip" not getting updated in state DB in ERSPAN Mirror (#1375)
> [intfmgr] Fix OA crash issue due to link local configurations (#1195)
> Fix the issue when persistent DVS is used to run pytest which has number of front-panel ports < 32 (#1373)
> [dvs] Refactor AsicDbValidator (#1402)
> [fec] Get FEC mode when port is already admin down (#1403)
> [fec] added logic that put port down before applying fec onfiguration (#1399)
> [dvs] Add performance test for adding and deleting routes (#1392)
> Ignore IPv6 link-local and multicast entries as Vnet routes (#1401)
> [vlanmgr] Support Jumbo Frame By Default (#1393)
> Fix log/syslog not being correct when last test fails for given module (#1395)
> Get initial speed from ASIC DB (#1390)
> [dvs] Add options to limit CPU usage (#1394)
> [intfsorch] Retrieve Port object before setting NAT zone on router interfaces. (#1372)
> [.gitignore] Ignore gearsyncd binary (#1381)
> Added Max Nexthopgroup/ECMP Count supported by device into State DB. (#1383)
> [dvs] Upload logs even if failure occurs during startup (#1389)
> [rates] fix issue with rates init (#1387)
> [dvs] Validate that SWSS is ready to receive input before starting tests (#1385)
> [dvs] Convert sflow and speed tests to use dvslib (#1382)
> [dvs_acl] Refactor and document dvs_acl library (#1378)
> [dvs] Fix install instructions in README (#1379)
> [dvs] Update README with new flags, options, and known issues (#1380)
> swss: gearsyncd should return 0 on exit (#1376)
> Remove 00-copp.config.json from swss debian package. (#1366)
> fix undefined var in rates lua scripts (#1365)
> [fdborch] Fixed Orchagent crash in FDB flush on port disable. (#1369)
> [tlm_teamd]: Try to add LAG again, when teamd is not ready first time (#1347)
> [vs] Incorporate python3 best practices into DVSLib (#1357)
> [dvs] Mark unstable tests as xfail (#1356)
- Why I did it
When SONiC is configured with the management framework and/or telemetry services, the applications running inside those containers need to access some functionality on the host system. The following is a non-exhaustive list of such functionality:
Image management
Configuration save and load
ZTP enable/disable and status
Show tech support
- How I did it
The host service is a Python process that listens for requests via D-Bus. It will then service those requests and send a response back to the requestor.
This PR only introduces the host service infrastructure. Applications that need access to the host services must add applets that will register on D-Bus endpoints to service the appropriate functionality.
- How to verify it
- Description for the changelog
Add SONiC Host Service for container to execute select commands in host
Signed-off-by: Nirenjan Krishnan <Nirenjan.Krishnan@dell.com>
* src/sonic-utilities d5fdd74...17fb378 (7):
> [sonic-installer] Import re module (#1061)
> [fast-reboot]: Fix fail to execute fast-reboot problem (#1047)
> [config] Reduce Calls to SONiC Cfggen (#1052)
> [filter-fdb] Call Filter FDB Main From Within Test Code (#1051)
> [sflow_test.py]: Fix show sflow display. (#1054)
> Change fast-reboot script to use swss and radv service script (#1036)
> Common functions for show CLI support on multi ASIC (#999)
- Ignore directories generated by building Python wheel package
- Move all sonic-config-engine ignores from the root .gitignore to src/sonic-config-engine/.gitignore
Update sonic-snmpagent submodule to include below commits:
1a2b62a [Namespace]: Fix SAI_ID key used in cpfcIfTable and csqIfQosGroupStatsTable implementation (#138)
d06f00c [pytest/coverage]: add coverage support (#156)
90e9f2e [Namespace]: Simplify sync_d functions to use higher order (#154)
b5815d9 [LLDP]: Modify OID index of LLDPRemTableUpdater MIB (#155)
d5f2b92 [Multiasic]: Provide namespace support for ipNetToMediaPhysAddress (#129)
166c221 [Namespace]: Fix interface counters in RFC 1213 (#145)
Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
Argument to write to config-db is not allowed when using template.
This PR allows cfggen to write to redis db when using template
mode.
signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
* Add sonic_interface.py in sonic-py-common for sonic interface utilities to keep this SONIC PREFIX naming convention in one place in py-common and all modules/applications use the functions defined here.
Changes:
-- Correct Pattern for VLAN.
-- Add Special Test support.
-- Add first special test to load VLAN 1-4094.
-- Add lanes in ports.
-- Make Sure None test gets no output from libyang while config load.
[yangModelTesting.py]: INCORRECT VLAN_NAME FIELD IN VLAN TABLE test.
Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com
[db_migrator] Support migrating database regarding buffer configuration for all Mellanox switches (#993)
[show/vlan]: fix show vlan config (#1050)
[PFCWD] Fix issue with "pfcwd show stats" command during SONiC init (#1018)
[config] Eliminate port breakout-related globals (#1045)
[teamshow]: refactor teamshow to use state db information (#1049)
[watermarkstat] Import print_function from __future__ (#1048)
[show]: split show interfaces commands into separate file (#1044)
Python 3 compliance (#1043)
Update all references to new 'sonic-installer' file name (#1033)
[config/show]: split vlan into separate file (#1038)
[pytest/coverage]: add coverage for all scripts (#1041)
enable watchdog before running platform specific reboot plugin (#1037)
[tests]:add coverage for config/main/utilities_common module (#1039)
[cli/feature]: split feature command into a separate file (#1034)
Signed-off-by: Stephen Sun <stephens@nvidia.com>
Co-authored-by: Stephen Sun <stephens@mellanox.com>
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan arlakshm@microsoft.com
The following common APIs are added for multi ASIC
- an API to check if a given port is a internal or external port
- an API to check if a given port-channel is internal or external
- an API to check if a bgp-session is internal or external
- an API to connect to the config and other dbs in the a given namespace
- added common APIs to the sonic_py_common library.
- update the sample port-config.ini with role column and add corresponding test to verify the ports configuration is - generated properly.
Calls to cfggen take considerable time. With batch mode, we will have the ability
to reduce number of calls from services.
Example of the batch mode command:
sonic-cfggen -t template-1.j2 -t template-2.j2,config-db -t template-3.j2,config-db -t template-4.j2,file1 -t template-5.j2,file2 --write-to-db.
template-1.j2 will be rendered to stdout since it is missing the dest part. stdout is default
config-db is a special keyword that will inject the rendered template into internal data structure. The internal data structure gets written to redis-db with --write-to-db switch. In the case the user would like to write to a file named config-db, it could be given as /config-db or ./config-db
signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
NUM_ASIC environment variable was added so that it could be used by other utilities.
This is not being used by any other utility or docker, hence removing the addition of NUM_ASIC environment variable.
Also, the environment variable was added by adding the variable value to /etc/environ file.
Upon each reboot, this file gets updated with the NUM_ASIC value but the existing value was not removed.
This causes multiple lines getting appended in /etc/environ file upon each reboot.
Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
* Bring up FPGA ports and test it
* Bring up those ports in neighbors dict
* Revert delete of a line
* Add test
* change code comment
* Change test name
* Revert submodule update
* src/sonic-utilities 9700e45...621aad0 (9):
> [Python] Migrate applications/scripts to import sonic-py-common package (#1008)
> [config qos] QoS and Buffer config genration for multi ASIC platforms (#978)
> [cli]: pass db connector as click context (#1029)
> [PDDF] Make utilities compatible to platform API 2.0, in accordance with pddf_2.0 (#940)
> [tests] Add unit tests for 'show platform ...' commands (#1021)
> Add secure fast/warm-reboot support for Aboot (#994)
> [sonic-installer] Create Envvars File for Incoming Image (#1011)
> [config load]: do not stop/reset/reload service if it is disabled (#1028)
> [tests]: move feature_test.py into tests folder (#1027)
* src/sonic-ztp c959371...dd025bc (2):
> Update all references to new 'sonic-installer' file name (#18)
> Filter out non-printable characters read from syseeprom (#17)
* src/sonic-platform-common be1cc24...f3f3573 (3):
> Remove sonic-config-engine dependency from setup.py (#109)
> Migrate from sonic-daemon-base package to sonic-py-common package (#103)
> [Cables] Add support for 'Extended Specification Compliance' for QSFP cables (#108)
> [sfputilbase.py] Add application_advertisement NA field to transceiver_info_dict for platform api 1.0 (#104)
* src/sonic-platform-daemons 49d145c...8e0704e (1):
> Proper fix for thermalctld using sonic-py-common (#80)
> Fix thermalctld tests which were broken by the transition to sonic-py-common (#79)
> Migrate from sonic-daemon-base package to sonic-py-common package (#74)
1. remove container feature table
2. do not generate feature entry if the feature is not included
in the image
3. rename ENABLE_* to INCLUDE_* for better clarity
4. rename feature status to feature state
5. [submodule]: update sonic-utilities
* 9700e45 2020-08-03 | [show/config]: combine feature and container feature cli (#1015) (HEAD, origin/master, origin/HEAD) [lguohan]
* c9d3550 2020-08-03 | [tests]: fix drops_group_test failure on second run (#1023) [lguohan]
* dfaae69 2020-08-03 | [lldpshow]: Fix input device is not a TTY error (#1016) [Arun Saravanan Balachandran]
* 216688e 2020-08-02 | [tests]: rename sonic-utilitie-tests to tests (#1022) [lguohan]
Signed-off-by: Guohan Lu <lguohan@gmail.com>
Applications running in the host OS can read the platform identifier from /host/machine.conf. When loading configuration, sonic-config-engine *needs* to read the platform identifier from machine.conf, as it it responsible for populating the value in Config DB.
When an application is running inside a Docker container, the machine.conf file is not accessible, as the /host directory is not mounted. So we need to retrieve the platform identifier from Config DB if get_platform() is called from inside a Docker
container. However, we can't simply check that we're running in a Docker container because the host OS of the SONiC virtual switch is running inside a Docker container. So I refactored `get_platform()` to:
1. Read from the `PLATFORM` environment variable if it exists (which is defined in a virtual switch Docker container)
2. Read from machine.conf if possible (works in the host OS of a standard SONiC image, critical for sonic-config-engine at boot)
3. Read the value from Config DB (needed for Docker containers running in SONiC, as machine.conf is not accessible to them)
- Also fix typo in daemon_base.py
- Also changes to align `get_hwsku()` with `get_platform()`
- Created the VS setup to test DPB functionality
- Created "platform.json" for VS docker
- Added test case for Breakout CLI
Signed-off-by: Sangita Maity <sangitamaity0211@gmail.com>
[xcvrd] Fix bailing out on platforms that do not support QSFP-DD (#78)
Fix psud and test_thermalctld (#75)
[Thermal] Fix for 'show platform fan' command (#71)
Signed-off-by: Volodymyr Boyko <volodymyrx.boiko@intel.com>
As part of consolidating all common Python-based functionality into the new sonic-py-common package, this pull request:
1. Redirects all Python applications/scripts in sonic-buildimage repo which previously imported sonic_device_util or sonic_daemon_base to instead import sonic-py-common, which was added in https://github.com/Azure/sonic-buildimage/pull/5003
2. Replaces all calls to `sonic_device_util.get_platform_info()` to instead call `sonic_py_common.get_platform()` and removes any calls to `sonic_device_util.get_machine_info()` which are no longer necessary (i.e., those which were only used to pass the results to `sonic_device_util.get_platform_info()`.
3. Removes unused imports to the now-deprecated sonic-daemon-base package and sonic_device_util.py module
This is the next step toward resolving https://github.com/Azure/sonic-buildimage/issues/4999
Also reverted my previous change in which device_info.get_platform() would first try obtaining the platform ID string from Config DB and fall back to gathering it from machine.conf upon failure because this function is called by sonic-cfggen before the data is in the DB, in which case, the db_connect() call will hang indefinitely, which was not the behavior I expected. As of now, the function will always reference machine.conf.
* [platform] Add Support For Environment Variable
This PR adds the ability to read environment file from /etc/sonic.
the file contains immutable SONiC config attributes such as platform,
hwsku, version, device_type. The aim is to minimize calls being made
into sonic-cfggen during boot time.
singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
To clarify error messages in case the ip address for Loopback is already set. It doesn't make sense to call correct ip address as ambiguous in this case
Consolidate common SONiC Python-language functionality into one shared package (sonic-py-common) and eliminate duplicate code.
The package currently includes three modules:
- daemon_base
- device_info
- logger
**sonic-platform-common:**
Commits on Jul 07, 2020
Changes in fan and psu base classes (1.0 platform API) related to pdd… 17292e4
Commits on Jul 10, 2020
update get_pcie_check() to use sysfs instead of lspci output (#95) d4eb804
Commits on Jul 11, 2020
[eeprom] Fix UnboundLocalError (#93) 7c8bed1
Commits on Jul 14, 2020
[Transceiver] Add parser for QSFP-DD cable type and dictionaries for … be1cc24
**sonic-platform-daemons:**
Commits on Jul 06, 2020
[psud] Store PSU temperature and voltage information to database (#61) ef9716a
Commits on Jul 14, 2020
[xcvrd] Add support for QSFP-DD cables (#66) c530587
Commits on Jul 18, 2020
Initial version of pcied (#60) e665ee8
[xcvrd] Return non-zero error code on SFP error (#67) 4f42a79
Commits on Jul 19, 2020
[README.md] Add LTGM badges (#69) 23757a3
Remove unused imports (#70) 029d5a5
Signed-off-by: Nazarii Hnydyn <nazariig@mellanox.com>
Update the sonic-swss-common submodule. The following are the commits in the submodule.
Fix sai api loglevel
ed9c497316e73bc5d4a6dfbe9f3507d5e92c15f3
Add support for attribute capability query in lua script
f58ea1e7a692eadd8695b4dcf89290c5e9018aae
Copp changes
4e50be7728d1e68582ac4dc369323ec5ab7b81dd
fix pytest for python3
c76d3b3a6dacae7fde14622c41e41b717876924a
use sysconfig instead of distutils.sysconfig
1772581f9b9f25e5ab2b790d7d517ea2b2010a78
CONFIG_DB table name for BUM Storm-control feature
c39c958bea445e650353651fcbae5c228114f86d
Fine Grained ECMP schema changes
1974d2e1547464a3b0279b5d63fc40eda94fa780
* src/sonic-telemetry fa8d498...3bd7ca3 (4):
> Update gnmi deps (#40)
> [testdata] Update SFP keys to align with new standard (#39)
> Fixed the parameters for subscribe APIs (#38)
> Azure ro mode (#34)
* src/sonic-mgmt-common 444aa9a...cc01ce4 (4):
> Make gnmi dep version the same as in telemetry repo (#17)
> Cleanup translib and cvl go test cases (#13)
> Package update and enhancements/fixes in YGOT, and Request Binder (#12)
> Translib phase I changes (#11)
Note: sonic-telemetry submodule update is dependent upon sonic-mgmt-common submodule update, thus updating both in this patch
This fix removes the requirement to have a NEIGHBOR_METADATA for dynamic peers. The change is made since it is not necessary for NEIGHBOR_METADATA entries be present for the dynamic neighbors
* [daemon_base] fix to not reregister signal handler
-src/sonic-daemon-base/sonic_daemon_base/daemon_base.py
Problem:
Currently all daemons inherit from daemon_base class, and for
signal handling functionality they register the signal_handler() by
overriding the siganl_handler() in daemon_base by their own
implmentation.
But some sonic_platform instances also can invoke the daemon_base
constructor while trying to instantiate the common utilities
for example
platform_chassis = sonic_platform.platform.Platform().get_chassis()
This will cause the re registration of signal_handler which will
cause base class signal_handler() to be invoked when the daemon
gets a signal, whereas their own signal_handler should have been
invoked.
Fix:
We only register the siganl_handler once, and if signal_handler has
been registered, not re register it.
Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
* [daemon_base] fix to not reregister signal handler
Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
* src/sonic-utilities 9715244...a23479e (7):
> [sonic-installer] Hyphens instead of underscores in command and subcommands (#983)
> Fix the None Type Exception when Interface Table does not exist (cold boot) as part of db migration (#986)
> Support to verify reboot for secure boot (#979)
> [filter-fdb] Fix For Vlan Defined With No CIDR (#976)
> Intf table migration for APP_DB entries during warmboot (#980)
> Enable HW watchdog before fast-reboot (#977)
> [Multi NPU] Time Improvements to the config reload/load_minigraph commands (#917)
This pull request was cherry picked from "#1238" to resolve the conflicts.
- Why I did it
Add support to specify source address for TACACS+
- How I did it
Add patches for libpam-tacplus and libnss-tacplus. The patches parse the new option 'src_ip' and store the converted addrinfo. Then the addrinfo is used for TACACS+ connection.
Add a attribute 'src_ip' for table "TACPLUS|global" in configDB
Add some code to adapt to the attribute 'src_ip'.
- How to verify it
Config command for source address PR in sonic-utilities
config tacacs src_ip <ip_address>
- Description for the changelog
Add patches to specify source address for the TACACS+ outgoing packets.
- A picture of a cute animal (not mandatory but encouraged)
**UT logs: **
UT_tacacs_source_intf.txt
Resubmitting the changes for (#4825) with fixes for sonic-bgpcdgd test failures
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
also update submodule
* 01f810f 2020-07-02 | fix compiling issue for gcc8.3 (#1339) [lguohan]
* 9b13120 2020-07-03 | Fix in script to avoid orchagent crash when port down followed by fdb delete (#1340) [rupesh-k]
* 9b01844 2020-07-01 | [qosorch] Update QoS scheduler params for shaping features (#1296) [Michael Li]
* 86b5e99 2020-07-02 | [mirrororch] Port Mirroring implementation (#1314) [rupesh-k]
* c05601c 2020-06-24 | [portsyncd]: add debug message if a port cannot be found in port able (#1328) [lguohan]
* a0b6412 2020-06-23 | COPP_DEL_fix: DEL for one trap group from SONIC is resetting all the trap IDs (#1273) [SinghMinu]
Signed-off-by: Guohan Lu <lguohan@gmail.com>
make swss build depends only on libsairedis instead of syncd. This allows to build swss without depending
on vendor sai library.
Currently, libsairedis build also buils syncd which requires vendor SAI lib. This makes difficult to build
swss docker in buster while still keeping syncd docker in stretch, as swss requires libsairedis which also
build syncd and requires vendor to provide SAI for buster. As swss docker does not really contain syncd
binary, so it is not necessary to build syncd for swss docker.
* [submodule]: update sonic-sairedis
* ccbb3bc 2020-06-28 | add option to build without syncd (HEAD, origin/master, origin/HEAD) [Guohan Lu]
* 4247481 2020-06-28 | install saidiscovery into syncd package [Guohan Lu]
* 61b8e8e 2020-06-26 | Revert "sonic-sairedis: Add support to sonic-sairedis for gearbox phys (#624)" (#630) [Danny Allen]
* 85e543c 2020-06-26 | add a README to tests directory to describe how to run 'make check' (#629) [Syd Logan]
* 2772f15 2020-06-26 | sonic-sairedis: Add support to sonic-sairedis for gearbox phys (#624) [Syd Logan]
Signed-off-by: Guohan Lu <lguohan@gmail.com>
* Loopback IP changes for multi ASIC devices
multi ASIC will have 2 Loopback Interfaces
- Loopback0 has globally unique IP address, which is advertised by the multi ASIC device to its peers.
This way all the external devices will see this device as a single device.
- Loopback4096 is assigned an IP address which has a scope is within the device. Each ASIC has a different ip address for Loopback4096. This ip address will be used as Router-Id by the bgp instance on multi ASIC devices.
This PR implements this change for multi ASIC devices
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
If some table with a list of tuples (interface name, ip prefix) has ip prefixes without a mask length, it will cause issues in SONiC. For example quagga and frr will treat ipv4 address without a mask, so "10.20.30.40" address will be treated as "10.0.0.0/8", which is dangerous.
The fix here is that when pfx_filter get a tuple (interface name, ip prefix), where the ip prefix doesn't have prefix mask length, add a mask by default: "/32 for ipv4 addresses, /128 for ipv6 addresses".
Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
Updated the NAT iptables patch for 4.19 buster
Depends on PR : Azure/sonic-linux-kernel#147
1 Known issue:
With both NAT patch files for 4.19 buster kernel, seeing 1 display issue in iptables like explained below
On Docker NAT, iptables supported version is 1.6.0 and on base OS it’s 1.8.2. So seeing an display issue of which fullcone option is not showing in version 1.8.2 iptables output and no issues in functionality.
Display issue – For example of comparsion:
NAT Docker:
root@sonic:/home/admin# docker exec -it nat bash
root@sonic:/# iptables -t nat -nvL
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DNAT all -- * * 0.0.0.0/0 0.0.0.0/0 to:1.1.1.1 fullcone
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 30 packets, 2749 bytes)
pkts bytes target prot opt in out source destination
Chain POSTROUTING (policy ACCEPT 30 packets, 2749 bytes)
pkts bytes target prot opt in out source destination
root@sonic:/#
Base OS:
root@sonic:/home/admin# iptables-legacy -t nat -nvL
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
1 36 DNAT all -- * * 0.0.0.0/0 0.0.0.0/0 to:1.1.1.1
Chain INPUT (policy ACCEPT 1 packets, 36 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 41 packets, 3572 bytes)
pkts bytes target prot opt in out source destination
Chain POSTROUTING (policy ACCEPT 41 packets, 3572 bytes)
pkts bytes target prot opt in out source destination
root@sonic:/home/admin#
To fix this issue, iptables need to update from 1.6.0 to 1.8.2 version and have to update the NAT docker from stretch to buster. Will raise a new PR with this.
Signed-off-by: Akhilesh Samineni akhilesh.samineni@broadcom.com
Signed-off-by: Akhilesh Samineni <akhilesh.samineni@broadcom.com>
* [systemd-generator]: Fix the code to make sure that dependencies
of host services are generated correctly for multi-asic platforms.
Add code to make sure that systemd timer files are also modified
to add the correct service dependency for multi-asic platforms.
Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
* [systemd-generator]: Minor fix, remove debug code and
remove unused variable.
* src/sonic-platform-common 75698a8...82bbeab (9):
> [sfputil] Make SfpUtilHelper.get_physical_to_logical noexcept as in SfpUtilBase (#96)
> [sfp_base] Update return value documentation of channel-specific methods (#98)
> [sfp] Tweak key names of some transceiver info fields (#97)
> fix typo: portconfig.ini to port_config.ini (#94)
> [chassis_base] Add platform API support for system LED (#91)
> Add PCIe check commad (#64)
> [sfputilbase.py] Don't try to print EEPROM sysfs file name if we failed to read from it (#81)
> [sfputilbase | sfputilhelper] Add support of platform.json (#72)
> [eeprom] Add try-except to catch the IOError (#85)
* src/sonic-platform-daemons 0f4fd83...abe115e (2):
> [xcvrd] Tweak some transceiver info key names (#62)
> [psud][thermalctld] Always get fan/PSU LED status from platform API to avoid status inconsistencies (#59)
* src/sonic-utilities fd7781b...16a33f2 (9):
> [config] Fix syntax error (#966)
> [config] Fix indentation level in _get_disabled_services_list() (#965)
> a4e64d1 [sonic_installer] Refactor sonic_installer code (#953)
> 90efd62 [Show | Command Reference] Add Port breakout Show Command (#859)
> [sfpshow][mock_state_db] Tweak key names of some transceiver info fields (#958)
> [show] Add missing verbose option to "show line" (#961)
> [filter-fdb] Check VLAN Presence When Filter FDB (#957)
> [master]fix #4716 show ipv6 interfaces neighbor_ip is N/A issue (#948)
> Fix for command. show interface transceiver eeprom -d Ethernet (#955)
Note: sonic-utilities update fixes#4716
* c60b1f4 2020-06-26 | e1000: Do not perform reset in reset_task if we are already down (#148) (HEAD -> master, origin/master, origin/HEAD) [lguohan]
* c6aeedd 2020-06-25 | Updated NAT kernel patch for 4.19 buster (#147) [Akhilesh Samineni]
Signed-off-by: Guohan Lu <lguohan@gmail.com>
To enable tagged vlan support by minigraph parser. This enables us to generate a config_db file that will enable SONiC device to operate using tagged and untagged vlan.
- Why I did it
New repo sonic-mgmt-common is introduced for the common translib related code. This commit adds build rules for this new repo.
- How I did it
Added sonic-mgmt-common submodule
Added build rules for the new sonic-mgmt-common repo. It creates two deb packages -- sonic-mgmt-common_1.0.0_{arch}.deb and sonic-mgmt-common-codegen_1.0.0_{arch}.deb. Package cache is enabled.
Added dependency on sonic-mgmt-common for mgmt-framework and telemetry debs and dockers.
- How to verify it
Full build and incremental builds
Basic ACL and interface opreations through REST, KLISH CLI and gNMI
- Description for the changelog
Git submodule and build rules for the new sonic-mgmt-common repo.
* Add secureboot support in boot0
* Initramfs changes for secureboot on Aboot devices
* Do not compress squashfs and gz in fs.zip
It doesn't make much sense to do so since these files are already
compressed.
Also not compressing the squashfs has the advantage of making it
mountable via a loop device.
* Add loopoffset parameter to initramfs-tools
