Commit Graph

1408 Commits

Author SHA1 Message Date
Tamer Ahmed
cbbda09599
[relay]: Prevent Buffer Overrun Of Malformed DHCP Packet (#6057)
[dhcp-relay]: Prevent Buffer Overrun Of Malformed DHCP Packet

The add/strip relay agent options does not take into account the buffer
length and so it is possible to overrun the buffer. The issue will
result in contents from previous packet being added to the current one.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-12-11 16:28:05 -08:00
Dong Zhang
6999ce5282
[sonic-utilities] update submodule for supportting MultiDB warmboot backing up database (#6181)
update sonic-utilites submodule, adding MultiDB warmboot backing up database support:

*    [MultiDB] add MultiDB warmboot support - backing up database (#1205) 
    * lua script to backup all database into one database and create rdb file - following earlier multiDB warmboot design at https://github.com/Azure/SONiC/blob/master/doc/database/multi_database_instances.md
    * copy this rdb file as before to WARM_DIR
    * restoring database part is in another PR at sonic-buildimage  https://github.com/Azure/sonic-buildimage/pull/5773, they depend on each other
2020-12-11 11:30:50 -08:00
Lawrence Lee
fd4433d836
[minigraph.py]: Remove prefix length from peer switch loopback address (#6174)
* PEER_SWITCH table in config DB expects a standalone IP address w/o a prefix length

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2020-12-10 14:43:39 -08:00
Sabareesh-Kumar-Anandan
8d00b57959
[sonic-utilities] Update sonic-utilities submodule (#6139)
[show] Moving some py imports inside specific cmds (#1202)
Fix python3 issue in ecnconfig (#1286)

Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>
2020-12-10 01:29:30 -08:00
Ying Xie
d9b9e4f7db
[swss] advance swss submodule head (#6157)
- Why I did it
Advance swss submodule to pick up latest changes.

- How I did it
Including folowing changes:

[portsorch] adjust port initialized event back to notice (#1532)
Signed-off-by: Ying Xie ying.xie@microsoft.com
2020-12-10 00:28:14 -08:00
Blueve
3d22019802
[sonic-config-engine/minigraph] Enable console mgmt feature for console device (#6166)
* Introduced a list console_device_types which contains the device types that support console management feature
* Inject CONSOLE_SWITCH:console_mgmt table with enabled:yes or enabled:no

Signed-off-by: Jing Kan jika@microsoft.com
2020-12-10 15:42:11 +08:00
Qi Luo
0e554e09ce
[makefile] Remove python-netsnmp deb package from makefile (#6161)
Because no one is using it in buildimage repo
2020-12-09 17:40:07 -08:00
Joe LeVeque
762dfd965b
[sonic-platform-daemons] Update submodule (#6158)
* src/sonic-platform-daemons 73e6ddd...4da0bfc (3):
  > Align style with PEP8 standards (#128)
  > Support python3 for xcvrd, psud, thermalctld and syseepromd (#132)
  > Import mock psu object for testing LED (#119)
2020-12-09 12:04:56 -08:00
Kamil Cudnik
275c5cfb07
[submodule] sonic-swss update: Add zmq sync mode flag (#1505) (#6144)
To support communication mode flag on orchagent
2020-12-08 11:37:04 -08:00
dflynn-Nokia
98a8753c11
[sonic-mgmt-common] Update submodule (#6129)
This update brings in the following commits.

86c1108 Enable arm architecture to build in addition to amd64 (#37)
4acb2c3 fix bugs and enhance Transformer (#35)
49e5a22 ygot related enhancements and fixes (#34)
51224de Fix ietf yang search path for cvl schema builds (#32)
3c6cdb3 CVL Changes #8: 'must' and 'when' expression evaluation (#31)
dabf231 CVL Changes #7: 'leafref' evaluation (#28)
6f9535f CVL Changes #6: Customized Xpath Engine integration (#27)
5e2466b DB-Layer fixes/enhancements (#26)
9a27302 CVL Changes #4: Implementation of new CVL APIs (#22)
dbf1093 Translib support for authorization, yang versioning and Delete flag (#21)
80f369e CVL Changes #5: YParser enhancement (#23)
904ce18 CVL Changes #3: Multi-db instance support (#20)
9d24a34 CVL Changes #2:  YValidator infra changes for evaluating xpath expression (#19)
f3fc40f CVL Changes #1: Initial CVL code reorganization and common infra changes (#18)
4922601 Bulk and RPC API support in translib (#16)
1d730df RFC7895 yang module library implementation (#15)
2020-12-08 04:31:16 -08:00
Myron Sosyak
3b04da97fb
[sonic-platform-common] Update submodule (#6141)
Update sonic-platform-common submodule:
* Make eeprom_tlvinfo.py Python3 compatible

**- Why I did it**
To get the latest changes which fix some python2 -> python3 migration errors.
2020-12-07 10:04:48 -08:00
yozhao101
3e717210e9
[sonic-telemetry] Update the submodule sonic-telemetry. (#6132)
[dataset] Add dataset "system uptime" into non-db client. (#52)
Adding new data set to query Sonic OS version. (#50)
[gnmi_server] Disregard EOF status for STREAM subs (#48)

Signed-off-by: Yong Zhao <yozhao@microsoft.com>
2020-12-06 06:01:46 -08:00
Kamil Cudnik
af357f3ec6
[sairedis] Advance sairedis pointer to support cmd zmq flag (#6064)
[sairedis] Advance sairedis pointer to support cmd zmq flag

[meta] Use memcpy instead of cast to prevent strict-aliasing error (#723)
[vslib]Add MACsec forward and filters to HostInterfaceInfo (#719)
[vslib] Add StateBase function for MACsec (#717)
Add support for default zmq synchronous mode flag (#711)
[syncd] Code clean (#720)
[sairedis] Remove custom bulk fdb methods (#710)
[vslib]Add MACsec Filters (#713)
[vslib]Add MACsec Forwarder (#714)
[vslib]Add MACsec Manager (#715)
Add helper functions, findObjects and dumpObject (#716)
Code clean refactor (#712)
[vslib] Fix CorePortIndexMap log line (#708)
[meta] Use custom hash in SaiObjectCollection (#709)
Fix LGTM localtime function warnings (#707)
[vs] VoQ Switch objects initialization - Local Port OID mapping to System Ports (#703)
Code style refactor (#705)
[vs] Initialization of VOQ switch objects (#702)
[vs] SAI support for VOQ switches - Switch State Initialization (#701)
Add MACsec meta methods (#704)
[vs] SAI support for VOQ switches (#698)
[vs] SAI support for VOQ switches - Core Port Index Map File parser (#700)
[vs] SAI support for VoQ switch - Core Port Index Map Container (#699)
[syncd][sairedis] Change pub/sub model to push/pull in zmq notification (#695)
[syncd] Use lua script to update db when using bulk api (#690)
[syncd] Fix bulk api object type for next hop group members (#685)
Add FlexCounter for MACsec SA (#684)
2020-12-05 12:13:34 +01:00
abdosi
59c1e3a78a
[multi-asic] Enhancing monit process checker for multi-asic. (#6100)
Added Support of process checker for work on multi-asic platforms.
2020-12-04 10:39:43 -08:00
rajendra-dendukuri
8a04487499
[kdump]: Implement Kdump configuration handler (#6122)
- Kdump configurations stored and manipulated in ConfigDB are now processed
by hostcfgd and applied asynchronously

Signed-off-by: Rajendra Dendukuri <rajendra.dendukuri@broadcom.com>
2020-12-04 10:15:30 -08:00
vdahiya12
03287cdbd7
[sonic-platform-common][sonic-platform-daemons] submodule update (#6124)
This PR updates the submodule sonic-platform-common, sonic-platform-daemons

For sonic-platform-common update following commits are added
a8823a3 [sonic_y_cable] add check_mux_direction api for y_cable (#151)
7c0301d Modular Chassis - APIs for midplane monitoring (#148)
6f7d8a0 Fix some LGTM alerts; Add LGTM badges to README.md (#146)

for sonic-platform-daemons update following commits are added
73e6ddd [xcvrd] replace check_active_linked_tor_side api to check_mux_direction (#131)

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
2020-12-04 04:27:42 -08:00
Stephen Sun
fdb51e7f33
Advance submodule head for sonic-swss and sonic-swss-common (#6111)
swss:
- [port_rates][rif_rates] Fix port_rates.lua (#1520)
- Improve the way of handling BUFFER_PG during PFC storm (#1480)
- [Orchagent] Vxlanorch and Portsorch changes for EVPN VXLAN (#1264)
- [sub intf] ecmp hardware convergence acceleration at parent port oper status changes (#1492)
- [dvs/copp] Adjust policer checks in VS tests for default COPP policers (#1519)
- Flush ARP/neighbor entry on FDB flush when port L2-L3 (#1506)
- [intfmgr]: Add GARP support (#1503)

swss-common:
- [Dynamic buffer calc] Support dynamic buffer calculation (#361)
- README.md: Update build requirements (#414)

Signed-off-by: Stephen Sun <stephens@nvidia.com>
2020-12-03 23:30:28 -08:00
Joe LeVeque
0e101247ac
[sonic-utilities] Update submodule (#6107)
* src/sonic-utilities 6fabbed...3749f5e (7):
  > [config vlan] Stop, reset-failed, then start dhcp_relay service (#1269)
  > [show] Break some groups out into their own modules (#1259)
  > Convert the output of 'arp -n' from bytes to str. (#1279)
  > Fixing python porting errors in fdbshow, natshow and nbrshow (#1280)
  > [pfcwd] Use floor division in Python3 (#1277)
  > Modify fast-reboot script to use teamd service script (#1251)
  > Add `show reboot-cause history` cli (#1210)
2020-12-03 15:57:21 -08:00
vdahiya12
356522bfcd
[sonic-platform-daemons] submodule update (#6116)
this PR updates the following Commits in sonic-platform-daemons
ab39059 [xcvrd] Fix y_cable state updates from 'failure' to 'unknown' on error conditions/events (#129)
5d74356 Clean some unused variables, check for multiAsic before calling initializeGlobalConfig (#130)
12b3628 [psud] Make sure psu leds are set on the first run (#125)

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
2020-12-03 09:47:04 -08:00
Dmytro Shevchuk
9a82dca483
[sonic-py-swsssdk] update submodule: fix fdbshow, show arp commands (#6074)
Update submodule for PR Azure/sonic-py-swsssdk#95
2020-12-02 18:17:24 -08:00
lguohan
4812953468
[ntp]: build ntp with various fixes (#6037)
- NTP Bug 1970 (UNLINK_EXPR_SLIST empty list) Fix
- ENOBUFS log message level set to WARN
- Fix audit message seen on console apparmor
- add force-confold option when install ntp

Signed-off-by: Guohan Lu <lguohan@gmail.com>
Co-authored-by: Prabhu Sreenivasan <prabhu.sreenivasan@broadcom>
2020-12-02 15:02:50 -08:00
trzhang-msft
c59975c7da
DHCP relay: support for dual ToR (#6066)
* DHCP relay: support for dual ToR

* update to use -U option

* update

* update
2020-12-02 14:43:20 -08:00
vganesan-nokia
19b592b8aa
[vstest]VS test for VOQ System ports (#5862)
Changes for supporting vstest for VOQ system ports. The changes include:
(1)Use of chassis_db.json is avoided since the SYSTEM_PORT is made
available in virtual chassis linecard's default_config.json which will
be loaded during bootup
(2)Core port index map file is introduced and is copied from virtual chassis
directory to hwsku direcory by start.sh
(3)vs sai profile is modified to include core port index map file name

Signed-off-by: vedganes <vedavinayagam.ganesan@nokia.com>
2020-12-02 10:31:43 -08:00
Xin Wang
e4ee07b79f
[bgp]: Fix bgp crash after BGP allow list configuration is added (#6088)
The issue was a typo introduced in #6006. In that change, the BGP allow list
configuration manager was updated to use a method of common ConfigMgr
for restarting peer groups. However, the method name 'restart_peers' was
used instead of the correct 'restart_peer_groups'.

This change updated the managers_allow_list.py to use correct method
'restart_peer_groups' for restarting peer groups.

Signed-off-by: Xin Wang <xiwang5@microsoft.com>
2020-12-02 09:15:06 -08:00
Ying Xie
443f81f9b7
[hostcfgd] check cached state instead of the next state (#6067)
- Why I did it
'always_enabled' feature can still be disabled/enabled.

- How I did it
When checking if a feature is 'always_enabled', check the cached state to prevent new change to be accepted.
Fix an issue where cache value is updated before all the check is done.
Restore 'always_enabled' value in config db if someone wants to change.

Signed-off-by: Ying Xie ying.xie@microsoft.com

- How to verify it
Without the fix, 'always_enabled' feature can be enabled or disabled without cli protection. With the protection, the change will be rejected properly.
2020-12-01 17:56:39 -08:00
abdosi
9265022502
[sonic-utilities] Update submodule (#6070)
6fabbedc8ba0f7961bec3ec0adf7bb9d5c71eb2b [consutil][test] Add unit tests for consutil clear command (#1274)
a4f663ea0d7b52eeb76d903b19119c34ab86b1e9 [show/fgnhg] Fix crash bug (#1272)
9f276b21b06423ad11d75fbe17f46a43723f62a5 [config][loopback] Fix Bug that cannot add loopback interface by CLI (#1276)
a5b78cf968e4e5c49b764f5a76bdea6d6532c8d3 [config][show] CLI support for interacting with muxcable (#1221)
8f3b22ed11e166ed24dcd2fb1ac61e227140c2af [connect][sonic-clear] Align the exit code with consutil for line commands (#1256)
cfb7a2269406e8cca354db8a579c401e97f66e9f Feature table cli command update (#1271)
5b8da561b8712e9785622a53a2898671ada9c76e Added Multi-ASIC support for show ip(v6) route (#1216)
a7f39b04eb4b14f56f14f9851d59b81ecd5285f8 Improve MockRedis _encode(): so it will work on all types of value (#1265)
8427caf28ea60af97d8eb1eb3ccb75e99bc54184 [config_mgmt.py]: Remove sort_data call, we do not need it now. (#1246)
9dc58eaa76d38a40f33d534b13442284f598852a [sonic-utilities] Fix show vnet [neighbors | routes [all | tunnel]] crashed issue (#1263)

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2020-12-01 09:37:01 -08:00
Garrick He
112eb08f39
[sflow] Bump hsflowd version in Makefile (#6065)
* Specify the hsflowd version in the Makefile using variables instead of hardcore.

Signed-off-by: Garrick He <garrick_he@dell.com>
2020-11-30 18:46:42 -08:00
abdosi
872c85d8e7
[lldp]: Lldp docker to use python3 version of sonic-db-syncd package. (#6046)
Made changes so that Lldp docker start using py3 of sonic-db-syncd

submodule update sonic-db-syncd
5cc29a1b32d8d1f4dfbc967bfea2727c50a49c76 (HEAD -> master, origin/master, origin/HEAD) Changes to convert sonic-dbsyncd from python 2 to 3

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2020-11-30 10:44:40 -08:00
Ying Xie
7f21c0be1a
[FRR] remove the whole block of outchannel properly (#6045)
- Why I did it
Fix issue #6043

- How I did it
We are disabling in container frr log. The log entries are sent to base image and are logged in /var/log/quagga/bgpd.log.

However, we need to remove the whole outchannel config block to avoid an error message raised by rsyslogd.

- How to verify it
Without the change, test_autorestart bgp container will fail on loganalyer errors. With the change, restarting bgp container is no longer generating error message and the test will pass.

The log generated by frr continued appearing in /var/log/quagga/bgpd.log
2020-11-26 16:50:32 -08:00
Joe LeVeque
00d3cb7a7c
[sonic-platform-common] Update submodule (#6041)
Commits include:

* src/sonic-platform-common d912db6...fa0aaa6 (3):
  > [sonic_sfp]: add logical_to_asic dict (#141)
  > [sonic_pcie][sonic_ssd] Make Python 3-compliant (#150)
  > [eeprom_base] Close lock file after unlocking (#149)
2020-11-26 13:14:56 -08:00
Joe LeVeque
829c116f13
[sonic-swss-common] Update submodule (#6042)
Commits include:

* src/sonic-swss-common f143b7f...3d400a9 (3):
  > ASIC internal temperature sensors support (#419)
  > SWIG functions accept unicode input (#422)
  > Introdcued RESOLVE_NEIGH APP DB table (#420)
2020-11-26 09:22:50 -08:00
Sujin Kang
d5cd4e7ea2
Fix the ExecStart path for process-reboot-cause (#6044)
- Change the ExecStart path for process-reboot-cause from /usr/bin to /usr/local/bin
- Correct description and dependencies.
2020-11-26 09:22:02 -08:00
pavel-shirshov
148436d42e
[bgpcfg]: Batch bgp updates (#6006)
* [bgpcfgd]: Batch bgp updates.

vtysh -f command is slow. It is sometimes takes about 3 seconds.
When we need to run many vtysh -f commands that slows down the system.
Batch vtysh -f updates.

* Use correct file to import run_command
2020-11-25 14:56:27 -08:00
Joe LeVeque
7f4ab8fbd8
[sonic-utilities] Update submodule; Build and install as a Python 3 wheel (#5926)
Submodule updates include the following commits:

* src/sonic-utilities 9dc58ea...f9eb739 (18):
  > Remove unnecessary calls to str.encode() now that the package is Python 3; Fix deprecation warning (#1260)
  > [generate_dump] Ignoring file/directory not found Errors (#1201)
  > Fixed porstat rate and util issues (#1140)
  > fix error: interface counters is mismatch after warm-reboot (#1099)
  > Remove unnecessary calls to str.decode() now that the package is Python 3 (#1255)
  > [acl-loader] Make list sorting compliant with Python 3 (#1257)
  > Replace hard-coded fast-reboot with variable. And some typo corrections (#1254)
  > [configlet][portconfig] Remove calls to dict.has_key() which is not available in Python 3 (#1247)
  > Remove unnecessary conversions to list() and calls to dict.keys() (#1243)
  > Clean up LGTM alerts (#1239)
  > Add 'requests' as install dependency in setup.py (#1240)
  > Convert to Python 3 (#1128)
  > Fix mock SonicV2Connector in python3: use decode_responses mode so caller code will be the same as python2 (#1238)
  > [tests] Do not trim from PATH if we did not append to it; Clean up/fix shebangs in scripts (#1233)
  > Updates to bgp config and show commands with BGP_INTERNAL_NEIGHBOR table (#1224)
  > [cli]: NAT show commands newline issue after migrated to Python3 (#1204)
  > [doc]: Update Command-Reference.md (#1231)
  > Added 'import sys' in feature.py file (#1232)

* src/sonic-py-swsssdk 9d9f0c6...1664be9 (2):
  > Fix: no need to decode() after redis client scan, so it will work for both python2 and python3 (#96)
  > FieldValueMap `contains`(`in`)  will also work when migrated to libswsscommon(C++ with SWIG wrapper) (#94)

- Also fix Python 3-related issues:
    - Use integer (floor) division in config_samples.py (sonic-config-engine)
    - Replace print statement with print function in eeprom.py plugin for x86_64-kvm_x86_64-r0 platform
    - Update all platform plugins to be compatible with both Python 2 and Python 3
    - Remove shebangs from plugins files which are not intended to be executable
    - Replace tabs with spaces in Python plugin files and fix alignment, because Python 3 is more strict
    - Remove trailing whitespace from plugins files
2020-11-25 10:28:36 -08:00
abdosi
fad481edc1
Enhanced Feature table to support 'always_enabled' value for state and auto-restart fields. (#6000)
Added new flag value 'always_enabled' for the state and auto-restart field of feature table

init_cfg.json is updated to initialize state field of database/swss/syncd/teamd feature and auto-restart field of database feature
as always_enabled

Once the state/auto-restart value is initialized as "always_enabled" it is immutable and cannot be change via feature config commands. (config feature..) PR#Azure/sonic-utilities#1271

hostcfgd will not take any action if state field value is 'always_enabled'

Since we have always_enabled field for auto-restart updated supervisor-proc-exit-listener
not to have special check for database and always rely on value from Feature table.
2020-11-25 08:41:11 -08:00
Sabareesh-Kumar-Anandan
ac609365f6
[libyang] Adding LFS support for arm32 (#6018)
In the emulated armhf environment, the function readdir()returns NULL on a ext4 file system directory. When running the libyang test cases, it will require to load the plugins from the files (such as metadata.so), because the readdir() is failing, the plugins can’t be loaded in the emulated armhf environment, so it causes libyang test error. This error is a combination of the following reasons.
• Emulation of a 32-bit target from a 64-bit host –> qemu from x86_64 to armhf
• Glibc version > 2.27 – Debian buster is using glibc 2.28

Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>
2020-11-25 01:10:53 -08:00
pavel-shirshov
fd87ba0aee
[bgpcfgd]: Add on-match next rule for set ipv6 next-hop prefer-global (#6011)
* Add 'on-match next' after every 'set ipv6 next-hop prefer-global'
* Check that 'set ipv6 next-hop prefer-global' rule has 'on-match' next
2020-11-24 08:33:31 -08:00
vdahiya12
962e838b16
[sonic-platform-common] sub_module update (#6007)
commits in this PR

d912db6 [sonic_y_cable] add error checking for eeprom read (#147)

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
2020-11-23 23:27:21 -08:00
Sudharsan Dhamal Gopalarathnam
b65fc7867e
Updating submodule pointers for sonic-utilities (#5921)
sonic-utilities:
[3f15835] [sflow] Add mVRF support (#1230)
[dadfc18] Copp Changes (#1004)
[b226159] [CLI][PFCWD] Fix issue with specifying ports in pfcwd start on masic platforms (#1203)
[40377d3] [chassis]: Configure and show for platform chassis_modules (#1145)
[d5eb2f8] [consutil][connect] Remove root need from connect line command (#1225)
[6cbec6b] [multi-ASIC] CRM show/config commands changes for multi-asic (#1127)

Co-authored-by: dgsudharsan <sudharsan_gopalarat@dell.com>
2020-11-23 14:55:23 -08:00
Qi Luo
5410290b26
[sonic-swss-common] Update submodule (#5998)
Included submodule commits:
    f143b7f 2020-11-22 | [swig] wrap logger.h (#418) [Qi Luo]
2020-11-23 03:58:06 -08:00
Shi Su
e0781f463f
[L2 switch mode] Update l2switch.j2 template (#5981)
- Why I did it
The l2switch.j2 template does not include all fields for PORT. This could be incompatible with the 201911 image or later.

- How I did it
Update l2switch.j2 template and add a unit test.
2020-11-21 11:35:35 -08:00
Sujin Kang
5b31996f7b
[reboot-history] Add reboot history to state db (#5933)
- Why I did it
Add reboot history to State db so that can be used telemetry service
- How I did it
Split the process-reboot-cause service to determine-reboot-cause and process-reboot-cause
determine-reboot-cause to determine the reboot cause
process-reboot-cause to parse the reboot cause files and put the reboot history to state db
Moved to sonic-host-service* packages
- How to verify it
Performed unit test and tested on DUT
2020-11-20 20:08:18 -08:00
Ying Xie
d3c1a5bf39
[frr] remove frr rsyslog file outchannel (#5962)
- Why I did it
frr is creating /var/log/frr/frr.log inside the frr docker and letting it grow. It will eventually exhaust hard drive space.

To fixe issue #5965

- How I did it
Remove rsyslog file outchannel so that frr won't generate /var/log/frr/frr.log inside the docker.

- How to verify it
Manually removed the outchannel and restart BGP docker, making sure that /var/log/frr/frr.log is no longer created inside the docker.

While restarting bgp docker, observed that base image /var/log/quagga/bgpd.log continued to grow and captured all FRR logs.
2020-11-20 19:36:04 -08:00
Lawrence Lee
3b27afe377
[minigraph.py]: Add VLAN_INTERFACE attributes for dual ToR devices (#5951)
* If a device has subtype = 'DualToR', then enable proxy ARP and gratuitous ARP.
2020-11-20 17:04:01 -08:00
Petro Bratash
31404dc43a
[sonic-platform-daemons] Update submodule (#5987)
- Fix test_updater_thermal_check_min_max()
f6e5e5902e

- [sonic-platform-daemons] Add more physical entity information to DB for physical entity mib
14e586d676

Signed-off-by: Petro Bratash <petrox.bratash@intel.com>
2020-11-20 12:25:13 -08:00
Joe LeVeque
23247514f9
Fix a number of LGTM alerts (#5952)
Fix 259 alerts reported by the LGTM tool:

- 245 for Unused import
- 7 for Testing equality to None
- 5 for Duplicate key in dict literal
- 1 for Module is imported more than once
- 1 for Unused local variable
2020-11-20 10:58:48 -08:00
Joe LeVeque
7bf05f7f4f
[supervisor] Install vanilla package once again, install Python 3 version in Buster container (#5546)
**- Why I did it**

We were building a custom version of Supervisor because I had added patches to prevent hangs and crashes if the system clock ever rolled backward. Those changes were merged into the upstream Supervisor repo as of version 3.4.0 (http://supervisord.org/changes.html#id9), therefore, we should be able to simply install the vanilla package via pip. This will also allow us to easily move to Python 3, as Python 3 support was added in version 4.0.0.

**- How I did it**

- Remove Makefiles and patches for building supervisor package from source
- Install Python 3 supervisor package version 4.2.1 in Buster base container
    - Also install Python 3 version of supervisord-dependent-startup in Buster base container
- Debian package installed binary in `/usr/bin/`, but pip package installs in `/usr/local/bin/`, so rather than update all absolute paths, I changed all references to simply call `supervisord` and let the system PATH find the executable to prevent future need for changes just in case we ever need to switch back to build a Debian package, then we won't need to modify these again.
- Install Python 2 supervisor package >= 3.4.0 in Stretch and Jessie base containers
2020-11-19 23:41:32 -08:00
Shi Su
2f2b7c2a5c
[sonic-swss] Update submodule (#5964)
Update the sonic-swss submodule. The following are the commits in the submodule.

[dvs] Re-add runcmd_async command
5ba548cd759be4f0b1d1830ea038bf3c84e6f3f6

[dvs] Re-add copy_file method to DVS
86e11715819ac183a68aaf1deed6beabb259384b

Add a check for warm-restart, and do a clear only when warm-restart is enable.
fcb6c9de918d93967440b78bf7d5cd3a51bf7129

[Routeorch] Fix next hop group reference count in bulk operation
7a92100c1c1a344e67d4501cb7b98de557d0ea52

[orchagent][port] In case of successful port creation set log level to INFO
fea7ade74941a529505681e384f3437e873b1f7a

Copp Manager Changes
7b76d2e201f9f7d26cec7460f1335f7762b4ad01

[orchagent] Arm 32-bit arch compilation warning Fixes
bed79702c9da7957b775a0c960f67dd426db34d5

Revert: swss: flush g_asicState after each event is done #570
b9084a720d41daf1d717bcbe06f56e7a95111fa6

[dvs] Clean-up conftest.py
d6e15e955c91a6f077a913a617a336a314c6c91b
2020-11-19 23:40:14 -08:00
Joe LeVeque
5fbb6ee613
[sonic-platform-daemons] Update submodule (#5976)
* src/sonic-platform-daemons be21d91...b90ddda (1):
  > [thermalctld] [psud] [chassisd] Fix crash with undefined environment variable (#120)
2020-11-19 22:43:10 -08:00
Petro Bratash
d2f143d66f
[sonic-platform-common] Update submodule (#5977)
[sonic_eeprom] Fix variable type: Azure/sonic-platform-common#145

Signed-off-by: Petro Bratash <petrox.bratash@intel.com>
2020-11-19 14:53:37 -08:00
Qi Luo
42e78a19c1
[sonic-swss-common] Update submodule (#5969)
Update the sonic-swss-common submodule. The following are the commits in the submodule.
```
95f9e11 2020-11-19 | [pyext] allow to catch exceptions raised in python (#415) [Stepan Blyshchak]
5a718f9 2020-11-18 | [swig] translate C++ `del` to python `delete` (#416) [Qi Luo]
40b255b 2020-11-12 | Fix: SWIG dict.get() should have optional default value parameter (#413) [Qi Luo]
91e484d 2020-11-07 | Reduce notice logging (#412) [Qi Luo]
f5945ae 2020-11-05 | Mux Cable schema definitions for interaction between linkmanager and xcvrd (#411) [vdahiya12]
602f9c2 2020-11-05 | [lua] load lua script on demand (#409) [Dong Zhang]
d88412b 2020-11-04 | Rename hdel to del when using multiple keys as param (#410) [Kamil Cudnik]
e0c229a 2020-11-04 | CHASSIS_STATE_DB on control-card for chassis state (#395) [mprabhu-nokia]
a4e3ac8 2020-11-04 | Chassisd config table to store admin state (#388) [mprabhu-nokia]
```
2020-11-19 10:57:07 -08:00
pavel-shirshov
a92732fe5d
[bgpcfgd]: Fixes for BBR (#5956)
* Add explicit default state into the constants.yml
* Enable/disable only peer-groups, available in the config
* Retrieve updates from frr before using configuration

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-11-19 00:07:58 -08:00
Praveen Chaudhary
2fe79c2c61
[yang_models]: yang models for device_metadata, device_neighbor, flex… (#4442)
Changes:
1.) yang models for device_metadata, device_neighbor, flex_counters, crm, versions.
2.) Test cases for above yang models.
Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com:
2020-11-18 01:59:55 -08:00
Joe LeVeque
6c4092d0a2
[sonic-platform-common] Update submodule (#5953)
Commits include:

* src/sonic-platform-common e4883e5...5db8c84 (1):
  > [sonic-platform-common] Fix name error and import error  (#143)
2020-11-18 01:37:05 -08:00
Joe LeVeque
ced11615a4
[sonic-py-common] Add 'universal_newlines=True' arg to all Popen calls (#5919)
The behavior of `subprocess.Popen()` changed in Python 3 such that stdin, stdout and stderr are treated as bytes by default. Adding the `universal_newlines=True` argument changes this behavior to return strings, matching the behavior of Python 2. The change is backward-compatible with Python 2, as well.
2020-11-16 17:07:37 -08:00
Lawrence Lee
261a81d379
[minigraph.py]: Parse mux cable table (#5876)
Parse device IPv6 loopback addresses when available
Add MUX_CABLE table to config DB:

'MUX_CABLE': {
	'Ethernet4': {
    	'state': 'auto',
        'server_ipv4': '10.10.10.1/32',
        'server_ipv6': 'fe80::0001/128'
	}
}
2020-11-16 13:14:34 -08:00
Joe LeVeque
e526d8895f
[sonic-platform-common][sonic-platform-daemons] Update submodules (#5924)
This should also fix the vsimage check build.

Commits include:

* src/sonic-platform-common a659219...e4883e5 (7):
  > [PsuBase] Reference inherited member constant (#142)
  > Thermalctld APIs for recording min and max temp (#131)
  > Common power consumption and supply APIs for modular chassis (#136)
  > [sonic-platform-base] Introduce APIs for modular chassis support (#124)
  > Remove shebangs from non-executable Python files (#140)
  > [sonic_eeprom] Make compatible with Python 2 and 3 (#127)
  > Add unit test infrastructure (#139)

* src/sonic-platform-daemons 600d043...be21d91 (9):
  > [psud] Import of mock libraries determined by environment variable (#117)
  > [chassisd][thermalctld] Set testing env var at top of file, not in setup_function() (#114)
  > [chassisd][thermalctld] Import os module, now needed for env var checking (#113)
  > [chassisd][thermalctld] Import mock libraries based on environment variable (#112)
  > [xcvrd] Fix the console-script xcvrd startup  (#111)
  > [Thermalctld] Update thermal info to CHASSIS_STATE_DB (#101)
  > PSUd changes to compute power-budget for Modular chassis (#104)
  > Introduce chassisd to monitor status of cards on chassis (#97)
  > [xcvrd] support for integrating Credo Y cable Ports initialization and status updates with xcvrd.  (#105)
2020-11-15 02:37:38 -08:00
pavel-shirshov
af654944bd
[bgp]: Update TSA functionality (#5906)
Fixed TSA bugs:
1. TSA didn't advertise Loopback ipv6 address
2. TSA and TSB changed BGP dynamic and BGP monitors sessions

**- How to verify it**
Build an image and run on your DUT.
```
admin@str-s6100-acs-1:~$ TSA
System Mode: Normal -> Maintenance
admin@str-s6100-acs-1:~$ vtysh -c 'show bgp ipv4 neighbors 10.0.0.1 advertised-routes'
BGP table version is 6, local router ID is 10.1.0.32, vrf id 0
Default local pref 100, local AS 64601
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 10.1.0.32/32     0.0.0.0                  0         32768 i

Total number of prefixes 1
admin@str-s6100-acs-1:~$ vtysh -c 'show bgp ipv6 neighbors fc00::a advertised-routes'
BGP table version is 6, local router ID is 10.1.0.32, vrf id 0
Default local pref 100, local AS 64601
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> fc00:1::/64      ::                       0         32768 i

Total number of prefixes 1
admin@str-s6100-acs-1:~$ TSB
System Mode: Maintenance -> Normal
```

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-11-13 17:54:20 -08:00
Lawrence Lee
bf50562670
[minigraph.py]: Parse peer switch info from minigraph (#5869)
* Create new `PEER_SWITCH` table in config DB with info from minigraph
* Add `subtype` field to `DEVICE_METADATA` table and set value to `DualToR` if device is in a dual ToR setup
2020-11-13 15:31:51 -08:00
Joe LeVeque
56fa3cf494
[sonic-bgpcfgd] Call Python 3 version of sonic-cfggen for testing (#5847)
sonic-bgpcfgd build fails in the absence of Python 2, as it attempts to explicitly call sonic-cfggen using `/usr/bin/python2.7`. Also, it attempts to call sonic-cfggen using a local, relative path. Since the sonic-config-engine package is not installed, neither are its dependencies.

Now, we configure the Python 3 sonic-config-engine as a dependency of sonic-bgpcfgd, which ensures the Python 3 sonic-config-engine package and its dependencies are installed before sonic-bgpcfgd is built/tested.
2020-11-13 11:39:16 -08:00
Ying Xie
48822be2aa
[dbsycnd] update submodule head of dbsyncd (#5916)
[lldp daemon] improve lldp daemon event when neighbor capability is not sent (#30)

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2020-11-13 03:12:12 -08:00
Joe LeVeque
d0c7bd8158
[sonic-py-swsssdk][sonic-snmpagent][sonic-utilities] Update submodules (#5849)
Submodule updates include the following commits. Also adding sonic-swsscommon build dependencies where necessary.

* src/sonic-py-swsssdk 1ea30d2...9d9f0c6 (1):
  > [SonicV2Connector] make decode_responses=True as default, so python2 application need no code change when migrated to python3(#93)

* src/sonic-snmpagent 6e4a796...57e54d9 (3):
  > Interact with Redis by str instead of bytes, migrate to SonicV2Connector with `decode_responses=True` (#171)
  > Add a test case for LLDP_LOC_CHASSIS.lldp_loc_man_addr has only one IPv6 address (#167)
  > [LLDP]: Update init_db to load global database config (#166)

* src/sonic-utilities acfa824...b693cf6 (11):
  > Remove SKU create pytest output directory before execution of the script (#1226)
  > [show][techsupport][multi-ASIC] Add support to collect tech support on multi ASIC platform (#1193)
  > [show] Fix `show ip bgp sum` (#1194)
  > [sonic_sku_create] Move tests from sonic-utilities-tests/ folder to tests/ folder (#1222)
  > Replace swsssdk.SonicV2Connector with swsscommon.SonicV2Connector (SWIG wrapper of C++ implementation) in production code (#1217)
  > Copy missing values from INIT_CFG to config_db as part of db_migration task (#1209)
  > [connect][clear] Support --devicename option for connect/clear line commands (#1214)
  > [consutil][show] Remove root need from show line command (#1218)
  > [Mellanox] SKU creator Tool (#1163)
  > SONiC installer - fix string formatting during image type check (#1197)
  > [show/fgnhg.py] Fix whitespace issues (#1211)
2020-11-12 17:58:56 -08:00
Guohan Lu
a98cf22162 [submodule]: downgrade sonic-platform-daemons
remove commit

850d0c6 2020-11-09 | [xcvrd] support for integrating Credo Y cable Ports initialization and status updates with xcvrd.  (#105) (HEAD, origin/master, origin/HEAD) [vdahiya12]

fix issue #5907

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-11-11 17:14:35 -08:00
Kebo Liu
4342cb0cea
[submodule]: update sonic-linux-kernel (#5894)
Pick up new kernel patches:
-  a28b399db609e9cab6748df28cf1482537f31560  [Mellanox] Add patches for i2c API 
- 18236531cb74d34fa5945d1c766a1a35b658569f [Mellanox] Add kernel patches with hw-mgmt V.7.0010.1300 
- cb4712f97dbfb4b7a5f867034317e1aa3ac75b45 Improve `patch/kconfig-exclusions`
2020-11-11 16:55:09 -08:00
carl-nokia
dbb6373e85
[Marvell] Utilize eeprom for system mac and remove dependency on get_hwsku (#5882)
Utilize eeprom for system mac and remove dependency on get_hwsku

Co-authored-by: Carl Keene <keene@nokia.com>
2020-11-11 15:11:00 -08:00
pavel-shirshov
83d090f42c
[teamd]: Remove extensive debug output for libteamdctl (#5870) 2020-11-10 19:12:51 -08:00
pavel-shirshov
cdc6879c3e
[bgpcfgd]: Implement BBR template test (#5850)
Add the test to check that all templates, which use CONFIG_DB__BBR are configured in constants.yml
2020-11-10 18:44:20 -08:00
Lawrence Lee
d0f16c0d79
Make backend device checking more robust (#5730)
Treat devices that are ToRRouters (ToRRouters and BackEndToRRouters) the same when rendering templates
 Except for BackEndToRRouters belonging to a storage cluster, since these devices have extra sub-interfaces created
Treat devices that are LeafRouters (LeafRouters and BackEndLeafRouters) the same when rendering templates

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2020-11-10 15:06:35 -08:00
Prince Sunny
1eaaf64ed2
Set preference for forced mgmt routes (#5844)
When forced mgmt routes are present, the issue fixed as part of #5754 is not complete. 
Added a preference(priority) field to forced mgmt route ip rules
2020-11-10 14:20:13 -08:00
Joe LeVeque
aee389e402
[sonic-config-engine] openconfig_acl.py: Make compliant with Python 3 (#5873)
Python 3 is more strict with `__slots__`. As per the [documentation](https://docs.python.org/3/reference/datamodel.html#notes-on-using-slots):

> \_\_slots\_\_ are implemented at the class level by creating descriptors (Implementing Descriptors) for each variable name. As a result, class attributes cannot be used to set default values for instance variables defined by \_\_slots\_\_; otherwise, the class attribute would overwrite the descriptor assignment.

This was apparently missed when making sonic-config-engine compliant with Python 3, and errors like the following would be seen:

```
tests/acl_loader_test.py:10: in <module>
    from acl_loader.main import *
acl_loader/main.py:8: in <module>
    import openconfig_acl
/usr/local/lib/python3.7/dist-packages/openconfig_acl.py:24: in <module>
    class yc_state_openconfig_acl__acl_state(PybindBase):
E   ValueError: '_pybind_generated_by' in __slots__ conflicts with class variable
```
2020-11-10 10:26:16 -08:00
madhanmellanox
b177ed9efd
Accomadating lower case ACL rules for Control Plane ACLs (#5748)
To make Control plane ACLs handle case insensitive ACL rules. Currently, it handles only upper case ACL rules.

Co-authored-by: Madhan Babu <madhan@arc-build-server.mtr.labs.mlnx>
2020-11-10 10:16:32 -08:00
judyjoseph
f2b22b5cd1
[multi-ASIC] util changes with the BGP_INTERNAL_NEIGHBOR table (#5874)
Reintroduce #5760, along with the fix needed in the template file for python3 compatibility.
2020-11-10 09:34:56 -08:00
Joe LeVeque
4669c3241b
Remove shebangs from various setup.py files (#5867)
setup.py files are not executable, and thus have no need for shebangs.

Also remove unnecessary comments
2020-11-10 00:07:37 -08:00
vdahiya12
e95483471e
[submodule]: update sonic-platform-daemons (#5868)
850d0c6 [xcvrd] support for integrating Credo Y cable Ports initialization and status updates with xcvrd.  (#105)
600d043 [xcvrd] Add enum dependence back; Install 'enum34' conditionally based on Python version (#107)
af79326 [xcvrd] Remove log errors in single ASIC platforms with init Global config (#108)

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
2020-11-10 00:06:48 -08:00
judyjoseph
b5121dcfd4
Revert "[multi-ASIC] util changes with the BGP_INTERNAL_NEIGHBOR table. (#5760)" (#5871)
This reverts commit c972052594.
2020-11-09 14:30:13 -08:00
judyjoseph
c972052594
[multi-ASIC] util changes with the BGP_INTERNAL_NEIGHBOR table. (#5760)
- Why I did it
Update the routine is_bgp_session_internal() by checking the BGP_INTERNAL_NEIGHBOR table.
Additionally to address the review comment #5520 (comment)
Add timer settings as will in the internal session templates and keep it minimal as these sessions which will always be up.
Updates to the internal tests data + add all of it to template tests.

- How I did it
Updated the APIs and the template files.

- How to verify it
Verified the internal BGP sessions are displayed correctly with show commands with this API is_bgp_session_internal()
2020-11-09 11:10:10 -08:00
Volodymyr Samotiy
3fa724d216
[submodule] Update sonic-swss submodule (#5825)
62e6c5a [bitmap_vnet] Remove BMTOR implementation (#1494)
582beae [FPMSYNCD] EVPN Type5 prefix handling in FPMSYNCD (#1326)
d9ca6e2 [vstest] Resolve intermittent timing failure in test_fgnhg.py (#1495)
a89b8ed [sub intf] Inherit parent port mtu changes (#1479)

Signed-off-by: Volodymyr Samotiy <volodymyrs@nvidia.com>
2020-11-08 23:57:50 -08:00
Longxiang Lyu
92704e8a93
[bgpmon] Fix dict_keys unsubscriptable error (#5859)
Simply looping through `dict_key` object instead of index-subscribing to
fix this.

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>
2020-11-08 13:42:38 -08:00
Lawrence Lee
23b0e07d48
[minigraph.py]: Parse IP-in-IP tunnels from minigraph (#5742)
Take tunnel info from `<TunnelInterface>` tag in minigraph, and create tables in config_DB:

```
"TUNNEL": {
    "MUX_TUNNEL_0": {
        "tunnel_type": "IPINIP",
        "dst_ip": "26.1.1.10",
        "dscp_mode": "uniform",
        "encap_ecn_mode": "standard",
        "ecn_mode": "copy_from_outer",
        "ttl_mode": "pipe"
    }
}
```

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2020-11-07 13:18:40 -08:00
Praveen Chaudhary
6156cb2805
[sonic-yang-mgmt] Build PY3 & PY2 packages (#5559)
Moving sonic-yang-mgmt to PY3 to support move of sonic-utilities to PY3.

Signed-off-by: Praveen Chaudhary<pchaudhary@linkedin.com>
2020-11-07 13:03:41 -08:00
Joe LeVeque
04d0e8ab00
[hostcfgd] Convert to Python 3; Add to sonic-host-services package (#5713)
To consolidate host services and install via packages instead of file-by-file, also as part of migrating all of SONiC to Python 3, as Python 2 is no longer supported.
2020-11-07 12:48:19 -08:00
Petro Bratash
32a832a8ac
[lldp]: Add verification IPv4 address on LLDP conf Jinja2 Template (#5699)
Fix #5812

LLDP conf Jinja2 Template does not verify IPv4 address and can use IPv6 version. This issue does not effect control LLDP daemon. Issue can be reproduced via `test_snmp_lldp` test. LLDP conf Jinja2 Template selects first item from the list of mgmt interfaces.

TESTBED_1 LLDP conf

```
# cat /etc/lldpd.conf 
configure ports eth0 lldp portidsubtype local eth0
configure system ip management pattern FC00:3::32
configure system hostname dut-1
```
TESTBED_2  LLDP conf

```
# cat /etc/lldpd.conf
configure ports eth0 lldp portidsubtype local eth0
configure system ip management pattern 10.22.24.61
configure system hostname dut-2
```
TESTBED_1  MGMT_INTERFACE

```
$ redis-cli -n 4 keys "*" | grep MGMT_INTERFACE
MGMT_INTERFACE|eth0|10.22.24.53/23
MGMT_INTERFACE|eth0|FC00:3::32/64
```
TESTBED_2  MGMT_INTERFACE

```
$ redis-cli -n 4 keys "*" | grep MGMT_INTERFACE
MGMT_INTERFACE|eth0|FC00:3::32/64
MGMT_INTERFACE|eth0|10.22.24.61/23

```

Signed-off-by: Petro Bratash <petrox.bratash@intel.com>
2020-11-07 10:30:41 -08:00
pavel-shirshov
cdcd20a7b5
[BGP]: Convert ip address to network address for the LOCAL_VLAN filter (#5832)
* [BGP]: Convert ip address to network address for the LOCAL_VLAN prefix filter
2020-11-06 17:47:08 -08:00
dflynn-Nokia
174b741dfd
[submodule]: update sonic-linux-kernel (#5829)
pick up follow commits:

  > hwmon-lm75: backport support for PCT2075 thermal sensor (#165)
  > [arista] Reassign prefetch memory per platform (#163)
  > Add .gitignore file (#157)
  > Enable PCA9541 I2C mux module (#160)
2020-11-06 12:37:41 -08:00
Danny Allen
cea364aa77
[utilities] Update sonic-utilities submodule (#5814)
Add a switch to route_check to control whether write log to syslog (#1215)
[consutil] Store console port state in STATE_DB (#1208)
[sfputil] Display 'N/A' for non-SFP ports (#1078)
Show FG_NHG CLI Commands Added (#1056)
[show] remove 'device2interface_dict' in show interface neighbor expected (#1188)
Add license file, remove third-party licenses file (#1192)
[fwutil]: Set min log priority to INFO. (#1191)
[CLI][show][platform] Added ASIC count in the output. (#1185)
fdbshow and nbrshow use SonicV2Connector with decode_responses=True, and remove all the decode() (#1187)
Remove stdeb.cfg; no longer used now that we build as wheel (#1182) 
[counterpoll] Disable Counter Poll When Entering Fast Reboot (#1174) 
Fixes the issue with show interface counters and for pfc and queue counters. (#1180)
[config/show] Add CLI support for proxy arp (#1168)
[consutil] Add brief option to show line command (#1176)
Modify fast-reboot script to use BGP service script to stop bgp service (#1177)
[config/console] Support update console configuration related commands (#1166)
[consutil] Fix issue where the show line command crash if no ttyUSB exists (#1173)
[watermarkstat] Add unit tests for watermarkstat show commands (#1157)
Fix exception for ipaddress in python2 (#1164) 
[celestica] consutil to support customize tty device name (#1155)

Signed-off-by: Danny Allen <daall@microsoft.com>
2020-11-06 00:58:49 -08:00
Joe LeVeque
2c9454e86a
[sonic-device-data] Convert Python files to Python 3 (#5816)
- Convert config_checker, media_checker and platform_json_checker scripts to Python 3
- Reorganize imports per PEP8 standard
- Two blank lines precede functions per PEP8 standard
2020-11-05 11:17:08 -08:00
pavel-shirshov
13f8e9ce5e
[bgpcfgd]: Convert bgpcfgd and bgpmon to python3 (#5746)
* Convert bgpcfgd to python3

Convert bgpmon to python3
Fix some issues in bgpmon

* Add python3-swsscommon as depends

* Install dependencies

* reorder deps

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-11-05 10:01:43 -08:00
Praveen Chaudhary
1f9132db60
[sonic-yang]: Yang model update fields (#5577)
Changes:
-- ACL stage allow lower and upper.
-- ACL include services leaf-list.
-- PORT include pfc_asym leaf.
-- PORT fec alloe none as per code.
-- 3 Tests for above changes.

Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com
2020-11-05 00:25:33 -08:00
Qi Luo
4ed2ff805f
[swss-common]: Advance submodule (#5799)
ec96868 2020-11-03 | Fix: treat DBConnector timeout=0 as infinite timeout (#408) [Qi Luo]
b4b8334 2020-11-03 | Add lua script for redis multi keys api hmset and del (#406) [Kamil Cudnik]
2020-11-04 05:49:52 -08:00
Qi Luo
215ce13890
[swss-common]: Advance submodule (#5780)
9b0e955 2020-10-30 | [schema]: Add MACsec support (#403) [Ze Gan]
f24dc97 2020-10-28 | Implementation of System ports initialization, Interface & Neighbor Synchronization... (#380) [minionatwork]
91e0885 2020-10-27 | Mux cable schema definitions (#398) [Prince Sunny]
d0cedea 2020-10-27 | Change log level (#402) [Prince Sunny]
64b3cfe 2020-10-24 | SonicV2Connector supports host and decode_responses in constructor parameters (#401) [Qi Luo]
f8b0065 2020-10-23 | Implement FieldValueMap update method (#400) [Qi Luo]
2020-11-03 14:28:42 -08:00
Joe LeVeque
84d3a26000
[sonic-py-swsssdk] Update submodule (#5757)
Commits included:

* src/sonic-py-swsssdk 748c404...1ea30d2 (1):
  > Fix bug: ConfigDBConnector.get_table does not work in python3 (#92)
2020-11-02 20:31:14 -08:00
lguohan
9d7355287f
[submodule]: update sairedis (#5772)
* b458e6f 2020-11-02 | [syncd] Disable use bulk api by default (#688) (HEAD, origin/master, origin/HEAD) [Kamil Cudnik]
* d978789 2020-11-01 | [syncd] Set modify redis flag in RedisSelectableChannel (#689) [Kamil Cudnik]
* 5df11f5 2020-11-01 | [syncd] Lower bulk missing api message level from error to info (#687) [Kamil Cudnik]
* dc73a1d 2020-10-30 | [saiplayer] Fix log messages (#686) [Kamil Cudnik]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-11-02 16:59:47 -08:00
lguohan
98d370bcef
[submodule]: swss/sairedis module update (#5765)
swss:

* d7643f2 2020-11-01 | [tlm_teamd]: Make the destionation for std::transform() to use std::back_inserter() for allocating new space for the copied objects (#1490) (HEAD, origin/master, origin/HEAD) [pavel-shirshov]
* 7fa7cd6 2020-10-31 | [vstest]: stablize fgnhg test (#1491) [lguohan]
* 9b0696e 2020-10-29 | Create vnet tunnel map only if it doesn't exist (#1482) [Prince Sunny]
* 0481e99 2020-10-29 | [acl] Update CRM to include LAG bindings for ACL tables (#1487) [Danny Allen]

sairedis

* 5df11f5 2020-11-01 | [syncd] Lower bulk missing api message level from error to info (#687) (HEAD, origin/master, origin/HEAD) [Kamil Cudnik]
* dc73a1d 2020-10-30 | [saiplayer] Fix log messages (#686) [Kamil Cudnik]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-11-01 20:08:55 -08:00
Blueve
698b5544c9
[openssh] Introduce custom openssh-server package for supporting reverse console SSH (#5717)
* Build and install openssh from source
* Copy openssh deb package to dest folder
* Update make rule
* Update sonic debian extension
* Append empty line before EOF
* Update openssh patch
* Add openssh-server to base image dependency
* Fix indent type
* Fix comments
* Use commit id instead of tag id and add comment

Signed-off-by: Jing Kan jika@microsoft.com
2020-11-02 10:31:15 +08:00
lguohan
c8a00eda95
[mgmt ip]: mvrf ip rule priority change to 32765 (#5754)
Fix Azure/SONiC#551

When eth0 IP address is configured, an ip rule is getting added for eth0 IP address through the interfaces.j2 template. 

This eth0 ip rule creates an issue when VRF (data VRF or management VRF) is also created in the system.
When any VRF (data VRF or management VRF) is created, a new rule is getting added automatically by kernel as "1000: from all lookup [l3mdev-table]".
This l3mdev IP rule is never getting deleted even if VRF is deleted.

Once if this l3mdev IP rule is added, if user configures IP address for the eth0 interface, interfaces.j2 adds an eth0 IP rule as "1000:from 100.104.47.74 lookup default ". Priority 1000 is automatically chosen by kernel and hence this rule gets higher priority than the already existing rule "1001:from all lookup local ".

This results in an issue "ping from console to eth0 IP does not work once if VRF is created" as explained in Issue 551.
More details and possible solutions are explained as comments in the Issue551.

This PR is to resolve the issue by always fixing the low priority 32765 for the IP rule that is created for the eth0 IP address.
Tested with various combinations of VRF creation, deletion and IP address configuration along with ping from console to eth0 IP address.

Co-authored-by: Kannan KVS <kannan_kvs@dell.com>
2020-10-31 20:45:59 -07:00
gechiang
908787d2a2
Added new method get_back_end_interface_set() to speed up back-end in… (#5731)
Added new MultiASIC util method "get_back_end_interface_set()" to speed up back-end interface check by allowing caller to cache the back-end intf into a set. This way the caller can use this set for all subsequent back-end interface check requests  instead of each time need to read from redis DB which become a scaling issue for cases such as checking for thousands of nexthop routes for filtering purpose.
2020-10-31 18:06:06 -07:00
abdosi
dddf96933c
[monit] Adding patch to enhance syslog error message generation for monit alert action when status is failed. (#5720)
Why/How I did:

Make sure first error syslog is triggered based on FAULT TOLERANCE condition.

Added support of repeat clause with alert action. This is used as trigger
for generation of periodic syslog error messages if error is persistent

Updated the monit conf files with repeat every x cycles for the alert action
2020-10-31 17:29:49 -07:00
Shi Su
279943c11f
[sonic-swss] Update submodule (#5745)
Update the sonic-swss submodule. The following are the commits in the submodule.

[neighorch] Remove pending DEL operation after SET operation for the same key
2265f548386929b7827d1079efd453128f1ec1f9

[NAT]: Update nat entries to use nat_type to support DNAT Pool changes.
8696e939f973895ead4731ad499a72f257a3b510

[intfsorch] Init proxy_arp variable while adding router interface.
1da3c773762fa637a5ea47017715361bede50a4a
2020-10-31 02:26:19 -07:00
Joe LeVeque
e111204206
[caclmgrd] Convert to Python 3; Add to sonic-host-services package (#5739)
To consolidate host services and install via packages instead of file-by-file, also as part of migrating all of SONiC to Python 3, as Python 2 is no longer supported, convert caclmgrd to Python 3 and add to sonic-host-services package
2020-10-29 16:29:12 -07:00
Volodymyr Boiko
fd7e2a12bc
[submodule-update][sonic-platform-daemon] Update submodule (#5741)
95b1696 [xcvrd] Remove dependence on enum; Add 'sonic-py-common' as dependencies in setup.py (#106)
61ed24e [thermalctld] Print exception using repr(e) to get more information (#103)
8507085 [psud] Fix psud logging (#98)

Signed-off-by: Volodymyr Boyko <volodymyrx.boiko@intel.com>
2020-10-29 09:46:55 -07:00
Shi Su
5ee5c13f32
Enable synchronous mode by default and add in minigraph parser (#5735) 2020-10-29 09:15:12 -07:00
Dong Zhang
d95e1969c8
[swsssdk] update submodule for adding new MultiDB API (#5737) 2020-10-28 21:47:01 -07:00
judyjoseph
6088bd59de
[multi-ASIC] BGP internal neighbor table support (#5520)
* Initial commit for BGP internal neighbor table support.
  > Add new template named "internal" for the internal BGP sessions
  > Add a new table in database "BGP_INTERNAL_NEIGHBOR"
  > The internal BGP sessions will be stored in this new table "BGP_INTERNAL_NEIGHBOR"

* Changes in template generation tests with the introduction of internal neighbor template files.
2020-10-28 16:41:27 -07:00
Shi Su
09d5a62fad
[sonic-sairedis] Update submodule (#5728)
Update the sonic-sairedis submodule. The following are the commits in the submodule.

[syncd_init_common.sh] Use template file to retrieve vars (#683)
0bf336a3e895167357d5d2e5a988471e115522e8

[syncd/FlexCounter]:Fix last remove bug (#679)
4d21a264d5956501bf69ad3a89ea2ebccd369654
2020-10-27 20:42:57 -07:00
Lawrence Lee
a639021af2
[minigraph.py]: Parse VLAN MAC address from minigraph when present (#5726) 2020-10-27 17:20:55 -07:00
Danny Allen
e0b09d0998
[swss] Update swss submodule (#5719)
Signed-off-by: Danny Allen <daall@microsoft.com>
2020-10-27 14:38:43 -07:00
vdahiya12
dfe005545a
[sonic-platform-common] update submodule (#5721)
a659219 [SONIC_SFP] adding abstract methods for reading and writing the eeprom address space within platform api (#126)
848f4a6 Add third-party licenses (#138)
c2ecd9a Add license file (#137)
403747a [sonic-platform-common] Add new platform API for SONiC Physical MIB Extension feature (#134)
19b8545 [sonic_y_cable] fix the unpacking (#135)

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
2020-10-27 09:36:26 -07:00
Joe LeVeque
9e34003136
[sonic-config-engine] Clean up dependencies, pin versions; install Python 3 package in Buster container (#5656)
To clean up the image build procedure, and let setuptools/pip[3] implicitly install Python dependencies. Also use ipaddress package instead of ipaddr.
2020-10-26 13:48:50 -07:00
shlomibitton
e66d49a57c
[LLDP] Fix for LLDP advertisements being sent with wrong information. (#5493)
* Fix for LLDP advertisments being sent with wrong information.
Since lldpd is starting before lldpmgr, some advertisment packets might sent with default value, mac address as Port ID.
This fix hold the packets from being sent by the lldpd until all interfaces are well configured by the lldpmgrd.

Signed-off-by: Shlomi Bitton <shlomibi@nvidia.com>

* Fix comments

* Fix unit-test output caused a failure during build

* Add 'run_cmd' function and use it

* Resume lldpd even if port init timeout reached
2020-10-26 19:38:09 +02:00
judyjoseph
c14b41dc30
[submodule]: Advance sonic-swss-common and sonic-sairedis module (#5703)
Advance sonic-swss-common submodule by adding the following  commits 

3ec30ef Deprecate RedisClient and remove unused header file (#399)
165a679 Schema update for BGP internal neighbor table (#389)
262e330 Fix SonicV2Connector interfaces (#396)

Advance sonic-sairedis submodule by adding the following  commits 

bc3e044  [Sai]: Change Sai::set log to level INFO (#680)
b16bc8b Clean code: remove unused header file (#678)
40439b4 [syncd] Remove depreacated dependency on swss::RedisClient (#681)
1b6fc2e [syncd] Add supports of bulk api in syncd (#656)
a9f69c1 [syncd] Add to handle FDB MOVE notification (#670)
c7ef5e9 [gbsyncd] exit with zero when platform has no gearbox (#676)
57228fd [gbsyncd]: add missing python dependency (#675)
02a57a6 [vs] Add CRM SAI attributes to virtual switch interface (#673)
609445a fix boot type for fast boot (#674)
1325cdf Add support for saiplayer bulk API and add performance timers (#666)
1d84b90 Add ZeroMQ communication channel between sairedis and syncd (#659)
017056a Support System ports config (#657)
0f3668f Enable fabric counter for syncd's FlexCounter (#669)
2020-10-26 09:19:16 -07:00
Lawrence Lee
c4f9bec562
[minigraph.py]: Add support for parsing mux cable (#5676)
Find LogicalLinks in minigraph and parse the port information. A new field called `mux_cable` is added to each port's entry in the Port table in config DB:

```
PORT|Ethernet0: {
	"alias": "Ethernet4/1"
	...
	"mux_cable": "true"
}
```

If a mux cable is present on a port, the value for `mux_cable` will be `"true"`. If no mux cable is present, the attribute will either be omitted (default behavior) or set to `"false"`.
2020-10-26 08:52:20 -07:00
Nazarii Hnydyn
ba6f012cc6
[sonic-py-common]: Fix syslog implicit min priority override (#5707)
Current implementation of logger class is based on standard python syslog library.  
Thus, logger class can be instantiated in different places and share the same context across the entire process.  
This means that reducing log severity level will affect other modules which use logging facility.

**- Why I did it**
* To fix syslog implicit min priority override

**- How I did it**
* Added per instance log severity check

**- How to verify it**
1. Run code snippet
```
from sonic_py_common import logger

log1 = logger.Logger(log_identifier='myApp1')
log1.set_min_log_priority_debug()
log1.log_error("=> this is error")
log1.log_warning("=> this is warning")
log1.log_notice("=> this is notice")
log1.log_info("=> this is info")
log1.log_debug("=> this is debug")

log2 = logger.Logger(
    log_identifier='myApp2',
    log_facility=logger.Logger.LOG_FACILITY_DAEMON,
    log_option=(logger.Logger.LOG_OPTION_NDELAY | logger.Logger.LOG_OPTION_PID)
)
log2.log_error("=> this is error")
log2.log_warning("=> this is warning")
log2.log_notice("=> this is notice")
log2.log_info("=> this is info")
log2.log_debug("=> this is debug")
```
2. Sample output:
```
Oct 23 15:08:30.447301 sonic ERR myApp1: => this is error
Oct 23 15:08:30.447908 sonic WARNING myApp1: => this is warning
Oct 23 15:08:30.448305 sonic NOTICE myApp1: => this is notice
Oct 23 15:08:30.448696 sonic INFO myApp1: => this is info
Oct 23 15:08:30.449063 sonic DEBUG myApp1: => this is debug

Oct 23 15:08:30.449442 sonic ERR myApp2[19178]: => this is error
Oct 23 15:08:30.449819 sonic WARNING myApp2[19178]: => this is warning
Oct 23 15:08:30.450183 sonic NOTICE myApp2[19178]: => this is notice
```

Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>
2020-10-24 12:36:22 -07:00
Joe LeVeque
3a4435eb53
Add sonic-host-services and sonic-host-services-data packages (#5694)
**- Why I did it**

Install all host services and their data files in package format rather than file-by-file

**- How I did it**

- Create sonic-host-services Python wheel package, currently including procdockerstatsd
  - Also add the framework for unit tests by adding one simple procdockerstatsd test case
- Create sonic-host-services-data Debian package which is responsible for installing the related systemd unit files to control the services in the Python wheel. This package will also be responsible for installing any Jinja2 templates and other data files needed by the host services.
2020-10-23 09:52:29 -07:00
judyjoseph
ace7f24cba
[docker-teamd]: Add teamd as a depedent service to swss (#5628)
**- Why I did it**
On teamd docker restart, the swss and syncd needs to be restarted as there are dependent resources present.

**- How I did it**
Add the teamd as a dependent service for swss
Updated the docker-wait script to handle service and dependent services separately.
Handle the case of warm-restart for the dependent service   

**- How to verify it**

Verified the following scenario's with the following testbed 
VM1 ----------------------------[DUT 6100] -----------------------VM2,  ping traffic continuous between VMs

1. Stop teamd docker alone  
      >  swss, syncd dockers seen going away
      >  The LAG reference count error messages seen for a while till swss docker stops.
      >  Dockers back up.

2. Enable WR mode for teamd. Stop teamd docker alone  
      >  swss, syncd dockers not removed.
      >  The LAG reference count error messages not seen
      >  Repeated stop teamd docker test - same result, no effect on swss/syncd.

3. Stop swss docker. 
      >  swss, teamd, syncd goes off - dockers comes back correctly, interfaces up

4. Enable WR mode for swss . Stop swss docker 
      >  swss goes off not affecting syncd/teamd dockers.

5. Config reload 
      > no reference counter error seen, dockers comes back correctly, with interfaces up

6. Warm reboot, observations below
	 > swss docker goes off first 
	 > teamd + syncd goes off to the end of WR process.
 	 > dockers comes back up fine.
	 > ping traffic between VM's was NOT HIT

7. Fast reboot, observations below
	 > teamd goes off first ( **confirmed swss don't exit here** )
	 > swss goes off next 
	 > syncd goes away at the end of the FR process
	 > dockers comes back up fine.
	 > there is a traffic HIT as per fast-reboot

8. Verified in multi-asic platform, the tests above other than WR/FB scenarios
2020-10-23 00:41:16 -07:00
Joe LeVeque
73825e4d4d
[system-health] Update .gitignore file (#5688)
Touch up .gitignore file to properly ignore all files generated when building a Python wheel package
2020-10-22 11:58:27 -07:00
pavel-shirshov
c94f93f046
[bgpcfgd]: Dynamic BBR support (#5626)
**- Why I did it**
To introduce dynamic support of BBR functionality into bgpcfgd.
BBR is adding  `neighbor PEER_GROUP allowas-in 1' for all BGP peer-groups which points to T0
Now we can add and remove this configuration based on CONFIG_DB entry 

**- How I did it**
I introduced a new CONFIG_DB entry:
 - table name: "BGP_BBR"
 - key value: "all". Currently only "all" is supported, which means that all peer-groups which points to T0s will be updated
 - data value: a dictionary: {"status": "status_value"}, where status_value could be either "enabled" or "disabled"

Initially, when bgpcfgd starts, it reads initial BBR status values from the [constants.yml](https://github.com/Azure/sonic-buildimage/pull/5626/files#diff-e6f2fe13a6c276dc2f3b27a5bef79886f9c103194be4fcb28ce57375edf2c23cR34). Then you can control BBR status by changing "BGP_BBR" table in the CONFIG_DB (see examples below).

bgpcfgd knows what peer-groups to change fron [constants.yml](https://github.com/Azure/sonic-buildimage/pull/5626/files#diff-e6f2fe13a6c276dc2f3b27a5bef79886f9c103194be4fcb28ce57375edf2c23cR39). The dictionary contains peer-group names as keys, and a list of address-families as values. So when bgpcfgd got a request to change the BBR state, it changes the state only for peer-groups listed in the constants.yml dictionary (and only for address families from the peer-group value).

**- How to verify it**
Initially, when we start SONiC FRR has BBR enabled for PEER_V4 and PEER_V6:
```
admin@str-s6100-acs-1:~$ vtysh -c 'show run' | egrep 'PEER_V.? allowas'
  neighbor PEER_V4 allowas-in 1
  neighbor PEER_V6 allowas-in 1
```

Then we apply following configuration to the db:
```
admin@str-s6100-acs-1:~$ cat disable.json                
{
        "BGP_BBR": {
            "all": {
                "status": "disabled"
            }
        }
}


admin@str-s6100-acs-1:~$ sonic-cfggen -j disable.json -w 
```
The log output are:
```
Oct 14 18:40:22.450322 str-s6100-acs-1 DEBUG bgp#bgpcfgd: Received message : '('all', 'SET', (('status', 'disabled'),))'
Oct 14 18:40:22.450620 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-f', '/tmp/tmpmWTiuq']'.
Oct 14 18:40:22.681084 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V4 soft in']'.
Oct 14 18:40:22.904626 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V6 soft in']'.
```

Check FRR configuraiton and see that no allowas parameters are there:
```
admin@str-s6100-acs-1:~$ vtysh -c 'show run' | egrep 'PEER_V.? allowas' 
admin@str-s6100-acs-1:~$
```

Then we apply enabling configuration back:
```
admin@str-s6100-acs-1:~$ cat enable.json 
{
        "BGP_BBR": {
            "all": {
                "status": "enabled"
            }
        }
}

admin@str-s6100-acs-1:~$ sonic-cfggen -j enable.json -w 
```
The log output:
```
Oct 14 18:40:41.074720 str-s6100-acs-1 DEBUG bgp#bgpcfgd: Received message : '('all', 'SET', (('status', 'enabled'),))'
Oct 14 18:40:41.074720 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-f', '/tmp/tmpDD6SKv']'.
Oct 14 18:40:41.587257 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V4 soft in']'.
Oct 14 18:40:42.042967 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V6 soft in']'.
```


Check FRR configuraiton and see that the BBR configuration is back:
```
admin@str-s6100-acs-1:~$ vtysh -c 'show run' | egrep 'PEER_V.? allowas'
  neighbor PEER_V4 allowas-in 1
  neighbor PEER_V6 allowas-in 1
```

*** The test coverage ***
Below is the test coverage
```
---------- coverage: platform linux2, python 2.7.12-final-0 ----------
Name                             Stmts   Miss  Cover
----------------------------------------------------
bgpcfgd/__init__.py                  0      0   100%
bgpcfgd/__main__.py                  3      3     0%
bgpcfgd/config.py                   78     41    47%
bgpcfgd/directory.py                63     34    46%
bgpcfgd/log.py                      15      3    80%
bgpcfgd/main.py                     51     51     0%
bgpcfgd/manager.py                  41     23    44%
bgpcfgd/managers_allow_list.py     385     21    95%
bgpcfgd/managers_bbr.py             76      0   100%
bgpcfgd/managers_bgp.py            193    193     0%
bgpcfgd/managers_db.py               9      9     0%
bgpcfgd/managers_intf.py            33     33     0%
bgpcfgd/managers_setsrc.py          45     45     0%
bgpcfgd/runner.py                   39     39     0%
bgpcfgd/template.py                 64     11    83%
bgpcfgd/utils.py                    32     24    25%
bgpcfgd/vars.py                      1      0   100%
----------------------------------------------------
TOTAL                             1128    530    53%
```

**- Which release branch to backport (provide reason below if selected)**

- [ ] 201811
- [x] 201911
- [x] 202006
2020-10-22 11:04:21 -07:00
abdosi
ef8a48a728
Everflow and Dataacl were getting binded to Backplane port-channels. (#5691)
Issue was because we were relying on port_alias_asic_map dictionary
but that dictionary can't be used as alias name format has changed.

Fix the port alias mapping as what is needed.

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2020-10-22 10:29:46 -07:00
Prince Sunny
63d40865ab
[Submodule] sonic-restapi (#5679)
37a7c61 - 2020-10-16 : Add to local route DB if ifname present in route add API (#51) [Sumukha Tumkur Vani]
2b4b6d7 - 2020-10-01 : Fix typo (#50) [Sumukha Tumkur Vani]
f0a527b - 2020-09-24 : Use RESTAPI_DB, cache reset_status (#49) [Prince Sunny]
2020-10-22 08:28:02 -07:00
pavel-shirshov
d19d1dd569
[bgpcfgd]: Change prefix-list generation for "Allow prefix" feature (#5639)
**- Why I did it**
I was asked to change "Allow list" prefix-list generation rule.
Previously we generated the rules using following method:
``` 
For each {prefix}/{masklen} we would generate the prefix-rule
permit {prefix}/{masklen} ge {masklen}+1
Example:
Prefix 1.2.3.4/24 would have following prefix-list entry generated
permit 1.2.3.4/24 ge 23
```
But we discovered the old rule doesn't work for all cases we have.

So we introduced the new rule:
```
For ipv4 entry,  
For mask  < 32 , we will add ‘le 32’ to cover all  prefix masks to be sent by T0  
For mask =32 , we will not add any ‘le mask’ 
For ipv6 entry, we will add le 128 to cover all the prefix mask to be sent by T0  
For mask < 128 , we will add ‘le 128’ to cover all prefix masks to be sent by T0 
For mask = 128 , we will not add any ‘le mask’ 
```    

**- How I did it**
I change prefix-list entry generation function. Also I introduced a test for the changed function.

**- How to verify it**
1. Build an image and put it on your dut.

2. Create a file test_schema.conf with the test configuration
```
{
    "BGP_ALLOWED_PREFIXES": {
        "DEPLOYMENT_ID|0|1010:1010": {
            "prefixes_v4": [
                "10.20.0.0/16",
                "10.50.1.0/29"
            ],
            "prefixes_v6": [
                "fc01:10::/64",
                "fc02:20::/64"
            ]
        },
        "DEPLOYMENT_ID|0": {
            "prefixes_v4": [
                "10.20.0.0/16",
                "10.50.1.0/29"
            ],
            "prefixes_v6": [
                "fc01:10::/64",
                "fc02:20::/64"
            ]
        }
    }
}
```

3. Apply the configuration by command 
```
sonic-cfggen -j test_schema.conf --write-to-db
```

4. Check that your bgp configuration has following prefix-list entries:
```
admin@str-s6100-acs-1:~$ show runningconfiguration bgp | grep PL_ALLOW
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V4 seq 10 deny 0.0.0.0/0 le 17
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V4 seq 20 permit 127.0.0.1/32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V4 seq 30 permit 10.20.0.0/16 le 32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V4 seq 40 permit 10.50.1.0/29 le 32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V4 seq 10 deny 0.0.0.0/0 le 17
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V4 seq 20 permit 127.0.0.1/32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V4 seq 30 permit 10.20.0.0/16 le 32
ip prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V4 seq 40 permit 10.50.1.0/29 le 32
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V6 seq 10 deny ::/0 le 59
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V6 seq 20 deny ::/0 ge 65
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V6 seq 30 permit fc01:10::/64 le 128
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_1010:1010_V6 seq 40 permit fc02:20::/64 le 128
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V6 seq 10 deny ::/0 le 59
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V6 seq 20 deny ::/0 ge 65
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V6 seq 30 permit fc01:10::/64 le 128
ipv6 prefix-list PL_ALLOW_LIST_DEPLOYMENT_ID_0_COMMUNITY_empty_V6 seq 40 permit fc02:20::/64 le 128

``` 

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-10-20 00:38:09 -07:00
Danny Allen
cdb82b9fc2
[swss] Update swss and swss-common submodule pointers (#5630)
[swss]
[acl] Replace IP_PROTOCOL with NEXT_HEADER for IPv6 ACL tables (#1458)
[acl] Refactor port OID retrieval into aclorch (#1462)
Fix issue #5157 by identifying the dependency among objects and avoiding releasing an object still being referenced (#1440)
[mock tests] Update MockDBConnector to match new swsscommon interface (#1465)

[swss-common]
netlink: Setting nl_socket buffer size to 3M from 2M (#391)
Added support in Swig file to cast Selectable object to Subscriber Table object (#394)
[warm reboot] Warm Reboot Support for EVPN VXLAN (#350)
Implement DBInterface/SonicV2Connector in C++ (#387)
Fix memory leak if a RedisCommand object were to be reused (#392)

Signed-off-by: Danny Allen <daall@microsoft.com>
2020-10-16 14:28:55 -07:00
Stepan Blyshchak
8df0e2b6ce
[sonic-cfggen] store jinja2 cache in log level db. (#5646)
This PR makes two changes:
    - Store Jinja2 cache in LOGLEVEL DB instead of STATE DB
    - Store bytecode cache encoded in base64

Tested with the following command: "redis-dump -d 3 -k JINJA2_CACHE"

Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
2020-10-16 12:00:57 -07:00
Joe LeVeque
d8363a99dc
[sonic-platform-common] Update submodule (#5625)
Includes the following commits:

* src/sonic-platform-common 111dcf7...b316f8d (5):
  > [sonic_platform_base] fix an indentation in fan_drawer_base.py (#133)
  > [SONIC_Y_CABLE] Adding new Y cable Package for accessing  Y cable QSFP's eeprom and configuring Y cable's mux through eeprom (#128)
  > Merge pull request #132 from vdahiya12/adding_changes_for_base_api
  > [ChassisBase][ModuleBase] Remove redundant 'get_serial_number()' method (#130)
  > Merge pull request #125 from vdahiya12/changes_for_mux_driver
2020-10-16 11:17:13 -07:00
Qi Luo
a5adb577f3
[sonic-snmpagent]: Advance submodule (#5614)
6e4a796 2020-10-12 | Fix: correctly handle that lldp_loc_man_addr contains only IPv6 address without IPv4 address (#164) [Qi Luo]
0f772ce 2020-09-28 | [ciscoSwitchQosMIB]: Remove invocation of update_data function during (#161) [SuvarnaMeenakshi]
2020-10-14 10:54:24 -07:00
Joe LeVeque
88c1d66c27
[python-click] No longer build our own package, let pip/setuptools install vanilla (#5549)
We were building our own python-click package because we needed features/bug fixes available as of version 7.0.0, but the most recent version available from Debian was in the 6.x range.

"Click" is needed for building/testing and installing sonic-utilities. Now that we are building sonic-utilities as a wheel, with Click specified as a dependency in the setup.py file, setuptools will install a more recent version of Click in the sonic-slave-buster container when building the package, and pip will install a more recent version of Click in the host OS of SONiC when installing the sonic-utilities package. Also, we don't need to worry about installing the Python 2 or 3 version of the package, as the proper one will be installed as necessary.
2020-10-14 10:16:35 -07:00
Nazarii Hnydyn
0155a15813
[submodule]: Advance swss submodule. (#5611)
- [portsorch]: Set proper initial forwarding state for LAG member #1448

Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>
2020-10-14 08:58:50 -07:00
pavel-shirshov
812e1a3489
[bgp]: Enable next-hop-tracking through default (#5600)
**- Why I did it**
FRR introduced [next hop tracking](http://docs.frrouting.org/projects/dev-guide/en/latest/next-hop-tracking.html) functionality. 
That functionality requires resolving BGP neighbors before setting BGP connection (or explicit ebgp-multihop command). Sometimes (BGP MONITORS) our neighbors are not directly connected and sessions are IBGP. In this case current configuration prevents FRR to establish BGP connections.  Reason would be "waiting for NHT". To fix that we need either add static routes for each not-directly connected ibgp neighbor, or enable command `ip nht resolve-via-default`

**- How I did it**
Put `ip nht resolve-via-default` into the config

**- How to verify it**
Build an image. Enable BGP_MONITOR entry and check that entry is Established or Connecting in FRR

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-10-13 22:21:28 -07:00
vdahiya12
3cd1d8e923
[sonic-py-common] Add platform.json to port_config files candidates only if interfaces are present within platform.json (#5538)
* [sonic-py-common] add platform.json to port_config

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
2020-10-13 16:58:06 -07:00
lguohan
4a390e93cf
[submodule]: sonic-utilities (#5606)
* b6af9f4 2020-10-12 | Fix expected neighbor when multiple ports connect to same neighbor (#1162) [lguohan]
* a71c72b 2020-10-12 | [show] Add 'show' CLI for system-health feature (#971) [shlomibitton]
* 561d133 2020-10-10 | [README.md] Add instructions for setting up a build/test environment (#1161) [Joe LeVeque]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-10-13 15:25:45 -07:00
Junchao-Mellanox
1c97a03b81
[system-health] Add support for monitoring system health (#4835)
* system health first commit

* system health daemon first commit

* Finish healthd

* Changes due to lower layer logic change

* Get ASIC temperature from TEMPERATURE_INFO table

* Add system health make rule and service files

* fix bugs found during manual test

* Change make file to install system-health library to host

* Set system LED to blink on bootup time

* Caught exceptions in system health checker to make it more robust

* fix issue that fan/psu presence will always be true

* fix issue for external checker

* move system-health service to right after rc-local service

* Set system-health service start after database service

* Get system up time via /proc/uptime

* Provide more information in stat for CLI to use

* fix typo

* Set default category to External for external checker

* If external checker reported OK, save it to stat too

* Trim string for external checker output

* fix issue: PSU voltage check always return OK

* Add unit test cases for system health library

* Fix LGTM warnings

* fix demo comments: 1. get boot up timeout from monit configuration file; 2. set system led in library instead of daemon

* Remove boot_timeout configuration because it will get from monit config file

* Fix argument miss

* fix unit test failure

* fix issue: summary status is not correct

* Fix format issues found in code review

* rename th to threshold to make it clearer

* Fix review comment: 1. add a .dep file for system health; 2. deprecated daemon_base and uses sonic-py-common instead

* Fix unit test failure

* Fix LGTM alert

* Fix LGTM alert

* Fix review comments

* Fix review comment

* 1. Add relevant comments for system health; 2. rename external_checker to user_define_checker

* Ignore check for unknown service type

* Fix unit test issue

* Rename user define checker to user defined checker

* Rename user_define_checkers to user_defined_checkers for configuration file

* Renmae file user_define_checker.py -> user_defined_checker.py

* Fix typo

* Adjust import order for config.py

Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com>

* Adjust import order for src/system-health/health_checker/hardware_checker.py

Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com>

* Adjust import order for src/system-health/scripts/healthd

Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com>

* Adjust import orders in src/system-health/tests/test_system_health.py

* Fix typo

* Add new line after import

* If system health configuration file not exist, healthd should exit

* Fix indent and enable pytest coverage

* Fix typo

* Fix typo

* Remove global logger and use log functions inherited from super class

* Change info level logger to notice level

Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com>
2020-10-12 11:12:49 +03:00
Joe LeVeque
5e97574e5d
[sonic-utilities] Update submodule (#5578)
* src/sonic-utilities 6e58dff...e1244a5 (5):
  > [setup.py] Pin external runtime dependencies at known working versions (#1151)
  > [config] Reload Monit configuration after potential hostname changes (#1132)
  > [NAT]:Updated natshow script to support DNAT Pool changes (#1001)
  > [pfcstat] Create new copies of Rx and Tx dictionaries (#1150)
  > [master]Fix show ip interface neighbor info not correct when add second ip (#1134)
2020-10-10 14:14:58 -07:00
Shi Su
c1add9a9c5
[sonic-sairedis] Update submodule (#5568)
Signed-off-by: shi-su <ssu@microsoft.com>
2020-10-08 23:37:36 -07:00
anish-n
42d9a44e05
Minigraph resource type changes (#5198)
* Parse sub_role from minigraph into DEVICE_METADATA
* Change minigraph sub_role to resource_type
2020-10-08 15:45:21 -07:00
Mahesh Maddikayala
744612d269
[ECMP][Multi-ASIC] Have different ECMP seed value on each ASIC (#5357)
* Calculate ECMP hash seed based on ASIC ID on multi ASIC platform. Each ASIC will have a unique ECMP hash seed value.
2020-10-08 09:05:37 -07:00
Petro Bratash
895f4e0bf7
[sonic-utilities] Submodule update (#5558)
- Fix show queue watermark command fail
     Azure/sonic-utilities@6e58dff

Signed-off-by: Petro Bratash <petrox.bratash@intel.com>
2020-10-07 11:32:38 -07:00
zzhiyuan
3da996bbf8
[arista]: Add disable_pcie_firmware_check soc property (#5543)
This is to fix pcie firmware check assert in Broadcom SDK once the SAI changes merges. This will be in the future but adding the soc property in the broadcom config now.

Co-authored-by: Zhi Yuan Carl Zhao <zyzhao@arista.com>
2020-10-06 18:35:15 -07:00
pavel-shirshov
63c1afbc68
[bgpcfgd]: Extract classes into their own files. Run bgpcfgd as a module (#5535)
1. Rename app module to bgpcfgd
2. Extract classes from one file to the module
2020-10-06 08:37:49 -07:00
Danny Allen
90d84143e8
[submodule update] Update sonic-utilities submodule pointer (#5545)
- [acl-loader] Revert fix for IP protocol == 0 (#1142)
- Show sflow interface to display all interfaces enabled for sflow (#1143)
- [sflow_test.py]: tests for config sflow commands. (#1112)
- [config/console] Support add/del console port setting commands (#1136)
- [cli][bgp]use vtysh in the show ip bgp summary command (#1137)
- [README.md] Describe new package creation schema (#1131)
- [consutil] Remove actual baud and refactor lib for future change (#1130)

Signed-off-by: Danny Allen <daall@microsoft.com>
2020-10-05 19:54:21 -07:00
Lawrence Lee
8c344095a8
[docker-orchagent]: Add NDP Proxy Daemon (#5517)
* Install ndppd during image build, and copy config files to image
* Configure proxy settings based on config DB at container start
* Pipe ndppd output to logger inside container to log output in syslog
2020-10-05 08:48:13 -07:00
Dong Zhang
6b0690adbb
[sonic-py-swsssdk] update submodule for using correct connect parameter number (#5524)
* [MultiDB] fix wrong parameter number in interface.py (#88)

Signed-off-by: Dong Zhang d.zhang@alibaba-inc.com
2020-10-03 23:03:13 -07:00
Mahesh Maddikayala
32a517f583
[sonic-swss] Update sonic-swss submodule (#5533) 2020-10-03 09:47:51 -07:00
pavel-shirshov
ffae82f8be
[bgp] Add 'allow list' manager feature (#5513)
implements a new feature: "BGP Allow list."

This feature allows us to control which IP prefixes are going to be advertised via ebgp from the routes received from EBGP neighbors.
2020-10-02 10:06:04 -07:00
abdosi
c0bbb7b63d
Fix python expception of missing subprocess (#5503)
Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2020-10-02 09:09:38 -07:00
abdosi
dda98023e3
[Submodule update] sonic swss (#5489)
be51ebc533cbe92a885c584b4611ffc92d10ea36 Add IPv6 key item support to request parser (#1449)
76e22516bffe45d577ab9cce4f5d96687140fe75 When teamd feature state is disabled the Netdevice created by teamd were (#1450)
6aa97ce7982b78a26803908c0ce12302e51df793 Use .clear() after std::move() (#1444)
d5757db5d896aa2405f6cbf9f4c150f46b27aa4e Add libzmq to README dependencies (#1447)
c7b262ee61973a48943603810c8246a2471d1862 Add libzmq to Makefiles (#1443)
0b2e59ab6b909e5ea06013da03215f0d1af1e618 [drop counters] Clarify log messages for initial counter setup (#1445)
003cf24353c845ef708d192b7da8d571537671e1 [dvs] Refactor and add buffer pool wm test (#1446)
2f5d2d99dfae506333bddda9de2f289f92bb889b [acl] Remove Ethertype from L3V6 qualifiers (#1433)
f7b974f7fc6474a6493a7b6d0676e17caffa6165 Fix issue: bufferorch only pass the first attribute to sai when setting attribute (#1442)

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2020-10-02 09:06:30 -07:00
Qi Luo
30f5557d36
Fix generate_l2_config: don't override hostname or device role (ToRRouter) (#5510)
* Fix generate_l2_config: don't override hostname because sonic-cfggen may not read from Redis. Fix test_l2switch_template test case to test preset l2 feature.
* Improve test script: compare json files with sort_keys
* Revert changes on sample_output
* Remove members field in VLAN section. Fix test assertTrue statement.
2020-10-01 23:24:35 -07:00
pavel-shirshov
1f0f751d4d
[bgpcfgd]: Don't use neighbor metadata in bgpmon sessions (#5506)
**- Why I did it**
BGP_MONITORS sessions don't have corresponding DEVICE_NEIGHBOR_METADATA CONFIG_DB entries in the minigraphs. Prevent bgpcfgd to wait on such entries for BGP_MONITORS sessions.

**- How I did it**
Set constructor argument to False that means - don't wait for device neighbors metadata info for BGP_MONITORS

**- How to verify it**
Build an image, write on your device, use a minigraph with BGP_MONITORS sessions. Check that sessions are populated in the config.
2020-09-30 22:35:52 -07:00
Dong Zhang
b01879b44b
[sonic-py-swsssdk] update submodule for changing redis_client index from db_id to db_name (#5512) 2020-09-30 13:42:20 -07:00
Tamer Ahmed
57a584997a [cfggen] Iterative Version Of Deep Update
Avoiding recursive update of maps as it consumes stack frames. This
PR introduces iterative version of deep_update method.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Tamer Ahmed
110f7b7817 [cfggen] Build Python 2 And Python 3 Wheel Packages
This builds Python 2&3 wheel packages for sonic-cfggen script.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Tamer Ahmed
99ffce6b57 [cfggen] Move Python 2 Test Data to Py2 Folder
Rendering templates show different orders when rendered using Python
2 vs Python 3. This PR prepare for Python 3 packaging by creating
new dir 'py2' for Python 2 rendered test cases.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Tamer Ahmed
6754635010 [cfggen] Make Jinja2 Template Python 3 Compatible
Jinja2 templates rendered using Python 3 interpreter, are required
to conform with Python 3 new semantics.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00