Signed-off-by: bingwang <wang.bing@microsoft.com>
Why I did it
This PR brings two changes
Add lossy PG profile for PG2 and PG6 on T1 for ports between T1 and T2.
After PR Update qos config to clear queues for bounced back traffic #10176 , the DSCP_TO_TC_MAP and TC_TO_PG_MAP is updated when remapping is enable
DSCP_TO_TC_MAP
Before After Why do this change
"2" : "1" "2" : "2" Only change for leaf router to map DSCP 2 to TC 2 as TC 2 will be used for lossless TC
"6" : "1" "6" : "6" Only change for leaf router to map DSCP 6 to TC 6 as TC 6 will be used for lossless TC
TC_TO_PRIORITY_GROUP_MAP
Before After Why do this change
"2" : "0" "2" : "2" Only change for leaf router to map TC 2 to PG 2 as PG 2 will be used for lossless PG
"6" : "0" "6" : "6" Only change for leaf router to map TC 6 to PG 6 as PG 6 will be used for lossless PG
So, we have two new lossy PGs (2 and 6) for the T2 facing ports on T1, and two new lossless PGs (2 and 6) for the T0 facing port on T1.
However, there is no lossy PG profile for the T2 facing ports on T1. The lossless PGs for ports between T1 and T0 have been handled by buffermgrd .Therefore, We need to add lossy PG profiles for T2 facing ports on T1.
We don't have this issue on T0 because PG 2 and PG 6 are lossless PGs, and there is no lossy traffic mapped to PG 2 and PG 6
Map port level TC7 to PG0
Before the PCBB change, DSCP48 -> TC 6 -> PG 0.
After the PCBB change, DSCP48 -> TC 7 -> PG 7
Actually, we can map TC7 to PG0 to save a lossy PG.
How I did it
Update the qos and buffer template.
How to verify it
Verified by UT.
- Why I did it
While doing config reload, FEATURE table may be removed and re-add. During this process, updating FEATURE table is not atomic. It could be that the FEATURE table has entry, but each entry has no field. This PR introduces a retry mechanism to avoid this.
- How I did it
Introduces a retry mechanism to avoid this.
- How to verify it
New unit test added to verify the flow as well as running some manual test.
#### Why I did it
Support the following tables which were introduced during dynamic buffer calculation
- LOSSLESS_TRAFFIC_PATTERN
- DEFAULT_LOSSLESS_BUFFER_PARAMETER
#### How I did it
- LOSSLESS_TRAFFIC_PATTERN
|name|type|range|mandatory|description|
|---|---|---|---|---|
|mtu|uint16|64~10240|true|The maximum packet size of a lossless packet|
|small_packet_percentage|uint8|0~100|true|The percentage of small packet|
- DEFAULT_LOSSLESS_BUFFER_PARAMETER
|name|type|range|mandatory|description|
|---|---|---|---|---|
|default_dynamic_th|int8|-8~7|true|The default dynamic_th for all buffer profiles that are dynamically generated for lossless PG|
|over_subscribe_ratio|uint16|-|false|The oversubscribe ratio for shared headroom pool.|
|||||Semantically, the upper bound is the number of physical ports but it can not be represented in the yang module. So we keep the upper bound open. As the type is (signed) integer whose lower bound is 0 by nature, we do not need to specify the range.|
#### How to verify it
Run unit test
Why I did it
To address internal build failures where the cable len for some of the skus is set to 300m for all tiers.
How I did it
For the buffers test, generate a new output file based off the original expected output with CABLE_LENGTH table updated to use 300m. In the comparison logic, compare against each of the expected output files and if any matches, the testcase is set to pass
Signed-off-by: Neetha John <nejo@microsoft.com>
Why I did it
As part of PCBB changes, we need to enable 2 extra lossless queues. The changes in this PR are done to adjust only the reserved sizes on Th2 for the additional 2 lossless queues
Calculations are done based on 40 downlinks for T1 and 16 uplinks for dual ToR
How to verify it
Verified that the rendering works fine on Th2 dut
Unit tests have been updated to reflect the modified buffer sizes when pcbb is enabled. There are existing testcases that will test the original buffer sizes when pcbb is disabled. With these changes, was able to build sonic-config-engine wheel successfully
Signed-off-by: Neetha John <nejo@microsoft.com>
#### Why I did it
There might be a case where service checker periodic operation determined that specific container is running but when it tries to perform an operation on it, it was already closed by the user. This is a valid flow and we should not log an error message, informative warning is enough.
#### How I did it
I reduce log severity.
#### How to verify it
I verified it manually.
Signed-off-by: bingwang <bingwang@microsoft.com>
Why I did it
This PR is to add two extra lossless queues for bounced back traffic.
HLD sonic-net/SONiC#950
SKUs include
Arista-7050CX3-32S-C32
Arista-7050CX3-32S-D48C8
Arista-7260CX3-D108C8
Arista-7260CX3-C64
Arista-7260CX3-Q64
How I did it
Update the buffers.json.j2 template and buffers_config.j2 template to generate new BUFFER_QUEUE table.
For T1 devices, queue 2 and queue 6 are set as lossless queues on T0 facing ports.
For T0 devices, queue 2 and queue 6 are set as lossless queues on T1 facing ports.
Queue 7 is added as a new lossy queue as DSCP 48 is mapped to TC 7, and then mapped into Queue 7
How to verify it
Verified by UT
Verified by coping the new template and generate buffer config with sonic-cfggen
Why I did it
Recently the nightly testing pipeline found that the autorestart test case was failed when it was run against master image. The reason is Restart= field in each container's systemd configuration file was set to Restart=no even the value of auto_restart field in FEATURE table of CONFIG_DB is enabled.
This issue introduced by #10168 can be reproduced by the following steps:
Issues the config command to disable the auto-restart feature of a container
Runs command config reload or config reload minigraph to enable auto-restart of the container
Checks Restart= field in the container's systemd config file mentioned in step 1 by running the command
sudo systemctl cat <container_name>.service
Initially this PR (#10168) wants to revert the changes proposed by this: #8861. However, it did not fully revert all the changes.
How I did it
When hostcfgd started or was restarted, the Restart= field in each container's systemd configuration file should be initialized according to the value of auto_restart field in FEATURE table of CONFIG_DB.
How to verify it
I verified this change by running auto-restart test case against newly built master image and also ran the unittest:
The following commits are pushed
1f112b8 (HEAD -> 202205, origin/202205) [sonic-ycabled] fix grpc logic for timeout,cli HWSTATUS value retrival logic for active-active cable (#264)
Signed-off-by: vaibhav-dahiya vdahiya@microsoft.com
* [Tunnel PFC] Tests for adding property 'sai_remap_prio_on_tnl_egress'
Add tests for adding property 'sai_remap_prio_on_tnl_egress', this
property should only be added in dual tor environment.
Test done:
Run test test_j2files.py
Co-authored-by: richardyu <richardyu@contoso.com>
Including change:
* 7ff8f75 2022-06-03 | Revert "[portsorch]: Prevent LAG member configuration when port has active ACL binding (#2165)" (#2306) (HEAD -> 202205, github/202205) [bingwang-ms]
Signed-off-by: Ying Xie <ying.xie@microsoft.com>
- Why I did it
Yang Model about password hardening feature, the sonic CLI of this feature was autogenerated from this Yang model
- How I did it
Create new Yang model in src/sonic-yang-models/yang-models/sonic-passwh.yang.
- How to verify it
There are unitests(yang test) in this P.R covering all the passwords policies with good and bad values cases.
Or is possible manually using the config/show password commands that were autogenerated from this Yang model. (this CLI code added in sonic-utilities)
Signed-off-by: Neetha John <nejo@microsoft.com>
Why I did it
There was a typo in hwsku specified as part of #10889
How I did it
Replaced with the correct hwsku
How to verify it
test_cfggen.py is passing
Updating sonic-utilities sub module with the following commits
84dbd93 Clear the fvs vector before popping the message out of notification
a90b2b7 selectabletimer: add mutex to start() and stop()
7ae22be Fix SIGTERM can't terminate PubSub::listen issue
#### Why I did it
To fix hostcfgd can't terminate by sigint issue, need update sonic-swss-common submodule.
#### How I did it
#### How to verify it
#### Which release branch to backport (provide reason below if selected)
#### Description for the changelog
84dbd93 Clear the fvs vector before popping the message out of notification
a90b2b7 selectabletimer: add mutex to start() and stop()
7ae22be Fix SIGTERM can't terminate PubSub::listen issue
#### A picture of a cute animal (not mandatory but encouraged)
#### Why I did it
For yang model, sample_config_db.json file was missing Sample data for the features SNAT/DNAT/IPMC
#### How I did it
Added the SNAT,DNAT,IPMC(low Threshold/high threshold/threshold_type)entries in CRM table.
#### How to verify it
With sanity Build/test only.
[muxorch] Handling optional attributes in muxorch (#2288)
Update netlink messages handler (#2233)
Broadcast Unknown-multicast and Unknown-unicast Storm-control (#1306)
[vstest]: Increase PollingConfig default timeout (#2285)
[FDB] Fix fbdorch to properly handle syncd FDB FLUSH Notif (#2254)
[macsecorch]: Support for non-default sa per sc (#2250)
Migrating the NAT vs tests from Click to direct DB access (#2278)
[neighsync] Ignoring IPv4 link local addresses (#2260)
[IntfMgrd] Retry adding ipv6 prefix by setting disabled_ipv6 flag (#2267)
Increase Redis Timeout value for Switch Create Opration for Packet (#2243)
Update fdborch.cpp (#2261)
Signed-off-by: dprital <drorp@nvidia.com>
- Why I did it
With SAI V.1.10.2, new interface types CR8/SR8/KR8/LR8 have been introduced, we should also support them from the CLI configuration.
- How I did it
Add new enum for the new interface types
- How to verify it
Run the "config interface type" command to verify new interface types can be accepted and handled correctly.
Signed-off-by: Kebo Liu <kebol@nvidia.com>
Why I did it
It is to improve the build performance, when building multiple targets.
The modified time of downloaded files should be not older than the file .platform.
If not, the file will be downloaded again, when building any dependent targets.
How I did it
When downloading the packages from web site, the modified time will be changed by the command "touch".
#### Why I did it
To ensure that some internal testcases do not break due to external changes
#### How to verify it
Ran test_cfggen.py with the changes and it passed
- Why I did it
YANG schema is missing for sonic-telemetry
- How I did it
Added YANG schema to sonic-yang-models and appropriate unit tests inside of test and test_config
- How to verify it
Build sonic-yang-models python wheels target and verify that unit tests are passing
Why I did it
Upgrade FRR to version 8.2.2. Build libyang2 required by FRR.
How I did it
Update FRR version and tag.
How to verify it
Following tests were performed on sonic-vs:
BGP docker status check
BGP configuration and session establishment
Route redistribution and ping
Issued show commands to check the bgp neighbor and routes
Checked app-db to ensure bgp routes are installed with correct interface and nexthop.
Create VRF and check FRR knows the VRF
Check VRF routes are installed in app-db with correct Vrf name and next-hop
Establish BGP Evpn session and check if Evpn routes (multicast, mac, prefix) are exchanged and installed correctly in app-db.
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan arlakshm@microsoft.com
Why I did it
resolves#10761.
For VOQ chassis, the Recirc port, which was added for the Everflow, stays admin down after load minigraph.
This PR add the fix to make the recirc port as admin up
How I did it
The PR adds a change in minigraph.py, if port has role as Rec make the the port as admin-status up.
How to verify it
UT
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
Why I did it
To further add cable_type and soc_ipv4 field to table MUX_CABLE, this PR tries to parse the minigraph like the following:
```
<Device i:type="SmartCable">
<ElementType>SmartCable</ElementType>
<SubType>active-active</SubType>
<Address xmlns:d5p1="Microsoft.Search.Autopilot.NetMux">
<d5p1:IPPrefix>192.168.0.3/21</d5p1:IPPrefix>
</Address>
<AddressV6 xmlns:d5p1="Microsoft.Search.Autopilot.NetMux">
<d5p1:IPPrefix>::/0</d5p1:IPPrefix>
</AddressV6>
<ManagementAddress xmlns:d5p1="Microsoft.Search.Autopilot.NetMux">
<d5p1:IPPrefix>0.0.0.0/0</d5p1:IPPrefix>
</ManagementAddress>
<ManagementAddressV6 xmlns:d5p1="Microsoft.Search.Autopilot.NetMux">
<d5p1:IPPrefix>::/0</d5p1:IPPrefix>
</ManagementAddressV6>
<SerialNumber i:nil="true" />
<Hostname>svcstr-7050-acs-1-Servers0-SC</Hostname>
</Device>
<Device i:type="Server">
<ElementType>Server</ElementType>
<Address xmlns:d5p1="Microsoft.Search.Autopilot.NetMux">
<d5p1:IPPrefix>192.168.0.2/21</d5p1:IPPrefix>
</Address>
<AddressV6 xmlns:d5p1="Microsoft.Search.Autopilot.NetMux">
<d5p1:IPPrefix>fc02:1000::2/64</d5p1:IPPrefix>
</AddressV6>
<ManagementAddress xmlns:d5p1="Microsoft.Search.Autopilot.NetMux">
<d5p1:IPPrefix>0.0.0.0/0</d5p1:IPPrefix>
</ManagementAddress>
<Hostname>Servers0</Hostname>
</Device>
```
Signed-off-by: Longxiang Lyu lolv@microsoft.com
How I did it
get_mux_cable_entries will try to get the mux cable device from the devices list and get the cable type and soc ip address from the device definition.
How to verify it
Pass the unit-test
Why I did it
At present, there is no mechanism in an event driven model to know that the system is up with all the essential sonic services and also, all the docker apps are ready along with port ready status to start the network traffic. With the asynchronous architecture of SONiC, we will not be able to verify if the config has been applied all the way down to the HW. But we can get the closest up status of each app and arrive at the system readiness.
How I did it
A new python based system monitor tool is introduced under system-health framework to monitor all the essential system host services including docker wrapper services on an event based model and declare the system is ready. This framework gives provision for docker apps to notify its closest up status. CLIs are provided to fetch the current system status and also service running status and its app ready status along with failure reason if any.
How to verify it
"show system-health sysready-status" click CLI
Syslogs for system ready
What I did:
Added support to create route-map action set tag <user define value>
when the the allow prefix list matches. The tag can ben define by user in
constants.yml.
Why I did:
Since for Allow List feature we call from base route-map allow-list route-map having set tag option provides way for base route-map to do match tag and take any further action if needed. Adding tag provide metadata that can used by base route-map
Why I did it
https://github.com/Azure/SONiC/blob/master/doc/vxlan/Overlay%20ECMP%20with%20BFD.md
From the design, need to advertise the route with community string, the PR is to implement this.
How I did it
To use the route-map as the profile for the community string, all advertised routes can be associated with one route-map.
Add one file, mangers_rm.py, which is to add/update/del the route-map. Modified the managers_advertise_rt.py file to associate profile with IP route.
The route-map usage is very flexible, by this PR, we only support one fixed usage to add community string for route to simplify this design.
How to verify it
Implement new unit tests for mangers_rm.py and updated unit test for managers_advertise_rt.py.
Manually verified the test case in the test plan section, will add testcase in sonic-mgmt later. Azure/sonic-mgmt#5581
Why I did it
Config db schema generated by minigraph should run yang validation.
How I did it
Modify run_script to add yang validation.
How to verify it
Run sonic-config-engine unit test.
Signed-off-by: Gang Lv ganglv@microsoft.com
This is part of HLD Azure/SONiC#925
#### Why I did it
Add link-training support
#### How I did it
Update SONiC YANG for port link-training support
#### Description for the changelog
Add "link_training" to sonic-port.yang
#### Link to config_db schema for YANG module changes
https://github.com/sonic-net/SONiC/wiki/Configuration#port
Why I did it
Previous subport unit tests uses port channel names like PortChannel01, so for subport name generated PortChannel01.10, it exceeds Linux network interface name 15 char limit.
Signed-off-by: Longxiang Lyu lolv@microsoft.com
How I did it
Modify PortChannel01 to PortChannel1.
Why I did it
Fixes#10793
How I did it
Removed the switch_type validation from the Yang model.
How to verify it
compile sonic_yang_mgmt-1.0-py3-none-any.whl and sonic_yang_mgmt-1.0-py3-none-any.whl
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
288c2d8 Revert "[scripts/fast-reboot] Shutdown remaining containers through systemd (#2133)" (#2161)
bce4694 [autoneg] add support for remote speed advertisement (#2124)
a73f156 [show][vrf]Fixing show vrf to include vlan subinterface (#2158)
7a06457 [auto_ts] Enable register/de-register auto_ts config for APP Extension (#2139)
083ebcc Add transceiver-info items advertised for cmis-supported moddules (#2135)
0811214 Validate destination port is not LAG (#2053)
6ab1c51 [minigraph] Consume golden_config_db.json while loading minigraph (#2140)
c37a957 [Kdump] Remove the duplicate logic if Kdump was disabled (#2128)
1143869 Ordering fix for sfpshow eeprom (#2113)
fdb79b8 Allow fw update for other boot type against on the previous "none" boot fw update (#2040)
a54a091 [GCU] Supressing YANG errors from libyang while sorting (#1991)
fbfa8bc [GCU] Enabling AddRack and adding RemoveRack tests (#2143)
d012be9 [Command-Reference] Add CLI docs for route flow counter (#2069)
8c07d59 [Mellanox] [reboot] [asan] stop asan-enabled containers on reboot (#2107)
697aae3 Fix speed parsing when speed is NOT fetched from APPL_DB (#2138)
22a388b [show] fix get routing stack routine (#2137)
cb3a047 Support option --ports of config qos reload for reloading ports' QoS and buffer configuration to default (#2125)
154a801 Enhance "config interface type/advertised-type" to be blocked on RJ45 ports (#2112)
3732ac5 Add CLI for route flow counter feature (#2031)
29771e7 [techsupport] improve robustness (#2117)
f9dc681 [intfutil] Display RJ45 port and portchannel speed in 'M' instead of 'G' when it's <= 1000M (#2110)
781ae9f [config] Do not enable pfcwd for BmcMgmtToRRouter (#2136)
23e9398 [scripts/fast-reboot] Shutdown remaining containers through systemd (#2133)
576c9ef [scripts/fast-reboot] stop timers in advance (#2131)
4dad79c bugfix: incorrect command for portchannel creation (#2134)
c17b1f4 [show][muxcable] Decrease the timeout for show mux status/hwmode (#2130)
49d61f8 [scripts/fast-reboot] cleanup (#2132)
52ca324 [config/config_mgmt.py]: Fix dpb issue with upper case mac in (#2066)
9e2fbf4 Update db_migrator to support `pfcwd_sw_enable` (#2087)
4010bd0 FGNHG CLI changes (#1588)
6bd54d0 Fix 'show mac' output when FDB entry for default vlan is None instead of 1 (#2126)
Signed-off-by: Ze Gan <ganze718@gmail.com>
#### Why I did it
The SSCI is wrong in the output of MACsec so that the virtual SAI cannot parse the output corretly.
The wrong output:
```
142: macsec_eth1: protect on validate strict sc off sa off encrypt on send_sci on end_station off scb off replay off
cipher suite: GCM-AES-XPN-256, using ICV length 16
TXSC: 5254008f4f1c0001 on SA 0
0: PN 103, state on, key 12cbc4b64e26c9a1ba14d810da20d16e
SSCI 33554432, RXSC: 525400edac5b0001, state on
0: PN 107, state on, key 12cbc4b64e26c9a1ba14d810da20d16e
offload: off
```
Expected
```
142: macsec_eth1: protect on validate strict sc off sa off encrypt on send_sci on end_station off scb off replay off
cipher suite: GCM-AES-XPN-256, using ICV length 16
TXSC: 5254008f4f1c0001 on SA 0
0: PN 252, state on, SSCI 33554432, key 12cbc4b64e26c9a1ba14d810da20d16e
RXSC: 525400edac5b0001, state on
0: PN 264, state on, key 12cbc4b64e26c9a1ba14d810da20d16e
```
#### How I did it
Move SSCI before the key so that SSCI will not be the front of SC information.
#### Why I did it
To pick up new commits:
* 60d2467 Add depends to p4rt debian package
#### How I did it
update sonic-p4rt/sonic-pins submodule pointer
#### How to verify it
should be able to build with p4rt enabled.
#### Why I did it
This function is critical for is_multi_asic() and SonicDBConfig initializing. No explicit reading ConfigDB. Otherwise it will implicitly trigger SonicDBConfig initializing.
#### How I did it
1. No explicit reading ConfigDB in get_asic_conf_file_path()
2. Collect asic_conf_path_candidates lazily to prevent any unnecessary side effect and improve the performance
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
Why I did it
submodule update for the following commits
7a203b1 [chassis] Add new tables in counter db for Voq counter support. (#530)
5effea3 add new table schema for bgp profile (#608)
130dca5 [ci] Update azure pipeline branch variable reference.
708ed39 [ci] Parameterize pipeline and improve azure pipeline (#599)
9c08456 Added new P4RT tables. (#604)
#### Why I did it
Fix issue: Non compliant leaf list in config_db schema: https://github.com/Azure/sonic-buildimage/issues/9801
#### How I did it
The basic flow of DPB is like:
1. Transfer config db json value to YANG json value, name it “yangIn”
2. Validate “yangIn” by libyang
3. Generate a YANG json value to represent the target configuration, name it “yangTarget”
4. Do diff between “yangIn” and “yangTarget”
5. Apply the diff to CONFIG DB json and save it back to DB
The fix:
• For step #1, If value of a leaf-list field string type, transfer it to a list by splitting it with “,” the purpose here is to make step#2 happy. We also need to save <table_name>.<key>.<field_name> to a set named “leaf_list_with_string_value_set”.
• For step#5, loop “leaf_list_with_string_value_set” and change those fields back to a string.
#### How to verify it
1. Manual test
2. Changed sample config DB and unit test passed
Signed-off-by: Neetha John nejo@microsoft.com
Why I did it
Address build failures due to sonic config engine unit tests failing. Failures are due to referencing format used in Arista 7800 sample output for buffer template
How I did it
Remove referencing format
How to verify it
Sonic config engine wheel should be built successfully
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan arlakshm@microsoft.com
Why I did it
Fixes#10158
How I did it
Add yang model for config_db table BGP_VOQ_CHASSIS_NEIGHBOR and UT
closes#10157
Why I did it
Add yang model for the bgp_internal_neighbor table in config_db
How I did it
Add new yang model file and unit tests
How to verify it
UT and compile sonic_yang_models-1.0-py3-none-any.whl and sonic_yang_mgmt-1.0-py3-none-any.whl
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
- Why I did it
To add support for 800G speed for port in the yang.
- How I did it
Change limitation from 400G to 800G.
- How to verify it
Set a port speed to 800G and run the yang DB validation. e.g. by using dynamic port breakout.
#### Why I did it
Need to pass LY_CTX_DISABLE_SEARCHDIR_CWD to Context in order to disable automatically searching for schemas in current working directory (which is by default searched automatically)
#### How I did it
add additional attribute into YANG context
#### How to verify it
Create some invalid link on switch :
1) **ln -s /usr/abc xxx**
2) run **spm list**
--> There should not be these messages:
```
libyang[1]: Unable to get information about "xxx" file in "/tmp" when searching for (sub)modules (No such file or directory)
libyang[1]: Unable to get information about "xxx" file in "/tmp" when searching for (sub)modules (No such file or directory)
libyang[1]: Unable to get information about "xxx" file in "/tmp" when searching for (sub)modules (No such file or directory)
libyang[1]: Unable to get information about "xxx" file in "/tmp" when searching for (sub)modules (No such file or directory)
```
Add the following commits:
- [orchagent, crm]: Reset crm threshold exceed count when threshold type changed 5ba6a54786c0fd9b155bb9ea2a7ed724a58aab74
- [pbh] [aclorch] Fixed a bug causes by updating the flow-counter value for the PBH rule 841f00389b338e91ddc4de460ace4ff96adfa796
- [ACL]Avoid incrementing crm count when ACL rule create fails 3d3364f9715fa05fbdf2d09b08676c3055903b84
- set remote vtep the netdev down before delete 7f53db782aed2973f4ff6807911b5a549461f3c7
- Removing Vnet with scope default 2ea8581da4ba6f97bebde4845a234d7c810e5515
#### Why I did it
Adding exceptlionList to validation exception
#### How I did it
Check code.
#### How to verify it
Ran manually.
- Run full config validation from a KVM
- Print the thrown exception
**Before**
```
Error: Data Loading Failed
All Keys are not parsed in FEATURE
dict_keys(['telemetry'])
```
**After**
```
Error: Data Loading Failed
All Keys are not parsed in FEATURE
dict_keys(['telemetry'])
exceptionList:["'status'"]
```
#### Which release branch to backport (provide reason below if selected)
<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
- [ ] 202111
#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->
#### Link to config_db schema for YANG module changes
<!--
Provide a link to config_db schema for the table for which YANG model
is defined
Link should point to correct section on https://github.com/Azure/SONiC/wiki/Configuration.
-->
#### A picture of a cute animal (not mandatory but encouraged)
Why I did it
Can not start sonic-hostservice
How I did it
Install python3-dbus and systemd-python, and replace invalid path
How to verify it
Start the service with below commands:
sudo systemctl start sonic-hostservice
sudo systemctl status sonic-hostservice
Signed-off-by: Gang Lv ganglv@microsoft.com
Why I did it
Migrate ptftests script to python3, in order to do an incremental migration, add python virtual environment firstly, install all required python packages in virtual env as well.
Then migrate ptftests scripts from python2 to python3 one by one avoid impacting non-changed scripts.
Signed-off-by: Zhaohui Sun zhaohuisun@microsoft.com
How I did it
Add python3 virtual environment for docker-ptf.
Add submodule ptf-py3 and install patched ptf 0.9.3 into virtual environment as well, two ptf issues were reported here:
p4lang/ptf#173p4lang/ptf#174
Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
Why I did it
Allow portchannel vlan sub intf long name format as long as it follows Linux interface name length limit(<16).
How I did it
Modify the leaf name check.
How to verify it
Test case passes.
Why I did it
Provide fix for comment: https://github.com/Azure/sonic-buildimage/pull/10475/files#r847753187;
How I did it
Try exception is not required in this scenario, so remove and modify to initial db config according to single or multi-asic platforms.
How to verify it
Verified on multi-asic device.
* [CG-Fix-CVE-2021-44906] Patching on thrift.0.14.1 for package minimist
Signed-off-by: richardyu-ms <richard.yu@microsoft.com>
* add more information in patch
Signed-off-by: richardyu-ms <richard.yu@microsoft.com>
* Update 0003-Remove-minimist-packages.patch
* change the thrift 0.14.1 to package download
Signed-off-by: richardyu-ms <richard.yu@microsoft.com>
* use the series file for patching
* fix a code defect
#### Why I did it
Fix several bugs:
1. If one vlan member belongs to multiple vlans, and if any of the vlans is "Tagged" type, we respect the tagged type
2. If one vlan member belongs to multiple vlans, and all of the vlans have no "Tagged" type, we override it to be a tagged member
3. make sure `vlantype_name` is assigned correctly in each iteration
#### How to verify it
1. Test the command line to parse a minigraph and make sure the output does not change.
```
./sonic-cfggen -m minigraph.mlnx20.xml
```
The minigraph is for HwSKU Mellanox-SN2700-D40C8S8.
2. Test on a DUT with HwSKU Mellanox-SN2700-D40C8S8
```
sudo config load_minigraph
show vlan brief
```
Checked the "Port Tagging" column in the output.
* [build]: Patch debootstrap to not unmount the host's /proc filesystem
Currently, when the final image is being built (sonic-vs.img.gz,
sonic-broadcom.bin, or similar), each invocation of sudo in the
build_debian.sh script takes 0.8 seconds to run and execute the actual
command. This is because the /proc filesystem in the slave container has
been unmounted somehow. This is happening when debootstrap is running,
and it incorrectly unmounts the host's (in our case, the slave
container's) /proc filesystem because in the new image being built,
/proc is a symlink to the host's (the slave container's) /proc. Because
of that, /proc is gone, and each invocation of sudo adds 0.8 seconds
overhead. As a side effect, docker exec into the slave container during
this time will fail, because /proc/self/fd doesn't exist anymore, and
docker exec assumes that that exists.
Debootstrap has fixed this in 1.0.124 and newer, so backport the patch
that fixes this into the version that Bullseye has.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
* [build_debian.sh]: Use eatmydata to speed up deb package installations
During package installations, dpkg calls fsync multiples times (for each
package) to ensure that tht efiles are written to disk, so that if
there's some system crash during package installation, then it is in at
least a somewhat recoverable state. For our use case though, we're
installing packages in a chroot in fsroot-* from a slave container and
then packaging it into an image. If there were a system crash (or even
if docker crashed), the fsroot-* directory would first be removed, and
the process would get restarted. This means that the fsync calls aren't
really needed for our use case.
The eatmydata package includes a library that will block/suppress the
use of fsync (and similar) system calls from applications and will
instead just return success, so that the application is not blocked on
disk writes, which can instead happen in the background instead as
necessary. If dpkg is run with this library, then the fsync calls that
it does will have no effect.
Therefore, install the eatmydata package at the beginning of
build_debian.sh and have dpkg be run under eatmydata for almost all
package installations/removals. At the end of the installation, remove
it, so that the final image uses dpkg as normal.
In my testing, this saves about 2-3 minutes from the image build time.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
* Change ln syntax to use chroot
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
9ac12bf (HEAD -> master, origin/master, origin/HEAD) Fix platform daemon chassisd to handle auto restart on fail (#247)
24fba04 [ycable] fix the logic to update cable_info values when ycable is not present; fix read side logic for ycable (#249)
Updating sonic-utilities sub module with the following commits
f09bd31 Fix UT failed cause by change pycommon to use swsscommon
c092300 Increased pcied unit test coverage to > 80%
7d7c85e Modular chassis: Psud set master led on first run
7195dcc Remove py2 from pipeline
c2e7393 [ycabled] increase UT coverage of ycabled daemon
#### Why I did it
When change pycommon to use swsscommon UT failed in sonic-platform-daemon, need submodule update with UT issue fix.
#### How I did it
#### How to verify it
#### Which release branch to backport (provide reason below if selected)
#### Description for the changelog
Fix UT failed cause by change pycommon to use swsscommon
Increased pcied unit test coverage to > 80%
Modular chassis: Psud set master led on first run
Remove py2 from pipeline
[ycabled] increase UT coverage of ycabled daemon
#### A picture of a cute animal (not mandatory but encouraged)
Why I did it
[Build]: Fix pip version constraint conflict issue
When a version is specified in the constraint file, if upgrading the version in build script, it will have conflict issue.
How I did it
If a specified version has specified in pip command line, then the version constraint will be skipped.
* [device config] Adding configuration for default route fallback
* Set sai_tunnel_underlay_route_mode attribute to fallback to default route if more specific route is unavailable.
Why I did it
Config db schema generated by minigraph can’t pass yang validation, PORT table does not have 'lanes' and 'speed' field.
How I did it
Make cfggen command fail when 'lanes' and 'speed' are not provided
How to verify it
Run 'sonic-cfggen -m xxx.xml --print-data' to make sure command fail when 'lanes' and 'speed' not in PORT table
Why I did it
minigraph parser has introduced new type.
How I did it
Update yang models to support BmcMgmtToRRouter.
How to verify it
Run unit test for sonic-yang-models
Signed-off-by: Gang Lv ganglv@microsoft.com
#### Why I did it
As of https://github.com/Azure/sonic-swss-common/pull/587 the blackout issue in ConfigDBConnector has been resolved.
In the past hostcfgd was refactored to use SubscriberStateTable instead of ConfigDBConnector for subscribing to CONFIG_DB updates due to a "blackout" period between hostcfgd pulling the table data down and running the initialization and actually calling `listen()` on ConfigDBConnector which starts the update handler.
However SusbscriberStateTable creates many file descriptors against the redis DB which is inefficient compared to ConfigDBConnector which only opens a single file descriptor.
With the new fix to ConfigDBConnector I refactored hostcfgd to take advantage of these updates.
#### How I did it
Replaced SubscriberStateTable with ConfigDBConnector
#### How to verify it
The functionality of hostcfgd can be verified by booting the switch and verifying that NTP is properly configured.
To check the blackout period you can add a delay in the hostcfgd `load()` function and also add a print statement before and after the load so you know when it occurs. Then restart hostcfgd and wait for the load to start, then during the load push a partial change to the FEATURE table and verify that the change is picked up and the feature is enabled after the load period finishes.
#### Description for the changelog
[hostcfgd] Move hostcfgd back to ConfigDBConnector for subscribing to updates
Why I did it
Running warm-reboot in a loop for 500 times leads to this error on 318-th iteration:
Apr 2 15:56:27.346747 sonic INFO swss#/supervisord: restore_neighbors Traceback (most recent call last):
Apr 2 15:56:27.346747 sonic INFO swss#/supervisord: restore_neighbors File "/usr/bin/restore_neighbors.py", line 24, in <module>
Apr 2 15:56:27.346747 sonic INFO swss#/supervisord: restore_neighbors from scapy.all import conf, in6_getnsma, inet_pton, inet_ntop, in6_getnsmac, get_if_hwaddr, Ether, ARP, IPv6, ICMPv6ND_NS, ICMPv6NDOptSrcLLAddr
Apr 2 15:56:27.346795 sonic INFO swss#/supervisord: restore_neighbors File "/usr/local/lib/python3.7/dist-packages/scapy/all.py", line 25, in <module>
Apr 2 15:56:27.346956 sonic INFO swss#/supervisord: restore_neighbors from scapy.route import *
Apr 2 15:56:27.346995 sonic INFO swss#/supervisord: restore_neighbors File "/usr/local/lib/python3.7/dist-packages/scapy/route.py", line 205, in <module>
Apr 2 15:56:27.347089 sonic INFO swss#/supervisord: restore_neighbors conf.iface = get_working_if()
Apr 2 15:56:27.347129 sonic INFO swss#/supervisord: restore_neighbors File "/usr/local/lib/python3.7/dist-packages/scapy/arch/linux.py", line 128, in get_working_if
Apr 2 15:56:27.347213 sonic INFO swss#/supervisord: restore_neighbors ifflags = struct.unpack("16xH14x", get_if(i, SIOCGIFFLAGS))[0]
Apr 2 15:56:27.347250 sonic INFO swss#/supervisord: restore_neighbors File "/usr/local/lib/python3.7/dist-packages/scapy/arch/common.py", line 31, in get_if
Apr 2 15:56:27.347345 sonic INFO swss#/supervisord: restore_neighbors return ioctl(sck, cmd, struct.pack("16s16x", iff.encode("utf8")))
Apr 2 15:56:27.347365 sonic INFO swss#/supervisord: restore_neighbors OSError: [Errno 19] No such device
The issue was reported to scapy devs secdev/scapy#3369, the fix is secdev/scapy#3371, however there is no released scapy version with this fix right now, thus decided to build scapy v2.4.5 from sources and apply the fix in a form of a patch.
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
In order to include the following commit:
0f06910 [PBH] Implement Edit Flows (Azure/sonic-swss#2169)
sonic-swss
50d5be2 Make changes to support compiling on Bullseye with GCC 10 (#2216)
0870cf5 [mirrororch]: Implement HW resources availability validation for SPAN/ERSPAN (#2187)
f4ec565 [vlanmgrd] fix use-after-free memory issue (#2211)
c2de7fc [QosOrch] The notifications cannot be drained in QosOrch in case the first one needs to retry (#2206)
5575935 [neighsyncd] increase neighsyncd timeout (#2209)
0f06910 [PBH] Implement Edit Flows (#2169)
6241bbf Remove redundant and problematic code to skip "pool" field in buffer profile handling (#2197)
a55343c [azp]: Set diff coverage threshhold to 80% (#2188)
390cae1 [portsorch]: Prevent LAG member configuration when port has active ACL binding (#2165)
c1d47e6 [VNET]Fixing nexthop group delete during route change (#2198)
8941cc0 [BFD]Registering BFD state change callback during session creation (#2202)
680c539 [vxlan] Remove tunnel map objects on VNET tunnel removal (#2150)
20dde0c Fix for handling broadcom DNX ASIC to have ipv4 and ipv6 ACL rules in separate tables. (#2178)
5b7c949 [FdbOrch] SAI_FDB_EVENT_MOVE generates update with empty update.entry.port_name (#2200)
7350d49 [Vxlanmgr] vnet netdev cleanup during config reload fix (#2191)
2bef62b Validate LAG has members before mirror session create (#2130)
1e4d4ce [VS test] Increase VS test time, skip dpb flaky test (#2195)
6eda965 [vstest]Migrating vs tests from using click commands to direct DB access (#2179)
Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>
Why I did it
Need to run yang validation for sonic-cfggen unit test, and many unit test does not provide speed for port table.
How I did it
Update minigraph xml.
How to verify it
Run sonic-cfggen unit test.
Signed-off-by: Gang Lv ganglv@microsoft.com
Why I did it
Fix#9746
How I did it
Split the check condition based on non-exist and zero length.
How to verify it
Run verification script when table contains empty value
890f32f LLDPLocalSystemDataUpdater Exception Log Handled (#249)
2151731 Handle error seen on system where vlan interface map is not present (#246)
c6141c7 [build] use Azure.sonic-buildimage.official.vs pipeline as artifact source (#248)
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
Why I did it
ASN range is from 1 to 4294967295, need to remove invalid ASN.
How I did it
Update unit test and replace ASN 0.
How to verify it
Run unit test for sonic-config-engine.
Signed-off-by: Gang Lv ganglv@microsoft.com
In order to include the following commit:
c752457 [PBH] Implement Edit Flows (Azure/sonic-utilities#2093)
sonic-utilities
f70dc27 [techsupport] Handle minor fixes of TS Lock and update auto-TS (#2114)
51d3550 Fix issues in clear_qos (#2122)
6d3aa1e [GCU] Optimizing moves by adding generators for keys/tables (#2120)
65a5a6b Fixing get port speed when oper status is down (#2123)
c752457 [PBH] Implement Edit Flows (#2093)
827358f [debug dump] dump interface module added (#2070)
Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>
Why I did it
Need to run yang validation for sonic-cfggen unit test, and many unit test does not provide lanes for port table.
How I did it
Update port config file.
How to verify it
Run sonic-cfggen unit test,
Use below PR to verify
#10228
Signed-off-by: Gang Lv ganglv@microsoft.com
*[Submodule update] sonic-py-swssdk
78f167e4728f939712b3f3ea550949e2ea675fec With the changes in PR:https://github.com/Azure/sonic-buildimage/pull/5289 access to redis unix socket is given to the redis group members. Many of sonic-util commands (especially in multi-asic) case use redis unix socket to connect to DB and thus those comamnd fails without providing sudo. This PR is continuation of PR: https://github.com/Azure/sonic-buildimage/pull/7002 where we default to use TCP for Redis if user is not root
Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
#### Why I did it
To take the following into sonic-buildimage
36e1f61 API endpoints to ConfigDBConnector to support pre-loading data without blackout (#587)
[sonic-swss-common](https://github.com/Azure/sonic-swss-common)
```
36e1f61 API endpoints to ConfigDBConnector to support pre-loading data without blackout (#587)
556f4ad Add SAI copy action to ACL schema (#591)
db18c3a Fix bitwidth of mac address in SAI ACL schema (#590)
e9442bb [ConfigDBPipeConnector]: Added set_entry API (#586)
6c007c0 Update README for gmock (#562)
```
[sonic-linkmgrd](https://github.com/Azure/sonic-linkmgrd)
```
83dfc99 Enforce uniform file/class naming convention (#52)
83004bf Shutdown ICMP heartbeats when default route state is missing and ToR is in `auto` mode (#44)
b81ea31 Add `LinkProberStateMachineActiveActive` (#47)
c43cf7a Decrease link probing interval after switchover to better determine the overhead of a toggle (#43)
2239fbe Enhance clang format (#46)
```
#### How I did it
* Advanced submodules pointers
#### How to verify it
1. make configure PLATFORM=mellanox
2. make target/sonic-mellanox.bin
Why I did it
dhcp_server is introduced, and need to update yang model.
How I did it
Update yang models and add unit test.
How to verify it
Run unit test for sonic-yang-models.
Signed-off-by: Gang Lv ganglv@microsoft.com
1ccda8d MACsec in Gearbox (#993)
7a2b824 Update SAI submodule (#1015)
418d5fb Add pre match logic for lag member (#1008)
Signed-off-by: Myron Sosyak <myronx.sosyak@intel.com>
#### Why I did it
When too many user login concurrently and run commands, SONiC may kernel panic on some device which has very limited memory.
#### How I did it
Add j2 template for setup pam_limit plugin for limit SSH session per-user.
#### How to verify it
Manually validate the j2 template can generate correct config file.
#### Which release branch to backport (provide reason below if selected)
- [x] 201811
- [ ] 201911
- [ ] 202006
- [x] 202012
- [x] 202106
- [x] 202111
#### Description for the changelog
Add j2 template for setup pam_limit plugin for limit SSH session per-user.
#### A picture of a cute animal (not mandatory but encouraged)
Why I did it
sonic-config-engine unit test is using invalid switch_type
How I did it
Update xml with correct switch_type
How to verify it
Run UT for sonic-config-engine
Signed-off-by: Gang Lv ganglv@microsoft.com
Why I did it
To support vlan sub intf short naming format, like subport Eth8.100 for parent portEthernet8 with vlan id 100.
How I did it
Add checks to must condition to verify there is a port or portchannel with the index same as the vlan sub intf.
How to verify it
Run the unttests.
Update sonic-utilities submodule to include below commits:
510a9b9 Add date to show version (#2086)
61b1396 [sfpshow][recycle_port] sfpshow script needs to skip recycle ports (#2109)
4681697 Support command `show ip fib` and `show ipv6 fib` (#2100)
2a982a1 Update `config mirror_session` CLI to support heximal gre type value (#2095)
b25f1e1 [generic-config-updater] Add caclrule validator (#2103)
968900c [sonic-package-manager] do not mod_config for whole config db when setting init_cfg (#2055)
bf55ceb [nvgre] Added auto-generated CLI plugins and test for NVGRE Tunnel feature (#1915)
8389c81 [sonic-cli-gen] fix failure "Error: digits_class" when field "digit_class" does not exist in DB (#2054)
f71ef64 Fix import statement in mclag.py (#2073)
For Bullseye, Python 2 isn't present at all. This means that in certain
build cases (such as building something only for Bullseye), the version
file may not exist, and so the sort command would fail.
For most normal build commands, this probably won't be an issue, because
the SONiC build will start with Buster (which has both Python 2 and
Python 3 wheels built), and so the py2 and py3 files will be present
even during the Bullseye builds.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Update sonic-py-swsssdk submodule to include below commits:
96c0590 [port_util] Fix issue: port_util.get_vlan_interface_oid_map should not raise exception when DB has not RIF data (#117)
96da8b9 [build] pin pytest-azurepipelines version (#119)
Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
#### Why I did it
Fixes https://github.com/Azure/sonic-utilities/issues/2049
from caclmgr:
- PRIORITY is a required field [code](3fa18d18d4/src/sonic-host-services/scripts/caclmgrd (L548))
- PACKET_ACTION is a required field [code](3fa18d18d4/src/sonic-host-services/scripts/caclmgrd (L581))
I think PRIORITY is a required field for ACLs not only CTRLPLANE ACLs
#### How I did it
Check code.
#### How to verify it
Unit-test
#### Which release branch to backport (provide reason below if selected)
<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
- [ ] 202111
#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->
#### Link to config_db schema for YANG module changes
<!--
Provide a link to config_db schema for the table for which YANG model
is defined
Link should point to correct section on https://github.com/Azure/SONiC/wiki/Configuration.
-->
#### A picture of a cute animal (not mandatory but encouraged)
#### Why I did it
To fix https://github.com/Azure/sonic-buildimage/issues/9643
#### How I did it
Instead of ast.literal_eval added python2 compat code for json strings unicode -> str convertion.
We need python2 compatibility since py2 sonic config engine (buster/sonic_config_engine-1.0-py2-none-any.whl target) is still included into the build (ENABLE_PY2_MODULES flag is set for buster). Once we abandon buster and python2, this compat and ast.literal_eval could be cleaned up all through the code base.
#### How to verify it
run steps from the linked issue
Why I did it
support to collect version when purging debian package
Support to collect version multiple times
How I did it
Add the collection action before purging.
Updating sonic-snmpagent submodule with the following commits
dae8146 [ci]: Support code diff coverage
6bd51c4 Fix: LAG counters, if LAG don't have L3 interface
2654f4a Fix snmp agent Initialize config DB multiple times issue
#### Why I did it
When change pycommon to use swsscommon UT failed in sonic-snmpagent, need submodule update with UT issue fix.
#### How I did it
#### How to verify it
#### Which release branch to backport (provide reason below if selected)
#### Description for the changelog
[ci]: Support code diff coverage
Fix: LAG counters, if LAG don't have L3 interface
Fix snmp agent Initialize config DB multiple times issue
#### A picture of a cute animal (not mandatory but encouraged)
Updating sonic-utilities sub module with the following commits
b00b870 [build] stop vstest in the Azure pipeline
2c56e92 [GCU] Marking fields under BGP_PEER_RANGE, BGP_MONITORS as create-only
6289987 Fix sonic-installer failure due to missing import
45e6ac1 [show] add support for hwstatus in show muxcable status
93384ed Try get port operational speed from STATE DB
483fc6e [techsupport] Added a lock to avoid running techsupport in parallel
398da58 Validation check correction while adding a member to PortChannel
a8a7edb [generate_dump] exclude mft and mlx folders from /etc
1cf1d03 Fix UT failed cause by change pycommon to use swsscommon
#### Why I did it
PR https://github.com/Azure/sonic-utilities/pull/1825 added validation for the input of `config mirror session add`, and only decimal value is accepted.
An issue https://github.com/Azure/sonic-buildimage/issues/10096 was raised to suggest accepting HEX value as well, and the suggestion makes sense to me.
To accept HEX value for GRE type, and keep backward compatibility as well, I updated the YANG model to support both decimal and hexadecimal input for GRE type.
#### How I did it
Update the regex for GRE type.
#### How to verify it
Verified by UT
```
platform linux -- Python 3.9.2, pytest-6.0.2, py-1.10.0, pluggy-0.13.0
rootdir: /sonic/src/sonic-yang-models
plugins: pyfakefs-4.5.4, cov-2.10.1
collected 3 items
tests/test_sonic_yang_models.py .. [ 66%]
tests/yang_model_tests/test_yang_model.py . [100%]
========================================================================================== 3 passed in 2.53s ==========================================================================================
```
#### Description for the changelog
Update YANG model for mirror session to support decimal value for GRE type.
f00efef Longxiang Lyu Wed Mar 16 09:12:46 2022 +0800 Add a command line option to store logs into a separate file (#41)
ff2e67d Longxiang Lyu Tue Mar 15 09:10:59 2022 +0800 Add default port cable type (#39)
ebbb4d8 Jing Zhang Mon Mar 14 15:41:11 2022 -0700 Prevent switching MUX to "Unknown" (#36)
c779b8f Longxiang Lyu Thu Mar 10 21:35:11 2022 +0800 [nonfunctional] Use LinkProberStateMachineBase (#38)
b9fedd0 Longxiang Lyu Wed Mar 9 13:03:58 2022 +0800 [NONFUNCTIONAL] Add LinkProberStateMachineBase (#37)
bedd42b Longxiang Lyu Wed Mar 9 10:03:00 2022 +0800 Add .clang-format file to format code (#28)
9fe4fc6 Guohan Lu Thu Mar 3 17:51:43 2022 -0800 [doc]: add lgtm badge in README.md
c1249d9 Longxiang Lyu Wed Mar 2 18:05:18 2022 +0800 Enable lgtm (#33)
b8514c6 Longxiang Lyu Wed Mar 2 13:34:39 2022 +0800 Collect port cable type to use corresponding state machine (#31)
9b59ef9 Longxiang Lyu Wed Mar 2 07:19:33 2022 +0800 Improve make clean (#32)
#### Why I did it
Fixing issue https://github.com/Azure/sonic-buildimage/issues/9991
The ACL RULE table field ETHER_TYPE can accept both hex as well as decimal values. However yang model didn't allow decimal values. Fixed it to allow decimal values (same pattern as in hex (1536-65535)
#### How I did it
Updated yang model to handle decimal values
#### How to verify it
Added UT to verify it.
#### Why I did it
```
Update submodule sonic-restapi
bd97dfe Fix urllib3 CVE-2021-33503 issue (#104)
f159bfa Upgrade the containers to be based on Debian Buster (#103)
a1830c1 (origin/201911) Fix OpenAPI spec to be readable by autorest (#101)
```
#### Why I did it
Moved Configuration.md from swss to yang folder. The configuration schema should be present along with yang models and when users add yang model they could update configuration document as well.
Update sonic-swss with the following changes:
29d5d8da Use abort instead of exit in case calling SAI API failure (#2170)
12f980c8 Fix issue config qos reload causing orchagent aborted via tracking dependencies among QoS tables (#2116)
6e5ed1c0 [chassis][syncd][sai] Adjusting response timeout during syncd init (#2159)
0a99f546 Try get port operational speed from STATE DB (#2119)
828cccfe [crm] Use sai_object_type_get_availability() API to get counters (#2098)
18c73a19 Allow IPv4 link-local nexthops (#1903)
Signed-off-by: Stephen Sun <stephens@nvidia.com>
- Why I did it
To implement blocking feature state change.
- How I did it
Record the actual feature state in STATE DB from hostcfg.
- How to verify it
UT + verification by running on the switch and checking STATE DB.
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
e56e9b4 Fix CVE-2021-3121 warning (#96)
bf1be4f [ci]: Support code diff coverage threshold 50% (#94)
64e516c Ported Marvell armhf build on x86 for debian buster to use cross-compilation instead of qemu emulation (#80)
e426388 [ci]: Support azp code coverage (#87)
* fix allow list issue
Signed-off-by: stormliang <stormliang@microsoft.com>
* add the ipaddress in the install list
* add unit test
Co-authored-by: Ubuntu <azureuser@SONIC-SH-STORM-02.5pu3m0fajw1edcfltykk1gauxa.gx.internal.cloudapp.net>
Why I did it
Failed to remove part of configuration of bgp allowed prefix list. The details in #10141
How I did it
There are two issues:
In FRR, ipv6 default route is ::/0, but in the configuration, it is 0::/0, string comparison would be false, but why ipv4 failed to remove the allowed prefix list, ipv6 works? Looks into next one for the answer.
The current managers_allow_list doesn’t support removal part of the prefix list. But why IPv6 works in 1? It is because the bug for the IPv6 default route comparison, it would do the update no matter what is the operation (the code will compare the prefix list in the FRR and configuration db, if all configurations in db are presented in FRR, it do nothing, otherwise it will update the prefix list based on the configuration from db).
How to verify it
Follow the step in #10141
47c243e [show][muxcable] fix the sudo access error for show muxcable metrics (#2083)
f872516 [muxcable][show] enhance show mux status to show last switchover time (#2067)
d440df7 [warmboot] Migrate 10G ports during warm-reboot on s6100 (#2064)
494c6d7 [counterpoll] Display the correct default poll interval for watermark counters (#2082)
499988e [show][config] add muxcable command line support for retrieve / reset ICMP packet loss data (#2046)
8b01d3e Remove the warning message appear when there are no ports on CONFIG DB (#2050)
ed6e66e [GCU] Supporting Groupings during path-xpath translation (#2044)
25b3455 [ci] Use official build debian pkg instead and parameterize source branch (#2079)
Why I did it
Config db schema generated by minigraph can’t pass yang validation, deployment_id can’t be none for yang validation.
How I did it
Update minigraph.py, skip deployment_id with None value
How to verify it
Run UT for sonic-config-enginue.
Run command 'sonic-cfggen -m tests/multi_npu_data/sample-minigraph-noportchannel.xml -p tests/multi_npu_data/sample_port_config-3.ini -n asic3 --print-data'.
Signed-off-by: Gang Lv ganglv@microsoft.com
Why I did it
Desired the log message destination to be syslog and it misses the critical info.
How I did it
Non logical code changes only.
Logging update, just for one message only
a) The log message is directed to /var/log/syslog, instead of /var/log/auth.log
b) Include user alias in the message
How to verify it
Pick a user alias that has not logged into the switch yet
Add this alias to /etc/tacplus_user
Attempt to login as that user
Look for the error message in /var/log/syslog
e.g. "Feb 18 19:16:41.592191 sonic ERR sshd[5233]: auth fail: Password incorrect. user: user_xyz"
Why I did it
Smartmontools 6.6 has an issue with reading SMART info of nvme SSD
Smartmontools can be installed with apt-get, no need to build and install
How I did it
Use apt-get to install smartmontools 7.2-1
Remove previous make files for smartmontools 6.6
How to verify it
verify with "smartctl" can read out correct SMART info on NVME ssd.
verify "show platform ssdhealth" can still work
Signed-off-by: Kebo Liu <kebol@nvidia.com>
Why I did it
Multi-asic platform add aisc_port_name and role to PORT table, and port_index range is changed.
How I did it
Update sonic-port.yang, add asic_port_name and role, and remove range limitation.
How to verify it
Run UT for sonic-yang-models.
Signed-off-by: Gang Lv ganglv@microsoft.com
Why I did it
ACL doesn't have mirror related action
How I did it
Add 'MIRROR_INGRESS_ACTION' and 'MIRROR_EGRESS_ACTION' to sonic-acl.yang.j2
How to verify it
Run the YANG model unit tests
Signed-off-by: Gang Lv ganglv@microsoft.com
<!--
Please make sure you've read and understood our contributing guidelines:
https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md
** Make sure all your commits include a signature generated with `git commit -s` **
If this is a bug fix, make sure your description includes "fixes #xxxx", or
"closes #xxxx" or "resolves #xxxx"
Please provide the following information:
-->
#### Why I did it
end2end test is blocked by Yang model for AAA login pattern.
#### How I did it
Add pattern to AAA yang models.
#### How to verify it
Run UT for sonc-yang-models.
#### Which release branch to backport (provide reason below if selected)
<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->
Fix#9713
#### A picture of a cute animal (not mandatory but encouraged)
ce72b0d Longxiang Lyu Thu Feb 24 06:05:12 2022 Put handler member functions as virtual in base (#30)
ef59e4f Jing Zhang Fri Feb 25 11:38:28 2022 Incrementing tolerance on mux state inconsistency (#27)
2d12892 Longxiang Lyu Wed Feb 16 03:32:06 2022 Rename LinkManagerStateMachine to ActiveStandbyStateMachine (#26)
f38634c Jing Zhang Thu Feb 17 17:23:56 2022 Update log level for mux probing and mux state chance (#23)
a8434dd Jing Zhang Thu Feb 17 17:21:01 2022 Handle xcvrd crashing scenarios (#22)
2ebdb2b Longxiang Lyu Mon Feb 14 13:26:07 2022 [make] Enable make extra includes (#24)
- Why I did it
NVGRE Tunnel feature extends the Config DB with new tables. These tables require a new YANG model.
- How I did it
Added a new YANG model sonic-nvgre-tunnel.yang
- How to verify it
Added YANG test cases.
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
Why I did it
During warm-reboot and fast-reboot the below error logs appear
Feb 3 22:05:15.187408 r-lionfish-13 ERR container: docker cmd: kill for nat failed with 404 Client Error for http+docker://localhost/v1.41/containers/nat/json: Not Found ("No such container: nat")
The container command when called for local mode doesn't check if it is enabled before calling docker kill which throws the above errors.
b6ca76b482/scripts/fast-reboot (L699)
How I did it
Checking feature state if local mode and returning error exit code along with valid debug message.
How to verify it
Manually tested with warm-reboot and fast-reboot
Added UT to verify it.
Why I did it
amrhf build fails while building sonic-config-engine whl package
https://dev.azure.com/mssonic/be1b070f-be15-4154-aade-b1d3bfb17054/_apis/build/builds/77089/logs/9
The reason for the failure is due to the fact that there is a new line generated at the top of the file in buffer config test cases while building for broadcom based platform and this issue is not seen in Marvell based platforms.
How I did it
Removed the new line for all the buffer test cases as there is no need to add it and accordingly changed the buffer_config.j2 where the new line is generated.
This change is adding asic_name, switch_id, switch_type and max_cores to sonic-device_metadata.yang
This should fix issue #9575
Co-authored-by: Maxime Lorrillere <mlorrillere@arista.com>
Why I did it
end2end test is blocked by Yang model for BGP_PEER_RANGE.
How I did it
Add new yang models.
How to verify it
Run UT for sonc-yang-models.
Signed-off-by: Gang Lv ganglv@microsoft.com
Why I did it
In the recent minigraph changes we add separate BGP session configuration for V4 and V6 internal VoQ neighbors.
This PR is adding different Peer groups for V4 and V6 neighbors
How I did it
Add VOQ_CHASSIS_V4_PEER and VOQ_CHASSIS_V6_PEER groups
Add extra Unit tests
How to verify it
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
- Why I did it
swsscommon.ConfigDBConnector does not automatically close connection when the instance is recycled by python. So, it should not create this instance each time calling check_services. It will cause error like Failed to read from file /var/run/hw-management/led/led_status_capability - OSError(24, 'Too many open files')
- How I did it
Only connect DB once in init
- How to verify it
Manual test
Signed-off-by: Ze Gan <ganze718@gmail.com>
b9337dc (HEAD, origin/master, origin/HEAD) [vslib]: Fix MACsec bug in SCI and XPN (#1003)
edbceb9 [syncd][vslib] Keep new warm boot discovered SERDES objects (#985)
af5c156 Fix build issues on gcc-10 (#999)
1445cd5 update SAI submoule (#1001)
48fe704 [ci] pipeline fixes for VS test (#1002)
f484cf9 Enable SAI_SWITCH_ATTR_UNINIT_DATA_PLANE_ON_REMOVAL attribute (#975)
5d0b22d Enable SAI_SWITCH_ATTR_UNINIT_DATA_PLANE_ON_REMOVAL attribute (#975)
1b8ce97 (origin/202111) [pipeline] Download swss common artifact in a separated directory (#995)
7a2e096 Change sonic-buildimage.vs artifact source from CI build to official build. (#992)
* Update sonic-swss
Signed-off-by: Ze Gan <ganze718@gmail.com>
* update swss
Signed-off-by: Ze Gan <ganze718@gmail.com>
* Update submodule update-swss
Signed-off-by: Ze Gan <ganze718@gmail.com>
```
Commits on Jan 31, 2022
[sonic-package-manager] implement blocking feature state change (#2035)
Commits on Feb 12, 2022
[ci] pipeline fixes for VS test (#2059)
Commits on Feb 14, 2022
[ci] allow download partially succeeded suilds for sonic-swss-pytests…
[build] allowPartiallySucceededBuilds true in the step of download so…
Commits on Feb 16, 2022
[config] Fix snmpagentaddress doesn't support the uppercase letters f…
Commits on Feb 17, 2022
[ci] Fix pipeline on build_and_install_module (#2074)
Enable diff coverage and set the threshold to 50% (#2061)
Commits on Feb 18, 2022
check for adding default vlan attempt added (#2075)
[GCU] Adding unit-test where path and ref paths are under the same YA…
[GCU] Turning port admin down before some critical port changes (#1998)
Commits on Feb 20, 2022
show_platfom_info not run for simx (#2042)
[aclshow] fix aclshow when clear is called before counters are popula…
```
#### Why I did it
1. Fix Build exception [example](https://dev.azure.com/mssonic/build/_build/results?buildId=73911&view=logs&jobId=88ce9a53-729c-5fa9-7b6e-3d98f2488e3f&j=cef3d8a9-152e-5193-620b-567dc18af272&t=ac3bce9f-b126-5a26-3fee-28ce0ec1679d)
```
2022-02-19T01:54:23.4200556Z ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.8/dist-packages/markupsafe/__init__.py)
```
This is because Jinja2 uses MarkupSafe without specifying an upper limit to the version, MarkupSafe version that was released today removed 'soft_unicode'. So now Jinja2 is complaining.
Related issues:
https://github.com/pallets/jinja/issues/1591https://github.com/aws/aws-sam-cli/issues/3661
2. Reverts #9136
Fixing build failures in SONiC utils [example](https://dev.azure.com/mssonic/build/_build/results?buildId=73784&view=logs&jobId=83516c17-6666-5250-abde-63983ce72a49&j=83516c17-6666-5250-abde-63983ce72a49&t=6177235f-d4f1-5f72-835a-90ebb93a1784)
One of the errors:
```
TestPathAddressing.test_find_ref_paths__ref_is_the_whole_key__returns_ref_paths
self = <tests.generic_config_updater.gu_common_test.TestPathAddressing testMethod=test_find_ref_paths__ref_is_the_whole_key__returns_ref_paths>
def test_find_ref_paths__ref_is_the_whole_key__returns_ref_paths(self):
# Arrange
path = "/PORT/Ethernet0"
expected = [
"/ACL_TABLE/NO-NSW-PACL-V4/ports/0",
"/VLAN_MEMBER/Vlan1000|Ethernet0",
]
# Act
actual = self.path_addressing.find_ref_paths(path, Files.CROPPED_CONFIG_DB_AS_JSON)
# Assert
> self.assertEqual(expected, actual)
E AssertionError: Lists differ: ['/ACL_TABLE/NO-NSW-PACL-V4/ports/0', '/VLAN_MEMBER/Vlan1000|Ethernet0'] != ['/ACL_TABLE/NO-NSW-PACL-V4/ports/0']
E
E First list contains 1 additional elements.
E First extra element 1:
E '/VLAN_MEMBER/Vlan1000|Ethernet0'
E
E - ['/ACL_TABLE/NO-NSW-PACL-V4/ports/0', '/VLAN_MEMBER/Vlan1000|Ethernet0']
E + ['/ACL_TABLE/NO-NSW-PACL-V4/ports/0']
```
The VLAN_MEMBER backlink (can be called referrer link or ref link) is not found.
Issue introduced by https://github.com/Azure/sonic-buildimage/pull/9136
I don't know how this PR passed the build system, it should have failed.
Known YANG issue https://github.com/Azure/sonic-buildimage/issues/9312
#### How I did it
The import to `sonic-vlan` is breaking the build
```
import sonic-vlan {
prefix vlan;
}
```
I am not sure if that's the only issue, so I think reverting the whole PR should be the safer option.
#### How to verify it
Ran sonic-utils tests locally.
sonic-swss-common:
[ci] Set diff coverage threshold to 50% (#567)
Added NVGRE tunnel related tables (#549)
Add option to suppress logging (#566)
[TableBase] Make channel name from both table name and database ID (#568)
Revert "[TableBase] Make channel name from both table name and database ID (#568)" (#574)
sonic-py-swsssdk:
Add option to suppress warnings when querying Redis (#115)
- Why I did it
NVGRE Tunnel feature extends the Config DB with new tables. These tables require a new YANG model.
- How I did it
Added a new YANG model sonic-nvgre-tunnel.yang
- How to verify it
Added YANG test cases.
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
Why I did it
Radvd.conf.j2 template creates two copies of the vlan interface when there are more than one ipv6 address assigned to a single vlan interface. Changed the format to add prefixes under the same vlan interface block.
How I did it
Modifies radvd.conf.j2 and added unit tests
How to verify it
Configure multiple ipv6 address to the same vlan, start radvd
Unit test will check if radvd.conf with multiple ipv6 addresses is formed correctly
#### Why I did it
To bump the Thrift version to 0.14.1
- To avoid [CVE-2020-13949](https://nvd.nist.gov/vuln/detail/CVE-2020-13949)
- to fix some dependencies issues
#### How I did it
- rename `src/thrfit_0_13_0` to `src/thrift_2` to remove version number in the path. (`src/thrift` contains rules to build thrift 0.11.0 )
- Add thrift sources as submodule as there are no prepared debian packages for version >0.13.0 on [debian.org](https://packages.debian.org/search?searchon=sourcenames&keywords=thrift)
- Added patches with fixes for original thrift debian rules:(remove unneeded packages, fix multi job build)
#### How to verify it
```
BLDENV=buster make -f Makefile.work target/debs/buster/libthrift-dev_0.14.1_amd64.deb
```
Updates include the following changes in order to support new Mellanox platforms and drivers (Azure/sonic-linux-kernel#259)
10ef390 Update kconfig to support / enable newly backported mellanox patches.
6a949e1 Add backported patches for Mellanox hw-mgmt V.7.0020.1300
e1913f7 Rename and reformat patch headers
#### Why I did it
Fixing the issue https://github.com/Azure/sonic-buildimage/issues/9915
#### How I did it
Added 'not-provisioned' as a supported value for type field in DEVICE_METADATA type. This value is set during initial ZTP bring up
#### How to verify it
Added UT to verify it.
Update submodule sonic-utilities that contains the following commits:
[build] allowPartiallySucceededBuilds: true (#2043)
[system-health] Remove booting stage in system health service (#2022)
[GCU] Mark children of bgp_neighbor as create-only (#2008)
[generic_config_updater] Minor update - No logical code change (#2028)
[generic-config-updater] Handle failed service restarts (#2020)
[debug dump] Missing Dict Key handled in the MatchOptimizer (#2014)
[Auto Techsupport] Added Event Driven TS to Command Reference (#1985)
#### Why I did it
PR9611 - sonic-scheduler.yang pattern issue
#### How I did it
Modified the scheduler name pattern string to accept any string
#### How to verify it
Sonic yang tests
