This change is adding asic_name, switch_id, switch_type and max_cores to sonic-device_metadata.yang
This should fix issue #9575
Co-authored-by: Maxime Lorrillere <mlorrillere@arista.com>
Why I did it
end2end test is blocked by Yang model for BGP_PEER_RANGE.
How I did it
Add new yang models.
How to verify it
Run UT for sonc-yang-models.
Signed-off-by: Gang Lv ganglv@microsoft.com
#### Why I did it
1. Fix Build exception [example](https://dev.azure.com/mssonic/build/_build/results?buildId=73911&view=logs&jobId=88ce9a53-729c-5fa9-7b6e-3d98f2488e3f&j=cef3d8a9-152e-5193-620b-567dc18af272&t=ac3bce9f-b126-5a26-3fee-28ce0ec1679d)
```
2022-02-19T01:54:23.4200556Z ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.8/dist-packages/markupsafe/__init__.py)
```
This is because Jinja2 uses MarkupSafe without specifying an upper limit to the version, MarkupSafe version that was released today removed 'soft_unicode'. So now Jinja2 is complaining.
Related issues:
https://github.com/pallets/jinja/issues/1591https://github.com/aws/aws-sam-cli/issues/3661
2. Reverts #9136
Fixing build failures in SONiC utils [example](https://dev.azure.com/mssonic/build/_build/results?buildId=73784&view=logs&jobId=83516c17-6666-5250-abde-63983ce72a49&j=83516c17-6666-5250-abde-63983ce72a49&t=6177235f-d4f1-5f72-835a-90ebb93a1784)
One of the errors:
```
TestPathAddressing.test_find_ref_paths__ref_is_the_whole_key__returns_ref_paths
self = <tests.generic_config_updater.gu_common_test.TestPathAddressing testMethod=test_find_ref_paths__ref_is_the_whole_key__returns_ref_paths>
def test_find_ref_paths__ref_is_the_whole_key__returns_ref_paths(self):
# Arrange
path = "/PORT/Ethernet0"
expected = [
"/ACL_TABLE/NO-NSW-PACL-V4/ports/0",
"/VLAN_MEMBER/Vlan1000|Ethernet0",
]
# Act
actual = self.path_addressing.find_ref_paths(path, Files.CROPPED_CONFIG_DB_AS_JSON)
# Assert
> self.assertEqual(expected, actual)
E AssertionError: Lists differ: ['/ACL_TABLE/NO-NSW-PACL-V4/ports/0', '/VLAN_MEMBER/Vlan1000|Ethernet0'] != ['/ACL_TABLE/NO-NSW-PACL-V4/ports/0']
E
E First list contains 1 additional elements.
E First extra element 1:
E '/VLAN_MEMBER/Vlan1000|Ethernet0'
E
E - ['/ACL_TABLE/NO-NSW-PACL-V4/ports/0', '/VLAN_MEMBER/Vlan1000|Ethernet0']
E + ['/ACL_TABLE/NO-NSW-PACL-V4/ports/0']
```
The VLAN_MEMBER backlink (can be called referrer link or ref link) is not found.
Issue introduced by https://github.com/Azure/sonic-buildimage/pull/9136
I don't know how this PR passed the build system, it should have failed.
Known YANG issue https://github.com/Azure/sonic-buildimage/issues/9312
#### How I did it
The import to `sonic-vlan` is breaking the build
```
import sonic-vlan {
prefix vlan;
}
```
I am not sure if that's the only issue, so I think reverting the whole PR should be the safer option.
#### How to verify it
Ran sonic-utils tests locally.
- Why I did it
NVGRE Tunnel feature extends the Config DB with new tables. These tables require a new YANG model.
- How I did it
Added a new YANG model sonic-nvgre-tunnel.yang
- How to verify it
Added YANG test cases.
Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
#### Why I did it
Fixing the issue https://github.com/Azure/sonic-buildimage/issues/9915
#### How I did it
Added 'not-provisioned' as a supported value for type field in DEVICE_METADATA type. This value is set during initial ZTP bring up
#### How to verify it
Added UT to verify it.
#### Why I did it
PR9611 - sonic-scheduler.yang pattern issue
#### How I did it
Modified the scheduler name pattern string to accept any string
#### How to verify it
Sonic yang tests
#### Why I did it
Include sonic-bgp-monitor to setup.py so it gets included in /usr/local/yang-models when installing the package
#### How I did it
#### How to verify it
install the package
#### Which release branch to backport (provide reason below if selected)
<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->
#### A picture of a cute animal (not mandatory but encouraged)
#### How I did it
Added mclag sonic yang file for the MCLAG enhancements as per HLD: Azure/SONiC#596
#### How to verify it
try rest APIs
#### Description for the changelog
Added mclag sonic yang
Why I did it
Minigraph parser added a new field 'cluster' to device_metadata, and then yang validation is blocked.
How I did it
Add 'cluster' to device_metadata yang models.
How to verify it
Run UT for sonc-yang-models.
Use minigraph parser to generate ConfigDB schema and run yang validation.
Signed-off-by: Gang Lv ganglv@microsoft.com
Why I did it
ACL have ACCEPT action indeed, but yang doesn't support it.
How I did it
Add 'ACCEPT' enum to sonic-types.yang.j2
How to verify it
Run the YANG model unit tests
Why I did it
end2end test is blocked by Yang model for BGP monitor.
How I did it
Create new yang files for BGP monitor, and add UT.
How to verify it
Follow the steps in #9711.
Run UT for sonic-yang-models.
Signed-off-by: Gang Lv ganglv@microsoft.com
#### Why I did it
AAA yang model is not up to date.
#### How I did it
Add fallback and trace field, and replace boolean_type
#### How to verify it
Run UT for sonic_yang_models.
Follow the steps from #9710
Fixes#9561Fixes#9570Fixes#9563
Partial fix for #9556
#### Why I did it
- Attributes for dual ToR configs lack YANG model support
#### How I did it
- Extend YANG tests to cover dual ToR use cases
- Extend YANG model to cover dual ToR use cases
- Reduce the default log level to warning so only test failures are printed
#### How to verify it
- Run the YANG model unit tests
Why I did it
Add YANG model file for table VLAN_SUB_INTERFACE
How I did it
Add YANG model file sonic-vlan-sub-interface.yang to describe data structure
modify existing unit-test to cover vlan sub interface
How to verify it
Build sonic-yang-models and sonic-yang-mgmt without errors
Why I did it
Config db schema generated by minigraph can’t pass yang validation, and there's no 'alias' field in yang model.
Minigraph parser supports 'alias' field for VLAN.
How I did it
Add 'alias' field to sonic-vlan.yang
How to verify it
Build sonic-yang-models.
Run command 'sonic-cfggen -m xxx.xml --print-data', and run yang validation.
Signed-off-by: Gang Lv ganglv@microsoft.com
- Why I did it
The feature state can be a jinja template, like in this file - https://github.com/Azure/sonic-buildimage/blob/master/files/build_templates/init_cfg.json.j2#L39.
Without this change it is not possible to validate a configuration file.
- How I did it
Relaxes the constraint on feature state. Feature state leaf can be any string.
- How to verify it
Run UT.
Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
#### Why I did it
POLL_INTERVAL cannot be set if any of the detection/restoration times in this table is less than the POLL_INTERVAL.
#### How I did it
Add "must" constraint to make sure detection/restoration times are greater than POLL_INTERVAL.
#### How to verify it
Use apply-patch command to update POLL_INTERVAL.
Build sonic-yang-model.
#### Why I did it
Sonic yang model for BUM storm control
#### How I did it
Added yang model and the corresponding test cases.
#### How to verify it
yang model test case for storm control
#### Why I did it
Fix issue https://github.com/Azure/sonic-utilities/issues/1962
The problem is current implementation of [sonic-yang-mgmt::find_data_dependencies](f2774b635d/src/sonic-yang-mgmt/sonic_yang.py (L518)) does not get referrers if they are using `must` statement, it has to use `leafref`.
For now we can convert `must` to `leafref` if possible. In the future we will investigate get referrers by `must` statements as well https://github.com/Azure/sonic-buildimage/issues/9534
#### How I did it
Instead of `must` use `leafref`
#### How to verify it
unit-test
#### Which release branch to backport (provide reason below if selected)
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
Fixes#9326
#### Why I did it
When we try execute DPB from CLI we have error:
`libyang[0]: Invalid value "False" in "has_global_scope" element. (path: /sonic-feature:sonic-feature/FEATURE/FEATURE_LIST[name='bgp']/has_global_scope)`
The reason for this issue is that has_global_scope and other have been stored in redis database with value False or True form capital letter:
```
"FEATURE":{
"bgp":{
"auto_restart":"enabled",
"has_global_scope":"False",
"has_per_asic_scope":"True",
"has_timer":"False",
"high_mem_alert":"disabled",
"state":"enabled"
}
```
But yang model support boolean just in lowercase letters (https://datatracker.ietf.org/doc/html/rfc6020#section-9.5.1).
#### How I did it
Added boolean to sonic-types as typedef with different literal cases.
#### How to verify it
Run the command config interface breakout <breakout_mode>
**NOTE:**
To verify this fix, the following PRs that fix other problems in SONiC must be merged into master:
1) Azure/sonic-buildimage/pull/9075
2) Azure/sonic-buildimage/pull/9276
Signed-off-by: Neetha John <nejo@microsoft.com>
Bring back the changes in #9226 that were reverted. Unable to do a revert-revert.
Why I did it
Few device types were missing in the DEVICE_METADATA type field
How I did it
Added missing device types to the device metadata yang
#### Why I did it
Fixing issue #9294
#### How I did it
Updating ACL yang model
#### How to verify it
Validating issue with `config patch-apply` is fixed.
- Start a KVM
- Add file `add-ctrl-plane-tbl.json-patch ` with content:
```json
[
{
"op": "add",
"path": "/ACL_TABLE/ACTRLPLANETABLE",
"value": {
"policy_desc": "ACTRLPLANETABLE",
"services": [
"SSH"
],
"stage": "ingress",
"type": "CTRLPLANE"
}
}
]
```
- Run `sudo config apply-patch add-ctrl-plane-tbl.json-patch`
Before:
```
Patch Applier: The patch was sorted into 4 changes:
Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/ACTRLPLANETABLE", "value": {"type": "CTRLPLANE"}}]
Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/ACTRLPLANETABLE/policy_desc", "value": "ACTRLPLANETABLE"}]
Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/ACTRLPLANETABLE/services", "value": ["SSH"]}]
Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/ACTRLPLANETABLE/stage", "value": "ingress"}]
```
After:
```
Patch Applier: The patch was sorted into 1 change:
Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/ACTRLPLANETABLE", "value": {"policy_desc": "ACTRLPLANETABLE", "services": ["SSH"], "stage": "ingress", "type": "CTRLPLANE"}}]
```
#### Which release branch to backport (provide reason below if selected)
<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->
#### A picture of a cute animal (not mandatory but encouraged)
#### Why I did it
Add the configuration for the set_owner in the `feature` yang model
#### How I did it
Add new leaf `set_pwner` to the `feature` yang model
#### How to verify it
compile `sonic_yang_mgmt-1.0-py3-none-any.whl`
#### Why I did it
Currently only IP ACL and related model is defined. Support for MAC ACL is missing. Added support for it.
#### How I did it
ACL_RULE table is added with new MAC ACL related fields namely Source MAC, Destination MAC, Ethertype (Pattern updated to match any valid Ethertypes), VLAN, PCP, DEI
#### How to verify it
Yang model tests are attached.
Why I did it
Add YANG model support for table CABLE_LENGTH
How I did it
Add the YANG model file
Add the test description file and config file
add list CABLE_LENGTH_LIST to the qos_maps_model list in sonic-yang-ext, as it has an inner list.
How to verify it
Build sonic-yang-model and sonic-yang-mgmt
Why I did it
#9122
DEVICE_METADATA does not have cloudtype and region.
How I did it
Add cloudtype and region to DEVICE_METADATA.
How to verify it
Follow the steps in #9122.
Build sonic-yang-model.
Signed-off-by: Gang Lv ganglv@microsoft.com
#### Why I did it
DPB falls due to missing POLL_INTERVAL in sonic-flex_counter yang model.
#### How I did it
Added POLL_INTERVAL leaf to ACL container in sonic-flex_counter yang model.
#### How to verify it
Run the command config interface breakout <interface> <breakout_mode>
**NOTE:**
To verify this fix, a PR ([add set_owner to feature yang](https://github.com/Azure/sonic-buildimage/pull/9075)) that fix another bug in SONiC should be merged to master.
Why I did it
Add yang model for syslog server
How I did it
Add new file sonic-syslog.yang and new files for tests
How to verify it
Compile target/python-wheels/sonic_yang_mgmt-1.0-py3-none-any.whl
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan arlakshm@microsoft.com
#### Why I did it
Created SONiC Yang model for Mirror.
Tables: MIRROR_SESSION
#### How I did it
Defined Yang models for COPP based on Guideline doc:
https://github.com/Azure/SONiC/blob/master/doc/mgmt/SONiC_YANG_Model_Guidelines.md
and
https://github.com/Azure/sonic-utilities/blob/master/doc/Command-Reference.md
#### How to verify it
'''
============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-3.10.1, py-1.7.0, pluggy-0.8.0
rootdir: /sonic/src/sonic-yang-models, inifile:
plugins: cov-2.6.0
collected 3 items
tests/test_sonic_yang_models.py .. [ 66%]
tests/yang_model_tests/test_yang_model.py . [100%]
=============================== warnings summary ===============================
module: sonic-mirror-session
+--rw sonic-mirror-session
+--rw MIRROR_SESSION
+--rw MIRROR_SESSION_LIST* [name]
+--rw name string
+--rw type? string
+--rw src_ip? inet:ipv4-address
+--rw dst_ip? inet:ipv4-address
+--rw gre_type? string
+--rw dscp? uint8
+--rw ttl? uint8
+--rw queue? uint8
+--rw dst_port? -> /port:sonic-port/PORT/PORT_LIST/name
+--rw src_port? union
+--rw direction? string
'''
* [sonic-acl] add ACL_TABLE_TYPE table
I implemented ACL table type concept. Till this change, there are predefined ACL table types orchagent knows about (L3, L3V6, etc.) and if other orch requires a custom table a new table type needs to be defined in aclorch.
This PR addresses this limitation by introducing AclTableType which can be constructed from a set of matches, actions and bpoint types user needs. There is also a new handler for ACL_TABLE_TYPE table which is used for user to define table types.
Currently, some of built-in ACL table types that requires special handling are distinguished from others by their names (TABLE_TYPE_MIRROR, TABLE_TYPE_MIRRORV6) and a special handling is performed by an AclOrch.
Signed-off-by: Stepan Blyshchak <stepanb@nvidia.com>
#### Why I did it
Changes required for feature "Event Driven TechSupport Invocation & CoreDump Mgmt". [HLD](https://github.com/Azure/SONiC/pull/818 )
Requires: https://github.com/Azure/sonic-utilities/pull/1796.
Merging in any order would be fine.
Summary of the changes:
- Added the YANG Models for the new tables introduces as a part of this feature.
- Enhanced init_cfg.json with the default config required
- Added a compile Time flag which enables/disables the config required for this feature inside the init_cfg.json
- Enhanced the supervisor-proc-exit-listener script to populate `<feature>:<critical_proc> = <comm>:<pid>` info in the STATE_DB when it observes an proc exit notification for the critical processes running inside the docker.
Why I did it
To add ACL FC to the YANG model.
How I did it
Added to the YANG model and added a UT.
How to verify it
Together with depends PRs. Run ACL/Everflow test suite.
Signed-off-by: Stepan Blyshchak <stepanb@nvidia.com>
Why I did it
Added Buffer yang files derived from sonic/mgmt-cvl/testdata/schema from sonic-mgmt-common, used by mgmt-framework.
Updated BUFFER_PG|({ifname},)*|{pg_num} to BUFFER_PG|{ifname}|{pg_num} in sonic-buffer-pg.yang.
This change is required for configuration migration for dynamic port breakout operation.
Added sonic-buffer-queue.yang for BUFFER_QUEUE
Tables: BUFFER_POOL, BUFFER_PROFILE, BUFFER_PG, BUFFER_QUEUE.
How I did it
Defined Yang models for BUFFER tables based on Guideline doc:
https://github.com/Azure/SONiC/blob/master/doc/mgmt/SONiC_YANG_Model_Guidelines.md
and
https://github.com/Azure/sonic-utilities/blob/master/doc/Command-Reference.md
How to verify it
sonic_yang_models package build.
Description for the changelog
MMU Buffer yang files (BUFFER_POOL/BUFFER_PROFILE/BUFFER_PG/BUFFER_QUEUE)
[TACACS+] Add Config DB schema and HostCfg Enforcer plugin to support TACACS+ per-command authorization&accounting. (#9029)
#### Why I did it
Support TACACS per-command authorization&accounting.
#### How I did it
Change ConfigDB schema and HostCfg enforcer.
Add UT to cover changed code.
#### How to verify it
Build following project and pass all UTs:
make target/python-wheels/sonic_host_services-1.0-py3-none-any.whl
#### Which release branch to backport (provide reason below if selected)
N/A
#### Description for the changelog
Add Config DB schema and HostCfg Enforcer plugin to support TACACS+ per-command authorization&accounting.
#### A picture of a cute animal (not mandatory but encouraged)
Why I did it
Add yang model for Feature configuration
How I did it
Add feature.yang and unit tests
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
Changes:
-- pre Process Grouping section from all yang models, so it can be used
from any yang model.
-- add jsondiff in setup.py, it is useful for test debugging in case of
failures.
-- use 'stypes' instead of head.
-- pass config DB table name in _createLeafDict().
-- added test config for grouping.
-- white spaces changes.
Note: Changes are done in the way that we can add support for other
Generic YANG statement easily for translation.
Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com
Why I did it
SONiC YANG model support for warm restart.
How I did it
Defined warm restart YANG containers and lists based on config-DB schema.
How to verify it
Successful build of the following packages:
make target/python-wheels/sonic_yang_models-1.0-py3-none-any.whl
make target/python-wheels/sonic_yang_mgmt-1.0-py3-none-any.whl
Signed-off-by: Arthi Sivanantham <arthi_sivanantham@dell.com>