#### Why I did it
Fix issue: Non compliant leaf list in config_db schema: https://github.com/Azure/sonic-buildimage/issues/9801
#### How I did it
The basic flow of DPB is like:
1. Transfer config db json value to YANG json value, name it “yangIn”
2. Validate “yangIn” by libyang
3. Generate a YANG json value to represent the target configuration, name it “yangTarget”
4. Do diff between “yangIn” and “yangTarget”
5. Apply the diff to CONFIG DB json and save it back to DB
The fix:
• For step #1, If value of a leaf-list field string type, transfer it to a list by splitting it with “,” the purpose here is to make step#2 happy. We also need to save <table_name>.<key>.<field_name> to a set named “leaf_list_with_string_value_set”.
• For step#5, loop “leaf_list_with_string_value_set” and change those fields back to a string.
#### How to verify it
1. Manual test
2. Changed sample config DB and unit test passed
#### Why I did it
Adding exceptlionList to validation exception
#### How I did it
Check code.
#### How to verify it
Ran manually.
- Run full config validation from a KVM
- Print the thrown exception
**Before**
```
Error: Data Loading Failed
All Keys are not parsed in FEATURE
dict_keys(['telemetry'])
```
**After**
```
Error: Data Loading Failed
All Keys are not parsed in FEATURE
dict_keys(['telemetry'])
exceptionList:["'status'"]
```
#### Which release branch to backport (provide reason below if selected)
<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
- [ ] 202111
#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->
#### Link to config_db schema for YANG module changes
<!--
Provide a link to config_db schema for the table for which YANG model
is defined
Link should point to correct section on https://github.com/Azure/SONiC/wiki/Configuration.
-->
#### A picture of a cute animal (not mandatory but encouraged)
Why I did it
Fix#9746
How I did it
Split the check condition based on non-exist and zero length.
How to verify it
Run verification script when table contains empty value
#### Why I did it
If the grouping is in the same file in the module, fetching the grouping fails
#### How I did it
fixing the `uses_module_name` when the grouping is under the same file
#### How to verify it
Enabled a grouping under the same file which is `lldp`, there is a test in sonic-yang-mgmt that translates `sample-config-db` into yang format. This test passes with grouping in `lldp` used.
#### Which release branch to backport (provide reason below if selected)
<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->
#### A picture of a cute animal (not mandatory but encouraged)
#### Why I did it
Fixes https://github.com/Azure/sonic-utilities/issues/1871
From [generic-config-updater](https://github.com/Azure/sonic-utilities/tree/master/generic_config_updater) we call `sonic-yang-mgmt` multiple times in order to check a certain change to ConfigDb is valid or not. It is expected for some changes to be invalid, so always printing errors from `sonic-yang-mgmt` makes the output hard to read.
In this PR, we are adding a way to control if logs should be printed or not.
#### How I did it
- Added `print_log_enabled` flag to sonic_yang ctor
- Converted all `print` statements to `sysLog(..., doPrint=True)`
#### How to verify it
unit-test passing means the change did not break logs.
#### Info about libyang logging
libyang provides an extensive logging logic which can support a lot of scenarios:
- ly_log_level: setting logging level
- LY_LLERR
- LY_LLWRN
- ...
- ly_set_log_clb: setting log callback to customize the default behavior which is printing the msgs
- ly_log_options: setting logging options
- LY_LOLOG: If callback is set use it, otherwise just print. If flag is not set, do nothing.
- ...
For more info refer to:
- https://netopeer.liberouter.org/doc/libyang/devel/html/group__logopts.html#gaff80501597ed76344a679be2b90a1d0a
- https://netopeer.liberouter.org/doc/libyang/devel/html/group__log.html#gac88b78694dfe9efe0450a69603f7eceb
#### What's next?
Consume the new flag `print_log_enabled` in [generic-config-updater](https://github.com/Azure/sonic-utilities/tree/master/generic_config_updater) to reduce the logging clutter.
#### Which release branch to backport (provide reason below if selected)
<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->
#### A picture of a cute animal (not mandatory but encouraged)
Why I did it
Add YANG model support for table CABLE_LENGTH
How I did it
Add the YANG model file
Add the test description file and config file
add list CABLE_LENGTH_LIST to the qos_maps_model list in sonic-yang-ext, as it has an inner list.
How to verify it
Build sonic-yang-model and sonic-yang-mgmt
Changes:
-- pre Process Grouping section from all yang models, so it can be used
from any yang model.
-- add jsondiff in setup.py, it is useful for test debugging in case of
failures.
-- use 'stypes' instead of head.
-- pass config DB table name in _createLeafDict().
-- added test config for grouping.
-- white spaces changes.
Note: Changes are done in the way that we can add support for other
Generic YANG statement easily for translation.
Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com
Why I did it
This PR adds changes in sonic-config-engine to consume configuration data in SONiC Yang schema and generate config_db entries
How I did it
Add a new file sonic_yang_cfg_generator .
This file has the functions to
parse yang data json and convert them in config_db json format.
Validate the converted config_db entries to make sure all the dependencies and constraints are met.
Add a new option -Y to the sonic-cfggen command for this purpose
Add unit tests
This capability is support only in sonic-config-engine Python3 package only
* [yang-models]: Remove PLY Extensions and change translation code.
With assumption that TABLE_SEPARATOR and ENTRY_SEPARATOR for configDB is always "|",
translation from configDB.json to sonicYang.json can be done based on keys specified
in YANG Lists inside YANG models. So removing extensions is good idea.
Changes:
-- Remove use of regex in Translation code.
-- Remove regex Extensions from YANG models.
-- Improved debugging i.e. log on stdout in case of any Exception from sonic-yang-mgmt,
so that failed tests can be debugged faster. Also this is good to debug Dynamic
port breakout issues.
-- Minor Test changes.
Co-authored-by: lguohan <lguohan@gmail.com>
Changes:
— Remove unnecessary file libyang-python-tests/sample_config_db.json
— Keep only config in format of sonic_yang.json in yang_model_tests/yangTest.json.
— Keep config in format of config_db.json in file tests/files/sample_config_db.json
— Add a test case to validate yang models.
**- Why I did it**
As per discussion in sonic Yang workgroup, arranging files better so that Yang model can be in force for new config, also adding a test case to validate young models upto some extent.
**- How I did it**
Changes:
— Remove unnecessary file libyang-python-tests/sample_config_db.json
— Keep only config in format of sonic_yang.json in yang_model_tests/yangTest.json.
— Keep config in format of config_db.json in file tests/files/sample_config_db.json
— Add a test case to validate yang models.
**- How to verify it**
Build PKGs----Building-----
Changes:
1.) yang models for device_metadata, device_neighbor, flex_counters, crm, versions.
2.) Test cases for above yang models.
Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com:
**- What I did**
#### wheel package Makefiles
- wheel package Makefiles for sonic-yang-mgmt package.
#### libyang Python APIs:
- python APIs based on libyang
- functions to load/merge yang models and Yang data files
- function to validate data trees based on Yang models
- functions to merge yang data files/trees
- add/set/delete node in schema and data trees
- find data/schema nodes from xpath from the Yang data/schema tree in memory
- find dependencies
- dump the data tree in json/xml
#### Extension of libyang Python APIs:
-- Cropping input config based on Yang Model.
-- Translate input config based on Yang Model.
-- rev Translate input config based on Yang Model.
-- Find xpath of port, portleaf and a yang list.
-- Find if node is key of a list while deletion if yes, then delete the parent.
Signed-off-by: Praveen Chaudhary pchaudhary@linkedin.com
Signed-off-by: Ping Mao pmao@linkedin.com
