Resubmitting the changes for (#4825) with fixes for sonic-bgpcdgd test failures
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
* Loopback IP changes for multi ASIC devices
multi ASIC will have 2 Loopback Interfaces
- Loopback0 has globally unique IP address, which is advertised by the multi ASIC device to its peers.
This way all the external devices will see this device as a single device.
- Loopback4096 is assigned an IP address which has a scope is within the device. Each ASIC has a different ip address for Loopback4096. This ip address will be used as Router-Id by the bgp instance on multi ASIC devices.
This PR implements this change for multi ASIC devices
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
If some table with a list of tuples (interface name, ip prefix) has ip prefixes without a mask length, it will cause issues in SONiC. For example quagga and frr will treat ipv4 address without a mask, so "10.20.30.40" address will be treated as "10.0.0.0/8", which is dangerous.
The fix here is that when pfx_filter get a tuple (interface name, ip prefix), where the ip prefix doesn't have prefix mask length, add a mask by default: "/32 for ipv4 addresses, /128 for ipv6 addresses".
Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
**- What I did**
Add support of **platform.json** parsing to **portconfig.py** file which is being used by **_sonic-cfggen_** and ***minigraph.py*** file under ***src/sonic-config-engine*** folder to get port config via get_port_config function.
**- How I did it**
1. **portconfig.py** file will first check whether the **platform.json** file is there or not. if not then whether port_config.ini file is there or not. Modified **get_port_config_file_name** for this purpose.
2. Added two separate functions i.e. **parse_platform_json_file** to get port attributes from **platform.json** and **gen_port_config** to generate port attributes.
3. Added another two functions i.e get_breakout_mode parse_breakout_mode to get breakout mode and parse breakout mode from platform.json respectively.
**- How to verify it**
rebuilt "sonic_config_engine-1.0" wheel package with all the test cases.All the below-mentioned test cases passed.
```
# Check whether all interfaces present or not as per platform.json
def test_platform_json_interfaces_keys(self):
# Check specific Interface with it's proper configuration as per platform.json
def test_platform_json_specific_ethernet_interfaces(self):
# Check all Interface with it's proper configuration as per platform.json
def test_platform_json_all_ethernet_interfaces(self):
```
Signed-off-by: Sangita Maity <sangitamaity0211@gmail.com>
* Multi DB with namespace support, Introducing the database_global.json file
for supporting accessing DB's in other namespaces for service running in
linux host
* Updates based on comments
* Adding the j2 templates for database_config and database_global files.
* Updating to retrieve the redis DIR's to be mounted from database_global.json file.
* Additional check to see if asic.conf file exists before sourcing it.
* Updates based on PR comments discussion.
* Review comments update
* Updates to the argument "-n" for namespace used in both context of parsing minigraph and multi DB access.
* Update with the attribute "persistence_for_warm_boot" that was added to database_config.json file earlier.
* Removing the database_config.json file to avioid confusion in future.
We use the database_config.json.j2 file to generate database_config.json files dynamically.
* Update the comments for sudo usage in docker_image_ctrl.j2
* Update with the new logic in PING PONG tests using sonic-db-cli. With this we wait till the
PONG response is received when redis server is up.
* Similar changes in swss and syncd scripts for the PING tests with sonic-db-cli
* Updated with a missing , in the database_config.json.j2 file, Do pip install of j2cli in docker-base-buster.
- Changes to minigraph.py to parse minigraph.xml of a multi asic platform
- Changes to portconfig.py to parse additional column "asic_port_name" in
port_config.ini
- Add a new option -n to sonic-cfggen for multi asic platforms
- Add unit tests for config generation for multi asic platforms
Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
The one big bgp configuration template was splitted into chunks.
Currently we have three types of bgp neighbor peers:
general bgp peers. They are represented by CONFIG_DB::BGP_NEIGHBOR table entries
dynamic bgp peers. They are represented by CONFIG_DB::BGP_PEER_RANGE table entries
monitors bgp peers. They are represented by CONFIG_DB::BGP_MONITORS table entries
This PR introduces three templates for each peer type:
bgp policies: represent policieas that will be applied to the bgp peer-group (ip prefix-lists, route-maps, etc)
bgp peer-group: represent bgp peer group which has common configuration for the bgp peer type and uses bgp routing policy from the previous item
bgp peer-group instance: represent bgp configuration, which will be used to instatiate a bgp peer-group for the bgp peer-type. Usually this one is simple, consist of the referral to the bgp peer-group, bgp peer description and bgp peer ip address.
This PR redefined constant.yml file. Now this file has a setting for to use or don't use bgp_neighbor metadata. This file has more parameters for now, which are not used. They will be used in the next iteration of bgpcfgd.
Currently all tests have been disabled. I'm going to create next PR with the tests right after this PR is merged.
I'm going to introduce better bgpcfgd in a short time. It will include support of dynamic changes for the templates.
FIX:: #4231
* Fix bugs - "show run acl/interfaces" returns traceback when no ACL_RULE/INTERFACE is present in configuration
- Change logic in main to take data only if the var_json appears in DB.
- Add unit testing for bug fix & fix comments
Signed-off-by: Noa Or <noaor@mellanox.com>
If sonic-cfggen is passed the -m argument (to load the minigraph file) along with one or more -j <json_file> arguments, load the JSON files before loading the minigraph file.
This ensures that the init_cfg.json file is loaded before the minigraph, therefore the minigraph can override any default configuration options specified in init_cfg.json. Currently, the behavior is reversed.
Note: This is not an issue if loading loading multiple JSON files, because sonic-cfggen loads them in the left-to-right order they were specified on the command line, therefore providing flexibility for loading JSON files in a specific order. As long as init_cfg.json is specified before config_db.json, the values specified in config_db.json will take precedence.
* [sonic-cfggen] optimize execution time
a lot of template rendering causes switch to start longer because jinja2
needs to parse them. Introducing RedisBytecodeCache to store parsed buckets of
internal template bytecode to speedup same template rendering during start
* [sonic-cfggen] do lazy regexp compilation to speedup sonic-cfggen
* [sonic-cfggen] address pep8 related comments
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
Now it's possible to add and remove peers based on ConfigDB
- What I did
Fixed functionality for dynamically adding/removing static bgp peers.
- How I did it
Split the bgp default template on bgp part and bgp peer part
Changed bgpcfgd to use 1.
- How to verify it
Build an image and run on your DUT
From 5.1 version of PyYAML python module, yaml.load() API is deprecated. Code should be compatible to support both the versions, else error/warning messages are seen like below,
2019-07-02 08:25:35,284 – INFO: [D1] /usr/local/lib/python2.7/dist-packages/sonic_device_util.py:44: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
* Adding --key option to sonic-cfggen script. This will help to display config DB with more granularity.
Signed-off-by: Vasant Patil <vapatil@linkedin.com>
* fix name conflict between sonic_platfrom package and sonic_platform.py
* update sonic-utility submodule to pickup lastest fix
* Revert "update sonic-utility submodule to pickup lastest fix"
This reverts commit f66aa99738.
* update sonic-utility sub module
* Unify qos config with qos_config.j2 template
Signed-off-by: Wenda <wenni@microsoft.com>
* Change 7050 to use qos config template
Signed-off-by: Wenda <wenni@microsoft.com>
modified: device/arista/x86_64-arista_7050_qx32/Arista-7050-QX32/qos.json.j2
modified: device/arista/x86_64-arista_7050_qx32s/Arista-7050-QX-32S/qos.json.j2
* Change a7060, a7260, s6000, s6100, z9100 to use qos config template
Signed-off-by: Wenda <wenni@microsoft.com>
* Change mlnx devices to use qos config template
Signed-off-by: Wenda <wenni@microsoft.com>
modified: ../../../mellanox/x86_64-mlnx_msn2100-r0/ACS-MSN2100/qos.json.j2
modified: ../../../mellanox/x86_64-mlnx_msn2410-r0/ACS-MSN2410/qos.json.j2
modified: ../../../mellanox/x86_64-mlnx_msn2700-r0/ACS-MSN2700/qos.json.j2
modified: ../../../mellanox/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-D48C8/qos.json.j2
* Change barefoot devices to use qos config template
Signed-off-by: Wenda <wenni@microsoft.com>
modified: barefoot/x86_64-accton_wedge100bf_32x-r0/montara/qos.json.j2
modified: barefoot/x86_64-accton_wedge100bf_65x-r0/mavericks/qos.json.j2
* Change accton as7212 to use qos config template
Signed-off-by: Wenda <wenni@microsoft.com>
modified: accton/x86_64-accton_as7212_54x-r0/AS7212-54x/qos.json.j2
* Apply PORT_QOS_MAP to active ports only
Signed-off-by: Wenda <wenni@microsoft.com>
* Update qos config test with qos_config.j2 template
Signed-off-by: Wenda <wenni@microsoft.com>
* Update sample output of qos-dell6100.json
Signed-off-by: Wenda <wenni@microsoft.com>
* Remove generating the default port name and index list, i.e., remove the generate_port_lists macro, because PORT is always defined
Signed-off-by: Wenda <wenni@microsoft.com>
* Include pfc_to_pg_map according to platform asic type obtained from
/etc/sonic/sonic_version.yml rather than specifying per hwsku
Signed-off-by: Wenda Ni <wenni@microsoft.com>
* Customize TC_TO_PRIORITY_GROUP_MAP and
PFC_PRIORITY_TO_PRIORITY_GROUP_MAP for barefoot
Signed-off-by: Wenda <wenni@microsoft.com>
* Unify PFC_PRIORITY_TO_PRIORITY_GROUP_MAP: remove "0":"0", "1":"1" as
these two pgs do not generate PFC frames.
Signed-off-by: Wenda <wenni@microsoft.com>
* [sonic-cfggen] Add support to generate sample t1 config
* Fix typo
* Fix space issue
* Add list of preset messages in help message
* Utilize choice support of argparse
* Fix missing module in setup.py
Just a simple change to make sonic's user life a little bit easier. Displaying the multiple elements of config_db.json in an orderly fashion allows a more intuitive user-experience.
For this change i'm simply intercepting the config-state that is about to be dumped to config_db.json, and i'm placing it in an ordered-dictionary, so that information is pushed to file in natural/alphanumerical ordering.
Example:
admin@lnos-x1-a-csw01:~$ sudo sonic-cfggen -m /etc/sonic/minigraph.xml --print-data
{
...
"PORT": {
"Ethernet0": {
"alias": "Eth1/1",
"lanes": "65"
},
"Ethernet1": {
"alias": "Eth1/2",
"lanes": "66"
},
"Ethernet2": {
"alias": "Eth1/3",
"lanes": "67"
},
"Ethernet3": {
"alias": "Eth1/4",
"lanes": "68"
},
"Ethernet4": {
"alias": "Eth2/1",
"lanes": "69"
},
...
...
"INTERFACE": {
"Ethernet0|10.0.0.0/31": {},
"Ethernet1|10.0.0.2/31": {},
"Ethernet2|10.0.0.4/31": {},
"Ethernet3|10.0.0.6/31": {},
"Ethernet4|10.0.0.8/31": {},
* [sonic build] Define folder macro for target folder /usr/sonic/share/templates
* [sonic-cfggen] allow templates to include from common folders
- Allow templates to include files under /usr/share/sonic/templates
- Allow templates to include files in the same folder as the root template
* [Buffer config] install the buffer configuration template
* [Arista7260cx3] Add buffer configuration for Arista7260CX3 T0 topology
- pg profile look up table is incomplete. Currently contains T0 default
values.
* [Arista7260cx3] Adding QoS configuration
* Address review comments
1. Stop generating ingress pg configuration for lossless pgs.
2. Stop putting ports into speed sets, put all ports in one set.
3. Remove ingress lossless profiles.
4. Added some tailing '-' back to remove leading spaces.
Modify minigraph parser output format so it fit DB schema
Modify configuration templates to fit new schema
Systemd services dependencies are modified so database starts before any configuration consumer
* [cfggen] Support reading from and writing to configdb
* [bgp] Move bgp_admin_state to configdb, support dynamic admin state change
* [sonic-utilities] Adapt configDB for admin status, support config save and config load
- Add -p --port-config option to feed sonic-cfggen with port_config.ini
file when necessary.
- Update minigraph.py file to accept the -p option
- Add test_j2files.py test to test config.sh and all .j2 templates
* Currently test_teamd is added to test both the config.sh and teamd.j2
file works well with the t0 sample minigraph and sample port config
file
* The sample output is added to the folder sample_output for comparison
Signed-off-by: Shuotian Cheng <shuche@microsoft.com>
* [files]: Add allow-hotplug stanza to interfaces files
- start interface <interface_name> when the kernel detects
a hotplug event from the interface
ref: https://www.debian.org/doc/manuals/debian-reference/ch05.en.html
* [interfaces]: Combine vlan_interfaces and lag_interfaces file and add allow-hotplug
1. Remove vlan_interfaces and lag_interfaces file and members in teamd.j2
2. Add all interfaces to /etc/network/interfaces file
3. Add allow-hotplug stanza
4. Add up <command> to automatically add interfaces to VLAN and LAG
5. Add unique_name filter to minigraph.py to remove duplicate interface names
6. Add brctl to base image
7. Update sonic-swss submodule
Signed-off-by: Shuotian Cheng <shuche@microsoft.com>
Add a sonic-config-engine to help generate config file based on minigraph and other data on runtime. Modify fpm, teamd, lldp, snmp, and platform-monitor docker to use sonic-config-engine to generate config in docker upon load.
