Commit Graph

488 Commits

Author SHA1 Message Date
judyjoseph
44f21a18ea
Port the fix to get the PCI_id from asic.conf file from 201911 --> master branch (#7513)
Port the fix to get the pci_device ID from asic.conf file and update the "asic_id" field in DEVICE_METADATA with the pci_device_Id.

Ref: https://github.com/Azure/sonic-buildimage/pull/4705
2021-05-07 09:47:26 -07:00
Lawrence Lee
1b39424520
[docker-orchagent]: Increase ndppd kernel poll interval (#7456)
Why I did it
ndppd by default reads /proc/net/ipv6_route ever 30 seconds. Since T1s advertise so many routes to ToRs, this file is extremely large, and reading it causes ndppd's CPU usage to spike every 30 seconds

How I did it
Increase the delay for reading this file to the maximum possible value (max integer value), which will result in CPU spikes every ~24 days instead of every 30 seconds

How to verify it
Start ndppd with the new config file, confirm that no CPU spikes are seen except at startup

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2021-04-30 16:30:30 -07:00
Maxime Lorrillere
a92da83047
[chassis] VoQ configuration using minigraph.xml file (#5991)
This commit contains the following changes to support for configuring a VoQ switch using a minigraph.xml file.:
- Add support for system ports configuration to minigraph
- Add support for SwitchId, SwitchType and MaxCores to minigraph
- Add support for inband vlan configuration in minigraph
- `asic_name` is now a mandatory attribute in CONFIG_DB on VoQ switches

Co-authored-by: Maxime Lorrillere <mlorrillere@arista.com>
2021-04-27 12:18:45 -07:00
jmmikkel
43342b33b8
[chassis] Add templates and code to support VoQ chassis iBGP peers (#5622)
This commit has following changes:

* Add templates and code to support VoQ chassis iBGP peers

* Add support to convert a new VoQChassisInternal element in the
   BGPSession element of the minigraph to a new BGP_VOQ_CHASSIS_NEIGHBOR 
   table in CONFIG_DB.
* Add a new set of "voq_chassis" templates to docker-fpm-frr
* Add a new BGP peer manager to bgpcfgd to add neighbors from the
  BGP_VOQ_CHASSIS_NEIGHBOR table using the voq_chassis templates.
* Add a test case for minigraph.py, making sure the VoQChassisInternal
  element creates a BGP_VOQ_CHASSIS_NEIGHBOR entry, but not if its
  value is "false".
* Add a set of test cases for the new voq_chassis templates in
  sonic-bgpcfgd tests.

Note that the templates expect the new
"bgp bestpath peer-type multipath-relax" bgpd configuration to be
available.

Signed-off-by: Joanne Mikkelson <jmmikkel@arista.com>
2021-04-16 11:11:32 -07:00
Lawrence Lee
39a027bceb
[l2-preset]: Add all dual ToR required fields (#7318)
- Add peer_switch field to DEVICE_METADATA table
- In PORT table:
   - Set used ports to admin status up
   - Set mux_cable to true for downlinks in use
- In MUX_CABLE table:
   - Only add entry if the downlink is in use

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2021-04-14 17:38:14 -07:00
anish-n
9d81524768
Add downstreamsubrole parsing to minigraph.py (#7193) 2021-04-12 10:33:41 -07:00
abdosi
8ec3bdc301
[multi-asic] Make sure Everflow tables always gets classified as Mirror table (#7241)
Make sure Everflow always gets classified as Mirror table and not as Control Plane on multi-asic platforms.

Why I did:
In Multi-asic platforms we generate Everflow acl table data from minigraph for both host and namespace.
It is possible in multi-asic minigraph if there are no external port-channel (Only Router Port IP Interface) then Everflow table will have no binded interface in host and will gets classified as Control Plane ACL while in namespace gets classified as Mirror Table.
For ACL Rule generation we read global db as source of truth for acl table information and so for everflow rule generation if tables gets classified as Control plane we can generate rules with invalid action causing orchagent to throw runtime error.

How I did:
If the table is attach to erspan interface in minigraph then it always gets classified as mirror table.
2021-04-07 13:35:34 -07:00
Prince Sunny
20c8dd2691
[IPinIP] Add Loopback2 interface, change dscp mode to uniform (#7234)
Co-authored-by: Ubuntu <prsunny>
2021-04-07 09:58:12 -07:00
Lawrence Lee
0a86699e73
[sonic-config-engine]: Update L2 preset for dualtor (#7215)
- When generating L2 preset, check for dual ToR setting from CLI option `-a '{"is_dualtor": true}'`
- When dual ToR is specified, add subtype field to DEVICE_METADATA table
- When dual ToR is specified, add MUX_CABLE, TUNNEL, LOOPBACK_INTERFACE, and PEER_SWITCH tables
2021-04-05 13:06:27 -07:00
Joe LeVeque
c651a9ade4
[dockers][supervisor] Increase event buffer size for process exit listener; Set all event buffer sizes to 1024 (#7083)
To prevent error [messages](https://dev.azure.com/mssonic/build/_build/results?buildId=2254&view=logs&j=9a13fbcd-e92d-583c-2f89-d81f90cac1fd&t=739db6ba-1b35-5485-5697-de102068d650&l=802) like the following from being logged:

```
Mar 17 02:33:48.523153 vlab-01 INFO swss#supervisord 2021-03-17 02:33:48,518 ERRO pool supervisor-proc-exit-listener event buffer overflowed, discarding event 46
```

This is basically an addendum to https://github.com/Azure/sonic-buildimage/pull/5247, which increased the event buffer size for dependent-startup. While supervisor-proc-exit-listener doesn't subscribe to as many events as dependent-startup, there is still a chance some containers (like swss, as in the example above) have enough processes running to cause an overflow of the default buffer size of 10.

This is especially important for preventing erroneous log_analyzer failures in the sonic-mgmt repo regression tests, which have started occasionally causing PR check builds to fail. Example [here](https://dev.azure.com/mssonic/build/_build/results?buildId=2254&view=logs&j=9a13fbcd-e92d-583c-2f89-d81f90cac1fd&t=739db6ba-1b35-5485-5697-de102068d650&l=802).

I set all supervisor-proc-exit-listener event buffer sizes to 1024, and also updated all dependent-startup event buffer sizes to 1024, as well, to keep things simple, unified, and allow headroom so that we will not need to adjust these values frequently, if at all.
2021-03-27 21:14:24 -07:00
dependabot[bot]
247ac7ba17
Bump lxml from 4.6.2 to 4.6.3 in /src/sonic-config-engine (#7122)
Bumps [lxml](https://github.com/lxml/lxml) from 4.6.2 to 4.6.3.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](https://github.com/lxml/lxml/compare/lxml-4.6.2...lxml-4.6.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-27 11:54:38 -07:00
Joe LeVeque
7f52abc621
[sonic-config-engine] Parse AutoNegotiation element from LinkMetadata section of minigraph file (#7031)
Parse the value `AutoNegotiation` element from the `LinkMetadata` section of minigraph file. If the element is present, an `autoneg` key will be added to the port in the `PORT` table of Config DB with a value of either `off` or `on`

If an `autoneg` value is present in port_config.ini, the value from the minigraph will take precedence, overriding that value.

Also remove `AutoNegotiation` and `EnableAutoNegotiation` elements from the `DeviceInfo` section, as we will use this data in the `LinkMetadata` section to determine whether to enable auto-negotiation for a port.
2021-03-12 16:34:27 -08:00
judyjoseph
a09a85b64a
[sonic-cfggen]: Use unix socket when reading from DB only if we are using sudo. (#7002)
Closes issue #6982.
The issue was root caused as we were using the unix_socket for reading from DB as a default mechanism (#5250). The redis unix socket is created as follows.

admin@str--acs-1:~$ ls -lrt /var/run/redis/redis.sock 
srwxrw---- 1 root redis 0 Mar  6 01:57 /var/run/redis/redis.sock
So it used to work fine for the user "root" or if user is part of redis group ( admin was made part of redis group by default )

Check if the user is with sudo permissions then use the redis unix socket, else fallback to tcp socket.
2021-03-10 09:07:47 -08:00
anish-n
a61888f7da
[minigraph] Adjust minigraph parsing per Fine Grained ECMP matchmode changes (#6885)
To adjust config db generated via minigraph per matchmode changes in fine grained ecmp. The changes are done so that nexthop IP based filtering can occur to determine routes as requiring Fine Grained ECMP, in the past the only mode was to use the IP prefix of the route for filtering, with this matchmode change we will use nexthop IP based filtering
Azure/SONiC#727

How I did it
Change will modify config db entry created for FG_NHG to include 'match_mode': 'nexthop-based' so that nexthop IP based filtering can occur to determine routes as requiring Fine Grained ECMP. Changes also remove FG_NHG_PREFIX entry since its not needed under matchmode nexthop-based.
2021-03-05 10:17:56 -08:00
Tamer Ahmed
bb03e5bb37
Start DHCP Relay When Helpers IPs Are Available (#6961)
#### Why I did it

It is possible to have DHCP relay configuration with no servers/
helpers which result in DHCP container to crash. This PR fixes this
issue by not starting DHCP relay for vlans with no DHCP helpers.

resolves: #6931 
closes: #6931 
#### How I did it
Do not add program group for dhcp relay with not dhcp helpers

#### How to verify it
Unit test
2021-03-04 20:43:08 -08:00
Danny Allen
880a743c43
[openconfig_acl] Allow setting ICMP type/code to 0 (#6932)
There is a bug in how pyangbind translates yang models into python bindings. The model always sets integer values to 0 by default, so there is no way to check if a user has provided a value that is equal to 0. This is problematic for ICMP and VLAN (among others) because 0 is a valid input value.

This change converts ICMP and VLAN fields to union types so that acl-loader will treat them as null values unless a user explicitly adds an integer value.

Signed-off-by: Danny Allen <daall@microsoft.com>
2021-03-02 14:32:00 -08:00
Danny Allen
abb3867b72
[openconfig_acl] Add SONiC ACL extension to open config ACL model (#6896)
- Add support for VLAN ID match
- Add support for ICMP type/code match

Signed-off-by: Danny Allen <daall@microsoft.com>
2021-02-28 15:02:31 -08:00
abdosi
30b6668b7d
Changes in FRR temapltes for multi-asic (#6901)
1. Made the command next-hop-self force only applicable on back-end asic bgp. This is done so that BGPL iBGP session running on backend can send e-BGP learn nexthop. Back end asic FRR is able to recursively resolve the eBGP nexthop in its routing table since it knows about all the connected routes advertise from front end asic.

2. Made all front-end asic bgp use global loopback ip (Loopback0) as router id and back end asic bgp use Loopbacl4096 as ruter-id and originator id for Route-Reflector. This is done so that routes learnt by external peer do not see Loopback4096 as router id in show ip bgp <route-prerfix> output.

3. To handle above change need to pass Loopback4096 from BGP manager for jinja2 template generation. This was missing and this change/fix is needed for this also https://github.com/Azure/sonic-buildimage/blob/master/dockers/docker-fpm-frr/frr/bgpd/templates/dynamic/instance.conf.j2#L27

4. Enhancement to add mult_asic specific bgpd template generation unit test cases.
2021-02-26 17:05:15 -08:00
Qi Luo
833584eff9
[minigraph] For egress ACL attaching to vlan, break them into vlan members (#6895)
#### Why I did it
Some platforms have difficult to attach egress ACL to vlan.

#### How I did it
For egress ACL attaching to vlan, break them into vlan members.

#### How to verify it
Unit test
Tested in DUT
2021-02-26 10:41:49 -08:00
Sangita Maity
18263c99dd
[DPB|master] Update Dynamic Port Breakout Logic for flexible alias support a… (#6831)
To fix [DPB| wrong aliases for interfaces](https://github.com/Azure/sonic-buildimage/issues/6024) issue, implimented flexible alias support [design doc](https://github.com/Azure/SONiC/pull/749)

> [[dpb|config] Fix the validation logic of breakout mode](https://github.com/Azure/sonic-utilities/pull/1440) depends on this

#### How I did it

1. Removed `"alias_at_lanes"` from port-configuration file(i.e. platfrom.json) 
2. Added dictionary to "breakout_modes" values. This defines the breakout modes available on the platform for this parent port, and it maps to the alias list. The alias list presents the alias names for individual ports in order under this breakout mode.
```
{
    "interfaces": {
        "Ethernet0": {
            "index": "1,1,1,1",
            "lanes": "0,1,2,3",
            "breakout_modes": {
                "1x100G[40G]": ["Eth1"],
                "2x50G": ["Eth1/1", "Eth1/2"],
                "4x25G[10G]": ["Eth1/1", "Eth1/2", "Eth1/3", "Eth1/4"],
                "2x25G(2)+1x50G(2)": ["Eth1/1", "Eth1/2", "Eth1/3"],
                "1x50G(2)+2x25G(2)": ["Eth1/1", "Eth1/2", "Eth1/3"]
            }
        }
}
```
#### How to verify it
`config interface breakout`

Signed-off-by: Sangita Maity <samaity@linkedin.com>
2021-02-26 00:13:33 -08:00
Qi Luo
c8ed3676ab
[minigraph] Support tagged VlanInterface if attached to multiple vlans (#6833)
It is possible that one interface attaches multiple vlans. The VlanInterface should be in tagged mode.

Signed-off-by: Qi Luo <qiluo-msft@users.noreply.github.com>
2021-02-22 10:14:55 -08:00
Renuka Manavalan
890545f317
Update minigraph.py to parse kubernetes config from minigraph.xml (#6633)
Update minigraph parser to retrieve kubernetes server info from minigraph.xml and update "KUBERNETES_MASTER|SERVER" in running config.
Update minigraph parser to include clusterName from minigraph.xml into "DEVICE_METADATA|localhost"
2021-02-02 11:16:37 -08:00
Qi Luo
1c8d5ec500
Bump pyyaml from 5.3.1 to 5.4.1 (#6511)
RCE resolved in new version https://github.com/yaml/pyyaml/issues/420
2021-01-28 10:46:56 -08:00
Tamer Ahmed
8d857fab16
[dhcp-relay]: Launch DHCP Relay On L3 Vlan (#6527)
Recent changes brought l2 vlan concept which do not have DHCP
clients behind them and so DHCP relay is not required. Also,
dhcpmon fails to launch on those vlans as their interfaces
lack IP addresses. This PR limit launch of both DHCP relay
and dhcpmon to L3 vlans only.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2021-01-25 10:48:48 -08:00
Dmytro Shevchuk
dd0e1100a5
[sonic-cfggen] parse optional fec and autoneg fields from hwsku.json (#6155)
**- Why I did it**

For now `hwsku.json` and `platform.json` dont support optional fields. For example no way to add `fec` or `autoneg` field using `platform.json` and `hwsku.json`.

**- How I did it**
Added parsing of optional fields from hwsku.json.

**- How to verify it**
Add optional field to `hwsku.json`. After first boot will be generated new `config_db.json` or you can generate it using `sonic-cfggen` command. In this file must be optional field from `hwsku.json` or check using command `redis-cli hgetall PORT_TABLE:Ethernet0`
Example of `hwsku.json`, that must be parsed:
```
{
    "interfaces": {
        "Ethernet0": {
            "default_brkout_mode": "1x100G[40G]",
            "fec": "rs",
            "autoneg": "0"
        },
...
}
```
Example of generated `config_db.json`:
```
    "PORT": {
        "Ethernet0": {
            "alias": "Ethernet0",
            "lanes": "0,1,2,3",
            "speed": "100000",
            "index": "1",
            "admin_status": "up",
            "fec": "rs",
            "autoneg": "0",
            "mtu": "9100"
        },
```
So, we can see this entries in redis db:
```
admin@sonic:~$ redis-cli hgetall PORT_TABLE:Ethernet0

 1) "alias"
 2) "Ethernet0"
 3) "lanes"
 4) "0,1,2,3"
 5) "speed"
 6) "100000"
 7) "index"
 8) "1"
 9) "admin_status"
10) "up"
11) "fec"
12) "rs"
13) "autoneg"
14) "0"
15) "mtu"
16) "9100"
17) "description"
18) ""
19) "oper_status"
20) "up"
```

Also its way to fix `show interface status`, `FEC` field but also need add `FEC` field to `hwsku.json`.
Before:
```
admin@sonic:~$ show interfaces status
  Interface            Lanes    Speed    MTU    FEC        Alias    Vlan    Oper    Admin             Type    Asym PFC
-----------  ---------------  -------  -----  -----  -----------  ------  ------  -------  ---------------  ----------
  Ethernet0          0,1,2,3     100G   9100     N/A    Ethernet0  routed      up       up  QSFP28 or later         N/A
```
After:
```
admin@sonic:~$ show interfaces status
  Interface            Lanes    Speed    MTU    FEC        Alias    Vlan    Oper    Admin             Type    Asym PFC
-----------  ---------------  -------  -----  -----  -----------  ------  ------  -------  ---------------  ----------
  Ethernet0          0,1,2,3     100G   9100     rs    Ethernet0  routed      up       up  QSFP28 or later         N/A
```
2021-01-24 17:46:33 -08:00
Lawrence Lee
8729fdc9ed
[minigraph.py]: Force /128 prefix for server IPv6 loopbacks (#6524)
Meet the requirement for the MUX_CABLE table that IPv6 loopbacks have a /128 prefix

Note that this change only affects the MUX_CABLE table, all other tables continue to use the loopback address provided in minigraph.

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2021-01-21 15:05:35 -08:00
yozhao101
be3c036794
[supervisord] Monitoring the critical processes with supervisord. (#6242)
- Why I did it
Initially, we used Monit to monitor critical processes in each container. If one of critical processes was not running
or crashed due to some reasons, then Monit will write an alerting message into syslog periodically. If we add a new process
in a container, the corresponding Monti configuration file will also need to update. It is a little hard for maintenance.

Currently we employed event listener of Supervisod to do this monitoring. Since processes in each container are managed by
Supervisord, we can only focus on the logic of monitoring.

- How I did it
We borrowed the event listener of Supervisord to monitor critical processes in containers. The event listener will take
following steps if it was notified one of critical processes exited unexpectedly:

The event listener will first check whether the auto-restart mechanism was enabled for this container or not. If auto-restart mechanism was enabled, event listener will kill the Supervisord process, which should cause the container to exit and subsequently get restarted.

If auto-restart mechanism was not enabled for this contianer, the event listener will enter a loop which will first sleep 1 minute and then check whether the process is running. If yes, the event listener exits. If no, an alerting message will be written into syslog.

- How to verify it
First, we need checked whether the auto-restart mechanism of a container was enabled or not by running the command show feature status. If enabled, one critical process should be selected and killed manually, then we need check whether the container will be restarted or not.

Second, we can disable the auto-restart mechanism if it was enabled at step 1 by running the commnad sudo config feature autorestart <container_name> disabled. Then one critical process should be selected and killed. After that, we will see the alerting message which will appear in the syslog every 1 minute.

- Which release branch to backport (provide reason below if selected)

 201811
 201911
[x ] 202006
2021-01-21 12:57:49 -08:00
Qi Luo
3c16f80382
sonic-config-engine uses libswsscommon instead of swsssdk (#6406)
**- Why I did it**
swsssdk will be deprecated. Migrate sonic-config-engine to use libswsscommon library instead

**- How to verify it**
Unit test
2021-01-20 12:06:08 -08:00
Lawrence Lee
063e11cce1
[minigraph.py]: Don't create mux table entries for servers w/o loopbacks (#6457)
Avoid sonic-cfggen crashing when a server does not have a configured loopback address in the minigraph

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2021-01-17 00:23:11 -08:00
Lawrence Lee
ffcef27eb1
[minigraph.py]: Check for empty cluster tag before parsing (#6440)
Some non-production minigraphs will have an empty ClusterName tag

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2021-01-15 08:03:47 -08:00
Lawrence Lee
6e63ecfa1d
[minigraph.py]: Force /32 prefix for mux cable server IPv4 loopbacks (#6418)
Server IPv4 loopbacks do not always arrive with /32 prefix, which is a requirement for the MUX_CABLE table in config DB

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2021-01-12 06:04:48 -08:00
Lawrence Lee
3dd993e019
[minigraph.py]: Add peer switch hostname to device metadata (#6405)
To make the peer switch hostname easily accessible from config DB. Add peer_switch field to DEVICE_METADATA table

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2021-01-11 10:18:39 -08:00
dependabot[bot]
72b635083d
Bump lxml from 4.6.1 to 4.6.2 in /src/sonic-config-engine (#6385)
Bumps [lxml](https://github.com/lxml/lxml) from 4.6.1 to 4.6.2.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](https://github.com/lxml/lxml/compare/lxml-4.6.1...lxml-4.6.2)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-01-08 15:47:29 -08:00
kktheballer
ba92a081ce
Minigraph ECMP parsing support (cleaner format) (#4985)
Why I did it
To support FG_ECMP  scenarios
- How I did it
Modified minigraph parser to parse ECMP fields in the case they are present in minigraph
- How to verify it
Loaded ensuing config_db file on a DUT to verify the fields are parsed and configure device correctly
2020-12-30 15:18:21 -08:00
Ubuntu
273846a412 FRR 7.5
Build libyang1 which is required for frr 7.5
2020-12-29 03:44:49 -08:00
Stepan Blyshchak
23f1d51de3
[ipinip.json.j2] align mellanox configuration dst_ip with other platforms (#6304)
Mellanox already supports multiple destination IPs in IPinIP tunnel configuration, thus removing mellanox
exception for IPinIP configuration.

- How I did it

Removed "dst_ip" field generation in mellanox platform condition.
Sorted the "dst_ip" list, so that it is easier to test against sample configuration in unit tests.
Aligned unit test sample.

Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
2020-12-28 20:53:12 -08:00
Guohan Lu
ed58684e36 [docker-frr]: add static ipv6 loopback route to allow bgp to advertise prefix
frr does not advertise route if local route is not reachable, as a result
loopback route /64 is not advertised to the neighbors. Add static route
allows frr to advertise the route to its peers

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-12-28 10:34:34 -08:00
Joe LeVeque
e39cc018c8
[sonic-config-engine] Pin 'importlib-resources' package to v3.3.1 for Python 2 (#6288)
importlib-resources v4.0.0 was released today (2020-12-23) and drops support for Python 2. This caused the sonic-config-engine Python 2 wheel build to fail.

Reference: https://pypi.org/project/importlib-resources/

Pin 'importlib-resources' package to v3.3.1 for Python 2
Unrelated: remove pinned version of zipp for sonic-bgpcfgd because we no longer build a Python 2 version of that package
2020-12-23 18:00:31 -08:00
Ying Xie
b4ea0c09f5
[minigraph parse] avoid crash when peer switch lo address is not available (#6266)
- Why I did it
Latest master image crashes when loading minigraph

Fixing #6265

- How I did it
Avoid converting 'None' to ipaddress.

- How to verify it
On a system crashing with the issue, manually patch minigraph.py with the change in PR and load minigraph succeeded.

Signed-off-by: Ying Xie ying.xie@microsoft.com
2020-12-21 14:39:34 -08:00
Lawrence Lee
290f66bbb8
[minigraph.py]: Prefer parsing device type from <ElementType> (#6184)
* Parse device type from <ElementType> first in <PngDec>
* Fall back to <Device> type attribute if no <ElementType> is found

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2020-12-15 10:20:44 -08:00
Lawrence Lee
fd4433d836
[minigraph.py]: Remove prefix length from peer switch loopback address (#6174)
* PEER_SWITCH table in config DB expects a standalone IP address w/o a prefix length

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2020-12-10 14:43:39 -08:00
Blueve
3d22019802
[sonic-config-engine/minigraph] Enable console mgmt feature for console device (#6166)
* Introduced a list console_device_types which contains the device types that support console management feature
* Inject CONSOLE_SWITCH:console_mgmt table with enabled:yes or enabled:no

Signed-off-by: Jing Kan jika@microsoft.com
2020-12-10 15:42:11 +08:00
Joe LeVeque
7f4ab8fbd8
[sonic-utilities] Update submodule; Build and install as a Python 3 wheel (#5926)
Submodule updates include the following commits:

* src/sonic-utilities 9dc58ea...f9eb739 (18):
  > Remove unnecessary calls to str.encode() now that the package is Python 3; Fix deprecation warning (#1260)
  > [generate_dump] Ignoring file/directory not found Errors (#1201)
  > Fixed porstat rate and util issues (#1140)
  > fix error: interface counters is mismatch after warm-reboot (#1099)
  > Remove unnecessary calls to str.decode() now that the package is Python 3 (#1255)
  > [acl-loader] Make list sorting compliant with Python 3 (#1257)
  > Replace hard-coded fast-reboot with variable. And some typo corrections (#1254)
  > [configlet][portconfig] Remove calls to dict.has_key() which is not available in Python 3 (#1247)
  > Remove unnecessary conversions to list() and calls to dict.keys() (#1243)
  > Clean up LGTM alerts (#1239)
  > Add 'requests' as install dependency in setup.py (#1240)
  > Convert to Python 3 (#1128)
  > Fix mock SonicV2Connector in python3: use decode_responses mode so caller code will be the same as python2 (#1238)
  > [tests] Do not trim from PATH if we did not append to it; Clean up/fix shebangs in scripts (#1233)
  > Updates to bgp config and show commands with BGP_INTERNAL_NEIGHBOR table (#1224)
  > [cli]: NAT show commands newline issue after migrated to Python3 (#1204)
  > [doc]: Update Command-Reference.md (#1231)
  > Added 'import sys' in feature.py file (#1232)

* src/sonic-py-swsssdk 9d9f0c6...1664be9 (2):
  > Fix: no need to decode() after redis client scan, so it will work for both python2 and python3 (#96)
  > FieldValueMap `contains`(`in`)  will also work when migrated to libswsscommon(C++ with SWIG wrapper) (#94)

- Also fix Python 3-related issues:
    - Use integer (floor) division in config_samples.py (sonic-config-engine)
    - Replace print statement with print function in eeprom.py plugin for x86_64-kvm_x86_64-r0 platform
    - Update all platform plugins to be compatible with both Python 2 and Python 3
    - Remove shebangs from plugins files which are not intended to be executable
    - Replace tabs with spaces in Python plugin files and fix alignment, because Python 3 is more strict
    - Remove trailing whitespace from plugins files
2020-11-25 10:28:36 -08:00
Shi Su
e0781f463f
[L2 switch mode] Update l2switch.j2 template (#5981)
- Why I did it
The l2switch.j2 template does not include all fields for PORT. This could be incompatible with the 201911 image or later.

- How I did it
Update l2switch.j2 template and add a unit test.
2020-11-21 11:35:35 -08:00
Lawrence Lee
3b27afe377
[minigraph.py]: Add VLAN_INTERFACE attributes for dual ToR devices (#5951)
* If a device has subtype = 'DualToR', then enable proxy ARP and gratuitous ARP.
2020-11-20 17:04:01 -08:00
Joe LeVeque
23247514f9
Fix a number of LGTM alerts (#5952)
Fix 259 alerts reported by the LGTM tool:

- 245 for Unused import
- 7 for Testing equality to None
- 5 for Duplicate key in dict literal
- 1 for Module is imported more than once
- 1 for Unused local variable
2020-11-20 10:58:48 -08:00
Joe LeVeque
7bf05f7f4f
[supervisor] Install vanilla package once again, install Python 3 version in Buster container (#5546)
**- Why I did it**

We were building a custom version of Supervisor because I had added patches to prevent hangs and crashes if the system clock ever rolled backward. Those changes were merged into the upstream Supervisor repo as of version 3.4.0 (http://supervisord.org/changes.html#id9), therefore, we should be able to simply install the vanilla package via pip. This will also allow us to easily move to Python 3, as Python 3 support was added in version 4.0.0.

**- How I did it**

- Remove Makefiles and patches for building supervisor package from source
- Install Python 3 supervisor package version 4.2.1 in Buster base container
    - Also install Python 3 version of supervisord-dependent-startup in Buster base container
- Debian package installed binary in `/usr/bin/`, but pip package installs in `/usr/local/bin/`, so rather than update all absolute paths, I changed all references to simply call `supervisord` and let the system PATH find the executable to prevent future need for changes just in case we ever need to switch back to build a Debian package, then we won't need to modify these again.
- Install Python 2 supervisor package >= 3.4.0 in Stretch and Jessie base containers
2020-11-19 23:41:32 -08:00
Lawrence Lee
261a81d379
[minigraph.py]: Parse mux cable table (#5876)
Parse device IPv6 loopback addresses when available
Add MUX_CABLE table to config DB:

'MUX_CABLE': {
	'Ethernet4': {
    	'state': 'auto',
        'server_ipv4': '10.10.10.1/32',
        'server_ipv6': 'fe80::0001/128'
	}
}
2020-11-16 13:14:34 -08:00
Lawrence Lee
bf50562670
[minigraph.py]: Parse peer switch info from minigraph (#5869)
* Create new `PEER_SWITCH` table in config DB with info from minigraph
* Add `subtype` field to `DEVICE_METADATA` table and set value to `DualToR` if device is in a dual ToR setup
2020-11-13 15:31:51 -08:00
Lawrence Lee
d0f16c0d79
Make backend device checking more robust (#5730)
Treat devices that are ToRRouters (ToRRouters and BackEndToRRouters) the same when rendering templates
 Except for BackEndToRRouters belonging to a storage cluster, since these devices have extra sub-interfaces created
Treat devices that are LeafRouters (LeafRouters and BackEndLeafRouters) the same when rendering templates

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2020-11-10 15:06:35 -08:00
Prince Sunny
1eaaf64ed2
Set preference for forced mgmt routes (#5844)
When forced mgmt routes are present, the issue fixed as part of #5754 is not complete. 
Added a preference(priority) field to forced mgmt route ip rules
2020-11-10 14:20:13 -08:00
Joe LeVeque
aee389e402
[sonic-config-engine] openconfig_acl.py: Make compliant with Python 3 (#5873)
Python 3 is more strict with `__slots__`. As per the [documentation](https://docs.python.org/3/reference/datamodel.html#notes-on-using-slots):

> \_\_slots\_\_ are implemented at the class level by creating descriptors (Implementing Descriptors) for each variable name. As a result, class attributes cannot be used to set default values for instance variables defined by \_\_slots\_\_; otherwise, the class attribute would overwrite the descriptor assignment.

This was apparently missed when making sonic-config-engine compliant with Python 3, and errors like the following would be seen:

```
tests/acl_loader_test.py:10: in <module>
    from acl_loader.main import *
acl_loader/main.py:8: in <module>
    import openconfig_acl
/usr/local/lib/python3.7/dist-packages/openconfig_acl.py:24: in <module>
    class yc_state_openconfig_acl__acl_state(PybindBase):
E   ValueError: '_pybind_generated_by' in __slots__ conflicts with class variable
```
2020-11-10 10:26:16 -08:00
Lawrence Lee
23b0e07d48
[minigraph.py]: Parse IP-in-IP tunnels from minigraph (#5742)
Take tunnel info from `<TunnelInterface>` tag in minigraph, and create tables in config_DB:

```
"TUNNEL": {
    "MUX_TUNNEL_0": {
        "tunnel_type": "IPINIP",
        "dst_ip": "26.1.1.10",
        "dscp_mode": "uniform",
        "encap_ecn_mode": "standard",
        "ecn_mode": "copy_from_outer",
        "ttl_mode": "pipe"
    }
}
```

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
2020-11-07 13:18:40 -08:00
Petro Bratash
32a832a8ac
[lldp]: Add verification IPv4 address on LLDP conf Jinja2 Template (#5699)
Fix #5812

LLDP conf Jinja2 Template does not verify IPv4 address and can use IPv6 version. This issue does not effect control LLDP daemon. Issue can be reproduced via `test_snmp_lldp` test. LLDP conf Jinja2 Template selects first item from the list of mgmt interfaces.

TESTBED_1 LLDP conf

```
# cat /etc/lldpd.conf 
configure ports eth0 lldp portidsubtype local eth0
configure system ip management pattern FC00:3::32
configure system hostname dut-1
```
TESTBED_2  LLDP conf

```
# cat /etc/lldpd.conf
configure ports eth0 lldp portidsubtype local eth0
configure system ip management pattern 10.22.24.61
configure system hostname dut-2
```
TESTBED_1  MGMT_INTERFACE

```
$ redis-cli -n 4 keys "*" | grep MGMT_INTERFACE
MGMT_INTERFACE|eth0|10.22.24.53/23
MGMT_INTERFACE|eth0|FC00:3::32/64
```
TESTBED_2  MGMT_INTERFACE

```
$ redis-cli -n 4 keys "*" | grep MGMT_INTERFACE
MGMT_INTERFACE|eth0|FC00:3::32/64
MGMT_INTERFACE|eth0|10.22.24.61/23

```

Signed-off-by: Petro Bratash <petrox.bratash@intel.com>
2020-11-07 10:30:41 -08:00
pavel-shirshov
cdcd20a7b5
[BGP]: Convert ip address to network address for the LOCAL_VLAN filter (#5832)
* [BGP]: Convert ip address to network address for the LOCAL_VLAN prefix filter
2020-11-06 17:47:08 -08:00
lguohan
c8a00eda95
[mgmt ip]: mvrf ip rule priority change to 32765 (#5754)
Fix Azure/SONiC#551

When eth0 IP address is configured, an ip rule is getting added for eth0 IP address through the interfaces.j2 template. 

This eth0 ip rule creates an issue when VRF (data VRF or management VRF) is also created in the system.
When any VRF (data VRF or management VRF) is created, a new rule is getting added automatically by kernel as "1000: from all lookup [l3mdev-table]".
This l3mdev IP rule is never getting deleted even if VRF is deleted.

Once if this l3mdev IP rule is added, if user configures IP address for the eth0 interface, interfaces.j2 adds an eth0 IP rule as "1000:from 100.104.47.74 lookup default ". Priority 1000 is automatically chosen by kernel and hence this rule gets higher priority than the already existing rule "1001:from all lookup local ".

This results in an issue "ping from console to eth0 IP does not work once if VRF is created" as explained in Issue 551.
More details and possible solutions are explained as comments in the Issue551.

This PR is to resolve the issue by always fixing the low priority 32765 for the IP rule that is created for the eth0 IP address.
Tested with various combinations of VRF creation, deletion and IP address configuration along with ping from console to eth0 IP address.

Co-authored-by: Kannan KVS <kannan_kvs@dell.com>
2020-10-31 20:45:59 -07:00
Shi Su
5ee5c13f32
Enable synchronous mode by default and add in minigraph parser (#5735) 2020-10-29 09:15:12 -07:00
judyjoseph
6088bd59de
[multi-ASIC] BGP internal neighbor table support (#5520)
* Initial commit for BGP internal neighbor table support.
  > Add new template named "internal" for the internal BGP sessions
  > Add a new table in database "BGP_INTERNAL_NEIGHBOR"
  > The internal BGP sessions will be stored in this new table "BGP_INTERNAL_NEIGHBOR"

* Changes in template generation tests with the introduction of internal neighbor template files.
2020-10-28 16:41:27 -07:00
Lawrence Lee
a639021af2
[minigraph.py]: Parse VLAN MAC address from minigraph when present (#5726) 2020-10-27 17:20:55 -07:00
Joe LeVeque
9e34003136
[sonic-config-engine] Clean up dependencies, pin versions; install Python 3 package in Buster container (#5656)
To clean up the image build procedure, and let setuptools/pip[3] implicitly install Python dependencies. Also use ipaddress package instead of ipaddr.
2020-10-26 13:48:50 -07:00
shlomibitton
e66d49a57c
[LLDP] Fix for LLDP advertisements being sent with wrong information. (#5493)
* Fix for LLDP advertisments being sent with wrong information.
Since lldpd is starting before lldpmgr, some advertisment packets might sent with default value, mac address as Port ID.
This fix hold the packets from being sent by the lldpd until all interfaces are well configured by the lldpmgrd.

Signed-off-by: Shlomi Bitton <shlomibi@nvidia.com>

* Fix comments

* Fix unit-test output caused a failure during build

* Add 'run_cmd' function and use it

* Resume lldpd even if port init timeout reached
2020-10-26 19:38:09 +02:00
Lawrence Lee
c4f9bec562
[minigraph.py]: Add support for parsing mux cable (#5676)
Find LogicalLinks in minigraph and parse the port information. A new field called `mux_cable` is added to each port's entry in the Port table in config DB:

```
PORT|Ethernet0: {
	"alias": "Ethernet4/1"
	...
	"mux_cable": "true"
}
```

If a mux cable is present on a port, the value for `mux_cable` will be `"true"`. If no mux cable is present, the attribute will either be omitted (default behavior) or set to `"false"`.
2020-10-26 08:52:20 -07:00
abdosi
ef8a48a728
Everflow and Dataacl were getting binded to Backplane port-channels. (#5691)
Issue was because we were relying on port_alias_asic_map dictionary
but that dictionary can't be used as alias name format has changed.

Fix the port alias mapping as what is needed.

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2020-10-22 10:29:46 -07:00
Stepan Blyshchak
8df0e2b6ce
[sonic-cfggen] store jinja2 cache in log level db. (#5646)
This PR makes two changes:
    - Store Jinja2 cache in LOGLEVEL DB instead of STATE DB
    - Store bytecode cache encoded in base64

Tested with the following command: "redis-dump -d 3 -k JINJA2_CACHE"

Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
2020-10-16 12:00:57 -07:00
pavel-shirshov
812e1a3489
[bgp]: Enable next-hop-tracking through default (#5600)
**- Why I did it**
FRR introduced [next hop tracking](http://docs.frrouting.org/projects/dev-guide/en/latest/next-hop-tracking.html) functionality. 
That functionality requires resolving BGP neighbors before setting BGP connection (or explicit ebgp-multihop command). Sometimes (BGP MONITORS) our neighbors are not directly connected and sessions are IBGP. In this case current configuration prevents FRR to establish BGP connections.  Reason would be "waiting for NHT". To fix that we need either add static routes for each not-directly connected ibgp neighbor, or enable command `ip nht resolve-via-default`

**- How I did it**
Put `ip nht resolve-via-default` into the config

**- How to verify it**
Build an image. Enable BGP_MONITOR entry and check that entry is Established or Connecting in FRR

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-10-13 22:21:28 -07:00
anish-n
42d9a44e05
Minigraph resource type changes (#5198)
* Parse sub_role from minigraph into DEVICE_METADATA
* Change minigraph sub_role to resource_type
2020-10-08 15:45:21 -07:00
Mahesh Maddikayala
744612d269
[ECMP][Multi-ASIC] Have different ECMP seed value on each ASIC (#5357)
* Calculate ECMP hash seed based on ASIC ID on multi ASIC platform. Each ASIC will have a unique ECMP hash seed value.
2020-10-08 09:05:37 -07:00
Lawrence Lee
8c344095a8
[docker-orchagent]: Add NDP Proxy Daemon (#5517)
* Install ndppd during image build, and copy config files to image
* Configure proxy settings based on config DB at container start
* Pipe ndppd output to logger inside container to log output in syslog
2020-10-05 08:48:13 -07:00
Qi Luo
30f5557d36
Fix generate_l2_config: don't override hostname or device role (ToRRouter) (#5510)
* Fix generate_l2_config: don't override hostname because sonic-cfggen may not read from Redis. Fix test_l2switch_template test case to test preset l2 feature.
* Improve test script: compare json files with sort_keys
* Revert changes on sample_output
* Remove members field in VLAN section. Fix test assertTrue statement.
2020-10-01 23:24:35 -07:00
Tamer Ahmed
57a584997a [cfggen] Iterative Version Of Deep Update
Avoiding recursive update of maps as it consumes stack frames. This
PR introduces iterative version of deep_update method.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Tamer Ahmed
110f7b7817 [cfggen] Build Python 2 And Python 3 Wheel Packages
This builds Python 2&3 wheel packages for sonic-cfggen script.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Tamer Ahmed
99ffce6b57 [cfggen] Move Python 2 Test Data to Py2 Folder
Rendering templates show different orders when rendered using Python
2 vs Python 3. This PR prepare for Python 3 packaging by creating
new dir 'py2' for Python 2 rendered test cases.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Tamer Ahmed
6754635010 [cfggen] Make Jinja2 Template Python 3 Compatible
Jinja2 templates rendered using Python 3 interpreter, are required
to conform with Python 3 new semantics.

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Tamer Ahmed
9122d16dd2 [cfggen] Remove NatSorted
Natural sorting of SONiC config gen output consumes lot of CPU cycles.
The sole use of natsorted was to make test comparison easier and so,
the natsorting logic is now relocated to the test suite. As a result
sonic-cfggen gained nearly 1 sec per call since we no longer import
natsorted module!

singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-30 07:07:43 -07:00
Sumukha Tumkur Vani
b5bcfef013
Update conf DB with CA cert & rename ca_crt field (#5448) 2020-09-25 09:20:09 -07:00
Samuel Angebault
9bf4b0a93e
[baseimage]: Change the loopback mask from /8 to /16 (#5353)
As per the VOQ HLDs, internal networking between the linecards and supervisor is required within a chassis.
Allocating 127.X/16 subnets for private communication within a chassis is a good candidate.
It doesn't require any external IP allocation as well as ensure that the traffic will not leave the chassis.

References:
https://github.com/Azure/SONiC/pull/622
https://github.com/Azure/SONiC/pull/639

**- How I did it**

Changed the `interfaces.j2` file to add `127.0.0.1/16` as the `lo` ip address.
Then once the interface is up, the post-up command removes the `127.0.0.1/8` ip address.
The order in which the netmask change is made matters for `127.0.0.1` to be reachable at all times.

**- How to verify it**

```
root@sonic:~# ip address show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/16 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
```

Co-authored-by: Baptiste Covolato <baptiste@arista.com>
2020-09-15 15:29:48 -07:00
Tamer Ahmed
1bf6fdc6d2
[dhcpmon] Monitor Mgmt Interface For DHCP Packets (#5317)
When BGP routes are missing, DHCP packets get relayed over mgmt
interface. This results in dhcpmon alerting that DHCP packets are
not being relayed. This is PR include mgmt interface as uplink
device, and so, if DHCP packet gets relayed over mgmt interface,
regular dhcpmon alert will not be issues. Instead, dhcpmon will
check the mgmt interface counts and issue a separate alert regarding
packets travelling through mgmt network.

In addition, this PR includes the following enhancements:
1. Add SIGUSR1 handler that prints out current packet counts
2. Increase alert grace window to 3 minutes from currently 2 minutes
3. Time is now computed more accurately
4. Print vlan name before counters

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-09-09 18:37:01 -07:00
Joe LeVeque
5b3b4804ad
[dockers][supervisor] Increase event buffer size for dependent-startup (#5247)
When stopping the swss, pmon or bgp containers, log messages like the following can be seen:

```
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,061 ERRO pool dependent-startup event buffer overflowed, discarding event 34
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,063 ERRO pool dependent-startup event buffer overflowed, discarding event 35
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,064 ERRO pool dependent-startup event buffer overflowed, discarding event 36
Aug 23 22:50:43.789760 sonic-dut INFO swss#supervisord 2020-08-23 22:50:10,066 ERRO pool dependent-startup event buffer overflowed, discarding event 37
```

This is due to the number of programs in the container managed by supervisor, all generating events at the same time. The default event queue buffer size in supervisor is 10. This patch increases that value in all containers in order to eliminate these errors. As more programs are added to the containers, we may need to further adjust these values. I increased all buffer sizes to 25 except for containers with more programs or templated supervisor.conf files which allow for a variable number of programs. In these cases I increased the buffer size to 50. One final exception is the swss container, where the buffer fills up to ~50, so I increased this buffer to 100.

Resolves https://github.com/Azure/sonic-buildimage/issues/5241
2020-09-08 23:36:38 -07:00
vdahiya12
679d8df36f
[sonic-utilities][sonic-py-common] Move logic to get port config file path to sonic-py-common and update sonic-utilities to comply (#5264)
* [sonic-utilities]update submodule with fix

This PR addresses fixes in sonic-py-common to imitate the behavior inside
sonic-cfggen. Essentially this is a fix for accessing the port-config file.
First check if there is a platform.json file for config generation
and then for legacy port_config.ini.
Also updating the sub-module sonic-utilities.

Fix pfcwd stats crash with invalid queue name (#1077)
[show][bgp]Display the Total number of neighbors in the show ip bgp(v6) summary. (#1079)
[config] Update SONiC Environment Vars When Loading Minigraph (#1073)
Multi asic platform changes for interface, portchannel commands (#878)
Update Command-Reference.md (#1075)
[filter-fdb] Fix Filter FDB With IPv6 Present in Config DB (#1059)
[config] Remove _get_breakout_cfg_file_name helper function (#1069)
[SHOW][BGP] support show ip(v6) bgp summary for multi asic platform (#1064)
[fanshow] Display other fan status, such as Updating (#1014)
Add ip_prefix len based on proxy_arp status (#1046)
Enable the platform specific ssd firmware upgrade during reboot (#954)
[show][cli[show interface portchannel support for Multi ASIC (#1005)
support show interface commands for multi ASIC platforms (#1006)

Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
2020-09-04 10:19:12 -07:00
arlakshm
17e78715ae
[Multi-ASIC]:Update the template to add ipinip entry for Loopback4096 (#5235)
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
The following changes are done. 
- Multi asic platform have 2 Loopback interfaces, Loopback0 and Loopback4096. IPinIP decap entries need to be added for both of them. Update the ipinip.json.j2 template to add decap entries for Loopback4096.
- Add corressponding unit test
2020-08-31 17:35:48 -07:00
Tamer Ahmed
a8016a6987
[cfggen] Use Redis Pipeline (#5250)
This PR enables cfggen to readr/write from Redis DB using pipelines.
Pipelines enables batch read/write from/to Redis DB.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-26 15:05:54 -07:00
Joe LeVeque
ce2c0781ab
[sonic-config-engine] Update .gitignore (#5223)
- Ignore directories generated by building Python wheel package
- Move all sonic-config-engine ignores from the root .gitignore to src/sonic-config-engine/.gitignore
2020-08-20 11:09:10 -07:00
Tamer Ahmed
ec11308a66
[cfggen] Conform With Python 3 Syntax (#5154)
Preparing sonic-cfggen for migration to Python 3.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-19 09:29:40 -07:00
Tamer Ahmed
f840013ed9
[cfggen] Allow Write To Redis DB With Template/Batch Mode (#5203)
Argument to write to config-db is not allowed when using template.
This PR allows cfggen to write to redis db when using template
mode.

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-18 18:24:36 -07:00
judyjoseph
52fe7ae365
[sonic-py-common] Add interface utilities (#5113)
* Add sonic_interface.py in sonic-py-common for sonic interface utilities to keep this SONIC PREFIX naming convention in one place in py-common and all modules/applications use the functions defined here.
2020-08-18 10:20:26 -07:00
Joe LeVeque
fa4af0d372
[sonic-config-engine] Ensure all test output files are ignored (#5197)
Also align the naming and cleanup of generated output files in test_cfggen.py with the rest of the tests.
2020-08-15 23:09:11 -07:00
arlakshm
6c895513ac
Common functions for Multi ASIC (#4973)
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan arlakshm@microsoft.com

The following common APIs are added for multi ASIC
- an  API to check if a given port is a internal or external port
- an  API to check if a given port-channel is internal or external
- an API to check if a bgp-session is internal or external
- an  API to connect to the config and other dbs in the a given namespace
- added common APIs to the sonic_py_common library.
- update the sample port-config.ini with role column and add corresponding test to verify the ports configuration is - generated properly.
2020-08-14 07:36:00 -07:00
Tamer Ahmed
f9edf6e5cc
[cfggen] Extend Template Argument to Support Batch Mode (#4941)
Calls to cfggen take considerable time. With batch mode, we will have the ability
to reduce number of calls from services.

Example of the batch mode command:
sonic-cfggen -t template-1.j2 -t template-2.j2,config-db -t template-3.j2,config-db -t template-4.j2,file1 -t template-5.j2,file2 --write-to-db.

template-1.j2 will be rendered to stdout since it is missing the dest part. stdout is default
config-db is a special keyword that will inject the rendered template into internal data structure. The internal data structure gets written to redis-db with --write-to-db switch. In the case the user would like to write to a file named config-db, it could be given as /config-db or ./config-db

signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-08-12 15:13:06 -07:00
Sumukha Tumkur Vani
9c364fbf93
Bring up all neighbor ports in DeviceInterfaceLinks (#5071)
* Bring up FPGA ports and test it

* Bring up those ports in neighbors dict

* Revert delete of a line

* Add test

* change code comment

* Change test name

* Revert submodule update
2020-08-11 07:29:33 -07:00
Joe LeVeque
966ac85e24
[sonic-config-engine] Remove sonic_device_util.py (#5130)
sonic_device_util.py has been deprecated in favor of the sonic-py-common package. All related functionality has been moved there.
2020-08-09 21:26:54 -07:00
Joe LeVeque
3b89e5d467
[Python] Migrate applications/scripts to import sonic-py-common package (#5043)
As part of consolidating all common Python-based functionality into the new sonic-py-common package, this pull request:
1. Redirects all Python applications/scripts in sonic-buildimage repo which previously imported sonic_device_util or sonic_daemon_base to instead import sonic-py-common, which was added in https://github.com/Azure/sonic-buildimage/pull/5003
2. Replaces all calls to `sonic_device_util.get_platform_info()` to instead call `sonic_py_common.get_platform()` and removes any calls to `sonic_device_util.get_machine_info()` which are no longer necessary (i.e., those which were only used to pass the results to `sonic_device_util.get_platform_info()`.
3. Removes unused imports to the now-deprecated sonic-daemon-base package and sonic_device_util.py module

This is the next step toward resolving https://github.com/Azure/sonic-buildimage/issues/4999

Also reverted my previous change in which device_info.get_platform() would first try obtaining the platform ID string from Config DB and fall back to gathering it from machine.conf upon failure because this function is called by sonic-cfggen before the data is in the DB, in which case, the db_connect() call will hang indefinitely, which was not the behavior I expected. As of now, the function will always reference machine.conf.
2020-08-03 11:43:12 -07:00
Mahesh Maddikayala
ee4197e9f8
+ Modified buffer config template to include internal ASIC with 5m ca… (#4959)
+ Modified buffer config template to include internal ASIC with 5m cable length
+ added unit test to verify the changes.
2020-07-27 10:57:07 -07:00
anish-n
da017f4ec9
[bgpcfgd]: Add Vlan prefix list to the FRR templates (#5005)
add the Vlan prefix list to the FRR templates
2020-07-21 19:26:19 -07:00
arlakshm
97fa2c087b "[config]: Multi ASIC loopback changes (#4895)
Resubmitting the changes for (#4825) with fixes for sonic-bgpcdgd test failures
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
2020-07-12 18:08:51 +00:00
Guohan Lu
f8da3e4c69 Revert "[config]: Loopback Interface changes for multi ASIC devices (#4825)"
This reverts commit cae65a451c.
2020-07-12 18:08:51 +00:00
arlakshm
002335a3d5 [config]: Loopback Interface changes for multi ASIC devices (#4825)
* Loopback IP changes for multi ASIC devices
multi ASIC will have 2 Loopback Interfaces

- Loopback0 has globally unique IP address, which is advertised by the multi ASIC device to its peers.
This way all the external devices will see this device as a single device.
- Loopback4096 is assigned an IP address which has a scope is within the device. Each ASIC has a different ip address for Loopback4096. This ip address will be used as Router-Id by the bgp instance on multi ASIC devices.

This PR implements this change for multi ASIC devices

Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
2020-07-12 18:08:51 +00:00
pavel-shirshov
2b137fb540 Tests of FRR templates which rendered by sonic-cfggen (#4875)
* Tests of FRR templates which rendered by sonic-cfggen
2020-07-12 18:08:51 +00:00
pavel-shirshov
7d0ea7383d [pfx_filter]: Add a prefix mask by default in pfx_filter, when there is no one (#4860)
If some table with a list of tuples (interface name, ip prefix) has ip prefixes without a mask length, it will cause issues in SONiC. For example quagga and frr will treat ipv4 address without a mask, so "10.20.30.40" address will be treated as "10.0.0.0/8", which is dangerous.

The fix here is that when pfx_filter get a tuple (interface name, ip prefix), where the ip prefix doesn't have prefix mask length, add a mask by default: "/32 for ipv4 addresses, /128 for ipv6 addresses".

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>
2020-07-12 18:08:51 +00:00
kktheballer
8339d5f4b1
[minigraph.py]: Add Tagged Vlan support in Minigraph (#4731)
To enable tagged vlan support by minigraph parser. This enables us to generate a config_db file that will enable SONiC device to operate using tagged and untagged vlan.
2020-06-23 16:36:47 -07:00
Sangita Maity
7f9523887d
[buildImage] Add support of platform.json parsing to portconfig.py file (#3909)
**- What I did**
Add support of **platform.json** parsing to **portconfig.py** file which is being used by **_sonic-cfggen_** and ***minigraph.py*** file under ***src/sonic-config-engine*** folder to get port config via get_port_config  function.

**- How I did it**

1.  **portconfig.py** file will first check whether the **platform.json** file is there or not. if not then whether port_config.ini file is there or not. Modified **get_port_config_file_name** for this purpose.
2. Added two separate functions i.e. **parse_platform_json_file**  to get port attributes from **platform.json** and **gen_port_config** to generate port attributes.
3. Added another two functions i.e  get_breakout_mode parse_breakout_mode to get breakout mode and parse breakout mode from platform.json respectively.

**- How to verify it**

rebuilt "sonic_config_engine-1.0" wheel package with all the test cases.All the below-mentioned test cases passed.

```
# Check whether all interfaces present or not as per platform.json
def test_platform_json_interfaces_keys(self):
       
# Check specific Interface with it's proper configuration as per platform.json
def test_platform_json_specific_ethernet_interfaces(self):

# Check all Interface with it's proper configuration as per platform.json
def test_platform_json_all_ethernet_interfaces(self):
```

Signed-off-by: Sangita Maity <sangitamaity0211@gmail.com>
2020-06-18 16:26:09 -07:00
Qi Luo
edf3160224
Fix bug: check port alias even when port_config_file parameter is not provided (#4787) 2020-06-17 12:12:07 -07:00
Joe LeVeque
9b27efdcc2
[dockers] Rename 'docker-lldp-sv2' to 'docker-lldp' (#4700)
The -sv2 suffix was used to differentiate SNMP Dockers when we transitioned from "SONiCv1" to "SONiCv2", about four years ago. The old Docker materials were removed long ago; there is no need to keep this suffix. Removing it aligns the name with all the other Dockers.
2020-06-09 09:09:56 -07:00
Sabareesh-Kumar-Anandan
17904969b6 [sonic-config-engine][marvell] read mac address fix (#4649)
Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>
2020-05-27 02:02:19 -07:00
Sumukha Tumkur Vani
40481f01af
Read cloudtype info from minigraph (#4642) 2020-05-26 10:59:56 -07:00
Guohan Lu
2e42a4ba0f [docker-dhcp-relay]: use service dependency in supervisord to start services 2020-05-22 11:01:28 -07:00
arlakshm
eb97dccb88
[config]: Fix the device type and internal bgp session status for multi NPU platforms (#4600)
* The following changes for multi-npu platforms are done
- Set the type in device_metadata for asic configuration to be same as host
- Set the admin-status of internal bgp sessions as up
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
2020-05-20 18:10:05 -07:00
abdosi
0542afb619
Moved utility functions for multi-npu platforms from sonic-utilities to sonic_device_util.py (#4559)
* Moved utility functions for multi-npu platforms from
sonic-utilities config/main.py to here so that they can be used
any module

* Fix the issue with test run during compilation with acl-uploader
PR#908 of sonic-utilities.

* Fix get_num_npu as it was retuning string and not int

* Address Review Comments

* Address Review Comments
2020-05-13 10:24:16 -07:00
abdosi
e3be45f2da
[sonic-buildimage] updated minigraph for ACL Table data and ACL Interface Binding for Multi-NPU platforms (#4491)
* [sonic-buildimage] updated minigraph for ACL Table data and ACL Interface
binding update for multu-npu platform based on subrole as "Frontend" or
"Backend". For backend npu no ACL table is associated. For frontend npu
only front-panel interface are associated.

Updated with test case and fix typo in sample-mingraph for npu

Address Review comments

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>

* Fixed the logic as per preview comment. Interface Filter logic
only applies to Everflow/Mirror tables.

* Address Review Comments.
2020-05-12 14:49:15 -07:00
abdosi
a96f9ecee9
Changes for LLDP docker to support multi-npu platforms (#4530)
* Changes for LLDP for Multi NPU Platoforms:-
a) Enable LLDP for Host namespace for Management Port
b) Make sure Management IP is avaliable in per asic namespace
   needed for LLDP Chassis configuration
c) Make sure chassis mac-address is correct in per asic namespace
d) Do not run lldp on eth0 of per asic namespace and avoid chassis
   configuration for same
e) Use Linux hostname instead from Device Metadata for lldp chassis
   configuration since in multi-npu platforms device metadata hostname
   will be differnt

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>

* Address Review Comment with following changes:
a) Use Device Metadata hostname even in per namespace conatiner.
   updated minigraph parsing for same to have hostname as system
   hostname and add new key for asic name

b) Minigraph changes to have MGMT_INTERFACE Key in per asic/namespace
   config also as needed for LLDP for setting chassis management IP.

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>

* Address Review Comments
2020-05-11 11:05:44 -07:00
Neetha John
286aa35ac6
[qos]: Alpha and ECN settings change for Th (#4564)
Dynamic threshold setting changed to 0 and WRED profile green min threshold set to 250000 for Tomahawk devices

Changed the dynamic threshold settings in pg_profile_lookup.ini
Added a macro for WRED profiles in qos.json.j2 for Tomahawk devices
Necessary changes made in qos.config.j2 to use the macro if present

Signed-off-by: Neetha John <nejo@microsoft.com>
2020-05-09 11:21:18 -07:00
judyjoseph
acf465b43b
Multi DB with namespace support, Introducing the database_global.json… (#4477)
* Multi DB with namespace support, Introducing the database_global.json file
for supporting accessing DB's in other namespaces for service running in
linux host

* Updates based on comments

* Adding the j2 templates for database_config and database_global files.

* Updating to retrieve the redis DIR's to be mounted from database_global.json file.

* Additional check to see if asic.conf file exists before sourcing it.

* Updates based on PR comments discussion.

* Review comments update

* Updates to the argument "-n" for namespace used in both context of parsing minigraph and multi DB access.

* Update with the attribute "persistence_for_warm_boot" that was added to database_config.json file earlier.

* Removing the database_config.json file to avioid confusion in future.
We use the database_config.json.j2 file to generate database_config.json files dynamically.

* Update the comments for sudo usage in docker_image_ctrl.j2

* Update with the new logic in PING PONG tests using sonic-db-cli. With this we wait till the
PONG response is received when redis server is up.

* Similar changes in swss and syncd scripts for the PING tests with sonic-db-cli

* Updated with a missing , in the database_config.json.j2 file, Do pip install of j2cli in docker-base-buster.
2020-05-08 21:24:05 -07:00
Qi Luo
d0099ed43e
[minigraph] Support FECDisabled in minigraph parser (#4556)
Signed-off-by: Qi Luo <qiluo-msft@users.noreply.github.com>
2020-05-08 19:47:05 -07:00
SuvarnaMeenakshi
7f14acbf03
[minigraph.py]: Fix undefined variable error. (#4555)
Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
2020-05-07 23:17:22 -07:00
Danny Allen
3ae5945cfc
[minigraph] Add tags for egress mirror tables (#4526)
Signed-off-by: Danny Allen <daall@microsoft.com>
2020-05-06 15:43:29 -07:00
SuvarnaMeenakshi
8ac1c60b2a
[config engine] Parser changes to support parsing of multi-asic device minigraph (#4222)
- Changes to minigraph.py to parse minigraph.xml of a multi asic platform 
- Changes to portconfig.py to parse additional column "asic_port_name" in
port_config.ini
- Add a new option -n to sonic-cfggen for multi asic platforms
- Add unit tests for config generation for multi asic platforms

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>
2020-05-04 16:15:15 -07:00
Sumukha Tumkur Vani
e1ba5b0f5f
[docker-restapi]: add restapi config to config Db (#4495) 2020-04-30 00:25:49 -07:00
Sumukha Tumkur Vani
72af555645
Parse Device region field in Minigraph (#4486) 2020-04-27 21:48:09 -07:00
pavel-shirshov
057ced0391
[bgpcfgd]: Split one bgp mega-template to chunks. (#4143)
The one big bgp configuration template was splitted into chunks.

Currently we have three types of bgp neighbor peers:

general bgp peers. They are represented by CONFIG_DB::BGP_NEIGHBOR table entries
dynamic bgp peers. They are represented by CONFIG_DB::BGP_PEER_RANGE table entries
monitors bgp peers. They are represented by CONFIG_DB::BGP_MONITORS table entries
This PR introduces three templates for each peer type:

bgp policies: represent policieas that will be applied to the bgp peer-group (ip prefix-lists, route-maps, etc)
bgp peer-group: represent bgp peer group which has common configuration for the bgp peer type and uses bgp routing policy from the previous item
bgp peer-group instance: represent bgp configuration, which will be used to instatiate a bgp peer-group for the bgp peer-type. Usually this one is simple, consist of the referral to the bgp peer-group, bgp peer description and bgp peer ip address.
This PR redefined constant.yml file. Now this file has a setting for to use or don't use bgp_neighbor metadata. This file has more parameters for now, which are not used. They will be used in the next iteration of bgpcfgd.

Currently all tests have been disabled. I'm going to create next PR with the tests right after this PR is merged.

I'm going to introduce better bgpcfgd in a short time. It will include support of dynamic changes for the templates.

FIX:: #4231
2020-04-23 09:42:22 -07:00
lguohan
46fad0e0c6
Merge pull request #4405 from lguohan/buster
[baseimage]: upgrade base image to debian buster

bring up the base image to debian buster 4.19 kernel.

using the merge commits to preserve the individual commits to better track the history.
2020-04-20 15:32:17 -07:00
taochengyi
744d33d502
[devices]: Centec x86 goldengate platforms and devices update (#4248)
* Centec x86 goldengate update (#8)
2020-04-17 00:38:48 -07:00
bsun-sudo
2a237c57e6 [mgmt-vrf]: mgmt vrf related change for Buster (#53)
Co-authored-by: Bing Sun <Bing_Sun@dell.com>
2020-04-17 04:51:51 +00:00
noaOrMlnx
1a0067ede0
Bug fix: #4011 #4012 - "show run acl", "show run interfaces" - traceback when no ACL_RULE/INTERFACE is present. (#4226)
* Fix bugs - "show run acl/interfaces" returns traceback when no ACL_RULE/INTERFACE is present in configuration
- Change logic in main to take data only if the var_json appears in DB.
- Add unit testing for bug fix & fix comments

Signed-off-by: Noa Or <noaor@mellanox.com>
2020-03-30 09:24:16 -07:00
lguohan
a0d213cc37
[telemetry]: move default certs location from device metadata to telemetry (#4307)
maintains backward compatibility to search original x509 location
when telemetry table does not have certs

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-03-24 00:04:36 -07:00
arheneus@marvell.com
94162679bb
[sonic-cfggen] MGMT Interface configuration (#4280)
update network and broadcast address in /etc/network/interfaces

Before:
root@sonic:/home/admin# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.28.32.73  netmask 255.255.254.0  broadcast 0.0.0.0 <<<<<

After:
root@sonic:~# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.28.32.73  netmask 255.255.254.0  broadcast 10.28.33.255 <<<<<

Signed-off-by: Antony Rheneus <arheneus@marvell.com>
2020-03-21 14:25:19 -07:00
Joe LeVeque
e05cd1135e
[minigraph.py] Add support for 'OutAcl' keyword and attaching ACLs to VLAN interfaces (#4229)
- Support parsing egress ACLs from minigraph file specified by the "OutAcl" element
- Support attaching ACLs to VLAN interfaces
2020-03-19 23:31:25 -07:00
Joe LeVeque
d19bba0ec0
[sonic-cfggen] Load JSON files before minigraph file (#4202)
If sonic-cfggen is passed the -m argument (to load the minigraph file) along with one or more -j <json_file> arguments, load the JSON files before loading the minigraph file.

This ensures that the init_cfg.json file is loaded before the minigraph, therefore the minigraph can override any default configuration options specified in init_cfg.json. Currently, the behavior is reversed.

Note: This is not an issue if loading loading multiple JSON files, because sonic-cfggen loads them in the left-to-right order they were specified on the command line, therefore providing flexibility for loading JSON files in a specific order. As long as init_cfg.json is specified before config_db.json, the values specified in config_db.json will take precedence.
2020-02-27 22:08:52 -08:00
yozhao101
91e5fb5602
[Service] Enable/disable container auto-restart based on configuration. (#4073) 2020-02-07 12:34:07 -08:00
Dong Zhang
5057ac3122 [MultiDB] (./dockers dir) : replace redis-cli with sonic-db-cli and use new DBConnector (#3923)
* [MultiDB] (./dockers dirs): replace redis-cli with sonic-db-cli and use new DBConnector

* remove unnecessary quota

* update typo
2020-01-22 11:27:21 -08:00
kannankvs
d150721fa1 modified down rules to pre-down rules to ensure that default route is… (#3853)
* modified down rules to pre-down rules to ensure that default route is deleted just before interface is made down
2020-01-16 19:36:49 -08:00
Tamer Ahmed
2658ab8add
[dhcp-relay]: Add DHCP Relay Monitor (#3886)
DHCP relay MONitor (dhcpmon) keeps track of DORA messages. If DHCP Relay
is detected to be not forwarding DORA message, dhcpmon will log such event
to syslog. Under the hood dhcpmon keeps counts of clients DR messages,
forwarded DR messages, DHCP server OA messages, and forwarded OA messages.
dhcpmon will check every 12 sec (configurable) if counts are monotonically
increasing and record snapshot of those counters. dhcpmon will report
discrepancies when detected between current counters and snapshot counters.

pull-request: https://github.com/Azure/sonic-buildimage/pull/3886
signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
2020-01-07 17:48:03 -08:00
Mykola F
fbcaaa498e [config engine] For l2 preset, use admin_status: up by default (#3902)
Updated the l2 preset config generator to specify 'admin_status': 'up' for every port by default.
The use of setdefault() ensures that if port already has some admin_status set, the original value will not be overwritten.

Signed-off-by: Mykola Faryma <mykolaf@mellanox.com>
2019-12-13 14:30:52 -08:00
pra-moh
45e13b9929 [sonic-telemetry]: correcting server key name in configdb (#3809) 2019-11-23 12:31:22 -08:00
pavel-shirshov
d44cc30191 [docker-fpm-frr]: Enable sending ipv6 prefixes over ipv4 BGPMON session (#3799)
* Enable ipv6 prefixes over ipv4 BGPMON session
2019-11-22 20:43:44 -08:00
Tyler Li
643ef82a2f [config] supplement loopback interface in mimigraph (#3792) 2019-11-21 17:49:31 -08:00
pra-moh
bad57cc51a [minigraph.py] Generate items needed by streaming telemetry service (#3759) 2019-11-21 12:54:28 -08:00
Tyler Li
c07ae3b16f Loopback ip addresses move to intfmgrd for supporting VRF 2019-11-10 02:27:33 -08:00
yozhao101
ed79f54569 [Services] Restart DHCP-Relay service upon unexpected critical process exit. (#3667)
Signed-off-by: Yong Zhao <yozhao@microsoft.com>
2019-11-05 18:32:14 -08:00
pavel-shirshov
815e2ef6f4
[minigraph.py]: Use default namespace for <Address> (#3695)
* [minigraph.py]: Use default namespace for <Address>
2019-11-01 07:36:19 -07:00
Stepan Blyshchak
064689d442 [sonic-cfggen] optimize sonic-cfggen startup (#3658)
* [sonic-cfggen] optimize execution time

a lot of template rendering causes switch to start longer because jinja2
needs to parse them. Introducing RedisBytecodeCache to store parsed buckets of
internal template bytecode to speedup same template rendering during start

* [sonic-cfggen] do lazy regexp compilation to speedup sonic-cfggen

* [sonic-cfggen] address pep8 related comments

Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
2019-10-31 09:17:29 -07:00
pra-moh
040672c93f [minigraph.py] Enable telemetry service by default (#3611) 2019-10-24 18:16:06 -07:00
pavel-shirshov
8457248d01 [bgpcfgd]: Split default bgp config into main config and peer template (#3627)
Now it's possible to add and remove peers based on ConfigDB

- What I did
Fixed functionality for dynamically adding/removing static bgp peers.

- How I did it

Split the bgp default template on bgp part and bgp peer part
Changed bgpcfgd to use 1.

- How to verify it

Build an image and run on your DUT
2019-10-24 07:35:14 -07:00
Wenda Ni
be52977aca Revert "Configure buffer profile to all ports (#3561)" (#3628)
This reverts commit 8861cbe98e.
2019-10-18 09:14:39 -07:00
Danny Allen
c1848153c3
[minigraph.py] Update minigraph parsing logic to include only active ports for mirror tables (#3592)
* Update minigraph.py to filter out front-panel ports that are not active
* Update cfggen tests to reflect new behavior

Signed-off-by: Danny Allen <daall@microsoft.com>

* Incorporate PR comments
- Update t0 tests to include additional device neighbors
- Refactor xml parsing logic
2019-10-17 16:29:07 -07:00
Wenda Ni
d1c4deb811 [minigraph]; Parse backend switches to vlan sub port interface (#3413)
* Parse backend switches to vlan sub port interface

Signed-off-by: Wenda Ni <wenni@microsoft.com>
2019-10-17 09:47:46 -07:00
fk410167
bdf7d24962 [sonic-cfggen]: yaml.load() is deprecated in latest versions of PyYAML module (#3526)
From 5.1 version of PyYAML python module, yaml.load() API is deprecated. Code should be compatible to support both the versions, else error/warning messages are seen like below,

2019-07-02 08:25:35,284 – INFO: [D1] /usr/local/lib/python2.7/dist-packages/sonic_device_util.py:44: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
2019-10-11 13:25:45 -07:00
pavel-shirshov
e69cb73fc7 [frr]: Implement BGP_MONITORS support (#3545)
Implement support of BGP Monitors

Add changes into minigraph to extract BGP Monitors information. Add new templates for FRR.
2019-10-07 16:05:07 -07:00
Wenda Ni
8861cbe98e
Configure buffer profile to all ports (#3561)
Signed-off-by: Wenda Ni <wenni@microsoft.com>
2019-10-04 11:20:57 -07:00
Wenda Ni
cf0465bf53
Adopt per-port buffer and qos profile (#3542)
Signed-off-by: Wenda Ni <wenni@microsoft.com>
2019-10-02 13:01:16 -07:00
Wenda Ni
7b0a5ba6ae Remove the divide by 4 operation to the under the hood SAI (#1532)
* Remove the divide by 4 operation to the under the hood SAI

This is to avoid the need and thus the confusion for application program to know
the mmu internal architecture

This change must have support from SAI change to reach the correct
config

Signed-off-by: Wenda <wenni@microsoft.com>

* Relegate the divide by 4 operation to the under the hood SAI for egress
lossless pool

Extend to 7060 and 6100

Signed-off-by: Wenda <wenni@microsoft.com>

* Add more TH/TH2 hwskus

Signed-off-by: Wenda Ni <wenni@microsoft.com>

* Update config test

Signed-off-by: Wenda Ni <wenni@microsoft.com>

* Add TH2 ingress lossy profile

Signed-off-by: Wenda Ni <wenni@microsoft.com>

* Move the divide by 4 operation to SAI internal

Signed-off-by: Wenda Ni <wenni@microsoft.com>

* [bcm SAI] Upgrade Broadcom SAI to version 3.5.3.1-15

- Broadcom SAI 3.5 GA release 20190924.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-09-25 15:57:07 -07:00
Harish Venkatraman
31d1a76197 [baseimage]: Management vrf ntp support (#3204)
This commit adds NTP support for management VRF using L3mdev. Config vrf add
mgmt will enable management VRF, enslave the eth0 device to the master device
mgmt, stop ntp service in default, restart interfaces-configs and restart ntp
service in mgmt-vrf context. Requirement and design are covered in mgmt vrf
design document.

Signed-off-by: Harish Venkatraman <harish_venkatraman@dell.com>
2019-09-16 10:21:06 -07:00
Joe LeVeque
0e62280725
[dhcp_relay] Properly wait for routed interfaces to be ready before starting relay agent (#3441) 2019-09-12 10:57:08 -07:00
Shuotian Cheng
063567c9aa [minigraph]: Ignore mirror session portion in minigraph (#3410)
This piece of information is currently not used. Revert this
pull request in the future to add back the default mirror
session information into the configuration database.

Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
2019-09-05 16:35:30 -07:00
shikenghua
bf08a2caf3 [docker-fpm-frr]: Generate separated staticd.conf for staticd (#3306)
* [docker-fpm-frr]: Generate separated staticd.conf for staticd

Generate staticd.conf by templates/staticd.conf.j2 with config DB data

* [docker-fpm-frr]: Remove default_route block from zebra.conf.j2

default_route block already moved to staticd.conf.j2

* [docker-fpm-frr]: Add test for staticd.conf.j2 template

* Add test for staticd.conf.j2 template

* Correct the sample output of zebra.conf.j2 template

* Fix a typo in test_zebra_frr

* [docker-fpm-frr]: Fix test_j2files test errors

* Fix test errors in test_j2files.py and test_j2files_t2_chassis_fe.py

* Fix typo in test_j2files_t2_chassis_fe.py
2019-08-16 14:29:42 -07:00
wangshengjun
70febea0d9 [dhcp_relay] fix the compile error introduced by the fixed issue of '#3316', change the sample_output file of 'wait_for_intf.sh' (#3325)
Signed-off-by: wangshengjun <wangshengjun@asterfusion.com>
2019-08-10 12:26:13 -07:00
arheneus@marvell.com
17b5fd2b5f [sonic-cfg] [Marvell] read system mac address from eeprom (#3309)
* [sonic-cfg] [Marvell] get_system_mac() to fetch valid mac address from Eeprom else
get mac address from eth0

* [platform] Marvell mac address fetch for different platform based on board

Signed-off-by: Antony Rheneus <arheneus@marvell.com>
2019-08-08 11:04:06 -07:00
Vasant Patil
63b5e0e14c [sonic-cfggen]: add --key option to show a specific key (#3248)
* Adding --key option to sonic-cfggen script. This will help to display config DB with more granularity.

Signed-off-by: Vasant Patil <vapatil@linkedin.com>
2019-08-06 08:04:33 -07:00
Wei Bai
2081c2e388 [sonic-cfggen]: Fix a wrong attribute in VXLAN_TUNNEL (#3225)
Change 'source_ip' to 'src_ip'
2019-07-26 23:47:04 -07:00
pavel-shirshov
b4517b9591
[bgp]: Implement Universal Traffic Shift for SONiC (#3209)
* [bgp]: Implement Universal Traffic Shift for SONiC

* Fix issue with ipv6 loopback match

* Add tests
2019-07-26 14:31:56 -07:00
Harish Venkatraman
3e69427ac0 [baseimage] management VRF support via l3mdev (#2585)
This commit adds support for New feature management VRF using L3mdev.  Added
commands to enable/disable management VRF. Config vrf add mgmt will enable
management VRF, enslave the eth0 device to the master device mgmt and restart
interfaces-configs in mgmt-vrf context.

management interface (eth0) can be configured using config interface eth0 ip
add command and removed using config interface eth0 ip remove command.

Requirement and design are covered in mgmt vrf design document.  Currently show
command displays linux command output; will update show command display in next
PR after concluding what would be the output for the show commands. Added
metric for default routes in dhcp and static, any changes for metric will be
addressed subsequently after discussing.

Signed-off-by: Harish Venkatraman <harish_venkatraman@dell.com>
2019-07-24 16:18:40 -07:00
Shuotian Cheng
dbda34f2f8 [config-engine]: Add admin_status up to l2switch.json template (#3210)
Bring up all ports by default by adding the admin_status:up
to each of the ports.

Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
2019-07-24 09:45:24 -07:00
Wei Bai
7e78d40343 [sonic-cfggen]: Improve T2 chassis frontend parsing (#3155)
* Remove unnecessary checks
2019-07-16 10:06:06 -07:00
Shuotian Cheng
97645896d6 [sonic-cfggen]: Remove trailing spaces (#3129)
Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
2019-07-09 08:08:59 -07:00
Wei Bai
0ed8c81464 [sonic-cfggen][FRR] Support T2 chassis frontend (#3100) 2019-07-08 17:32:20 -07:00
Kebo Liu
331866dbe3 [sonic-cfggen] Add Mellanox platform specific code to read base mac from machine.conf (#2991)
* add code to read base mac from machine.conf

* rewording the comments

* add mac validation with re

* fix review comments

* remove empty line
2019-06-25 08:56:35 +03:00
Pavlo Yadvichuk
817c6370bf [frr.conf] Advertise /64 prefix for ipv6 lo addresses in case 'unified' config mode (#3045)
Signed-off-by: Pavlo Yadvychuk <pyadvychuk@barefootnetworks.com>
2019-06-22 10:29:41 -07:00
Michel Moriniaux
18544530d3 [FRR] Enable SNMP support (#2981)
This is a follow-up of sonic-snmpagent PR 92
Now that licensing issues have been solved FRR is distributed with SNMP
support compiled-in. This PR adds the last bits of configuration to get
the frr-snmp debian packages added to the docker container and the
config bits to enable the snmp module in FRR

This PR brings the functionality of being able to poll bgpd for routes
and peer status.

Signed-off-by: Michel Moriniaux <m.moriniaux@criteo.com>
2019-06-19 01:24:42 -07:00
Stepan Blyshchak
383fc0fa4b [FRR] fix pfx_filter to fix bgpd.conf.j2 rendering when no vlan interfaces (#2994) 2019-06-13 19:04:45 -07:00
Prince Sunny
231d309b69
Generate interface table to have an entry designated to default VRF. (#2848)
* Generate default VRF table for router interfaces

* Updated jinja2 template to have prefix filter
2019-06-10 14:02:55 -07:00
Joe LeVeque
552684fc08
[dhcp_relay] Add support for DHCP client(s) on one VLAN and DHCP server(s) on another (#2946) 2019-06-03 14:26:45 -07:00
Sumukha Tumkur Vani
2e9b18db10 [sonic-cfggen]: Fix for management port speed issue (#2945) 2019-06-01 12:31:14 -07:00
lguohan
2a5bfe2bd7
[docker-frr]: change default routing mode to separated and fix a bug in separated mode (#2961)
In separated mode, frr requires to run vtysh_b to load individual configuration

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-05-31 07:39:40 -07:00
xiongjihai
6aad2da475 [sonic-cfggen]: Add a unit-test in sonic-cfggen to test argument "--var-json" (#2906) 2019-05-18 22:10:54 -07:00
xiongjihai
71d0f46672 [sonic-cfggen]: fix bug in file sonic-cfggen (#2834)
Fixing bug of file sonic-cfggen. Error occurs if argument --var-json is set when running sonic-cfggen,for example:

Command: sonic-cfggen -d --var-json VLAN_MEMBER
Configuration in config_db.json: 
            "VLAN_MEMBER": {
            ......
                "Vlan11|Ethernet32": {
                 "tagging_mode": "untagged"
             },
             ......
Error occurs because FormatConverter.to_serialized(data) in file sonic-cfggen doesn't serialize
keys correctly
2019-05-18 10:40:57 -07:00
Kebo Liu
aa90cae1be [sonic-cfggen] fix name conflict between sonic_platform package and sonic_platform.py (#2875)
* fix name conflict between sonic_platfrom package and sonic_platform.py

* update sonic-utility submodule to pickup lastest fix

* Revert "update sonic-utility submodule to pickup lastest fix"

This reverts commit f66aa99738.

* update sonic-utility sub module
2019-05-15 16:22:56 -07:00
Shuotian Cheng
f82ed25063 [sonic-cfggen]: Add -w as short for --write-to-db (#2900)
--write-to-db is frequently used.

Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
2019-05-15 07:48:20 -07:00
Shuotian Cheng
0b745ed946 [minigraph]: Add mirror type v6 condition (#2836)
Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
2019-05-01 12:04:33 -07:00
Joe LeVeque
6eca27e564 [services] Restart SwSS service upon unexpected critical process exit (#2845)
* [service] Restart SwSS Docker container if orchagent exits unexpectedly

* Configure systemd to stop restarting swss if it attempts to restart more than 3 times in 20 minutes

* Move supervisor-proc-exit-listener script

* [docker-dhcp-relay] Enhance wait_for_intf.sh.j2 to utilize STATEDB

* Ensure dependent services stop/start/restart with SwSS

* Change 'StartLimitInterval' to 'StartLimitIntervalSec', as Stretch installs systemd 232 (>= v230)

* Also update journald.conf options

* Remove 'PartOf' option from unit files

* Add '$(SUPERVISOR_PROC_EXIT_LISTENER_SCRIPT)' to new shared docker-orchagent makefile

* Make supervisor-proc-exit-listener script read from 'critical_processes' file inside container

* Update critical_processes file for swss container
2019-05-01 08:02:38 -07:00
Shuotian Cheng
ad2c1b26dd [minigraph]: Fix bug in copying list in Python (#2831)
'=' cannot be used for copying the list

Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
2019-04-26 15:25:09 -07:00
pavel-shirshov
9b1663ce8a [sonic-cfggen]: Output differences for bgp configurations (#2768) 2019-04-10 22:55:54 -07:00
Shuotian Cheng
61760185e1 [config-engine]: Remove unused sample minigraph (#2769)
Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
2019-04-10 22:55:30 -07:00
pavel-shirshov
602369126c [docker-fpm-quagga]: Add support for PeerAsn and UpdateAddress (#2766) 2019-04-10 21:50:36 -07:00
Ying Xie
af64fd66d2 [bgp quagga] increase BGP graceful restart timeout to 240 seconds (#2754)
There are some platforms with less powerful CPU/hard-drive could take
longer to get ready for BGP. For these platforms, 240 seconds would be
a safer threshold.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2019-04-10 19:13:03 -07:00
Shuotian Cheng
4791701752 [minigraph]: Mark both ERSPAN and ERSPANv6 as mirror ACL tables (#2732)
- Support ERSPAN and ERSPANv6 two types of ACL tables.
- Update the unit test to cover this case.

Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
2019-04-02 16:59:00 -07:00
Pavlo Yadvichuk
11c2e9ee3d [barefoot]: Allow configuration of platform-specific interfaces used for internal purposes (#2631)
- Why it is required
since SONiC master switches ifupdown package to the new implementation (ifupdown2), it is required to change the configuration of a platform-specific interface for wedge100bf_32x and wedge100bf_65x platforms (bc of ifupdown2 doesn't support auto mode for inet6 protocol).

Also, need to make some refactoring and remove if platform == smth then.. from the system level scripts.

- What I did

removed customization of /usr/bin/interfaces-config.sh
explicitly created directory /etc/network/interfaces.d
added "source" to the /etc/network/interfaces generation template (to include platform-specific interfaces processing)
added platform-specific interfaces config itself (for wedge100bf_32x and wedge100bf_65x)
fixed testcase in sonic-config-engine
- How to verify it

build image for wedge100bf_32x
perform sudo config reload -y on new installation
check the correct configuration of usb0 interface
- Description for the changelog

Allow configuration of platform-specific interfaces
2019-03-09 06:22:32 -08:00
Wenda Ni
f9c9fa8ba1 [qos]: Map tc 1, 2, 5, and 6 back to pg 0 (#2650)
Lossy traffic does not need to be mapped to different ingress PGs. They can all share the same ingress PG.

Signed-off-by: Wenda Ni <wenni@microsoft.com>
2019-03-08 02:23:32 -08:00
Nazarii Hnydyn
f5cabcb55b [mellanox]: Remove MAC alignment WA for Mellanox platforms. (#2632)
Signed-off-by: Nazarii Hnydyn <nazariig@mellanox.com>
2019-03-02 15:48:28 -08:00
Mykola F
b86dc061c1 [config engine] update test_cfggen test_port_description test (#2593)
Signed-off-by: Mykola Faryma <mykolaf@mellanox.com>
2019-02-21 17:00:45 +02:00
Mykola F
85c16b3702 [minigraph] asymmetric pfc is disabled by default (#2481)
Signed-off-by: Mykola Faryma <mykolaf@mellanox.com>

Conflicts:
	src/sonic-config-engine/tests/sample_output/ports.json
	src/sonic-config-engine/tests/test_cfggen.py
2019-02-20 08:38:09 +02:00
RAMA CHANDRA REDDY GADDAM
fa12a5cc0f 'show vlan config' is not displaying the VLAN members, after the clear config and reload with default l2 configuration. (#2536)
'show vlan config' not displaying the VLAN members, after clear config done loaded with default l2 configuration.
2019-02-18 12:23:08 -08:00
Guohan Lu
2543b711ea setup port description if minigraph has neighbor information
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-02-12 09:41:15 +00:00
Guohan Lu
dd07e2afd4 configure the hostname for lldpd
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-02-12 09:40:51 +00:00
Mykola F
19846be7c2 [minigraph.py] generate port description for every port (#2395)
* [minigraph.py] generate mandatory default port description

Signed-off-by: Mykola Faryma <mykolaf@mellanox.com>

* use port name as default description

* [config-engine] update test exaple output

Signed-off-by: Mykola Faryma <mykolaf@mellanox.com>

* [minigraph.py] use alias/port name as default description instead of neighbor data

Signed-off-by: Mykola Faryma <mykolaf@mellanox.com>
2019-02-04 15:08:00 +02:00
Wenda Ni
58adf06cc0 [QoS]: Link pg 2 and 6 to lossy buffer profile (#2511)
* Link pg 2 and 6 to lossy buffer profile

Signed-off-by: Wenda <wenni@microsoft.com>
2019-01-31 23:27:58 -08:00
Wenda Ni
ce9a3f0c5a [QoS]: QoS Config change for multiple devices (#2505)
* QoS config change: 1) DSCP mapping; 2) link pg/queue 6 to lossy buffer;
3) redistribute scheduler

Signed-off-by: Wenda <wenni@microsoft.com>

* Add scheduling weight to queue 2

Signed-off-by: Wenda <wenni@microsoft.com>

* Link pg/queue 2 to lossy buffer

Signed-off-by: Wenda <wenni@microsoft.com>

* Update the pg headroom for a7060-D48C8 50G

Signed-off-by: Wenda <wenni@microsoft.com>

* Update config gen test for qos

Signed-off-by: Wenda <wenni@microsoft.com>

* Update pg headroom size, and update egress lossy pool size accordingly

Signed-off-by: Wenda <wenni@microsoft.com>

* Update headroom pool size; Update ingress service pool and egress lossy
pool sizes accordingly;

Signed-off-by: Wenda <wenni@microsoft.com>

* a7260: update headroom pool size; Update ingress service pool and egress lossy pool sizes accordingly;

Signed-off-by: Wenda <wenni@microsoft.com>

* Update config gen test for buffer

Signed-off-by: Wenda <wenni@microsoft.com>
2019-01-30 19:00:13 -08:00
Mykola F
52c533fe01 [lldpmgr] add mgmt ip to lldpd conf template, handle port description/alias changes (#2396)
* [lldpmgr] add mgmt ip to lldpd conf templte, handle port description/alias config

Signed-off-by: Mykola Faryma <mykolaf@mellanox.com>

* fix typo

* [config-engine] update test sample output lldpd.conf

Signed-off-by: Mykola Faryma <mykolaf@mellanox.com>

* fix the log message

* fix lldpd.conf.j2
2019-01-30 11:05:36 +02:00
Ying Xie
d9c076dada [quagga bgp] set quagga graceful restart timeout to 180 seconds (#2362)
Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2018-12-08 11:38:31 -08:00
Wenda Ni
f5e678cf84 Port QoS & buffer changes in 0330 to master (#2239)
* 1) DSCP 46 to 5; 2) ecn config for lossless traffic; 3) ecn on by default; 4) DWRR equal weight;

Signed-off-by: Wenda <wenni@microsoft.com>

* 1) link pg & queue 5 to lossy buffer profile; 2) ingress lossless alpha 1/8

Signed-off-by: Wenda <wenni@microsoft.com>

* Update the test case for qos & buffer json template

Signed-off-by: Wenda <wenni@microsoft.com>

* Migrate a7050-qx32 and s6000 to use pg_profile lookup architecture

Signed-off-by: Wenda <wenni@microsoft.com>

* Update pg headroom egress service pool for a7050-qx-32s, a7050-qx32, and s6000

Signed-off-by: Wenda <wenni@microsoft.com>

* Link queue 5 to lossy profile

Signed-off-by: Wenda <wenni@microsoft.com>
2018-12-04 20:51:55 -08:00
Taoyu Li
aedfd6e708 [sonic-cfggen] Multi-key should be in form of (a,b) instead of 'a|b' (#2337) 2018-12-04 10:07:44 -08:00
Nikos
7056b49af7 Routing application split config support (#2286)
* Routing application split config support

Signed-off-by: nikos <ntriantafillis@gmail.com>

* Routing application split config support
Routing application split config support

Signed-off-by: nikos <Nikos Triantafillis>
2018-11-26 18:19:12 -08:00
Shuotian Cheng
e5a031b5d5
[minigraph.py]: Set default admin_status up to LAG/VLAN members (#2209)
By default, port channel members and VLAN members are set to admin
status UP. This configuration could be updated via the command:
config interface <interface_name> startup/shutdown

Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
2018-10-30 16:42:38 -07:00