As part of consolidating all common Python-based functionality into the new sonic-py-common package, this pull request:
1. Redirects all Python applications/scripts in sonic-buildimage repo which previously imported sonic_device_util or sonic_daemon_base to instead import sonic-py-common, which was added in https://github.com/Azure/sonic-buildimage/pull/5003
2. Replaces all calls to `sonic_device_util.get_platform_info()` to instead call `sonic_py_common.get_platform()` and removes any calls to `sonic_device_util.get_machine_info()` which are no longer necessary (i.e., those which were only used to pass the results to `sonic_device_util.get_platform_info()`.
3. Removes unused imports to the now-deprecated sonic-daemon-base package and sonic_device_util.py module
This is the next step toward resolving https://github.com/Azure/sonic-buildimage/issues/4999
Also reverted my previous change in which device_info.get_platform() would first try obtaining the platform ID string from Config DB and fall back to gathering it from machine.conf upon failure because this function is called by sonic-cfggen before the data is in the DB, in which case, the db_connect() call will hang indefinitely, which was not the behavior I expected. As of now, the function will always reference machine.conf.
use lsb_release -cs to get build env within the docker
for jessie: jessie
for stretch: stretch
for buster: buster
Signed-off-by: Guohan Lu <lguohan@gmail.com>
following command add additional mount in the sonic:
DOCKER_BUILDER_USER_MOUNT=/data2:/data2,/data:/data make sonic-slave-bash
Signed-off-by: Guohan Lu <lguohan@gmail.com>
* [platform] Add Support For Environment Variable
This PR adds the ability to read environment file from /etc/sonic.
the file contains immutable SONiC config attributes such as platform,
hwsku, version, device_type. The aim is to minimize calls being made
into sonic-cfggen during boot time.
singed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
* Changes to add template support for copp.json.
This is needed so that we can install differnt type of
Traps based on Device Role (Tor/Leaf/Mgmt/etc...).
Initial use case is to install DHCP/DHCPv6 tarp only
for tor router.
Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
* Fixed based on review comments.
Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
* Fixed based on review comment.
* Update the buffer size based on the latest excel
Signed-off-by: Stephen Sun <stephens@mellanox.com>
* Align the buffer configuration with the latest formula:
- reduce redundant "*2" in formula
- use port MTU for local sending the PFC frame and peer lossless MTU for peer sending lossless traffic
Buffer pool size updated accordingly.
Signed-off-by: Stephen Sun <stephens@mellanox.com>
To clarify error messages in case the ip address for Loopback is already set. It doesn't make sense to call correct ip address as ambiguous in this case
--init: Run an init inside the container that forwards signals and
reaps processes
Before the change, process 1 is make, which does not do well to forward
signals and reaps process. We could see zombie process left if user
issues ctrl+c to interrupt the make process. with --init option,
a docker-init process will forwards the signals and reaps processes.
zombie process is no longer observed, and ctrl+c can reliably interrupt
the make process.
Before:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
lgh 1 0.3 0.0 12604 11908 pts/0 S+ 10:54 0:00 make ...
After:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
lgh 1 0.0 0.0 1080 4 pts/0 Ss 10:54 0:00 /sbin/docker-init -- make ...
lgh 7 0.3 0.0 12604 11908 pts/0 S+ 10:54 0:00 make ...
Signed-off-by: Guohan Lu <lguohan@gmail.com>
virtual-chassis test uses multiple vs instances to simulate a
modular switch and a redis-chassis service is required to run on
the vs instance that represents a supervisor card.
This change allows vs docker start redis-chassis service according
to external config file.
**- Why I did it**
To support virtual-chassis setup, so that we can test distributed forwarding feature in virtual sonic environment, see `Distributed forwarding in a VOQ architecture HLD` pull request at https://github.com/Azure/SONiC/pull/622
**- How I did it**
The sonic-vs start.sh is enhanced to start new redis_chassis service if external chassis config file found. The config file doesn't exist in current vs environment, start.sh will behave like before.
**- How to verify it**
The swss/test still pass. The chassis_db service is verified in virtual-chassis topology and tests which are in following PRs.
Signed-off-by: Honggang Xu <hxu@arista.com>
(cherry picked from commit c1d45cf81ce3238be2dcbccae98c0780944981ce)
Co-authored-by: Honggang Xu <hxu@arista.com>
Verify that /etc/apt/sources.list points to buster using docker exec bgp cat /etc/apt/sources.list
BGP neighborship is established.
root@sonic:~# show ip bgp summary
IPv4 Unicast Summary:
BGP router identifier 10.1.0.1, local AS number 65100 vrf-id 0
BGP table version 1
RIB entries 1, using 184 bytes of memory
Peers 1, using 20 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
6.1.1.1 4 100 96 96 0 0 0 01:32:04 0
Total number of neighbors 1
root@sonic:~#
Signed-off-by: Joyas Joseph <joyas_joseph@dell.com>
Copy proper fancontrol config file to the proper destination. Also some minor refactoring for code reuse to help prevent issues like this in the future.
Fixes a bug introduced by #4599
when parallel build is enabled, both docker-fpm-frr and docker-syncd-brcm
is built at the same time, docker-fpm-frr requires swss which requires to
install libsaivs-dev. docker-syncd-brcm requires syncd package which requires
to install libsaibcm-dev.
since libsaivs-dev and libsaibcm-dev install the sai header in the same
location, these two packages cannot be installed at the same time. Therefore,
we need to serialize the build between these two packages. Simply uninstall
the conflict package is not enough to solve this issue. The correct solution
is to have one package wait for another package to be uninstalled.
For example, if syncd is built first, then it will install libsaibcm-dev.
Meanwhile, if the swss build job starts and tries to install libsaivs-dev,
it will first try to query if libsaibcm-dev is installed or not. if it is
installed, then it will wait until libsaibcm-dev is uninstalled. After syncd
job is finished, it will uninstall libsaibcm-dev and swss build job will be
unblocked.
To solve this issue, _UNINSTALLS is introduced to uninstall a package that
is no longer needed and to allow blocked job to continue.
Signed-off-by: Guohan Lu <lguohan@gmail.com>
Consolidate common SONiC Python-language functionality into one shared package (sonic-py-common) and eliminate duplicate code.
The package currently includes three modules:
- daemon_base
- device_info
- logger
fixes https://github.com/Azure/sonic-buildimage/issues/5026
Explanation:
In the log from the issue I found:
```
I see following in the log
Jul 22 21:13:06.574831 vlab-01 WARNING bgp#bgpd[49]: [EC 33554499] sendmsg_nexthop: zclient_send_message() failed
```
Analyzing source code I found that the error message could be issues only when `zclient_send_rnh()` return less than 0.
```
ret = zclient_send_rnh(zclient, command, p, exact_match,
bnc->bgp->vrf_id);
/* TBD: handle the failure */
if (ret < 0)
flog_warn(EC_BGP_ZEBRA_SEND,
"sendmsg_nexthop: zclient_send_message() failed");
```
I checked [zclient_send_rnh()](88351c8f6d/lib/zclient.c (L654)) and found that this function will return the exit code which the function gets from [zclient_send_message()](88351c8f6d/lib/zclient.c (L266)) But the latter function could return not 0 in two cases:
1. bgpd didn’t connect to the zclient socket yet [code](88351c8f6d/lib/zclient.c (L269))
2. The socket was closed. But in this case we would receive the error message in the log. (And I can find the message in the log when we reboot sonic) [code](88351c8f6d/lib/zclient.c (L277))
Also I see from the logs that client connection was set later we had the issue in bgpd.
Bgpd.log
```
Jul 22 21:13:06.574831 vlab-01 WARNING bgp#bgpd[49]: [EC 33554499] sendmsg_nexthop: zclient_send_message() failed
```
Vs
Zebra.log
```
Jul 22 21:13:12.713249 vlab-01 NOTICE bgp#zebra[48]: client 25 says hello and bids fair to announce only static routes vrf=0
Jul 22 21:13:12.820352 vlab-01 NOTICE bgp#zebra[48]: client 30 says hello and bids fair to announce only bgp routes vrf=0
Jul 22 21:13:12.820352 vlab-01 NOTICE bgp#zebra[48]: client 33 says hello and bids fair to announce only vnc routes vrf=0
```
So in our case we should start zebra first. Wait until it is started and then start bgpd and other daemons.
**- How I did it**
I changed a graph to start daemons in the following order:
1. First start zebra
2. Then starts staticd and bgpd
3. Then starts vtysh -b and bgpeoi after bgpd is started.
Fix for the host unmount issue through PR https://github.com/Azure/sonic-buildimage/pull/4558 and https://github.com/Azure/sonic-buildimage/pull/4865 creates the timeout of syslog.socket closure during reboot since the journald socket closure has been included in syslog.socket
Removed the journal socket closure. The host unmount is fixed with just stopping the services which gets restarted only after /var/log unmount and not causing the unmount issues.
**sonic-platform-common:**
Commits on Jul 07, 2020
Changes in fan and psu base classes (1.0 platform API) related to pdd… 17292e4
Commits on Jul 10, 2020
update get_pcie_check() to use sysfs instead of lspci output (#95) d4eb804
Commits on Jul 11, 2020
[eeprom] Fix UnboundLocalError (#93) 7c8bed1
Commits on Jul 14, 2020
[Transceiver] Add parser for QSFP-DD cable type and dictionaries for … be1cc24
**sonic-platform-daemons:**
Commits on Jul 06, 2020
[psud] Store PSU temperature and voltage information to database (#61) ef9716a
Commits on Jul 14, 2020
[xcvrd] Add support for QSFP-DD cables (#66) c530587
Commits on Jul 18, 2020
Initial version of pcied (#60) e665ee8
[xcvrd] Return non-zero error code on SFP error (#67) 4f42a79
Commits on Jul 19, 2020
[README.md] Add LTGM badges (#69) 23757a3
Remove unused imports (#70) 029d5a5
Signed-off-by: Nazarii Hnydyn <nazariig@mellanox.com>
This PR updates kubernetes version to 1.18.6 and kubernetes-cni version to 0.8.6
signed-off by: Isabel Li isabel.li@microsoft.com
Why I did it
Previous kubernetes-cni version (0.7.5) introduced Kubernetes Man In The Middle Vulnerability. “A vulnerability was found in all versions of containernetworking/plugins before version 0.8.6, that allows malicious containers in Kubernetes clusters to perform man-in-the-middle (MitM) attacks. A malicious container can exploit this flaw by sending rogue IPv6 router advertisements to the host or other containers, to redirect traffic to the malicious container.”
How I did it
Defined kubernetes-cni version to be 0.8.6 and updated kubernetes version to be 1.18.6
How to verify it
Check versions by running dpkg -l | grep kube
Update the sonic-swss-common submodule. The following are the commits in the submodule.
Fix sai api loglevel
ed9c497316e73bc5d4a6dfbe9f3507d5e92c15f3
Add support for attribute capability query in lua script
f58ea1e7a692eadd8695b4dcf89290c5e9018aae
Copp changes
4e50be7728d1e68582ac4dc369323ec5ab7b81dd
fix pytest for python3
c76d3b3a6dacae7fde14622c41e41b717876924a
use sysconfig instead of distutils.sysconfig
1772581f9b9f25e5ab2b790d7d517ea2b2010a78
CONFIG_DB table name for BUM Storm-control feature
c39c958bea445e650353651fcbae5c228114f86d
Fine Grained ECMP schema changes
1974d2e1547464a3b0279b5d63fc40eda94fa780
* src/sonic-telemetry fa8d498...3bd7ca3 (4):
> Update gnmi deps (#40)
> [testdata] Update SFP keys to align with new standard (#39)
> Fixed the parameters for subscribe APIs (#38)
> Azure ro mode (#34)
* src/sonic-mgmt-common 444aa9a...cc01ce4 (4):
> Make gnmi dep version the same as in telemetry repo (#17)
> Cleanup translib and cvl go test cases (#13)
> Package update and enhancements/fixes in YGOT, and Request Binder (#12)
> Translib phase I changes (#11)
Note: sonic-telemetry submodule update is dependent upon sonic-mgmt-common submodule update, thus updating both in this patch
Added required packages to enabled YANG dependency check for Dynamic Port Breakout in VS container.
[sonic-utilities PR #766](https://github.com/Azure/sonic-utilities/pull/766) has a dependency on it.
Getting error like the following without this fix: `ImportError: No module named yang - required module not found`
Signed-off-by: Sangita Maity <sangitamaity0211@gmail.com>
This fix removes the requirement to have a NEIGHBOR_METADATA for dynamic peers. The change is made since it is not necessary for NEIGHBOR_METADATA entries be present for the dynamic neighbors
`sonic-installer list` is a read-only command. Specify it as such in the sudoers file.
This will also ensure the new `show boot` command, which calls `sudo sonic-installer list` under the hood doesn't fail due to permissions.
Otherwise, it may cause issues for warm restarts, warm reboot.
Warm restart of swss will start nat which is not expected for warm
restart. Also it is observed that during warm-reboot script execution
nat container gets started after it was killed. This causes removal of
nat dump generated by nat previously:
A check [ -f /host/warmboot/nat/nat_entries.dump ] || echo "NAT dump
does not exists" was added right before kexec:
```
Fri Jul 17 10:47:16 UTC 2020 Prepare MLNX ASIC to fastfast-reboot:
install new FW if required
Fri Jul 17 10:47:18 UTC 2020 Pausing orchagent ...
Fri Jul 17 10:47:18 UTC 2020 Stopping nat ...
Fri Jul 17 10:47:18 UTC 2020 Stopped nat ...
Fri Jul 17 10:47:18 UTC 2020 Stopping radv ...
Fri Jul 17 10:47:19 UTC 2020 Stopping bgp ...
Fri Jul 17 10:47:19 UTC 2020 Stopped bgp ...
Fri Jul 17 10:47:21 UTC 2020 Initialize pre-shutdown ...
Fri Jul 17 10:47:21 UTC 2020 Requesting pre-shutdown ...
Fri Jul 17 10:47:22 UTC 2020 Waiting for pre-shutdown ...
Fri Jul 17 10:47:24 UTC 2020 Pre-shutdown succeeded ...
Fri Jul 17 10:47:24 UTC 2020 Backing up database ...
Fri Jul 17 10:47:25 UTC 2020 Stopping teamd ...
Fri Jul 17 10:47:25 UTC 2020 Stopped teamd ...
Fri Jul 17 10:47:25 UTC 2020 Stopping syncd ...
Fri Jul 17 10:47:35 UTC 2020 Stopped syncd ...
Fri Jul 17 10:47:35 UTC 2020 Stopping all remaining containers ...
Warning: Stopping telemetry.service, but it can still be activated by:
telemetry.timer
Fri Jul 17 10:47:37 UTC 2020 Stopped all remaining containers ...
NAT dump does not exists
Fri Jul 17 10:47:39 UTC 2020 Rebooting with /sbin/kexec -e to
SONiC-OS-201911.140-08245093 ...
```
With this change, executed warm-reboot 10 times without hitting this
issue, while without this change the issue is easily reproducible almost
every warm-reboot run.
Signed-off-by: Stepan Blyschak <stepanb@mellanox.com>
