* This was a temporary fix for orchagent spamming log messages and causing rate limiting, leading to critical messages being dropped for the syslog. No longer needed since Azure/sonic-sairedis#680 was merged.
swss:
* d7643f2 2020-11-01 | [tlm_teamd]: Make the destionation for std::transform() to use std::back_inserter() for allocating new space for the copied objects (#1490) (HEAD, origin/master, origin/HEAD) [pavel-shirshov]
* 7fa7cd6 2020-10-31 | [vstest]: stablize fgnhg test (#1491) [lguohan]
* 9b0696e 2020-10-29 | Create vnet tunnel map only if it doesn't exist (#1482) [Prince Sunny]
* 0481e99 2020-10-29 | [acl] Update CRM to include LAG bindings for ACL tables (#1487) [Danny Allen]
sairedis
* 5df11f5 2020-11-01 | [syncd] Lower bulk missing api message level from error to info (#687) (HEAD, origin/master, origin/HEAD) [Kamil Cudnik]
* dc73a1d 2020-10-30 | [saiplayer] Fix log messages (#686) [Kamil Cudnik]
Signed-off-by: Guohan Lu <lguohan@gmail.com>
* Build and install openssh from source
* Copy openssh deb package to dest folder
* Update make rule
* Update sonic debian extension
* Append empty line before EOF
* Update openssh patch
* Add openssh-server to base image dependency
* Fix indent type
* Fix comments
* Use commit id instead of tag id and add comment
Signed-off-by: Jing Kan jika@microsoft.com
On Arista platforms, sonic_platform packages are not installed in the PMon container, but are rather mounted into the container from the host OS. Therefore, pip show sonic_platform will fail in the PMon container. This change will first check if we can import sonic_platform. If this fails, it will then fall back to checking if the package is installed. If both fail, it will attempt to install the package.
FixAzure/SONiC#551
When eth0 IP address is configured, an ip rule is getting added for eth0 IP address through the interfaces.j2 template.
This eth0 ip rule creates an issue when VRF (data VRF or management VRF) is also created in the system.
When any VRF (data VRF or management VRF) is created, a new rule is getting added automatically by kernel as "1000: from all lookup [l3mdev-table]".
This l3mdev IP rule is never getting deleted even if VRF is deleted.
Once if this l3mdev IP rule is added, if user configures IP address for the eth0 interface, interfaces.j2 adds an eth0 IP rule as "1000:from 100.104.47.74 lookup default ". Priority 1000 is automatically chosen by kernel and hence this rule gets higher priority than the already existing rule "1001:from all lookup local ".
This results in an issue "ping from console to eth0 IP does not work once if VRF is created" as explained in Issue 551.
More details and possible solutions are explained as comments in the Issue551.
This PR is to resolve the issue by always fixing the low priority 32765 for the IP rule that is created for the eth0 IP address.
Tested with various combinations of VRF creation, deletion and IP address configuration along with ping from console to eth0 IP address.
Co-authored-by: Kannan KVS <kannan_kvs@dell.com>
Added new MultiASIC util method "get_back_end_interface_set()" to speed up back-end interface check by allowing caller to cache the back-end intf into a set. This way the caller can use this set for all subsequent back-end interface check requests instead of each time need to read from redis DB which become a scaling issue for cases such as checking for thousands of nexthop routes for filtering purpose.
Why/How I did:
Make sure first error syslog is triggered based on FAULT TOLERANCE condition.
Added support of repeat clause with alert action. This is used as trigger
for generation of periodic syslog error messages if error is persistent
Updated the monit conf files with repeat every x cycles for the alert action
- Why I did it
The update_all_feature_states can run in the range of 20+ seconds to one minute. With load of AAA & Tacacs preceding it, any DB updates in AAA/TACACS during the long running feature updates would get missed. To avoid, switch the order.
- How I did it
Do a load after after updating all feature states.
- How to verify it
Not a easy one
Have a script that
restart hostcfgd
sleep 2s
run redis-cli/config command to update AAA/TACACS table
Run the script above and watch the file /etc/pam.d/common-auth-sonic for a minute.
- When it repro:
The updates will not reflect in /etc/pam.d/common-auth-sonic
Update the sonic-swss submodule. The following are the commits in the submodule.
[neighorch] Remove pending DEL operation after SET operation for the same key
2265f548386929b7827d1079efd453128f1ec1f9
[NAT]: Update nat entries to use nat_type to support DNAT Pool changes.
8696e939f973895ead4731ad499a72f257a3b510
[intfsorch] Init proxy_arp variable while adding router interface.
1da3c773762fa637a5ea47017715361bede50a4a
Increase startretires value from default of 10 to 50 to prevent supervisor from placing thermalctld in FATAL state during regression testing. Also ensures supervisord tries hard to get thermalctld running in production, as thermalctld is critical to prevent device from overheating.
As part of the transition from Python 2 to Python 3, we are installing both pip2 and pip3 in the slave and config-engine containers. This PR replaces calls to `pip` in these containers with an explicit call to `pip2` to ensure the proper version of pip is executed, no matter which version of pip is aliased to `pip`, as we no longer rely on that alias.
Also some other pip-related cleanup
- Change `/run/arista` mount to pmon by `/var/run/platform_cache`
- Python3 by default for Arista platform initialisation
- Fix outstanding py2/3 compatibility issues (eeprom mostly)
- Use pytest for unit testing
- Miscellaneous modular fixes
Upgrading pip3 after pip2 caused the pip command to be aliased to the pip3 command. However, since we are still transitioning from Python 2 to Python 3, most pip commands in the codebase are expecting pip to alias to pip2. The proper solution here is to explicitly call pip2 and pip3, and no longer call pip, however this will require extensive changes and testing, so to quickly fix this issue, we upgraded pip2 after pip3 to ensure that pip2 is installed after pip3.
- Implement FanDrawer and get_high_critical_threshold Platform API for S6000, S6100, Z9100 and Z9264F.
- Fix incorrect fan direction values in S6100, Z9100
To consolidate host services and install via packages instead of file-by-file, also as part of migrating all of SONiC to Python 3, as Python 2 is no longer supported, convert caclmgrd to Python 3 and add to sonic-host-services package
We don't need a custom platform reboot on Clearwater2(Ms). They are expected to be rebooted via a normal linux soft reboot.
Remove symlink to the arista common platform reboot for those 2 platforms.
95b1696 [xcvrd] Remove dependence on enum; Add 'sonic-py-common' as dependencies in setup.py (#106)
61ed24e [thermalctld] Print exception using repr(e) to get more information (#103)
8507085 [psud] Fix psud logging (#98)
Signed-off-by: Volodymyr Boyko <volodymyrx.boiko@intel.com>
**- Why I did it**
Converted two SP model to single pool model and modified the buffer size.
**- How I did it**
Changed buffer_default settings for all the DellEMC Z9264f HWSKU's.
**- How to verify it**
Check SP register values in NPU shell.
**- Which release branch to backport (provide reason below if selected)**
Need to be cherry picked for 201911 branch.
* Initial commit for BGP internal neighbor table support.
> Add new template named "internal" for the internal BGP sessions
> Add a new table in database "BGP_INTERNAL_NEIGHBOR"
> The internal BGP sessions will be stored in this new table "BGP_INTERNAL_NEIGHBOR"
* Changes in template generation tests with the introduction of internal neighbor template files.
Update the sonic-sairedis submodule. The following are the commits in the submodule.
[syncd_init_common.sh] Use template file to retrieve vars (#683)
0bf336a3e895167357d5d2e5a988471e115522e8
[syncd/FlexCounter]:Fix last remove bug (#679)
4d21a264d5956501bf69ad3a89ea2ebccd369654
remove syncd from critical process list because
gbsyncd process will exit for platform without
gearbox.
closes#5623
Signed-off-by: Guohan Lu <lguohan@gmail.com>
a659219 [SONIC_SFP] adding abstract methods for reading and writing the eeprom address space within platform api (#126)
848f4a6 Add third-party licenses (#138)
c2ecd9a Add license file (#137)
403747a [sonic-platform-common] Add new platform API for SONiC Physical MIB Extension feature (#134)
19b8545 [sonic_y_cable] fix the unpacking (#135)
Signed-off-by: vaibhav-dahiya <vdahiya@microsoft.com>
The psutil library used in process_checker create a cache for each
process when calling process_iter. So, there is some possibility that
one process exists when calling process_iter, but not exists when
calling cmdline, which will raise a NoSuchProcess exception. This commit
fix the issue.
Signed-off-by: bingwang <bingwang@microsoft.com>
* Fix for LLDP advertisments being sent with wrong information.
Since lldpd is starting before lldpmgr, some advertisment packets might sent with default value, mac address as Port ID.
This fix hold the packets from being sent by the lldpd until all interfaces are well configured by the lldpmgrd.
Signed-off-by: Shlomi Bitton <shlomibi@nvidia.com>
* Fix comments
* Fix unit-test output caused a failure during build
* Add 'run_cmd' function and use it
* Resume lldpd even if port init timeout reached
Advance sonic-swss-common submodule by adding the following commits
3ec30ef Deprecate RedisClient and remove unused header file (#399)
165a679 Schema update for BGP internal neighbor table (#389)
262e330 Fix SonicV2Connector interfaces (#396)
Advance sonic-sairedis submodule by adding the following commits
bc3e044 [Sai]: Change Sai::set log to level INFO (#680)
b16bc8b Clean code: remove unused header file (#678)
40439b4 [syncd] Remove depreacated dependency on swss::RedisClient (#681)
1b6fc2e [syncd] Add supports of bulk api in syncd (#656)
a9f69c1 [syncd] Add to handle FDB MOVE notification (#670)
c7ef5e9 [gbsyncd] exit with zero when platform has no gearbox (#676)
57228fd [gbsyncd]: add missing python dependency (#675)
02a57a6 [vs] Add CRM SAI attributes to virtual switch interface (#673)
609445a fix boot type for fast boot (#674)
1325cdf Add support for saiplayer bulk API and add performance timers (#666)
1d84b90 Add ZeroMQ communication channel between sairedis and syncd (#659)
017056a Support System ports config (#657)
0f3668f Enable fabric counter for syncd's FlexCounter (#669)
Find LogicalLinks in minigraph and parse the port information. A new field called `mux_cable` is added to each port's entry in the Port table in config DB:
```
PORT|Ethernet0: {
"alias": "Ethernet4/1"
...
"mux_cable": "true"
}
```
If a mux cable is present on a port, the value for `mux_cable` will be `"true"`. If no mux cable is present, the attribute will either be omitted (default behavior) or set to `"false"`.
Current implementation of logger class is based on standard python syslog library.
Thus, logger class can be instantiated in different places and share the same context across the entire process.
This means that reducing log severity level will affect other modules which use logging facility.
**- Why I did it**
* To fix syslog implicit min priority override
**- How I did it**
* Added per instance log severity check
**- How to verify it**
1. Run code snippet
```
from sonic_py_common import logger
log1 = logger.Logger(log_identifier='myApp1')
log1.set_min_log_priority_debug()
log1.log_error("=> this is error")
log1.log_warning("=> this is warning")
log1.log_notice("=> this is notice")
log1.log_info("=> this is info")
log1.log_debug("=> this is debug")
log2 = logger.Logger(
log_identifier='myApp2',
log_facility=logger.Logger.LOG_FACILITY_DAEMON,
log_option=(logger.Logger.LOG_OPTION_NDELAY | logger.Logger.LOG_OPTION_PID)
)
log2.log_error("=> this is error")
log2.log_warning("=> this is warning")
log2.log_notice("=> this is notice")
log2.log_info("=> this is info")
log2.log_debug("=> this is debug")
```
2. Sample output:
```
Oct 23 15:08:30.447301 sonic ERR myApp1: => this is error
Oct 23 15:08:30.447908 sonic WARNING myApp1: => this is warning
Oct 23 15:08:30.448305 sonic NOTICE myApp1: => this is notice
Oct 23 15:08:30.448696 sonic INFO myApp1: => this is info
Oct 23 15:08:30.449063 sonic DEBUG myApp1: => this is debug
Oct 23 15:08:30.449442 sonic ERR myApp2[19178]: => this is error
Oct 23 15:08:30.449819 sonic WARNING myApp2[19178]: => this is warning
Oct 23 15:08:30.450183 sonic NOTICE myApp2[19178]: => this is notice
```
Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>
The orchagent and syncd need to have the same default synchronous mode configuration. This PR adds a template file to translate the default value in CONFIG_DB (empty field) to an explicit mode so that the orchagent and syncd could have the same default mode.
When detecting a new SFP insertion, read its SFP type and DOM capability from EEPROM again.
SFP object will be initialized to a certain type even if no SFP present. A case could be:
1. A SFP object is initialized to QSFP type by default when there is no SFP present
2. User insert a SFP with an adapter to this QSFP port
3. The SFP object fail to read EEPROM because it still treats itself as QSFP.
This PR fixes this issue.
- Enable thermalctld support for our platforms
- Fix Chassis.get_num_sfp which had an off by one
- Implement read_eeprom and write_eeprom in SfpBase
- Refactor of Psus and PsuSlots. Psus they are now detected and metadata reported
- Improvements to modular support
Co-authored-by: Zhi Yuan Carl Zhao <zyzhao@arista.com>
**- Why I did it**
Install all host services and their data files in package format rather than file-by-file
**- How I did it**
- Create sonic-host-services Python wheel package, currently including procdockerstatsd
- Also add the framework for unit tests by adding one simple procdockerstatsd test case
- Create sonic-host-services-data Debian package which is responsible for installing the related systemd unit files to control the services in the Python wheel. This package will also be responsible for installing any Jinja2 templates and other data files needed by the host services.
**- Why I did it**
On teamd docker restart, the swss and syncd needs to be restarted as there are dependent resources present.
**- How I did it**
Add the teamd as a dependent service for swss
Updated the docker-wait script to handle service and dependent services separately.
Handle the case of warm-restart for the dependent service
**- How to verify it**
Verified the following scenario's with the following testbed
VM1 ----------------------------[DUT 6100] -----------------------VM2, ping traffic continuous between VMs
1. Stop teamd docker alone
> swss, syncd dockers seen going away
> The LAG reference count error messages seen for a while till swss docker stops.
> Dockers back up.
2. Enable WR mode for teamd. Stop teamd docker alone
> swss, syncd dockers not removed.
> The LAG reference count error messages not seen
> Repeated stop teamd docker test - same result, no effect on swss/syncd.
3. Stop swss docker.
> swss, teamd, syncd goes off - dockers comes back correctly, interfaces up
4. Enable WR mode for swss . Stop swss docker
> swss goes off not affecting syncd/teamd dockers.
5. Config reload
> no reference counter error seen, dockers comes back correctly, with interfaces up
6. Warm reboot, observations below
> swss docker goes off first
> teamd + syncd goes off to the end of WR process.
> dockers comes back up fine.
> ping traffic between VM's was NOT HIT
7. Fast reboot, observations below
> teamd goes off first ( **confirmed swss don't exit here** )
> swss goes off next
> syncd goes away at the end of the FR process
> dockers comes back up fine.
> there is a traffic HIT as per fast-reboot
8. Verified in multi-asic platform, the tests above other than WR/FB scenarios
**- Why I did it**
If we ran the CLI commands `sudo config feature autorestart snmp disabled/enabled` or `sudo config feature autorestart swss disabled/enabled`, then SNMP container will be stopped and started. This behavior was not expected since we updated the `auto_restart` field not update `state` field in `FEATURE` table. The reason behind this issue is that either `state` field or `auto_restart` field was updated, the function `update_feature_state(...)` will be invoked which then starts snmp.timer service.
The snmp.timer service will first stop snmp.service and later start snmp.service.
In order to solve this issue, the function `update_feature_state(...)` will be only invoked if `state` field in `FEATURE` table was
updated.
**- How I did it**
When the demon `hostcfgd` was activated, all the values of `state` field in `FEATURE` table of each container will be
cached. Each time the function `feature_state_handler(...)` is invoked, it will determine whether the `state` field of a
container was changed or not. If it was changed, function `update_feature_state(...)` will be invoked and the cached
value will also be updated. Otherwise, nothing will be done.
**- How to verify it**
We can run the CLI commands `sudo config feature autorestart snmp disabled/enabled` or `sudo config feature autorestart swss disabled/enabled` to check whether SNMP container is stopped and started. We also can run the CLI commands `sudo config feature state snmp disabled/enabled` or `sudo config feature state swss disabled/enabled` to check whether the container is stopped and restarted.
Signed-off-by: Yong Zhao <yozhao@microsoft.com>
**- Why I did it**
To introduce dynamic support of BBR functionality into bgpcfgd.
BBR is adding `neighbor PEER_GROUP allowas-in 1' for all BGP peer-groups which points to T0
Now we can add and remove this configuration based on CONFIG_DB entry
**- How I did it**
I introduced a new CONFIG_DB entry:
- table name: "BGP_BBR"
- key value: "all". Currently only "all" is supported, which means that all peer-groups which points to T0s will be updated
- data value: a dictionary: {"status": "status_value"}, where status_value could be either "enabled" or "disabled"
Initially, when bgpcfgd starts, it reads initial BBR status values from the [constants.yml](https://github.com/Azure/sonic-buildimage/pull/5626/files#diff-e6f2fe13a6c276dc2f3b27a5bef79886f9c103194be4fcb28ce57375edf2c23cR34). Then you can control BBR status by changing "BGP_BBR" table in the CONFIG_DB (see examples below).
bgpcfgd knows what peer-groups to change fron [constants.yml](https://github.com/Azure/sonic-buildimage/pull/5626/files#diff-e6f2fe13a6c276dc2f3b27a5bef79886f9c103194be4fcb28ce57375edf2c23cR39). The dictionary contains peer-group names as keys, and a list of address-families as values. So when bgpcfgd got a request to change the BBR state, it changes the state only for peer-groups listed in the constants.yml dictionary (and only for address families from the peer-group value).
**- How to verify it**
Initially, when we start SONiC FRR has BBR enabled for PEER_V4 and PEER_V6:
```
admin@str-s6100-acs-1:~$ vtysh -c 'show run' | egrep 'PEER_V.? allowas'
neighbor PEER_V4 allowas-in 1
neighbor PEER_V6 allowas-in 1
```
Then we apply following configuration to the db:
```
admin@str-s6100-acs-1:~$ cat disable.json
{
"BGP_BBR": {
"all": {
"status": "disabled"
}
}
}
admin@str-s6100-acs-1:~$ sonic-cfggen -j disable.json -w
```
The log output are:
```
Oct 14 18:40:22.450322 str-s6100-acs-1 DEBUG bgp#bgpcfgd: Received message : '('all', 'SET', (('status', 'disabled'),))'
Oct 14 18:40:22.450620 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-f', '/tmp/tmpmWTiuq']'.
Oct 14 18:40:22.681084 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V4 soft in']'.
Oct 14 18:40:22.904626 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V6 soft in']'.
```
Check FRR configuraiton and see that no allowas parameters are there:
```
admin@str-s6100-acs-1:~$ vtysh -c 'show run' | egrep 'PEER_V.? allowas'
admin@str-s6100-acs-1:~$
```
Then we apply enabling configuration back:
```
admin@str-s6100-acs-1:~$ cat enable.json
{
"BGP_BBR": {
"all": {
"status": "enabled"
}
}
}
admin@str-s6100-acs-1:~$ sonic-cfggen -j enable.json -w
```
The log output:
```
Oct 14 18:40:41.074720 str-s6100-acs-1 DEBUG bgp#bgpcfgd: Received message : '('all', 'SET', (('status', 'enabled'),))'
Oct 14 18:40:41.074720 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-f', '/tmp/tmpDD6SKv']'.
Oct 14 18:40:41.587257 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V4 soft in']'.
Oct 14 18:40:42.042967 str-s6100-acs-1 DEBUG bgp#bgpcfgd: execute command '['vtysh', '-c', 'clear bgp peer-group PEER_V6 soft in']'.
```
Check FRR configuraiton and see that the BBR configuration is back:
```
admin@str-s6100-acs-1:~$ vtysh -c 'show run' | egrep 'PEER_V.? allowas'
neighbor PEER_V4 allowas-in 1
neighbor PEER_V6 allowas-in 1
```
*** The test coverage ***
Below is the test coverage
```
---------- coverage: platform linux2, python 2.7.12-final-0 ----------
Name Stmts Miss Cover
----------------------------------------------------
bgpcfgd/__init__.py 0 0 100%
bgpcfgd/__main__.py 3 3 0%
bgpcfgd/config.py 78 41 47%
bgpcfgd/directory.py 63 34 46%
bgpcfgd/log.py 15 3 80%
bgpcfgd/main.py 51 51 0%
bgpcfgd/manager.py 41 23 44%
bgpcfgd/managers_allow_list.py 385 21 95%
bgpcfgd/managers_bbr.py 76 0 100%
bgpcfgd/managers_bgp.py 193 193 0%
bgpcfgd/managers_db.py 9 9 0%
bgpcfgd/managers_intf.py 33 33 0%
bgpcfgd/managers_setsrc.py 45 45 0%
bgpcfgd/runner.py 39 39 0%
bgpcfgd/template.py 64 11 83%
bgpcfgd/utils.py 32 24 25%
bgpcfgd/vars.py 1 0 100%
----------------------------------------------------
TOTAL 1128 530 53%
```
**- Which release branch to backport (provide reason below if selected)**
- [ ] 201811
- [x] 201911
- [x] 202006
Issue was because we were relying on port_alias_asic_map dictionary
but that dictionary can't be used as alias name format has changed.
Fix the port alias mapping as what is needed.
Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
