Commit Graph

2496 Commits

Author SHA1 Message Date
Shilong Liu
49ca56dd60 [CI] Fix azp targets for innovium platform (#7870)
There is no rule to build swi target. The correct target is *.bin.
2021-07-19 18:23:49 +00:00
xumia
29311dc44e
Fix CVE-2017-1000487 security alert (#7173) (#7278)
Plexus-utils before 3.0.16 is vulnerable to command injection because it does not correctly process the contents of double quoted strings.
2021-07-08 07:42:07 +08:00
xumia
8cfa9260f3
[ci]: Support PR checks for 201811 (#8114)
Enable PR checks for 201811
Fix target/docker-sonic-vs.gz build break.
2021-07-07 21:59:35 +08:00
xumia
a7725e6480
Fix vtysh shell-ingestion security issue (#7991)
Fix vtysh shell-ingestion security issue
Only expose the limited parameters of the command vtysh show.
2021-06-30 19:32:21 +08:00
Ying Xie
76bef999fd [7050] updating 7050 MMU configurations (#7801)
Why I did it
7050 S4Q31 mmu configuration is missing ALPM configurations, causing not enough memory reserved for routes. Orchagent crashes on a nightly testbed with 6400 route entries.

How I did it
Add the missing ALPM configurations.

How to verify it
Load the configuration on testbed and verified new configuration exists and no more crash.

Signed-off-by: Ying Xie ying.xie@microsoft.com
2021-06-06 04:50:37 +00:00
Neetha John
66dea00c44 Rename AristaQX-32S skus (#7751)
This PR contains the following changes
Original Arista-7050-QX-32S sku (32x40G ports) has been renamed to Arista-7050QX32S-Q32
Arista-7050-QX-32S is symlinked to Arista-7050QX-32S-S4Q31 (4x10G, 31x40G ports)

Signed-off-by: Neetha John <nejo@microsoft.com>
2021-05-29 05:55:58 +00:00
Neetha John
5b7d2651a8 Update MMU and QOS settings for Arista-7050QX-32S-S4Q31 (#7672)
Signed-off-by: Neetha John <nejo@microsoft.com>

Why I did it
Need proper MMU and Qos settings for Arista-7050QX-32S-S4Q31

How I did it
Updated the settings based on Arista-7050-QX-32S
2021-05-29 05:55:40 +00:00
Ying Xie
dfa9913986
[201811] address build error in sonic-config-engine (#7750)
Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2021-05-28 13:57:04 -07:00
Neetha John
d4b4257915
Parse bandwidth for DeviceMgmtLinks (#7749)
Why I did it
Backport #7744

How to verify it
Ran sonic-cfggen on a minigraph and verified that interface of type DeviceMgmtLink has speed set in the PORT table from the bandwidth attribute in the minigraph

Signed-off-by: Neetha John <nejo@microsoft.com>
2021-05-28 12:22:17 -07:00
Shilong Liu
7c4aeb694c
adjust official build platform (#7703)
Why I did it
Adjust azp official build artifacts and platform to keep same with Jenkins.
2021-05-27 10:34:40 +08:00
Ying Xie
871c78244a
[201811] upgrace broadcom SAI to version 3.5.3.7-4 (#7722)
- CS00012108770, CS00012115932: Sesto phy WB support + WB recovery + log messages

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2021-05-26 16:59:11 -07:00
Neetha John
4f3d755ece
Update PG profile settings for Arista-7050QX-32S-S4Q31 (#7675)
Why I did it
PG profile settings need to be aligned with Arista-7050-QX-32S

How I did it
Copy over the current settings from Arista-7050-QX-32S and define params for 10G and 1G speeds as well

Signed-off-by: Neetha John <nejo@microsoft.com>
2021-05-25 11:38:56 -07:00
Shilong Liu
1511baf5fb enable official build in release branch
Signed-off-by: Shilong Liu <shilongliu@microsoft.com>
2021-05-24 18:34:11 +08:00
xumia
78f90ac7a9
Support readonly vtysh for sudoers (#7383) (#7573)
* Support readonly vtysh for sudoers (#7383)

Why I did it
Support readonly version of the command vtysh

How I did it
Check if the command starting with "show", and verify only contains single command in script.

* Fix the type issue in rvtysh
2021-05-19 09:02:33 +08:00
Tamer Ahmed
cdca8da7dd
[201811][dhcp-relay]: Launch DHCP Relay On L3 Vlan Only (#7085)
Recent changes brought l2 vlan concept which does not have DHCP
clients behind them and so DHCP relay is not required. Also,
dhcpmon fails to launch on those vlans as their interfaces
lack IP addresses. This PR backposts #6527 that limits launch
of both DHCP relay and dhcpmon to L3 vlans only.

original-pr: #6527
singed-off-by: Tamer Ahmed tamer.ahmed@microsoft.com
2021-05-12 09:21:22 -07:00
Shilong Liu
f5bd97aef1 remove unuse file
Signed-off-by: Shilong Liu <shilongliu@microsoft.com>
2021-05-12 15:44:37 +08:00
shilongliu
a21bd201e7 [CI] Update azp files with master branch
Signed-off-by: shilongliu <shilongliu@microsoft.com>
2021-05-12 15:44:37 +08:00
xumia
59c120c18c Export the azure pipeline build id for SONiC version (#7406)
Improve the SONiC version, fix the "azure pipeline build id" part

<target branch name>-<pullrequest id>.<azure pipelines build id>-<merge commit id>
Example: master-7381.11668-43df5c87
2021-05-12 15:44:37 +08:00
xumia
e01ceffbab [ci]: Fix official build not existing issue (#7408)
When submitting a new official build for broadcom, vs, it prompts a error message, which says the job is not defined.
It was caused by the default option "[]", which is not empty, it is used as the jobGroups parameter.
2021-05-12 15:44:37 +08:00
xumia
0acf6c2f01 Improve the PR build version (#7381)
Why I did it
Improve the version of the Pull Request build by changing the local branch name.

How I did it
Change the default branch name merge to [target_branch_name]-[pullrequestid].

How to verify it
For official build, the version is not changed.
For pull request build, the version as below:
2021-05-12 15:44:37 +08:00
xumia
773ddbf433 [ci] Fix the boolean value case sensitive issue in Azure Pipelines (#7399)
Why I did it
Fix the boolean value case sensitive issue in Azure Pipelines

When passing parameters to a template, the "true" or "false" will have case sensitive issue, it should be a type casting issue.
To fix it, we change the true/false to yes/no, to escape the trap.

Support to override the job groups in the template, so PR build has chance to use different build parameters, only build simple targets. For example, for broadcom, we only build target/sonic-broadcom.bin, the other images, such as swi, debug bin, etc, will not be built.
2021-05-12 15:44:37 +08:00
liushilongbuaa
a8910607a3 [CI] Use the same template in PR-build pipeline and official-build pipeline
Signed-off-by: liushilongbuaa <shilongliu@microsoft.com>
2021-05-12 15:44:37 +08:00
Shilong Liu
c4c9661984 [CI] Support 202012 branch generating version files for reproducible build (#7303)
for reproducible build azp

Signed-off-by: Shilong Liu <shilongliu@microsoft.com>
2021-05-12 15:44:37 +08:00
liushilongbuaa
51bc97e4fc [CI] Fix pipeline template artifacts struct bug and cleanup bug
Signed-off-by: liushilongbuaa <shilongliu@microsoft.com>
2021-05-12 15:44:37 +08:00
xumia
9930ba0df0 [ci] Fix no enough space issue in docker root in multi arch build (#7321)
Why I did it
Failed to build the centec-arm64 for no space in docker data root.

How I did it
Change to use the data root to the folder under /data.
See detail info about DOCKER_DATA_ROOT_FOR_MULTIARCH in the file Makefile.work.

How to verify it
Set the environment variable, then the /data used as docker root.
2021-05-12 15:44:37 +08:00
xumia
2132d16ec4 Fix the SONiC version starting with HEAD issue (#7319) 2021-05-12 15:44:37 +08:00
xumia
0cd5c2a770 [ci] Fix the wrong build options issue (#7287)
Why I did it
Fix the wrong build options and improve display name for some tasks
1. Fix the wrong build architecture for arm64 and armhf
2. Fix the build timeout parameter
2021-05-12 15:44:37 +08:00
Shilong Liu
a375b9b7a5 [CI] Fix the wrong environment variable PLATFORM passing into the slave container issue (#7262)
* fix

* fix
2021-05-12 15:44:37 +08:00
Shilong Liu
9cc3721150 [CI] Update sonic image official build template. (#7198)
* draft

* fix

* Update new-build-template.yml

* Update azure-pipelines-build.yml

* Update azure-pipelines-job-groups.yml

* fix

* fix

* draft

* fix

* fix

* fix

* fix

* fix

* fix

Signed-off-by: Shilong Liu <shilongliu@microsoft.com>

Co-authored-by: Sonic Automation <svc-acs@microsoft.com>
2021-05-12 15:44:37 +08:00
lguohan
413103c9e4 [ci]: get build logs when build fails (#7219)
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2021-05-12 15:44:37 +08:00
lguohan
1dd2d94db7 [ci]: gzip the vm image disk and memdmp (#7131)
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2021-05-12 15:44:37 +08:00
Santhosh Kumar T
1952afdeed
[201811] DellEMC S6100 SSD Monitor porting changes (#7290)
Why I did it
To monitor the SSD health condition in DellEMC S6100 platform post upgrade.
A daemon is introduced to monitor the SSD every one hour.
To check for SSD status at boot time and at the time of cold-reboot.
All these changes are supported only for newer SSD firmware.
Porting changes from 201911 branch
Added a platform_reboot_pre_check script to prevent cold-reboot based on SSD status.
Depends on Azure/sonic-utilities#1557
2021-05-04 09:44:24 -07:00
Aravind Mani
8e31179538
[201811] Dell S6100: Modify transceiver change event from interrupt to poll mode (#7332)
#### Why I did it

- xcvrd crash was seen in latest 201811 images.
- For Dell S6100,API 2.0 uses poll mode while 1.0 was still using interrupt mode.

#### How I did it

- Modified get_transceiver_change_event in 1.0 to poll mode in all the related branches.
2021-05-03 12:01:37 -07:00
Ying Xie
399991fbc0
[201811][build] install python-arptable and psutil in sonic slave docker (#7489)
Why I did it
sonic snmp subagent build was failing recently.

How I did it
install python-arptable and psutil in sonic slave docker to address sonic snmp subagent build issue.

How to verify it
build 201811 image locally.

Signed-off-by: Ying Xie ying.xie@microsoft.com
2021-04-30 07:14:16 -07:00
Sangita Maity
b437b77ec9
[201811] Fix swsssdk build by installing redis explicitly (#7414)
Why I did it
for 201811 build image, swsssdk wheel package is not getting build due to redis==2.10.6 requirement issue. Notied that after upgrading pip to 20.3.3, we are experiencling this issue.

the issue

21:10:41  /sonic/src/sonic-py-swsssdk /sonic
21:10:41  running test
21:10:41  Searching for redis==2.10.6
21:10:41  Reading https://pypi.python.org/simple/redis/
21:10:41  Couldn't find index page for 'redis' (maybe misspelled?)
21:10:41  Scanning index of all packages (this may take a while)
21:10:41  Reading https://pypi.python.org/simple/
21:10:41  No local packages or download links found for redis==2.10.6
21:10:41  error: Could not find suitable distribution for Requirement.parse('redis==2.10.6')
21:10:41  [  FAIL LOG END  ] [ target/python-wheels/swsssdk-2.0.1-py3-none-any.whl ]
21:10:41  slave.mk:422: recipe for target 'target/python-wheels/swsssdk-2.0.1-py3-none-any.whl' failed
21:10:41  make: *** [target/python-wheels/swsssdk-2.0.1-py3-none-any.whl] Error 1
21:10:43  Makefile.work:132: recipe for target 'target/sonic-aboot-broadcom.swi' failed
21:10:43  make[1]: *** [target/sonic-aboot-broadcom.swi] Error 2
21:10:43  make[1]: Leaving directory '/data/johnar/workspace/broadcom/buildimage-brcm-201811'
21:10:43  Makefile:6: recipe for target 'target/sonic-aboot-broadcom.swi' failed
So, what I have understood till now, if pip v20.3.3 is able to produce a wheel that is not installable because it raises pip._vendor.packaging.version.InvalidVersion or some error like that (resource- > pypa/pip#9206), it just raises an exception when building the wheel.

SO, this issue occurs when we pinned down pip to 20.3.3 in short.

As of now, there are two solutions mentioned below.

pin down pip to 20.3.3(which it is) and explicitly install packages.
pin down pip to 20.3.4(pip wheel now verifies the built wheel contains valid metadata, and can be installed by a subsequent pip install.)(resource-> https://pip.pypa.io/en/stable/news/) -- didn't try yet
How I did it
Install nose explicitly for mockredispy
Install redis==2.10.6 for swsssdk tests.

How to verify it
Run local build after removing all previously built dockers.
2021-04-28 23:09:52 -07:00
lguohan
f4fdf310b9
[build] Fix the snmp docker build error. (#7452) (#7459)
* [build] Fix the snmp docker build error. (#7452)

Issue is get_pip.py is moved to pip 21.1 (https://github.com/pypa/get-pip/commits/main) which is not compatible with 3.6.
Issue of pip itself is fixed as part of 21.1.1 in pip community (pypa/pip#9835).
However get-pip.py is still not updated to latest pip. Also get.pip.py does not support python 3.6 version explicitly (pypa/get-pip#88)

Step 15/29 : RUN curl https://bootstrap.pypa.io/get-pip.py | python3.6
 ---> Running in bece31f49267
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 1891k  100 1891k    0     0  9564k      0 --:--:-- --:--:-- --:--:-- 9600k
Traceback (most recent call last):
  File "<stdin>", line 24298, in <module>
  File "<stdin>", line 139, in main
  File "<stdin>", line 115, in bootstrap
  File "<stdin>", line 96, in monkeypatch_for_cert
  File "/tmp/tmp5fnxrz0a/pip.zip/pip/_internal/commands/__init__.py", line 9, in <module>
  File "/tmp/tmp5fnxrz0a/pip.zip/pip/_internal/cli/base_command.py", line 12, in <module>
  File "/tmp/tmp5fnxrz0a/pip.zip/pip/_internal/cli/cmdoptions.py", line 30, in <module>
  File "/tmp/tmp5fnxrz0a/pip.zip/pip/_internal/utils/hashes.py", line 2, in <module>
ImportError: cannot import name 'NoReturn'
The command '/bin/sh -c curl https://bootstrap.pypa.io/get-pip.py | python3.6' returned a non-zero code: 1
How I did:

Got the file from https://github.com/pypa/get-pip/tree/21.0 and added to the buildimage
pin pip to the previous release 21.0.1. (Similar is done in other public repos eg: grpc/grpc-java#8115)

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2021-04-28 23:08:26 -07:00
Ying Xie
b8137369c7
[device] add phy_an_lt_msft to permitted list (#7390)
Why I did it
201811 build was failing due to the newly added bcm config file key word was not on the permitted list

How I did it
add phy_an_lt_msft to permitted list.

How to verify it
Build device package.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2021-04-20 21:37:50 -07:00
Joe LeVeque
4a3324b73e
[201811] Add SOC property to enable AN/LT on some platforms (#7388)
Why I did it
To enable autonegotiation/link training on some Broadcom-based platforms (Arista 7060 and 7260, Celestica DX010)

How I did it
Add appropriate SOC property for enabling the feature to the Broadcom config files of appropriate platforms
Also convert line endings to UNIX format for one Celestica file
2021-04-20 18:56:45 -07:00
Ying Xie
fa036c0ead
[201811][bcm sai] ugprade Broadcom SAI to 3.5.3.7-2 (#7385)
* 1d588e38 2021-03-25 | Pending changes for CS00011808451 (bcm_sai/REL_3.5.3, INT_3.5) [BrcmSAI]
* 53efbe91 2021-03-18 | Fix for CS00011810218 [BRCM-DevOps]
* ca0dc86c 2021-03-11 | Fix for CS00011808451 (msazure/INT_3.5, bcm_sai/REL_3.5.3, INT_3.5) [BrcmSAI]
* 208ea62e 2021-01-08 | Update changelog [BRCM-DevOps]
* 3961673c 2021-01-08 | Update version after sdk patch and egress obj leak fixes. [BRCM-DevOps]
* ce3818ad 2021-01-08 | Fix for CS00011581499 [BrcmSAI]
* 1e06157f 2021-01-08 | Merged SDk patch for CS00011651922 [BrcmSAI]
* 4d0c2c86 2021-04-06 | Merged PR 4328641: Add support for AN/LT (HEAD -> 3.5-201811, msazure/3.5-201811) [Joe LeVeque]

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2021-04-20 15:49:14 -07:00
Ying Xie
98c3bd757b
[201811][utilities] advance utilities submodule head (#7373)
* 72510da 2021-04-19 | [201811] Stop PMON docker before cold boot and soft reboot (#1515) (HEAD -> 201811, github/201811) [Aravind Mani]

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2021-04-20 15:33:46 -07:00
abdosi
a5b9abcd4b [201911] Fix easy_install error when installing pip (#7272)
see below error:

+ sudo https_proxy= LANG=C chroot ./fsroot easy_install pip==20.3.3
Searching for pip==20.3.3
Reading https://pypi.python.org/simple/pip/
Couldn't find index page for 'pip' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
No local packages or working download links found for pip==20.3.3
error: Could not find suitable distribution for Requirement.parse('pip==20.3.3')

How I fix:

Install python-pip via apt-get
Pin the version to 20.3.3
Master has same changes.

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>
2021-04-16 00:09:13 +00:00
Ying Xie
ae401ff92a
[201811][utilities] advance submodule head (#7280)
Utilities:
* c96f19d 2021-04-08 | [load_minigraph]: Avoid starting PFCWD for EPMS devicetype (#1552) (HEAD -> 201811, github/201811) [Sumukha Tumkur Vani]

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2021-04-09 19:07:33 -07:00
Sumukha Tumkur Vani
b6ca3bd5bb
add EPMS devicetype (#7255) 2021-04-09 12:44:31 -07:00
Ying Xie
273f99d0e5
[201811][utilities] advance submodule head (#7266)
* d58ba82 2021-04-05 | Fix fast-reboot fdb filtering for IPv6 addresses (#1548) (github/201811) [Vaibhav Hemant Dixit]
* cc7ad9e 2021-03-24 | Fast reboot: check if the files are present before accessing (#1526) [Vaibhav Hemant Dixit]
* 410323a 2021-03-24 | [201811] Add missing variable DEVPATH to fast-reboot script (#1525) [Vaibhav Hemant Dixit]
* eb5023a 2021-03-21 | [201811] soft-reboot (#1487) [Sujin Kang]
* 3f10540 2020-12-08 | [fast-reboot] Fix fast-reboot when NDP entries are present (#1295) (HEAD -> 201811) [shlomibitton]

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2021-04-08 18:57:05 -07:00
Ying Xie
d2b9b1ec0c [Arista] add MMU configuration for Arista 7260 C64 (#7027)
Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2021-04-08 16:29:19 +00:00
Nazarii Hnydyn
72c506061c
[Mellanox] Improve FW upgrade logging: add syslog logger (#7221)
Basically mlnx-fw-upgrade.sh is used in two places:
1. https://github.com/Azure/sonic-buildimage/blob/201811/files/scripts/syncd.sh#L109
```bash
        /usr/bin/mst start
        /usr/bin/mlnx-fw-upgrade.sh
        /etc/init.d/sxdkernel start
        /sbin/modprobe i2c-dev
```
2. https://github.com/Azure/sonic-buildimage/blob/201811/device/mellanox/x86_64-mlnx_msn2700-r0/platform_reboot#L32
```bash
ParseArguments "$@"

${FW_UPGRADE_SCRIPT} --upgrade --verbose
EXIT_CODE="$?"
```

In first case the `stdout` is redirected to `syslog` directly by `systemd`.
Thus, the `syslog` logger is only required in second case.

#### Why I did it
* To improve ASIC/CPLD FW upgrade logging
* To improve CPLD upgrade time

#### How I did it
* Added `syslog` logger support
* Replaced `_pciconf0` -> `_pci_cr0` to reduce CPLD upgrade time

#### How to verify it
1. mlnx-fw-upgrade.sh --upgrade
2021-04-07 10:57:12 -07:00
Ying Xie
161bca602f
[202018][utilities] set the utilities submodule head properly (#7079)
Signed-off-by: Ying Xie <ying.xie@microsoft.com>
2021-03-18 13:58:11 -07:00
Joe LeVeque
93fbb8fd96
[201811][sonic-config-engine] Parse AutoNegotiation element from LinkMetadata section of minigraph file (#7048)
Backport of https://github.com/Azure/sonic-buildimage/pull/7031 to the 201811 branch

#### Why I did it

To enable parsing the `AutoNegotiation` element from the LinkMetadata section of minigraph file

#### How I did it

Parse the value `AutoNegotiation` element from the `LinkMetadata` section of minigraph file. If the element is present, an `autoneg` key will be added to the port in the `PORT` table of Config DB with a value of either `0` or `1`

If an `autoneg` value is present in port_config.ini, the value from the minigraph will take precedence, overriding that value.

Also remove `AutoNegotiation` and `EnableAutoNegotiation` elements from the `DeviceInfo` section, as we will use this data in the `LinkMetadata` section to determine whether to enable auto-negotiation for a port.
2021-03-18 13:17:51 -07:00
rkdevi27
6c2fd18f51
Fixed S6000 abrupt reboot in 201811 (#6923)
Why I did it
The S6000 devices, the cold reboot is abrupt and it is likely to cause issues which will cause the device to land into EFI shell. Hence the platform reboot will happen after graceful unmount of all the filesystems as in S6100.

How I did it
Moved the platform_reboot to platform_reboot_override and hooked it to the systemd shutdown services as in S6100.
Fixed the "/host unmount failed" issue as well in 201811.

How to verify it
Issue "reboot" command to verify if the reboot is happening gracefully.
2021-03-12 11:09:54 -08:00
Vaibhav Hemant Dixit
2877cc9ff6
[submodule]: update sonic-utilities (#6991)
[201811] Warmboot script improvements - timeout in exec and disable swss autorestart Azure/sonic-utilities#1474
2021-03-11 20:19:20 -08:00