Commit Graph

39 Commits

Author SHA1 Message Date
Liu Shilong
979516633d
[ci] Support pensando platform build in pipeline. (#17512)
Why I did it
Update pipeline file to support pensando's build.

Work item tracking
Microsoft ADO (number only): 26087700
How I did it
How to verify it
2023-12-15 16:52:51 +08:00
lixiaoyuner
b2c8ad8b10
Fix K8S_OPTIONS maybe empty issue (#16891)
Why I did it
K8S_OPTIONS maybe empty, so there will be syntax error. Need to fix this issue.

Work item tracking
Microsoft ADO (number only): 25495020
How I did it
Add "" for K8S_OPTIONS to avoid exception.

How to verify it
No more exception is throwed in PR build validation pipeline.
2023-10-23 14:02:07 +08:00
Liu Shilong
78415800a5
[ci] Disable building broadcom raw image because of S6100 device disk space limit. (#16516)
* [ci] Disable building broadcom raw image because of S6100 device disk space limit.
2023-09-11 15:10:03 -07:00
lixiaoyuner
10b65d9826
Add k8s master code new (#15716)
Why I did it
Currently, k8s master image is generated from a separate branch which we created by ourselves, not release ones. We need to commit these k8s master related code to master branch for a better way to do k8s master image build out.

Work item tracking
Microsoft ADO (number only):
19998138
How I did it
Install k8s dashboard docker images
Install geneva mds and mdsd and fluentd docker images and tag them as latest, tagging latest will help create container always with the latest version
Install azure-storage-blob and azure-identity, this will help do etcd backup and restore.
Install kubernetes python client packages, this will help read worker and container state, we can send these metric to Geneva.
Remove mdm debian package, will replace it with the mdm docker image
Add k8s master entrance script, this script will be called by rc-local service when system startup. we have some master systemd services in compute-move repo, when VMM service create master VM, VMM will copy all master service files inside VM, the entrance script will setup all services according to the service files.
When the entrance script content changed, the PR build will set include_kubernetes_master=y to help do validation for k8s master related code change. The default value of include_kubernetes_master should be always n for public master branch. We will generate master image from internal master branch
How to verify it
Build with INCLUDE_KUBERNETES_MASTER = y
2023-07-25 07:44:59 +08:00
Liu Shilong
3b10334e10
[ci] Remove saiserverv2 build in official build. (#15191)
Why I did it
libsaithriftv2 build fails and nobody is maintaining saiserverv2's build.
Remove them from official build.

Work item tracking
Microsoft ADO (number only): 23764652
How I did it
How to verify it
2023-05-23 10:22:31 +00:00
xumia
50fb266c16
[Ci] Support marvell/marvell-arm64 build (#14875)
Why I did it
Support marvell/marvell-arm64 build

Work item tracking
Microsoft ADO (number only): 19995559
How I did it
2023-05-05 12:40:35 +08:00
Liu Shilong
f19c8a7be5
[ci] Fix build issue for vs vhdx image. #14427
Why I did it
sonic-slave-stretch build failed for mmh3 version update to 3.10 on Mar 24.

How I did it
Enable reproducible build for vhdx image.

How to verify it
2023-03-27 16:58:37 +00:00
Liu Shilong
5a2a95998c
[ci] Fix docker hang issue and change template reference branch (#13894)
Why I did it
Azure pipeline change.
Use common template to make it easy to change common steps.
Fix docker hang issue.

How I did it
2023-02-22 13:00:06 +08:00
Liu Shilong
22e46207c8
[ci] Kill hanged docker build process to avoid build timeout issue. (#13726)
Why I did it
Docker build has a low rate of hanging up.
It hangs on different steps. So, it looks like a bug in docker daemon.

How I did it
Start a daemon process to scan running time more than 1 hours, and kill the process.

How to verify it
2023-02-16 21:58:14 +08:00
Richard.Yu
689f7d2c43
[SAI-PTF][BFN]Enable saiserver test container on bfn container (#13166)
Why I did it
Enable Test sai api on bfn container with a lightweight container(saiserver).

How I did it
enable saiserver container on barefoot platform.

add docker-saiserver-bfn.mk for building saiserver container
in platform/barefoot/docker-saiserver-bfn, add necessary files that needs in saiserver container
How to verify it
Tested on Intel platform ec9516

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>
2023-01-06 11:45:03 +08:00
zitingguo-ms
b774ebfdc2
[SAI-PTF] Publish docker saiserverv2 in master branch (#12842)
Why I did it
Publish docker saiserverv2 in the build pipeline.

How I did it
Add docker saiserverv2 target in the build template.

How to verify it
Test this by running this pipeline: https://dev.azure.com/mssonic/build/_build/results?buildId=182134&view=results
2022-11-30 22:26:54 -08:00
Richard.Yu
47d63bcd06
[SAI PTF] SAI PTF docker support sai-ptf v2 (#12719)
* [SAI PTF] SAI PTF docker support sai-ptf v2

Publish the sai-ptf docker.

Take part of the change from previous PR #11610 (already reverted as some cache issue)
Cause in #11610, added two new target in it, one is sai-ptf another one is syncd-rpc with sai-ptf v2, to make the upgrade with more clear target, use this one take the sai-ptf one.

Test one:
NOSTRETCH=y NOJESSIE=y make configure PLATFORM=vs
NOSTRETCH=y NOJESSIE=y NOBULLSEYE=y SAITHRIFT_V2=y make target/docker-ptf-sai.gz

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* remove useless change

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* remove useless parameters

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* remove useless change

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* Update azure-pipelines-build.yml

remove a useless option

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>
2022-11-17 04:42:51 -08:00
Liu Shilong
6d78199d6f
Revert "[SAI PTF]Syncd-rpc and PTF docker support sai ptf v2 (#11610)" (#12677)
This reverts commit f0873f29d8.
2022-11-14 09:56:10 +08:00
Richard.Yu
f0873f29d8
[SAI PTF]Syncd-rpc and PTF docker support sai ptf v2 (#11610)
* support sai-ptf-v2 in libsaithrift vs

* add build target docker-ptf-sai syncd-rpcv2 and saiserverv2

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* add docker ptf sai

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* add build condition for broadcom

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* add docker syncd dbg and add debug symbol to docker-saiserverv2

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* correct the build option

* change the azure pipeline build template

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* change build option for docker-ptf-sai

* enable ptf-sai docker build

* remove the build for syncd-rpcv2

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* fix issue in build tempalte

* ignore useless package build when build sai-ptf

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* remove scapy version contraint

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* remove duplicated target docker-ptf

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* change template for testing the pipeline

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* remove duplicated target

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* fix error in make script

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* add shel to setup env

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* replace with certain platform name

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* disable cache for syncd-rpcv2

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* test without cache

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* disable cache

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* testing: disable the cache for build syncd-rpcv2

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* add cache back and get the code ready for testing

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* refactor code

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* add workaround for issue in rules/sairedis.dep

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

* refactor code

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>

Signed-off-by: richardyu-ms <richard.yu@microsoft.com>
2022-11-07 21:47:52 +08:00
Saikrishna Arcot
adffbd4643
Build the Broadcom DNX RPC container as part of the official build (#11829)
With the Broadcom syncd containers getting upgraded to Bullseye, the DNX
RPC container is no longer automatically built. Explicitly add a make
command to build it.

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
2022-08-24 11:42:15 -07:00
xumia
6f323b3e83
[ci] Support the cross build for armhf/arm64 (#11587)
Why I did it
[ci] Support the cross build for armhf/arm64
2022-08-01 11:31:02 +08:00
Yakiv Huryk
1568667011
[ci] add docker-sonic-vs with ASAN build (#11297)
The asan-enabled docker image will be used in other CIs that run DVS tests.

Why I did it
To add a possibility to run DVS tests with ASAN for other CIs (e.g. swss).

How I did it
Added the 'asan_image' flag to the vs job group.

How to verify it
Run the CI and check the docker-sonic-vs-asan.gz artifact.
2022-07-04 08:40:04 +08:00
Shilong Liu
2851884c8b
[ci] Support to skip vstest using include/exclude config file. (#11086)
example:
├── folderA
│  ├──  fileA (skip vstest)
│  ├──  fileB
│  └──  fileC
If we want to skip vstest when changing /folderA/fileA, and not skip vstest when changing fileB or fileC.

vstest-include:
^folderA/fileA

vstest-exclude:
^folderA
2022-06-17 15:39:41 +08:00
xumia
f24a04fa3d
[Ci]: check if there is a sonic dirty version issue (#10445)
Why I did it
[Ci]: check if there is a sonic dirty version issue
If there is a dirty version issue in PR build, the build will be failed.
2022-04-08 10:49:21 +08:00
xumia
baa24b3cb1
[Ci]: exclude innovium build and increase broadcom build timeout (#10482)
Why I did it
Exclude the innovium build in upgrading version build, currently, the builds are always failed, exclude the build temporarily.
Increase the broadcom build timeout.
2022-04-07 20:04:43 +08:00
Shilong Liu
5d23035333
[CI] Fix Azure pipeline set -e not work. (#9282)
In azure pipeline template 'set -e' not works as expected.
2021-11-17 21:01:41 -08:00
Guohan Lu
0c68585ecb [ci]: build marvel armhf on sonicbld-armhf pool [Guohan Lu]
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2021-10-27 13:00:00 -07:00
Guohan Lu
150673704c [ci]: build centec arm64 to sonicbld-arm64 pool
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2021-10-25 16:26:48 -07:00
Shilong Liu
e6699a0132
Add pipeline to build vhdx image. (#8665)
* Add pipeline to build vhdx image
2021-09-17 16:18:36 +08:00
Shilong Liu
66089647b9
[CI] lengthen time limit for arm platform (#8295)
#### Why I did it
Building image for arm platform needs more time.
#### How I did it
lengthen time limit from 1.5 days to 2 days
2021-08-04 01:45:38 -07:00
xumia
0b53a60412
[ci]: build rpc image for mellanox (#7905) 2021-06-20 23:12:25 -07:00
Shilong Liu
aeeef2d382
[CI] Fix azp targets for innovium platform (#7870)
There is no rule to build swi target. The correct target is *.bin.
2021-06-15 16:38:02 -07:00
Myron Sosyak
9a7b83c693
[BFN] Add missed param for syncd-rpc build (#7792)
To fix syncd-rpc build for BFN paltform
2021-06-08 09:43:00 -07:00
Myron Sosyak
05dddea9da
[BFN] Enable syncd-rpc build (#7646)
Why I did it
To enable syncd-rpc for Barefoot build

How I did it
Set the flag

How to verify it
ENABLE_SYNCD_RPC=y make configure PLATFORM=barefoot
ENABLE_SYNCD_RPC=y make all
2021-06-02 20:01:35 +08:00
Guohan Lu
853c214951 [ci]: set -ex for official build to exit on any build failures
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2021-05-03 12:31:52 -07:00
Guohan Lu
e1ff8b6ad6 [ci]: change timeout to 36 hours for armhf/arm64 build
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2021-05-02 09:55:15 -07:00
xumia
45d59a59ef
Export the azure pipeline build id for SONiC version (#7406)
Improve the SONiC version, fix the "azure pipeline build id" part

<target branch name>-<pullrequest id>.<azure pipelines build id>-<merge commit id>
Example: master-7381.11668-43df5c87
2021-04-23 17:58:46 +08:00
xumia
4bf118006a
[ci]: Fix official build not existing issue (#7408)
When submitting a new official build for broadcom, vs, it prompts a error message, which says the job is not defined.
It was caused by the default option "[]", which is not empty, it is used as the jobGroups parameter.
2021-04-22 21:56:09 -07:00
xumia
39d877c1ad
[ci] Fix the boolean value case sensitive issue in Azure Pipelines (#7399)
Why I did it
Fix the boolean value case sensitive issue in Azure Pipelines

When passing parameters to a template, the "true" or "false" will have case sensitive issue, it should be a type casting issue.
To fix it, we change the true/false to yes/no, to escape the trap.

Support to override the job groups in the template, so PR build has chance to use different build parameters, only build simple targets. For example, for broadcom, we only build target/sonic-broadcom.bin, the other images, such as swi, debug bin, etc, will not be built.
2021-04-23 09:26:16 +08:00
xumia
ac99b8fee8
[ci] Fix no enough space issue in docker root in multi arch build (#7321)
Why I did it
Failed to build the centec-arm64 for no space in docker data root.

How I did it
Change to use the data root to the folder under /data.
See detail info about DOCKER_DATA_ROOT_FOR_MULTIARCH in the file Makefile.work.

How to verify it
Set the environment variable, then the /data used as docker root.
2021-04-15 14:32:08 +08:00
xumia
7df4c0ad7c
[ci] Fix the wrong build options issue (#7287)
Why I did it
Fix the wrong build options and improve display name for some tasks
1. Fix the wrong build architecture for arm64 and armhf
2. Fix the build timeout parameter
2021-04-12 07:43:06 +08:00
Shilong Liu
1c220d923a
[CI] Fix the wrong environment variable PLATFORM passing into the slave container issue (#7262)
* fix

* fix
2021-04-09 07:36:53 +08:00
Shilong Liu
dcb9ef3d44
[CI] Update sonic image official build template. (#7198)
* draft

* fix

* Update new-build-template.yml

* Update azure-pipelines-build.yml

* Update azure-pipelines-job-groups.yml

* fix

* fix

* draft

* fix

* fix

* fix

* fix

* fix

* fix

Signed-off-by: Shilong Liu <shilongliu@microsoft.com>

Co-authored-by: Sonic Automation <svc-acs@microsoft.com>
2021-04-08 07:05:31 +08:00
xumia
d3040bd804
Add azure pipelines for reproducible build (#6870)
* Add azure pipelines for reproducible build

* Add always build flag for upgrade build

* Fix the build version ubuntu-20.04
2021-02-25 13:44:58 +08:00