Why I did it
Innovium platform has build issue.
Remove it from upgrade version pipeline.
Work item tracking
Microsoft ADO (number only):
How I did it
How to verify it
#### Why I did it
Cherry-pick commits from master to support the snapshot based mirror, and fix the code conflicts. And add the last commit to fix the build broken issue according to the mirror change.
ad162ae0e [Build] Optimize the version control for Debian packages (https://github.com/sonic-net/sonic-buildimage/pull/14557)
38c5d7fce [Build] Support j2 template for debian sources for docker ptf (https://github.com/sonic-net/sonic-buildimage/pull/13198)
5e4826ebf [Ci] Support to use the same snapshot for all platform builds (#13913)
820692563 [Build] Change the default mirror version config file (#13786)
5e4a866e3 [Build] Support Debian snapshot mirror to improve build stability (#13097)
ac5d89c6a [Build] Support j2 template for debian sources (#12557)
Why I did it
sonic-slave-stretch build failed for mmh3 version update to 3.10 on Mar 24.
How I did it
Enable reproducible build for vhdx image.
How to verify it
Why I did it
Docker build has a low rate of hanging up.
It hangs on different steps. So, it looks like a bug in docker daemon.
How I did it
Start a daemon process to scan running time more than 1 hours, and kill the process.
How to verify it
The display of azure pipeline is not specific now, such as when the step Run test fails, the display of itself shows successful, but the display of step Kvmdump shows fails, but actually, the step Kvmdump doesn't fail. I improve the display of azure pipeline in this pr, each step has its own success or failure, and is shown in azure pipeline.
Why I did it
The display of azure pipeline is not specific now, such as when the step Run test fails, the display of itself shows successful, but the display of step Kvmdump shows fails, but actually, the step Kvmdump doesn't fail. I improve the display of azure pipeline in this pr, each step has its own success or failure, and is shown in azure pipeline.
How I did it
Each step has its own signature of success or failure.
Using the chain of responsibility pattern to manage all status.
Modify the expected-state in each step.
Why I did it
Currently sonic-slave-* tag is confusing. Set correct tag on sonic-slave-* image.
Fix job name to fit the build.
How I did it
build amd image in amd64:
sonic-slave-bullseye:cfe29bff67c
sonic-slave-bullseye:latest
sonic-slave-bullseye:master
build armhf image in amd64:
sonic-slave-bullseye-march-armhf:33614806dc3
sonic-slave-bullseye-march-armhf:latest
sonic-slave-bullseye-march-armhf:master
build arm64 image in amd64:
sonic-slave-bullseye-march-arm64:f3b1b16c801
sonic-slave-bullseye-march-arm64:latest
sonic-slave-bullseye-march-arm64:master
build arm64 image in arm64:
sonic-slave-bullseye:75cb326c9a7
sonic-slave-bullseye-arm64:latest
sonic-slave-bullseye:master
build armhf image in armhf:
sonic-slave-bullseye:64d178951fc
sonic-slave-bullseye-armhf:latest
sonic-slave-bullseye:master
How to verify it
Why I did it
Makefile needs some dependencies from the Internet. It will fail for network related issue.
Retries will fix most of these issues.
How I did it
Add retries when running commands which maybe related with networking.
How to verify it
Previously, we set timeout in each step such as Lock testbed, Prepare testbed, Run test and KVM dump. When some issue suck like retry happens in one step, it will cause timeout error, but actually, it only needs more time to success. In this pr, we remove the timeout limit in each step and control the timeout outside in each job. When the job runs more than four hours, it will be cancelled.
Why I did it
Previously, we set timeout in each step such as Lock testbed, Prepare testbed, Run test and KVM dump. When some issue suck like retry happens in one step, it will cause timeout error, but actually, it only needs more time to success. In this pr, we remove the timeout limit in each step and control the timeout outside in each job. When the job runs more than four hours, it will be cancelled.
How I did it
Remove the timeout parameter in each step, and control the timeout outside in each job.
Signed-off-by: Yutong Zhang <yutongzhang@microsoft.com>
# Why I did it
Publish docker saiserverv2 in the build pipeline.
# How I did it
Add docker saiserverv2 target in the build template.
# How to verify it
Run test in #12836 and the target has been built out successfully.
Signed-off-by: zitingguo-ms <zitingguo@microsoft.com>
Previously, we hard code the min and max numbers of instance in a plan. In this pr, we support passing the instance numbers of a testplan.
Why I did it
Previously, we hard code the min and max numbers of instance in a plan. In this pr, we support passing the instance numbers of a testplan.
How I did it
Use a variable to set the instance number.
Add dualtor test jobs using TestbedV2 in 202012 branch.
Why I did it
Add dualtor test jobs using TestbedV2 in 202012 branch.
How I did it
Add dualtor test jobs using TestbedV2 in 202012 branch.
Signed-off-by: Yutong Zhang <yutongzhang@microsoft.com>
* [TestbedV2]Migrate t0 and t1-lag to TestbedV2 (#12383)
signed-off-by: jianquanye@microsoft.com
Migrate the t0 and t1-lag test jobs in buildimage repo to TestbedV2.
Why I did it
Migrate the t0 and t1-lag test jobs in buildimage repo to TestbedV2.
How I did it
Migrate the t0 and t1-lag test jobs in buildimage repo to TestbedV2.
Remove ceos type setting
Use 202012 branch as sonic-mgmt branch
* Modify template to use 202012 branch
Why I did it
202012 PR test is failing due to some recent change in sonic-mgmt master branch.
How I did it
Use matching sonic-mgmt branch to run 202012 branch PR tests.
How to verify it
this PR test.
Signed-off-by: Ying Xie ying.xie@microsoft.com
* [build] Add version files to docker image dependencies
* [ci] Support to skip vstest using include/exclude config file. (#11086)
example:
├── folderA
│ ├── fileA (skip vstest)
│ ├── fileB
│ └── fileC
If we want to skip vstest when changing /folderA/fileA, and not skip vstest when changing fileB or fileC.
vstest-include:
^folderA/fileA
vstest-exclude:
^folderA
Why I did it
Fix the target directory not empty issue when publishing artifacts.
Some of the artifacts are published to $(Build.ArtifactStagingDirectory)/target/ before source code checked out.
Why I did it
Introduce 2 sub jobs for kvmtest t0 job in sonic-mgmt repo in PR Azure/sonic-mgmt#4861
But in sonic-buildimage repo, because section parameter is null, it always run the part 2 test scripts in kvmtest t0 job.
It missed the part 1 test scripts in kvmtest.sh.
How I did it
Split kvmtest t0 job into two sub jobs such as sonic-mgmt repo and run them in parallel to save time.
How to verify it
Submit PR will trigger the pipeline to run.
Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
(cherry picked from commit 44028723ef)
Co-authored-by: Zhaohui Sun <94606222+ZhaohuiS@users.noreply.github.com>
Why I did it
It is not necessary to trigger the publish pipeline when build is failed.
How I did it
Remove the condition in the azp task, change to use template condition.
Why I did it
Support to trigger a pipeline to download and publish artifacts to storage and container registry.
Support to specify the patterns which docker images to upload.
How I did it
Pass the pipeline information and the artifact information by pipeline parameters to the pipeline which will be triggered a new build. It is to decouple the artifacts generation and the publish logic, how and where the artifacts/docker images will be published, depends on the triggered pipeline.
How to verify it
This reverts commit f4bbcd1cf1. The original one was missing one file ".azure-pipelines/azure-pipelines-repd-build-variables.yml" and break the Azure pipeline.
Why I did it
docker hub will limit the pull rate.
Use ACR instead to pull debian related docker image.
How I did it
Set DEFAULT_CONTAINER_REGISTRY in pipeline.
#### Why I did it
Merged from master branch: https://github.com/Azure/sonic-buildimage/pull/9443
Fix the nodesource.list cannot read issue, it is cased by the full path not used.
```
2021-12-03T06:59:26.0019306Z Removing intermediate container 77cfe980cd36
2021-12-03T06:59:26.0020872Z ---> 528fd40e60f6
2021-12-03T06:59:26.0021457Z Step 81/81 : RUN post_run_buildinfo
2021-12-03T06:59:26.0841136Z ---> Running in d804bd7e1b06
2021-12-03T06:59:29.1626594Z [91mDEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
2021-12-03T06:59:34.2960105Z [0m[91m/usr/bin/sed: can't read nodesource.list: No such file or directory
2021-12-03T06:59:34.5094880Z [0mThe command '/bin/sh -c post_run_buildinfo' returned a non-zero code: 2
```
Support marvell-armhf dpkg cache and the azp check.
Waiting for merging PR #9381 to 202012 branch, so only azp template change in this PR.
Move the VS build to a new stage BuildVS, change the Test stage only depending on BuildVS, running the BuildVS and the other platform's build in parallel. The Test stage do not has dependency on the marvel-armhf build, reduce the overall build time caused by longer build time of marvel-armhf build.
