Why I did it
Docker build has a low rate of hanging up.
It hangs on different steps. So, it looks like a bug in docker daemon.
How I did it
Start a daemon process to scan running time more than 1 hours, and kill the process.
* Enable marvell-armhf saiserver docker
Signed-off-by: zitingguo-ms <zitingguo@microsoft.com>
* fix libsaithriift build env
Signed-off-by: zitingguo-ms <zitingguo@microsoft.com>
* fix thrift 014 dependent issue in armhf
* fix build env
Signed-off-by: zitingguo-ms <zitingguo@microsoft.com>
* [sai_ptf]fix thrift armhf build
in armhf buidl failed as no python command
how
add a checker for different python command, python/python3 and base on result use the right command
verify
container build
* [Thrift_014[armhf]]Fix libboost_unit_test_framework.a not found during build
Why
error happen build thirft in armhf
How
fix this issue, add a soft link for the dependent file
Verify
Build pipeline
Signed-off-by: richardyu-ms <richard.yu@microsoft.com>
* add metadata dependence
Signed-off-by: zitingguo-ms <zitingguo@microsoft.com>
* change build pipeline
Signed-off-by: zitingguo-ms <zitingguo@microsoft.com>
---------
Signed-off-by: zitingguo-ms <zitingguo@microsoft.com>
Signed-off-by: richardyu-ms <richard.yu@microsoft.com>
Co-authored-by: richardyu-ms <richard.yu@microsoft.com>
Why I did it
Add multi-asic test jobs using TestbedV2 in 202205 branch.
How I did it
1. Add a job in azure-pipelines.yml to add a multi-asic test job using TestbedV2 in 202205 branch.
2. Add a parameter num_asic in .azure-pipelines/run-test-scheduler-template.yml to appiont the specific image version.
The display of azure pipeline is not specific now, such as when the step Run test fails, the display of itself shows successful, but the display of step Kvmdump shows fails, but actually, the step Kvmdump doesn't fail. I improve the display of azure pipeline in this pr, each step has its own success or failure, and is shown in azure pipeline.
Why I did it
The display of azure pipeline is not specific now, such as when the step Run test fails, the display of itself shows successful, but the display of step Kvmdump shows fails, but actually, the step Kvmdump doesn't fail. I improve the display of azure pipeline in this pr, each step has its own success or failure, and is shown in azure pipeline.
How I did it
Each step has its own signature of success or failure.
Using the chain of responsibility pattern to manage all status.
Modify the expected-state in each step.
Why I did it
Currently sonic-slave-* tag is confusing. Set correct tag on sonic-slave-* image.
Fix job name to fit the build.
How I did it
build amd image in amd64:
sonic-slave-bullseye:cfe29bff67c
sonic-slave-bullseye:latest
sonic-slave-bullseye:master
build armhf image in amd64:
sonic-slave-bullseye-march-armhf:33614806dc3
sonic-slave-bullseye-march-armhf:latest
sonic-slave-bullseye-march-armhf:master
build arm64 image in amd64:
sonic-slave-bullseye-march-arm64:f3b1b16c801
sonic-slave-bullseye-march-arm64:latest
sonic-slave-bullseye-march-arm64:master
build arm64 image in arm64:
sonic-slave-bullseye:75cb326c9a7
sonic-slave-bullseye-arm64:latest
sonic-slave-bullseye:master
build armhf image in armhf:
sonic-slave-bullseye:64d178951fc
sonic-slave-bullseye-armhf:latest
sonic-slave-bullseye:master
How to verify it
Why I did it
Makefile needs some dependencies from the Internet. It will fail for network related issue.
Retries will fix most of these issues.
How I did it
Add retries when running commands which maybe related with networking.
How to verify it
Previously, we set timeout in each step such as Lock testbed, Prepare testbed, Run test and KVM dump. When some issue suck like retry happens in one step, it will cause timeout error, but actually, it only needs more time to success. In this pr, we remove the timeout limit in each step and control the timeout outside in each job. When the job runs more than four hours, it will be cancelled.
Why I did it
Previously, we set timeout in each step such as Lock testbed, Prepare testbed, Run test and KVM dump. When some issue suck like retry happens in one step, it will cause timeout error, but actually, it only needs more time to success. In this pr, we remove the timeout limit in each step and control the timeout outside in each job. When the job runs more than four hours, it will be cancelled.
How I did it
Remove the timeout parameter in each step, and control the timeout outside in each job.
How to verify it
Set the timeout of one job to 4 hours, and when timeout happens, azure pipeline will cancel this job.
Signed-off-by: Yutong Zhang <yutongzhang@microsoft.com>
Why I did it
Publish docker saiserverv2 in the build pipeline.
How I did it
Cherry-pick #12842 from master to 202205 branch.
How to verify it
Run test in #12843 and it has been built out successfully.
Previously, we hard code the min and max numbers of instance in a plan. In this pr, we support passing the instance numbers of a testplan.
Why I did it
Previously, we hard code the min and max numbers of instance in a plan. In this pr, we support passing the instance numbers of a testplan.
How I did it
Use a variable to set the instance number.
Migrate t0-sonic test jobs to TestbedV2.
Why I did it
Migrate t0-sonic test jobs to TestbedV2.
How I did it
Add two parameters to create testplan.
Modify azure-pipelines.yml to run t0-sonic on tbv2.
Signed-off-by: Yutong Zhang <yutongzhang@microsoft.com>
Signed-off-by: jianquanye@microsoft.com
Why I did it
use the pr_test_scripts.yaml from sonic-mgmt's 202205 branch
How I did it
use the pr_test_scripts.yaml from sonic-mgmt's 202205 branch
co-authorized by: jianquanye@microsoft.com
Migrate the t0 and t1-lag test jobs in buildimage repo to TestbedV2.
Why I did it
Migrate the t0 and t1-lag test jobs in buildimage repo to TestbedV2.
How I did it
Migrate the t0 and t1-lag test jobs in buildimage repo to TestbedV2.
Remove ceos type setting
Use 202205 branch as sonic-mgmt branch
With the Broadcom syncd containers getting upgraded to Bullseye, the DNX
RPC container is no longer automatically built. Explicitly add a make
command to build it.
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
The asan-enabled docker image will be used in other CIs that run DVS tests.
Why I did it
To add a possibility to run DVS tests with ASAN for other CIs (e.g. swss).
How I did it
Added the 'asan_image' flag to the vs job group.
How to verify it
Run the CI and check the docker-sonic-vs-asan.gz artifact.
Why I did it
The t0-sonic pool has been fixed, so add it back to azp checker.
How I did it
Remove continueOnError in run-test-template.yml.
Signed-off-by: Ze Gan <ganze718@gmail.com>
* [ci] Support to skip vstest using include/exclude config file. (#11086)
example:
├── folderA
│ ├── fileA (skip vstest)
│ ├── fileB
│ └── fileC
If we want to skip vstest when changing /folderA/fileA, and not skip vstest when changing fileB or fileC.
vstest-include:
^folderA/fileA
vstest-exclude:
^folderA
* [build] Add version files to docker image dependencies
Why I did it
The docker storage driver vfs is not a good option for build, it uses the “deep copy” when building a new layer, leads to lower performance and more space used on disk than other storage drivers.
A better docker storage driver is the default one overlay2, it is a modern union filesystem.
Why I did it
Fix the target directory not empty issue when publishing artifacts.
Some of the artifacts are published to $(Build.ArtifactStagingDirectory)/target/ before source code checked out.
Why I did it
It is not necessary to trigger the publish pipeline when build is failed.
How I did it
Remove the condition in the azp task, change to use template condition.
Why I did it
Support to trigger a pipeline to download and publish artifacts to storage and container registry.
Support to specify the patterns which docker images to upload.
How I did it
Pass the pipeline information and the artifact information by pipeline parameters to the pipeline which will be triggered a new build. It is to decouple the artifacts generation and the publish logic, how and where the artifacts/docker images will be published, depends on the triggered pipeline.
How to verify it
Why I did it
Exclude the innovium build in upgrading version build, currently, the builds are always failed, exclude the build temporarily.
Increase the broadcom build timeout.
Fix the generating version file failure issue caused by artifacts folder change.
When changing to use the same template for PR build, official build and packages version upgrade, the artifacts folder adding a "target" folder, the version upgrade task should be changed accordingly.
Why I did it
docker hub will limit the pull rate.
Use ACR instead to pull debian related docker image.
How I did it
Set DEFAULT_CONTAINER_REGISTRY in pipeline.
Why I did it
Fix ENABLE_DOCKER_BASE_PULL not working issue in armhf/arm64
For build in native armhf/arm64, the expected container registry repo name is sonic-slave-<stretch|buster|bullseye>
How I did it
Publish the slave image to sonic-slave-<stretch|buster|bullseye>.
Why I did it
[Build]: Enable marvell-armhf PR check
Improve the azp dependencies, make the Test stage only depended on BuildVS stage. The Test stage will be triggered once the BuildVS stage finished, reduce the waiting time.
