Why I did it
Use remote PR test template from sonic-mgmt master to run PR test.
How I did it
Modify PR test azure pipeline yml file.
How to verify it
PR test executing normally.
Signed-off-by: Chun'ang Li <chunangli@microsoft.com>
Why I did it
Refine PR test template format.
How I did it
Refine PR test template format.
How to verify it
PR test executed normally.
Signed-off-by: Chun'ang Li <chunangli@microsoft.com>
Why I did it
Add marvell-arm64 platform build in PR checks to avoid build break.
Work item tracking
Microsoft ADO (number only): 17257160
How I did it
How to verify it
Why I did it
Remove 'kvmtest-t0' and 'kvmtest-t1-lag' test jobs since all the test jobs are required (continueOnError: false) already, and will only enable one of classical and testbedV2 tests, no need to do an unnecessary 'or' compute test job.
Change classic multi-asic and t0-sonic test jobs as required (continueOnError: false) following current testbedV2's logic.
Recently, the job of t0-sonic, multi-asic and wan run stably in master branch, so in this pr, I set them mandatory in azure pipeline.
Why I did it
Recently, the job of t0-sonic, multi-asic and wan run stably in master branch, so in this pr, I set them mandatory in azure pipeline.
How I did it
Modify the value of continueOnError in each job from `true` to `false`.
Signed-off-by: Yutong Zhang <yutongzhang@microsoft.com>
Previously, we set timeout in each step such as Lock testbed, Prepare testbed, Run test and KVM dump. When some issue suck like retry happens in one step, it will cause timeout error, but actually, it only needs more time to success. In this pr, we remove the timeout limit in each step and control the timeout outside in each job. When the job runs more than four hours, it will be cancelled.
Why I did it
Previously, we set timeout in each step such as Lock testbed, Prepare testbed, Run test and KVM dump. When some issue suck like retry happens in one step, it will cause timeout error, but actually, it only needs more time to success. In this pr, we remove the timeout limit in each step and control the timeout outside in each job. When the job runs more than four hours, it will be cancelled.
How I did it
Remove the timeout parameter in each step, and control the timeout outside in each job.
How to verify it
Set the timeout of one job to 4 hours, and when timeout happens, azure pipeline will cancel this job.
Previously, we hard code the min and max numbers of instance in a plan. In this pr, we support passing the instance numbers of a testplan.
Why I did it
Previously, we hard code the min and max numbers of instance in a plan. In this pr, we support passing the instance numbers of a testplan.
How I did it
Use a variable to set the instance number.
Add wan topo test using TestbedV2.
Why I did it
Add wan topo test using TestbedV2.
How I did it
Add a job to run wan topo in pr test.
Signed-off-by: Yutong Zhang <yutongzhang@microsoft.com>
Migrate multi-asic test jobs to TestbedV2.
Why I did it
Migrate multi-asic test jobs to TestbedV2.
How I did it
Add one parameter num_asic to create testplan.
Modify azure-pipelines.yml to run multi-asic on tbv2.
Signed-off-by: Yutong Zhang <yutongzhang@microsoft.com>
Migrate t0-sonic test jobs to TestbedV2.
Why I did it
Migrate t0-sonic test jobs to TestbedV2.
How I did it
Add two parameters to create testplan.
Modify azure-pipelines.yml to run t0-sonic on tbv2.
Signed-off-by: Yutong Zhang <yutongzhang@microsoft.com>
Add dualtor test using TestbedV2 in buildimage repo.
Why I did it
Add dualtor test using TestbedV2 in buildimage repo.
How I did it
Add dualtor test using TestbedV2 in buildimage repo.
Signed-off-by: Yutong Zhang <yutongzhang@microsoft.com>
co-authorized by: jianquanye@microsoft.com
Migrate the t0 and t1-lag test jobs in buildimage repo to TestbedV2.
Why I did it
Migrate the t0 and t1-lag test jobs in buildimage repo to TestbedV2.
How I did it
Migrate the t0 and t1-lag test jobs in buildimage repo to TestbedV2.
Why I did it
Fix PR merge failed because 'vstest' step does not install libyang.
How I did it
Install libyang in azure pipeline.
How to verify it
Pass vstest step.
Why I did it
With continueOnError: true, a failed job returns the result: partiallySuccess, which cause it can't be rerun, since AZP consider it as passed. Then we can't only rerun t0 jobs when it fails.
How I did it
Mark t0 part1 and part2 as continueOnError: false.
How to verify it
The pipeline will verify it.
* Define whether a test is required by code
Why I did it
Define whether a test job is required before merging by code.
Let the failure of multi-asic and t0-sonic don't block pr merge.
The 'required' configuration can be modified by the owner in the future.
How I did it
Required:
t1-lag, t0
Not required:
multi-asic, t0-sonic
How to verify it
AZP itself verifies it.
Signed-off-by: jianquanye@microsoft.com
Why I did it
Require multi-asic VS tests to be run during PR checks and merges in master branch.
How I did it
Add job to run multi-asic VS tests in sonic-buildimage pipeline. Currently pipeline will run basic bgp fact test to ensure the testbed comes up, load minigraph works and bgp sessions are up.
Use new multi-asic VS agent pool sonictest-ma in official-multi-asic-vs pipeline
Make multi-asic VS test optional for now. There are two known issues:
Announce route failure during refresh-dut in setup testbed stage.
bgp sessions not getting established.
How to verify it
Tested using test azure pipelines.
Why I did it
[multi-asic]: Add build param to build multi-asic KVM image.
Require sonic-4asic-vs.img.gz to run multi-asic tests.
BUILD_MULTIASIC_KVM=y has to be added to build parameter to generate multi-asic KVM image which can be eventually used in multi-asic VS testbed.
This is a pipeline that we currently have to generate multi-asic KVM image:
https://github.com/Azure/sonic-buildimage/blob/master/.azure-pipelines/official-build-multi-asic.yml
How I did it
Add BUILD_MULTIASIC_KVM=y to build parameter in pipeline script to generate multi-asic KVM image.
How to verify it
The build param is already used in official-build-multi-asic.yml pipeline.
Why I did it
When any of the test job failed in the test stage, the rerun will not work, the test stage will be skipped automaticall, so we do not have chance to rerun the test stage again, and the checks of the test will be always in failed status, block the PR to merge forever.
It should be caused by the condition in the Test stage, we should specify the status of the BuildVS stage.
How I did it
Fix stage dependency logic.
example:
├── folderA
│ ├── fileA (skip vstest)
│ ├── fileB
│ └── fileC
If we want to skip vstest when changing /folderA/fileA, and not skip vstest when changing fileB or fileC.
vstest-include:
^folderA/fileA
vstest-exclude:
^folderA
Why I did it
Some PR test are timing out on T1-lag kvm test.
How I did it
Increase the timeout to 5 hours.
How to verify it
Test on this PR.
Signed-off-by: Ying Xie ying.xie@microsoft.com
Why I did it
docker hub will limit the pull rate.
Use ACR instead to pull debian related docker image.
How I did it
Set DEFAULT_CONTAINER_REGISTRY in pipeline.
Why I did it
[Build]: Enable marvell-armhf PR check
Improve the azp dependencies, make the Test stage only depended on BuildVS stage. The Test stage will be triggered once the BuildVS stage finished, reduce the waiting time.
Why I did it
Introduce 2 sub jobs for kvmtest t0 job in sonic-mgmt repo in PR Azure/sonic-mgmt#4861
But in sonic-buildimage repo, because section parameter is null, it always run the part 2 test scripts in kvmtest t0 job.
It missed the part 1 test scripts in kvmtest.sh.
How I did it
Split kvmtest t0 job into two sub jobs such as sonic-mgmt repo and run them in parallel to save time.
How to verify it
Submit PR will trigger the pipeline to run.
Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
Why I did it
Fix the boolean value case sensitive issue in Azure Pipelines
When passing parameters to a template, the "true" or "false" will have case sensitive issue, it should be a type casting issue.
To fix it, we change the true/false to yes/no, to escape the trap.
Support to override the job groups in the template, so PR build has chance to use different build parameters, only build simple targets. For example, for broadcom, we only build target/sonic-broadcom.bin, the other images, such as swi, debug bin, etc, will not be built.
