Commit Graph

5984 Commits

Author SHA1 Message Date
Liu Shilong
4570365762
[202111] [build] Use public storage for public resources. (#18201)
* [build] Use public storage for public resources. (#18038)
2024-02-29 13:10:04 -08:00
Liu Shilong
f2a2906b66
Pin typing extension (#16513)
Why I did it
Fix UT failure caused by typing-extensions version update.

Work item tracking
Microsoft ADO (number only): 25123371
How I did it
How to verify it
2023-09-13 16:12:15 +08:00
xumia
8fd9ce758b
[Build][202111] Fix the PyYang python package installation issue (#15890) (#15905)
* [Build] Fix the PyYang python package installation issue (#15890)

Why I did it
Fix the armhf build failure.
How to reproduce the issue:

docker run -it debain:bullseye bash
apt-get update && apt-get install -y python3-pip
pip3 install PyYAML==5.4.1
Error message:

Collecting PyYAML==5.4.1
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /tmp/tmp6xabslgb_in_process.py get_requires_for_build_wheel /tmp/tmp_er01ztl
....
      raise AttributeError(attr)
  AttributeError: cython_sources
  ----------------------------------------
WARNING: Discarding d63f2d7597/PyYAML-5.4.1.tar.gz (sha256)=607774cbba28732bfa802b54baa7484215f530991055bb562efbed5b2f20a45e (from https://pypi.org/simple/pyyaml/) (requires-python:>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*). Command errored out with exit status 1: /usr/bin/python3 /tmp/tmp6xabslgb_in_process.py get_requires_for_build_wheel /tmp/tmp_er01ztl Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement PyYAML==5.4.1
ERROR: No matching distribution found for PyYAML==5.4.1
root@fa2fa92edcfd:/#
But if adding the option --no-build-isolation, then it is good, see fix.

install "PyYAML==5.4.1" --no-build-isolation
The same error can be found in the multiple builds.

Work item tracking
Microsoft ADO (number only): 24567457

How I did it
Add a build option --no-build-isolation.

* Fix docker-platform-monitor python2 issue

* Fix wheel dependency issue
2023-07-22 09:25:38 +08:00
mssonicbld
3f50123cbf
[ci/build]: Upgrade SONiC package versions (#15622) 2023-06-25 22:45:45 +08:00
mssonicbld
700571c77e
[ci/build]: Upgrade SONiC package versions (#15616) 2023-06-24 22:22:24 +08:00
mssonicbld
fd244514ff
[Build] Remove the additional space character in the mirrors.list file (#13812) (#15475)
Why I did it
Fix all mirror is commented out in sources.list in slave image issue. It will have an issue when installing more packages in the slave container.

It will add additional space character after running add-apt-repository command.

For example:
The original config in /etc/apt/sources.list

#deb [arch=amd64] http://deb.debian.org/debian/ bullseye main contrib non-free
Run the following command:

add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable"
Then the setting changed to: (added a new space character after #)

# deb [arch=amd64] http://deb.debian.org/debian/ bullseye main contrib non-free
How I did it
Fix the regex string to add the space pattern. After fixed, whether there is a space character or not, it will not be an issue.

How to verify it

Co-authored-by: xumia <59720581+xumia@users.noreply.github.com>
2023-06-19 17:10:33 +08:00
mssonicbld
a61044404b
[ci/build]: Upgrade SONiC package versions (#15529) 2023-06-18 22:52:06 +08:00
mssonicbld
7fda2dbb03
[ci/build]: Upgrade SONiC package versions (#15524) 2023-06-17 22:31:09 +08:00
mssonicbld
0bab1c8253
[ci/build]: Upgrade SONiC package versions (#15460) 2023-06-14 22:11:22 +08:00
mssonicbld
28b5f2862e
[ci/build]: Upgrade SONiC package versions (#15428) 2023-06-11 21:30:47 +08:00
mssonicbld
700df4fc7e
[ci/build]: Upgrade SONiC package versions (#15427) 2023-06-10 23:22:06 +08:00
Liu Shilong
e1291514db
[ci] Add OVERRIDE_BUILD_OPTIONS in image build template. (#15309) (#15340)
Why I did it
Set build options in pipeline UI.
Support setting reproducible build options to py2,py3 in release branch and none in master branch.

Work item tracking
Microsoft ADO (number only): 22335854
How I did it
How to verify it
2023-06-08 09:42:45 +08:00
mssonicbld
a4059c03aa
[ci/build]: Upgrade SONiC package versions (#15363) 2023-06-07 03:53:59 +08:00
mssonicbld
c5fb6bde30
[ci/build]: Upgrade SONiC package versions (#15330) 2023-06-04 23:55:53 +08:00
mssonicbld
2b75d62ea9
[ci/build]: Upgrade SONiC package versions (#15327) 2023-06-03 23:23:07 +08:00
mssonicbld
d2fca16f17 [ci/build]: Upgrade SONiC package versions 2023-06-01 00:35:57 +08:00
mssonicbld
18be3ba279 [ci/build]: Upgrade SONiC package versions 2023-05-29 00:34:07 +08:00
mssonicbld
f137a9c3b9
[ci/build]: Upgrade SONiC package versions (#15244) 2023-05-27 23:04:40 +08:00
mssonicbld
bbbd54b0cc
[ci/build]: Upgrade SONiC package versions (#15201) 2023-05-24 23:14:02 +08:00
xumia
39241decdd [Build] Fix the reproducible build variable display error in the slave container (#14543)
Why I did it
Enable the reproducible build for PR build for master branch

Fix the reproducible build variable display error in the slave container.
The below config is none, although the config is set and takes effect.

"SONIC_VERSION_CONTROL_COMPONENTS": "none"
How I did it
Passing the variable through the slave container command line.
The variable has been passed to the slave container and the other docker container by a config file, it is only used to display the value during the build.

How to verify it
See https://dev.azure.com/mssonic/build/_build/results?buildId=247960&view=logs&j=88ce9a53-729c-5fa9-7b6e-3d98f2488e3f&t=88f376cf-c35d-5783-0a48-9ad83a873284

"SONIC_VERSION_CONTROL_COMPONENTS": "deb,py2,py3,web,git,docker"
2023-05-23 18:16:41 +08:00
mssonicbld
6bbe2b63c5
[ci/build]: Upgrade SONiC package versions (#15175) 2023-05-22 00:27:52 +08:00
mssonicbld
01558db46a
[ci/build]: Upgrade SONiC package versions (#15166) 2023-05-20 23:31:55 +08:00
Cédric Ollivier
90a0cdc720 [build]: Force xz as compression type when building sonic-build-hooks debs (#12823)
Ubuntu 22.04 leverages Zstandard compression to dpkg by default.
Debian doesn't support it yet
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=892664

Fix #12822

Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2023-05-19 18:32:13 +08:00
xumia
2e40fd4e11 [Build] Fix the mirror gpg key expired issue (#14206)
Why I did it
[Build] Fix the mirror gpg key expired issue
See vs build: https://dev.azure.com/mssonic/build/_build/results?buildId=231680&view=logs&j=cef3d8a9-152e-5193-620b-567dc18af272&t=cf595088-5c84-5cf1-9d7e-03331f31d795

How I did it
Add the apt option not to check the valid until, the option is set to the SONiC docker base image, docker ptf missing the option.

Acquire::Check-Valid-Until "false";
How to verify it
The build of docker-ptf is succeeded after fixed.

2023-03-11T17:26:35.1801999Z [ building ] [ target/docker-ptf.gz ] 
2023-03-11T17:38:10.1608536Z [ finished ] [ target/docker-ptf.gz ]
2023-05-19 18:32:08 +08:00
Konstantin Vasin
7a8717073e [Build] update python package docker in host image to 6.1.1 (#14993)
Fix #14974
Refs: https://github.com/docker/docker-py/pull/3116
2023-05-19 16:32:39 +08:00
Liu Shilong
6efb96c040
Clear /etc/resolv.conf before building image (#12592) (#15104)
Cherry pick PR#12592

Why I did it
nameserver and domain entries from build system fsroot gets into sonic image.

How I did it
Clear /etc/resolv.conf before building image

How to verify it
Built image with it and verified with install that /etc/resolv.conf is empty

Co-authored-by: Devesh Pathak <54966909+devpatha@users.noreply.github.com>
2023-05-18 14:47:17 -07:00
mssonicbld
7ec9a1f6f1
[ci/build]: Upgrade SONiC package versions (#14924) 2023-05-04 00:16:43 +08:00
mssonicbld
c4c7f497b2
[ci/build]: Upgrade SONiC package versions (#14890) 2023-04-29 22:58:32 +08:00
mssonicbld
220d87eb54
[ci/build]: Upgrade SONiC package versions (#14811) 2023-04-23 22:40:00 +08:00
mssonicbld
7ce9a7f3bb
[ci/build]: Upgrade SONiC package versions (#14802) 2023-04-22 23:59:41 +08:00
mssonicbld
98d6c46ce0
[ci/build]: Upgrade SONiC package versions (#14721) 2023-04-19 23:29:42 +08:00
mssonicbld
3a2c16c5dd [ci/build]: Upgrade SONiC package versions 2023-04-19 06:32:08 +08:00
xumia
40780c072e
[Build][202111] Support Debian snapshot mirror to improve build stability (#14664)
Why I did it
Cherry-pick commits from master to support the snapshot based mirror, and fix the code conflicts.

ad162ae [Build] Optimize the version control for Debian packages (#14557)
38c5d7f [Build] Support j2 template for debian sources for docker ptf (#13198)
5e4826e [Ci] Support to use the same snapshot for all platform builds (#13913)
8206925 [Build] Change the default mirror version config file (#13786)
5e4a866 [Build] Support Debian snapshot mirror to improve build stability (#13097)
ac5d89c [Build] Support j2 template for debian sources (#12557)

Work item tracking
Microsoft ADO (number only): 18018114
How I did it
How to verify it
2023-04-18 14:32:31 +08:00
mssonicbld
0e5a2b9b3e
[ci/build]: Upgrade SONiC package versions (#14682) 2023-04-16 23:16:36 +08:00
mssonicbld
cbb77909c1
[ci/build]: Upgrade SONiC package versions (#14675) 2023-04-16 01:23:12 +08:00
mssonicbld
cb21b2126e
Pin mmh3 package version in sonic-slave-stretch docker (#14463) (#14470)
Why I did it
mmh3's new version 3.1.0 breaks pipeline build.
bullseye/buster/jessie pined the version to 2.5.1

How I did it
Pin mmh3's version as other dists.

How to verify it

Co-authored-by: Liu Shilong <shilongliu@microsoft.com>
2023-04-14 18:42:14 +08:00
Ravi [Marvell]
c73b81eb95
Add platform files for Innovium platform (#12652)
Why I did it
Add platform files for critical processes and default qos config for Innovium platforms

How I did it
Added default files for critical processes and qos config

How to verify it
Tested with autorestart/test_container_autorestart.py::test_containers_autorestart

Signed-off-by: rck-innovium rck@innovium.com
2023-03-30 11:33:51 -07:00
Liu Shilong
969de02c62
[ci] Fix build issue for vs vhdx image. #14426
Why I did it
sonic-slave-stretch build failed for mmh3 version update to 3.10 on Mar 24.

How I did it
Enable reproducible build for vhdx image.

How to verify it
2023-03-29 16:13:01 +08:00
Liu Shilong
502fe49ae9 [ci] Fix build issue for vs vhdx image. #14427
Why I did it
sonic-slave-stretch build failed for mmh3 version update to 3.10 on Mar 24.

How I did it
Enable reproducible build for vhdx image.

How to verify it
2023-03-28 20:53:23 +08:00
mssonicbld
fb319b4b36
[ci/build]: Upgrade SONiC package versions (#14315) 2023-03-19 23:09:51 +08:00
mssonicbld
a86de8aee1 [ci/build]: Upgrade SONiC package versions 2023-03-18 09:51:33 +08:00
Liu Shilong
8b8b6bd2b7 [build] Fix unexpected warnings: No names found, cannot describe anything. (#12963)
Why I did it
In PR check pipelines, there are too many duplicated warnings:
fatal: No names found, cannot describe anything.
SONIC_IMAGE_VERSION will not change in one build. We don't need to calculate in every reference. We just need calculate one time, then record it.
In Makefile, '=' will calculate again and again when it is referred.

How I did it
Fix it in Makefile.

How to verify it
Check this PR's check pipeline result.
2023-03-10 18:37:35 +08:00
mssonicbld
19e86f9f60
[ci/build]: Upgrade SONiC package versions (#14082) 2023-03-05 10:04:08 +08:00
mssonicbld
a86cadf13c
[ci/build]: Upgrade SONiC package versions (#13944) 2023-02-27 02:09:19 +08:00
mssonicbld
5ae5c053fc [ci/build]: Upgrade SONiC package versions 2023-02-22 22:33:20 +08:00
Liu Shilong
041dc55eb5 [ci] Fix docker hang issue and change template reference branch (#13894)
Why I did it
Azure pipeline change.
Use common template to make it easy to change common steps.
Fix docker hang issue.

How I did it
2023-02-22 20:55:37 +08:00
Liu Shilong
9d20dda40a
[ci] Kill hanged docker build process to avoid build timeout issue. (#13726) (#13597)
Why I did it
Docker build has a low rate of hanging up.
It hangs on different steps. So, it looks like a bug in docker daemon.

How I did it
Start a daemon process to scan running time more than 1 hours, and kill the process.

How to verify it
2023-02-22 13:15:53 +08:00
Liu Shilong
0798518244 [build] Check if patches are applied before applying patches (#13386)
Why I did it
If make fails, we can't rerun the make process, because existing patches can't apply again.

How I did it
Check if patches are applied. if yes, don't apply patches again.

How to verify it
2023-02-03 04:32:32 +08:00
jcaiMR
ed63f92b34
[202111] fix relay-reply dhcpv6 packet counter issue 2023-01-30 17:35:55 -08:00
Liu Shilong
f1f42c9a6a
Fix sonic slave pipeline to set correct tag on sonic slave image. (#13177) (#13384) 2023-01-18 11:32:19 +08:00