Commit Graph

30 Commits

Author SHA1 Message Date
Shilong Liu
5b22cba8ca [build] Fix issues found in reproducible build. (#10407) 2022-03-31 20:39:49 +08:00
xumia
8d0aea0f1b [build][Bug]: Fix the command set_reproducible_mirrors not found issue (#10398)
Why I did it
Fix the command set_reproducible_mirrors not found issue during the build.
2022-03-31 06:12:20 +00:00
xumia
674dc6e4c4 [Build]: Fix host image debian package version issue (#10358)
Why I did it
Fix host image debian package version issue.
The package dependencies may have issue, when some of debian packages of the base image are upgraded. For example, libc is installed in base image, but if the mirror has new version, when running "apt-get upgrade", the package will be upgraded unexpected. To avoid such issue, need to add the versions when building the host image.

How I did it
The package versions of host-image should contain host-base-image.
2022-03-29 04:36:43 +00:00
Shilong Liu
3455e99d45
Add a config variable to override default container registry instead of dockerhub. (#10166) (#10262)
* Add variable to reset default docker registry
* fix bug in docker version control
2022-03-18 12:01:52 +08:00
xumia
5670f25416 [Build]: Fix tmpfs space not enough issue when building vs image (#9438)
Fix no space left on device issue in tmpfs.
2021-12-01T06:30:40.1651742Z cp: write error: No space left on device
2021-12-01T06:30:40.1652225Z Failure: local_fs_run():/dev/vdb Unable to copy /tmp/tmp.gl4Sgp/onie-installer.bin to tmpfs
2021-12-06 21:32:29 +00:00
xumia
1e8fe7dd6c Fix armhf version issue (#9382)
Why I did it
Fix some of the version files not used issue.
One of example version file version-py3-all-armhf, when building marvell-armhf, the version is used as expected, but it not use.
2021-12-01 02:29:07 +00:00
Shilong Liu
6b69ddc861 [build] Fix reproducible build issues (#8548)
* [build] Fix reproducible build issues
2021-08-26 10:22:02 +00:00
Shilong Liu
e5428ecb2d Reproducible build add docker image debian* to white list. (#8330)
#### Why I did it
1. Add version control for debian* docker image to white list.
2. Always record docker image sha256 value, regardless of white list.
2021-08-12 07:09:01 +00:00
Shilong Liu
af8cd03d18 Reproducible build support docker image (#8039)
when build docker image, we need a base image. now base image is specified by tag. we can use sha256 to specify docker image.
2021-07-13 05:14:02 +00:00
xumia
1f75c4fbd2 Fix the apt-get download package disorder issue (#8021)
Why I did it
Multiple build failed in 202012 branch
It is caused by the disorder of the package urls retrieved from the command "apt-get download --print-urls "
2021-06-30 07:47:14 +00:00
Yilan
7424a13775 [build]: Update versions_manager.py to make versions map key unique (#7146)
py2/py3/deb packages names are case insensitive, and the versions map
key should be the same for packages whose name can have different cases.

For example, in files/build/versions/default/versions-py3, package
"click==7.1.2" is pinned; and in
files/build/versions/dockers/docker-sonic-vs/versions-py3, package
"Click==7.0" is pinned.
Without this fix, the aggregated versions-py3 file used for building
docker-sonic-vs looks like below:
...
click==7.1.2
Click==7.0
...
However, we actually want "click==7.0" to overwrite "click==7.1.2" for
docker-sonic-vs build.
2021-05-14 04:32:41 +00:00
xumia
79dd3b15e7
Fix the /var wrong permission issue (#7571)
Why I did it
Build broken when reproducible build enabled, relative to backport package systemd.
The root cause is the bad permission of /var.
2021-05-11 18:04:01 +08:00
guxianghong
a0fde3a626 [arm] support compile sonic arm image on arm server (#7285)
- Support compile sonic arm image on arm server. If arm image compiling is executed on arm server instead of using qemu mode on x86 server, compile time can be saved significantly.
- Add kernel argument systemd.unified_cgroup_hierarchy=0 for upgrade systemd to version 247, according to #7228
- rename multiarch docker to sonic-slave-${distro}-march-${arch}

Co-authored-by: Xianghong Gu <xgu@centecnetworks.com>
Co-authored-by: Shi Lei <shil@centecnetworks.com>
2021-05-02 08:11:56 -07:00
lguohan
17148d70fc [kvm]: add debug cmd for build_kvm_image.sh (#6472)
dump netstat info on error

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2021-01-19 01:11:22 -08:00
xumia
a79eb88052 Install the latest version of the sonic build hooks in slave container (#6348) 2021-01-06 06:18:27 -08:00
xumia
0ac81e3c96 Fix the hostimage version path permission issue (#6337) 2021-01-03 06:04:05 -08:00
xumia
0a36de3a89
Recover "Support SONiC Reproduceable Build-debian/pip/web packages (#6255)
* Revert "Revert "Support SONiC Reproduceable Build-debian/pip/web packages (#5718)""

This reverts commit 17497a65e3.

* Revert "Revert "Remove unnecessary sudo authority in build Makefile (#6237)""

This reverts commit 163b7111b5.
2020-12-21 15:31:10 +08:00
Guohan Lu
17497a65e3 Revert "Support SONiC Reproduceable Build-debian/pip/web packages (#5718)"
This reverts commit 55a707586b.
2020-12-18 23:37:27 -08:00
xumia
55a707586b
Support SONiC Reproduceable Build-debian/pip/web packages (#5718)
* Support SONiC reproduceable build for deb/py2/py3/web

* Remove j2 files

* Fix bug

* Fix some issues

1. Change some code format issues
2. Fix curl calling wget command, pip2 calling pip3 issue
3. Fix wget/curl downloading multiple urls issue

* Fix some code format issue

* Fix bug

* Fix bug

* Fix command path hard code in build info scripts issue

* Add debian package sonic-build-tools

* Fix auto debian package removed issue

* Change build debian package name, and change the folder

* Collect the pre-versions and post-versions

* Change to use debian:buster

* Remove apt-mark and improve code

* Remove set_build_hooks

* Change docker trusted gpg files

* Fix docker build COPY directory name issue

* Move the trusted gpg files into the sonic-build-hooks package
2020-12-17 13:06:53 +08:00
xumia
918ce2a633
Simplify the function to covert pfx certificate (#5086) 2020-08-01 20:07:20 +08:00
xumia
81e0777b47
Support to convert certificate from pfx to pem format (#5073)
* Support to convert certificate from pfx to pem format

* Fix code issue
2020-07-31 12:24:54 +08:00
xumia
e28cb8f58e
[Secure Boot] Support to sign swi image (#4627)
* [secure boot] Support to sign swi image

* Fix build issue

* fix tab format issue

* Fix typing issue

* Change the sign_image.sh command line

* Remove SONIC_CETIFICATE_PATH

* Fix bugs
2020-06-09 15:25:17 +08:00
Guohan Lu
f1cc577c75 [build_kvm]: mount /proc to show memory
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-04-17 04:51:51 +00:00
lguohan
5299644dc0
[kvmbuild]: print out kvm_log on error (#4384)
Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-04-07 12:31:20 -07:00
lguohan
20260ceb1d
[build]: add SONIC_CONFIG_BUILD_LOG_TIMESTAMP to add timestamp in build log (#4269)
add timestamp in each job build log

example:

   [01:39:21] dh clean  --with autotools-dev
   [01:39:22]    dh_auto_clean
   [01:39:27]      make -j16 distclean

Signed-off-by: Guohan Lu <lguohan@gmail.com>
2020-03-21 14:21:26 -07:00
Guohan Lu
bd72844f75 [kvm]: increase the kvm installer size to 2G for dbg image
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-12-28 21:21:09 +00:00
lguohan
8d4516ca3d
[kvm]: increase mem to 3G to avoid OOM during onie installation (#3811)
Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-11-23 12:30:22 -08:00
Renuka Manavalan
14458b79ba
Create the src archive in target/ when INSTALL_DEBUG_TOOLS=y (#3323)
* Create the src archive in target/ when INSTALL_DEBUG_TOOLS=y

* Dropped -v (verbose flag) from tar command
2019-08-28 09:29:48 -07:00
Renuka Manavalan
76bf5a0bc4 [build]: Added debug symbols to many debug dockers. (#3098)
* Added debug symbols to many debug dockers.

* For debug images *only*:
1) Archive source files into debug image
2) Archived source is copied into /src
3) Created an empty dir /debug
4) Mount both /src as ro & /debug as rw into every docker
5) Login banner will give some details on /src & /debug
6) Devs can copy core file into /debug and view it from inside a container.
7) Dev may create all gdb logs and other data directly into /debug.

* Dropped redundant REDIS_TOOLS per review comments.

* Added debug symbols to frr package and hence FRR based BGP docker.

* 1) Moved dbg_files.sh to scripts/
2) Src directories to archive are now collected from individual Makefiles.
3) Added few more debug symbols
4) Added few more debug dockers.

Here after no more changes except per review comments.

To debug:
Install required version of debug image in Switch or VM.
Copy core file into /debug of host
Get into Docker
gdb /usr/bin/<daemon> -c /debug/<your core file>
set directory /src/... <-- inside gdb to get the source

For non-in-depth debugging:

Download corresponding debug Docker image (docker-...-dbg.gz) to your VM
Load the image
Run image with entrypoint as 'bash' with dir containing core mapped in.
Run gdb on the core.
2019-07-03 22:13:55 -07:00
lguohan
40330bcbdf
[build]: wait up to 60 seconds for docker engine to start (#3067)
* [build]: wait 60 seconds for docker engine to start

On some platforms, it can take more than 1 second for docker
engine to start.

Signed-off-by: Guohan Lu <gulv@microsoft.com>
2019-06-22 14:40:05 -07:00